Displace USGS elevation

When you grab one of those 16-bit grayscale elevation maps from USGS, how high do you need to set the Displacement modifier on your mesh for an accurate elevation?