No Load Maps when Disp OFF Globally

Is it possible to not load the displacement maps when they are turned off globally.

Thanks,
--Jon