Try adding VrayLighting and VrayGlobalIllumination as render elements when doing these tests and you’ll see how Vray treats the different methods. As DADAL says, when doing IBL from override you only get indirect light. This is the reason the lighting is smooth, but also the reason why you won’t get the nice, sharp direct light and shadows. When adding the HDRi into the VrayDomeLight, add more shadow samples and the noise should go away. To get faster, nicer and more balanced lighting with this method, I recommend our own Akin Bilgic’s Vray Optimization Guide which step-by-step shows how to get the best results with these lighting setups (and most others).