Well, V-Ray takes exposure into account when doing its calculations. So if you have a scene where you make a transition from exterior to interior, you might get issues when parts of the interior are visible from the exterior shots. This is because they will be very dark, and V-Ray will use low-precision samples there. Later on, when the camera moves in the interior space and these areas are brightened, the lower precision will show up as splotches.
There are several ways to deal with this; one is to use a constant exposure when calculating the irradiance map; another is to decrease the Noise threshold of the DMC sampler and increase the Min. samples.