For a scene like this (exterior, very diffuse lighting, almost no bouncing light), you might even get away without using GI, which would solve all your problems in a snap.
Did you manage to sort out your problem with an animation with moving objects ? I recently did one using high irr settings (high animation) and brute force with samples set to 30. You may want to look at the new ambientlight option to help you out with vraydirt AO.