5th update: at least on the ram increase between frames, it seems to be related to the tree material. if i put a flat grey vraymtl on the trees, the ram usage stays constant. i tried just removing the 2sided from the leaves, but still had the issue.. i also found a “forest colour” map in the leaf diffuse, but removing that didnt help either.
Please send us your file to test it here. You can send it directly to our support mail or post it here.
ive sent an archive of the scene to support@chaosgroup.
there are a few issues as i see it:
1) forest support seems completely broken ( this scene has had the forest object converted to instances, so it will at least attempt to render)
2) RT seems to fail on a large number of instances even if they are well within the ram limits of the card
3) even when the number of instances is reduced significantly so the scene will render, the ram usage of vray.exe is enormous compared to a standard vray render, or an rt cpu render.
4) there is something about the materials on the tree that are causing the ram usage to increase with each frame rendered. ive not had time to test further, but 1, 2 and 3 might be related to 4 in some way.