3dsMax Process vs VRay Memory Usage

[mitchgates]

I'm running into some very strange behavior trying to figure out how to reduce memory usage while rendering a fairly large and complex scene. I've optimized most of the geometry, reduced texture sizes, converted some stuff to VrayProxy and using Xrefs where appropriate. What's most strange is that even though the Vray Memory Tracker reports around 37gb total usage, the memory footprint of 3dsMax during the render soars past 100gb. I'm running Max 2019.3 and Vray Next Update 2 and rendering via cpu engine.

Another very strange behavior is that the overall memory usage during rendering the same test frame with the same settings can vary quite a bit between 48gb and 120gb. The IDLE Max process between these test renders will sometimes show usage as high as 80gb and other times as low as 25gb. This changes without me doing anything in my scene or UI other than hitting render again and again to test things. I know Windows memory reporting is not the greatest, so not sure I can trust it, but the Vray memory tracker log consistently clocks in right at ~37gb. On the farm Deadline reports peak memory usage @ 120gb for this frame, but I'm assuming that relies on Windows as well.

Here's the last bit of the log file with the overall summary. Unsure how to reduce overhead of the 3dsMax process while rendering to be more in line with what Vray appears to be actually using.

Total
Type Peak memory consumption
Bitmap 14.04 GiB
Bucket image sampler 3.61 GiB
Dynamic displaced geometry 62.84 MiB
Dynamic geometry 608.18 MiB
Dynamic moving geometry 355.94 MiB
GI < 1 MiB
Lights 76.65 MiB
Misc. 9.00 GiB
Misc. geometry < 1 MiB
Misc. moving geometry < 1 MiB
Static geometry 3.92 GiB
Static moving geometry 3.13 GiB
Tiled bitmap 1020.33 MiB
VFB 937.47 MiB
Frame startup 23.32 GiB
Frame total 36.69 GiB

[^Lele^]

What bitmap loaders are you using, Mitch?
It'd look like either through xrefs or simple scene loading, max is polluting RAM by pre-loading stuff (the max bitmap loader would do this, depending also on viewport settings.).

[mitchgates]

Hi Lele!

Unfortunately it's a mix of VrayHDR and Max Bitmap loaders. There were multiple artists creating the assets in the scene, as well as use of existing models from legacy projects, etc. Kind of all over the map (no pun intended)

I'm going to try a test of merging my Xref objects into the master scene to see if that changes anything. Not really a long term solution since it would make the storage space and saving times extremely brutal.

Thanks,
Mitch

[^Lele^]

Before attempting the xref merging, try the (admittedly lengthy) conversion to .tx loaded with vraybitmap loaders only.
On the plus side, it's a process which will take you only a few clicks (i's he usual RMC menu), and won't make any tangible difference to scene file size.
It will, however, definitely cut down on file loading times and RAM usage with an open scene, even if shaded with textures is the chosen viewport mode.
It will make complex materials loading in the material editor much quicker than with full-size originals, and automatically save you heaps of RAM at rendertime (you can control how much to devote to TX files with the texture caching spinner.).
Maybe, to start with, try it on he master scene without any xRef, and see if that helps any.

If it works, you'll likely have to go through each xref scene and convert those too.
On a multi-core modern machine, you should be able to process a few concurrently, though.

Last but no least, push conversion to proxy as far as you feasibly can (i.e. don't go bake super-high detail deformations. but go to town with anything static, f.e., or simply transforming).
That too is only a few clicks, it *will* greatly reduce scene size and loading/saving times, RAM occupation with an open scene, RAM usage at rendertime (check "conserve memory" in the embree settings!) and that too can be done concurrently on modern hardware.
Just ensure you use one of the preview modes other than "load whole mesh".

[mitchgates]

Thanks for the tips, Lele. "Conserve Memory" under Embree is already checked. We did some conversion to TX in this scene a while back but didn't end up doing everything. I'll try converting all of the maps and see what that buys me. It's interesting to consider that Max is probably loading all the legacy texture maps into it's own memory overhead then offloading duplicates to VRay at render time. No wonder things get pushed beyond 100gb even though VRay is only reporting 38gb being used.

[^Lele^]

You have the gift i sorely lack, that of synthesis.
Yep, exactly what you said is my suspicion.

[mitchgates]

Any idea if the TX converter properly handles bitmaps loaders that have things like Output adjustments, UV offsets, blur amounts and cropping applied?

[mitchgates]

Hmmm...this might explain why windows reports such wildly changing memory usage. https://www.howtogeek.com/319933/wha...in-windows-10/

[^Lele^]

Any idea if the TX converter properly handles bitmaps loaders that have things like Output adjustments, UV offsets, blur amounts and cropping applied?

We do copy over every little bit of the bitmap loader: including the UV and the output section.
Unfortunately, output *curves* can't be duplicated (old max thing: they aren't accessible without a UI), so something to be aware of.

EDIT: wrong! Output curves CAN be duplicated. Conversion will work. It's the editing that cannot be done.

[mitchgates]

We do copy over every little bit of the bitmap loader: including the UV and the output section.
Unfortunately, output *curves* can't be duplicated (old max thing: they aren't accessible without a UI), so something to be aware of.

Good to know! Unfortunately there's a profuse amount of that going on in this scene, probably on the order of hundreds of maps!

[^Lele^]

edit: i was wrong.
Go ahead and convert, as we copy over the whole "output" object, the curve edits are kept verbatim.
Zoom on the curve section may be whacky, but that's a ax thing. zoom extents will fix the viewing of the curve.