A guide to reducing RAM usage in V-Ray

[joconnell]

Heya Matt!

It's a command line program with no user interface so you won't get anything appearing and staying open. You browse to whatever folder the maketx files are in from your command line and then run it with whatever options you want. Sharktacos on the forum has a guide here, hit the "click to show command line usage" button - https://docs.sharktacos.com/texture/mipmap.html

I thiiiiiiink what you can also do is take an image that you want to convert to tx and drag it on to the maketx.exe file in window explorer and it'll auto convert. You won't get a choice over some of the options though and I'm not sure if it's smart enough to put the converted file back into the same folder as the original non-tx texture.

[^Lele^]

We have modified the v-ray utility which swaps the VRayHDRI loaders (and optionally converts to tiled textures).
It will now recognise the presence of the makeTX.exe file in the bin folder of your V-Ray installation (where img2tiledexr.exe resides as well) and use that instead.
It'll be in the sp1.

p.s.: i am totally not sure the makeTX from the arnold deploy matches ours. Regardless, we have our most performing (in conjunction with the vrayHDRI loader) conversion parameters as default, so the user isn't asked for a thing.

p.p.s.: in fact, we adopt the original bitmap's gamma value (ie. 1/2.2 for color textures, and 1 for data ones) as set in the max bitmap loader, further saving the user the need to rename maps to include a driving string (f.e. "_diff_").

[FlynnAD]

Joconnell and Lele,

Thank you for the responses; I'm using the maketx.exe now. Unfortunately, it would be helpful if there was a script to "post-convert" all vrayHDRI loader textures to .tx. I already converted the 3dsmax bitmap nodes to vrayHDRI ones without simultaneously converting them to tiled exrs. It seems that if the textures are already in the vrayHDRI loader, rather than the 3dsmax bitmap node, then vray doesn't recognize them to convert them again to tx. Am I wrong?

Also, if you are using a .tx file within a 3dsmax bitmap node and not a vrayHDRI loader, will vray benefit at all or use the associated mipmap sizes from the 3dsmax bitmap's .tx file? I ask that because one plugin in particular, Multitexture, automatically creates a slew of 3dsmax bitmap nodes for all the textures you select in its explorer window. A user CAN go manually swap these 3dsmax bitmap nodes to an identical vrayHDRI loader .tx textures and the rendering will still work. Except that doing it this way is manual, slow, and also breaks the interactivity of the Multitexture node. If you try going back into the Multitexture's explorer window to select or deselect some more bitmaps, it will not work.

Matt

[^Lele^]

Joconnell and Lele,

Thank you for the responses; I'm using the maketx.exe now. Unfortunately, it would be helpful if there was a script to "post-convert" all vrayHDRI loader textures to .tx. I already converted the 3dsmax bitmap nodes to vrayHDRI ones without simultaneously converting them to tiled exrs. It seems that if the textures are already in the vrayHDRI loader, rather than the 3dsmax bitmap node, then vray doesn't recognize them to convert them again to tx. Am I wrong?

Not insofar as i recall, no.
It's not difficult to extend the script in that direction, however, we'll look into it.

Also, if you are using a .tx file within a 3dsmax bitmap node and not a vrayHDRI loader, will vray benefit at all or use the associated mipmap sizes from the 3dsmax bitmap's .tx file? I ask that because one plugin in particular, Multitexture, automatically creates a slew of 3dsmax bitmap nodes for all the textures you select in its explorer window. A user CAN go manually swap these 3dsmax bitmap nodes to an identical vrayHDRI loader .tx textures and the rendering will still work. Except that doing it this way is manual, slow, and also breaks the interactivity of the Multitexture node. If you try going back into the Multitexture's explorer window to select or deselect some more bitmaps, it will not work.

The max bitmap loader is very very old, so you'll need the vrayHDRI loader to benefit from the improvements (with, or without tiled textures, that is.).
The converter script will take care of swapping the nodes for you (without making .tx files, if you so choose), but i have no idea of what that does to MultiTexture, it'll have to be looked into.

[Ihno]

Could you possibly indicate what applies to CPU only and what to CPU + GPU? Maybe with a color, something like:

B) VRayHDRI and Tiled Textures(GPU+CPU):
[...]
As such, the best way to optimize for ram usage is with Bucket rendering (CPU only).

It would take alot of guess work out and could be edited when things get implemented down the road.

[^Lele^]

The guide was written with the CPU engine only in mind.
I shall endeavor to update it for GPU.

[Ihno]

Awesome thanks! Maybe a new thread would be less confusing. Either way, I'm looking forward to it!

[Karol.Osinski]

Thank you so much ^Lele^ for sending me to this topic. The amount of technical info I found here feels like a present I've never asked for but got it anyway hahaha. Thanks again, great job!

[Joelaff]

A very important tip for using both VRayProxies and any tiled image formats when net rendering is to clone these resources to each render node. Why? Because if you net render with, say, 40 machines all accessing the shared drive with the tiled texture or proxy object it quickly eats up TONS of network resources, and slows down the render considerably. This network latency will actually become your bottleneck in many cases and the rendering will come to a crawl. !0GigE will help a lot, but it will still tax your server quite a bit.

We use a program called GoodSync to synchronize textures across render nodes. Every node shares a directory for use with these tiled resources. Before net rendering we run GoodSync to PUSH all the resources from one machine out to every node (this technique also avoids Windows arbitrary connection limits for non-"server" versions of Windows). So now every node has its own local copy of the tiled textures and proxies. The scene file then references the local copy.

Most scenes don't need VRayProxies or tiled textures. In fact, they usually slow things down vs. static geometry and fully loaded texturemaps.. But when your start running out of memory you have to make sacrifices and use these techniques.

[^Lele^]

I wholly agree on caching locally, and ideally on a fast drive.
When this is the case, not only there's a case to be made for memory savings, but often enough the tiled textures will outperform the non-tiled ones, and proxies, in the case of massive geo, will also perform on par or even quicker, than static geo.
It's ofc scene dependent, but i have measured the benefits myself a few times (f.e. when a scene is modifiers-heavy. then caches will fly, instead of the single-threaded modifiers evaluation).

[TheScope]

Nice write up. Thanks.
I have one gotcha to add for the tiled, bufferless rendering. It render result is a tiled EXRs. While this is nice for textures (as mentioned above), it is very bad for compositing apps like Nuke. They can only read them much much slower than scanline EXRs. So it is needed to convert them afterwards.

[^Lele^]

Yes indeed, thanks ever so much for the note!
The speed of multipart EXRs of the v 2.x type (i.e. anything modern) is however much better, compared to the old ones, but still a wee bit slower than scanline.
Will add this to the relevant chapter.

[Karol.Osinski]

Joelaff Good to know, I believe the GoodSync would help us to avoid missing proxies when DR on local network. Fact that we use both Macs and Window machines complicate things even further...Plus I think Windows 10 screws the proxy loading, it is something about admin/network rights which windows restarts every 90 min. so if you render few frames of animation of still renders you most likely find that proxies are missing from 2nd or 3rd frame...very annoying.