Light Cache RAM limit

[tysonibele]

I've got a scene with about 70 Phoenix containers. It renders pretty speedily with light caching enabled in the smoke options of all the containers, but RAM usage becomes a bit unwieldy (around 100 gigs) and on some machines on our renderfarm with less than 100 gigs of RAM, the machines start to stall or crash while rendering as their RAM fills and they begin switching to the disk swap or simply become unstable during light cache calcs.

It would be nice if a max RAM usage limit could be enabled globally for a scene, so that subsequent containers would not perform a light cache if the total RAM used by container light caches thus far exceeds a certain value. At first I thought a simpler solution would be to skip light caching if RAM usage exceeds a certain percentage of total RAM, but that would mean computers with different amounts of RAM would render frames differently since some might allow for more light caching then others. Putting a hard limit on total light cache usage would keep all frames consistent, so long as container cache ordering is deterministic (which I believe it is). Ie, if each container takes 1 gig to cache, I have 100 containers and set the limit to 50 gigs, containers after the 50th container will have light caching skipped during processing, regardless of total RAM on a particular machine.

Anyways, for now my solution was just to disable caching on a random selection of containers manually, but some kind of automated limiter would be very helpful.

[Svetlin.Nikolov]

Hey, yes, there are a total of 3 grid volume rendering options that create extra grids per each instance:
- Fire Lights -> Self-shadowing = Grid-based: This one can be reused between instances, but it currently isn't.
- Smoke Color -> Volume Light Cache: Unfortunately this one has to be unique per each instance; This is pretty much a hack though, and is not implemented at all for V-Ray GPU, or Corona for example. With time, more scenes are showing less benefit from using the grid volume light cache, so at one point it's probably gonna be OFF by default...
- Smoke Opacity -> Optimize Big Volumetric Grids: This one also creates a dense grid per each instance and it also can be reused between instances. The code has to mature a bit in order for this to happen, I hope not long from now.

We don't have a memory manager yet in Phoenix that would be able to tell how much memory goes where, so it's gonna take some time until we can actually have a sustainable automatic solution. Unfortunately until then it will be in the hands of the user to balance these options with the RAM usage..