Clearing exporter memory issues

[AlexOddbratt]

We are experiencing some weird issues with VRay randomly taking forever to clear/dump the memory, below is an excerpt from our render logs for a frame that just got stuck and as you can see it took about an hour to actually render the frame
and then it spent over 1.5 hours trying to clear the memory before our farm killed it because of inactivity. This seems to happen to random jobs and random frames so it's really hard to reproduce or debug on our side so I wanted to ask if this
has happened to any one else. I found an old thread from 2015 talking about a windows memory issue in 3dsmax and we seem to be getting the same error.
https://forums.chaosgroup.com/forum/...windows+memory

We are using windows 10, Maya 2017 Update 4 and VRay 3.60.01

[2017/Nov/20|18:10:21] V-Ray: Total sequence time 0h 58m 47.4s (3527.4 s)
[2017/Nov/20|18:10:21] V-Ray: Render complete
[2017/Nov/20|18:10:21] V-Ray: ========================
[2017/Nov/20|18:10:21] V-Ray: Clearing exporter memory...
Current time:11.20 19:49.11
Total processing time: 02:38.13
Max memory usage: 16.42GB
Max CPU usage: 16.0cores 100% (of 16 cores)
************************* CLIENT REPORTS DURING THIS RENDER *************************
Frame pre-processing time: 00.06 m.s
Job froze (low CPU usage since 99 minutes (18:10). Limit is set to 1.9 cores. Current system usage on one core: 0.0, current job usage 0.0, max job usage 16.0, non-RR usage: 0.0. Abortminutes set to 99min)
Error: Job frozen

[Svetlozar Draganov]

Can we have more information about the scene and the RAM usage.
There are certain cases where V-Ray release RAM slower than normally, this usually happen during bucket rendering when RAM is insufficient, for examples scenes with extreme displacement or subdivisions.
In some cases this could be a bug, we already fixed a few similar bugs but there might be another case which we haven't foreseen.

Can you explain when the issue takes place? How often does it happen? Is it on every scene or just on particular ones? As much information is provided as easier would be for us to troubleshoot it.

V-Ray has a special variable that could be helpful, its name is VRAY_TERMINATE_ON_FRAME_END, more information about it can be found on the following page:
https://docs.chaosgroup.com/display/...ment+Variables

[AlexOddbratt]

The scenes that are causing the most frequent issues are all of the same sort, we have a character that's subdivided and displaced quite a lot and our scenes have everything from 1 to 18 versions of this character loaded.
As the log shows this scene used 16+GB of ram and the heaviest version of these scene uses about 35-40GB. All our renderfarm machine have between 64-256GB of ram so any sort of swapping wouldn't be an issue.

The problem is that these errors occur randomly both in jobs, frames and machines and re-rendering the same frame tends to work, so the images comes out but it's a huge wast of time.
Roughly estimating the occurance I would probably put it at 1-2% of the frames rendered, which isn't a lot but the wasted time tends to add up when we're render thousands of frames per day.

Regarding the VRAY_TERMINATE_ON_FRAME_END variable. I would love to try it out as it might be a great temp fix for now. I do have a question about how it handles the shutdown process, ie does it shut the process down right after V-Ray: Render complete
and what sort of signal does it send. I want to make sure our render farm doesn't register it as an error or crash.