Crash when using Light Cache and batch render

[oliver.sullivan]

Maya 2014 SP2
VRay 3 Final

I have two different scenes, both of which I'd like to render with GI using Irr Map / Light Cache. The scenes render happily within Maya, but if I do a batch render (or submit to our farm) it fails. Locally, I just get a "Rendering Completed" message after a minute or so, but no output. On the farm I get:

2014-11-06 09:58:37.191026 | Segmentation fault (core dumped)
2014-11-06 09:58:37.192901 | // Maya exited with status 139

This happens either during, or immediately after the Light Cache has been generated (I can't tell exactly, as I don't get any Light Cache progress info in a batch render).

If I disable GI or use methods other than Light Cache it renders fine. On the farm, the renders fail on ~98% of nodes, but there is no consistency (i.e. one node might render correctly once, then fail if tried again).

I did not encounter this issue with 3.0504 (this may be a coincidence). The scenes are not very memory intensive (peak usage during render <2GB)

Is there any way this could be a permissions issue? Does VRay write the Light Cache to disk during a batch render, and if so, where does it put it?

[ivaylo.ivanov]

Can you send the scene to our support? Or at least describe what you have there? OS?

[oliver.sullivan]

Hi. I can't send the scenes across as that'd break my NDA... :/

OS is Linux, Ubuntu 12.04. All machines (local and farm) have at least 32GB. Let me know what else I can provide to try to diagnose?

[ivaylo.ivanov]

What do you have in the scene - i.e. subdivisions, hair, proxy, fluids?

[oliver.sullivan]

Geo with approx 100K polys, subdivision, refl, refr, SSS, VRayBlendMtls
Multi UV-tile displacement and textures
VRayRectLight based lighting
Adaptive sampler

Renders locally in about 12 mins using Irr Map/Light Cache.

[oliver.sullivan]

If I disable Maps globally it renders ok. (Just disabling Displacement does not help, so it's not that).

Edit - it seems to stop rendering immediately when it starts to calculate the Light Cache:

// Result: Rendering with V-Ray... //
// Result: Percentage of rendering done: 1 (Building light cache...
/path/to/file.exr) //
// Result: Rendering Completed. See mayaRenderLog.txt for information. //

[tashko.zashev]

If I disable Maps globally it renders ok. (Just disabling Displacement does not help, so it's not that).

Edit - it seems to stop rendering immediately when it starts to calculate the Light Cache:

// Result: Rendering with V-Ray... //
// Result: Percentage of rendering done: 1 (Building light cache...
/path/to/file.exr) //
// Result: Rendering Completed. See mayaRenderLog.txt for information. //

It would be very helpful if you could send us the mayarenderlog.txt and vray4maya_log.txt from one machine.
I see that your setup is quite complicated and it will be difficult without a scene file to replicate it in our environment, so any information about the crash that you could share will be useful.

[oliver.sullivan]

Files attached. Thanks.

[tashko.zashev]

In order to get more details about the crash, please set Maya debug variable export MAYA_DEBUG_ENABLE_CRASH_REPORTING=1
Then run another render on this machine and send us the log mayaRenderLog.txt again.

I see from the V-Ray log that Yeti plugins are loaded. Could the crash be related to it? Would it be possible to run a test without hair systems?

It will be best if you can send us any scene that reproduces the issue, so we will be able to troubleshoot in our environment.

Thank you very much.

[scorpes]

Exact same problem here with our renderfarm with the new 3.05.07. Random segmentation faults. Also we did not encounter this with vray 3.05.04. This happened with more than one project.
I attached a crash report. This scene is a simple product packshot with VRayLightRects and HDR environment.
OS: linux Ubuntu x64 14.04.

Small note: We found out in one project that the reason that vray was giving segmentation faults was because there was a simple nurbs plane in the scene.


We also would like to get access to newer nightly builds but Vray 3 is not added to our nightlies section?
Our login is: HECTIC_ELECTRIC

[t.petrov]

Can you please share the revision numbers of the builds you're using?
When using the night builds the version numbers change rarely and so there are many different builds that have the same version numbers, and only the revision numbers differ.

[scorpes]

We are using: V-Ray for Maya version 3.00.01 from Oct 16 2014 with V-Ray core version is 3.05.07 and Maya 2014

[oliver.sullivan]

We are using: V-Ray for Maya version 3.00.01 from Oct 16 2014 with V-Ray core version is 3.05.07 and Maya 2014

Exactly the same here! Glad to see I'm not the only one with the error. Apologies for not replying recently but in the midst of 3 productions...

[t.petrov]

scorpes: Is there any chance you can send us a scene for inspection?

[scorpes]

Sorry for the late response, very busy
But we found out that rendering the scene frame by frame (packed one frame) instead of rendering packets of 5 frames on the farm removed the fatal errors. So I guess it has something to do with translating from frame to frame.
I cannot send the scene because of copyrights, maybe I make a stripped down scene in the future and test exactly what is causing the errors, but it takes a lot of time.