Vray Beta 2 build crashing consistently on Distributed rendering on farm

[BadRobotProductions]

Odd behavior with the beta:

It seems to render the frame fine but at the end during the exporter cleanup all of the render nodes suddenly switch from rendering from the host machine ip to ip 0.0.0.0 and the maya scene on the user machine gets a fatal error and shuts down.

The host machine crashes at
[2011/Apr/3|18:25:23] V-Ray: ========================
[2011/Apr/3|18:25:23] V-Ray: Clearing exporter memory...

Below is the crash report on the render.

This happens constantly on scenes with shave and a haircut as well as with a shot with just procedurals and two 2k displacement maps on a standard vray material.

Machines are all 12 core xeon's with 24 gig of ram on windows 7.

This didn't happen on build 14535 but seems to be the case with 2.00.01

Here is the crash on one of the slaves (sometimes they go to 0.0.0.0 and other times they crash out...)


Batch rendering works fantastically.

[Metzger]

When are we going to see Nightly builds keep rolling with bugfixes that are being addressed in 2.0?

[BadRobotProductions]

I've finally gotten it to write a crashdump file.
Here is the error before crash.
[2011/Apr/3|21:55:11] V-Ray: Clearing exporter memory...
Writing crash dump to "C:\Users\Brandon\AppData\Local\Temp\VRayMaya. dmp"
[2011/Apr/3|21:55:12] V-Ray error: Fatal error freeing exporter memory.
[2011/Apr/3|21:55:12] V-Ray error: Fatal error rendering image.

I can send you the file if you'd like?

[vlado]

Yes, if you can send me both the scene and the dump that produced the errors to vlado@chaosgroup.com it will be very helpful!

Best regards,
Vlado

[vlado]

When are we going to see Nightly builds keep rolling with bugfixes that are being addressed in 2.0?

The beta will be updated quite often before the release (our goal was about once a week); we want to take the time until the release to organize the nightly builds a little better.

Best regards,
Vlado

[BadRobotProductions]

Sure thing!
Will upload first thing in the morning!
Thanks for the quick reply!

[BadRobotProductions]

Not sure if the scene helped out at all but I'm getting consistent crashing in this build 2.00.01 at the freeing memory portion of the end frame all across the farm (no distributed obviously, just a pure render) across all nodes.

Here is the output message (same type of crashing as the scene I submitted earlier last week)

I've tried reverting a build back to the March 28th nightly but that doesn't seem to save things...there are no rigs in the scene and everything is a converted mipmapped tiled Exr for all textures optimized for Vray (also there is no fur or any extraneous plugins.)

Error Message
================================================== =====
Exception during render: An error occurred in RenderTasks(): Deadline caught error "Error: There was a fatal error rendering the scene.". If this error message is unavoidable but not fatal to your render, please email deadline-support@primefocusworld.com with the error message, and disable the Maya job setting Strict Error Checking.
at Deadline.Plugins.ScriptPlugin.RenderTasks(Int32 startFrame, Int32 endFrame, String& outMessage)

================================================== =====
Slave Log
================================================== =====
y: GI rays: 21029281
0: STDOUT: [2011/Apr/11|21:37:14] V-Ray: Reflection rays: 21562282
0: STDOUT: [2011/Apr/11|21:37:14] V-Ray: Refraction rays: 0
0: STDOUT: [2011/Apr/11|21:37:14] V-Ray: Unshaded rays: 12387
0: STDOUT: [2011/Apr/11|21:37:16] V-Ray error: Exception [module=1] : Clearing ray server (freeing resource manager)
0: INFO: Deadline is ignoring error: "Error: Exception [module=1] : Clearing ray server (freeing resource manager)" because plugin setting Strict Error Checking is enabled and this error is not usually fatal.
0: STDOUT: Error: Exception [module=1] : Clearing ray server (freeing resource manager)
0: STDOUT: [2011/Apr/11|21:37:16] V-Ray: Number of intersectable primitives: 964385
0: STDOUT: [2011/Apr/11|21:37:16] V-Ray: SD triangles: 2772
0: STDOUT: [2011/Apr/11|21:37:16] V-Ray: MB triangles: 0
0: STDOUT: [2011/Apr/11|21:37:16] V-Ray: Static primitives: 961612
0: STDOUT: [2011/Apr/11|21:37:16] V-Ray: Moving primitives: 0
0: STDOUT: [2011/Apr/11|21:37:16] V-Ray: Infinite primitives: 1
0: STDOUT: [2011/Apr/11|21:37:16] V-Ray error: There was a fatal error rendering the scene.
0: STDOUT: Error: There was a fatal error rendering the scene.
0: STDOUT: [2011/Apr/11|21:37:16] V-Ray: Render interrupted
0: STDOUT: [2011/Apr/11|21:37:16] V-Ray: ========================
0: STDOUT: [2011/Apr/11|21:37:16] V-Ray: Clearing exporter memory...
0: STDOUT: Writing crash dump to "C:\Users\User\AppData\Local\Temp\VRayMaya.dmp "
Scheduler Thread - Render Thread 0 threw an error:
Scheduler Thread - Exception during render: An error occurred in RenderTasks(): Deadline caught error "Error: There was a fatal error rendering the scene.". If this error message is unavoidable but not fatal to your render, please email deadline-support@primefocusworld.com with the error message, and disable the Maya job setting Strict Error Checking.

at Deadline.Plugins.ScriptPlugin.RenderTasks(Int32 startFrame, Int32 endFrame, String& outMessage)

[vlado]

We could not reproduce the error right away with your scene, but there are a few more things to try with it...

BTW is there any possibility that you are using the VRAY_TEXTURE_CACHE environment variable?

Best regards,
Vlado

[vlado]

Ok nevermind, I could get it to crash on one of the machines... it's too late for Beta 2, but it will be fixed in Beta 3. Thanks a lot for the scene!

Best regards,
Vlado

[BadRobotProductions]

Any idea what may be causing the crash? I'd love to find a way to circumvent it and get the renders to push through before beta 3...

[vlado]

The crash should go away if you disable the V-Ray frame buffer and use the Maya Render View instead.

Best regards,
Vlado

[BadRobotProductions]

Great! That's the easiest fix yet.
Thanks.

[BadRobotProductions]

Looks like crash is still happening, even with the Vray frame buffer disabled and updated to Beta 2.
Same symptoms it seems.

================================================== =====
Error Message
================================================== =====
Exception during render: An error occurred in RenderTasks(): Deadline caught error "Error: There was a fatal error rendering the scene.". If this error message is unavoidable but not fatal to your render, please email deadline-support@primefocusworld.com with the error message, and disable the Maya job setting Strict Error Checking.
at Deadline.Plugins.ScriptPlugin.RenderTasks(Int32 startFrame, Int32 endFrame, String& outMessage)

================================================== =====
Slave Log
================================================== =====
ing resource manager)
0: STDOUT: [2011/Apr/12|09:11:37] V-Ray: Number of intersectable primitives: 256707
0: STDOUT: [2011/Apr/12|09:11:37] V-Ray: SD triangles: 2358
0: STDOUT: [2011/Apr/12|09:11:37] V-Ray: MB triangles: 2
0: STDOUT: [2011/Apr/12|09:11:37] V-Ray: Static primitives: 57938
0: STDOUT: [2011/Apr/12|09:11:37] V-Ray: Moving primitives: 196408
0: STDOUT: [2011/Apr/12|09:11:37] V-Ray: Infinite primitives: 1
0: STDOUT: [2011/Apr/12|09:11:37] V-Ray error: There was a fatal error rendering the scene.
0: STDOUT: Error: There was a fatal error rendering the scene.
0: STDOUT: [2011/Apr/12|09:11:37] V-Ray: Render interrupted
0: STDOUT: [2011/Apr/12|09:11:37] V-Ray: ========================
0: STDOUT: [2011/Apr/12|09:11:37] V-Ray: Clearing exporter memory...
0: STDOUT: Writing crash dump to "C:\Users\User\AppData\Local\Temp\VRayMaya.dmp "
0: STDOUT: [2011/Apr/12|09:11:37] V-Ray error: Fatal error freeing exporter memory.
0: INFO: Deadline is ignoring error: "Error: Fatal error freeing exporter memory." because plugin setting Strict Error Checking is enabled and this error is not usually fatal.
0: STDOUT: Error: Fatal error freeing exporter memory.
0: STDOUT: [2011/Apr/12|09:11:37] V-Ray error: Fatal error rendering image.
0: INFO: Deadline is ignoring error: "Error: Fatal error rendering image." because plugin setting Strict Error Checking is enabled and this error is not usually fatal.
0: STDOUT: Error: Fatal error rendering image.
Scheduler Thread - Render Thread 0 threw an error:
Scheduler Thread - Exception during render: An error occurred in RenderTasks(): Deadline caught error "Error: There was a fatal error rendering the scene.". If this error message is unavoidable but not fatal to your render, please email deadline-support@primefocusworld.com with the error message, and disable the Maya job setting Strict Error Checking.

at Deadline.Plugins.ScriptPlugin.RenderTasks(Int32 startFrame, Int32 endFrame, String& outMessage)


================================================== =====
Error Type
================================================== =====
RenderPluginException

================================================== =====
Error Stack Trace
================================================== =====
at Deadline.Plugins.Plugin.RenderTask(Int32 startFrame, Int32 endFrame)
at Deadline.Slaves.SlaveRenderThread.RenderCurrentTas k()

[vlado]

Ok; I'm looking for a fix now; I can get this to you when it is ready so please email me to vlado@chaosgroup.com

Best regards,
Vlado

[BadRobotProductions]

Emailed you!
Thank you!
I had the beginnings of a panic attack slowly rising as the frames froze....
=)