Vray error not interrupting network render as it should

[Dynedain]

When rendering through backburner, I am frequently getting a problem where Vray fails (and generates an error in the Vray log) but instead of marking the frame as an error, it continues and renders an empty junk frame with just the background, and flags the task as done in backburner. It does it an incredibly short period of time, so all of a sudden one machine will finish all the available tasks in a matter of minutes instead of days.

Here is a sample from the VrayLog when this happens:

[2005/May/6|10:30:43]           Creating render mesh for 'MER_GEAR_SH01': fixed mesh
[2005/May/6|10:30:43]           Sampling displacement maps.
[2005/May/6|10:30:43]           Updating render lights
[2005/May/6|10:30:43]           Updating atmospheric effects.
[2005/May/6|10:30:43]           Updating render effects.
[2005/May/6|10:30:43]           Updating the tone operator.
[2005/May/6|10:30:43]         Scene setup: 1.4 s
[2005/May/6|10:30:43]         Compiling render instances geometry.
[2005/May/6|10:30:43]         error: UNHANDLED EXCEPTION: Compiling geometry
Last marker is at P:\Vlado\SVN\vray\vrender\src\renderlight.cpp, line 185: Preparing lights (Driveway Sunlight)

[2005/May/6|10:30:43]         Broadcasting NOTIFY_PRE_RENDERFRAME.
[2005/May/6|10:30:43]       beginFrame() done.
[2005/May/6|10:30:43]       Broadcasting NOTIFY_BEGIN_RENDERING_ACTUAL_FRAME.
[2005/May/6|10:30:43]     Setting up DoneCallback.
[2005/May/6|10:30:43]     Rendering frame region (0,0)-(800,450).
[2005/May/6|10:30:43]       Lightmap sample size is 72 bytes.
[2005/May/6|10:30:43]       Building light cache.
[2005/May/6|10:30:43]       Tracing 640000 image samples for light map in 2 passes.
[2005/May/6|10:30:43]       Setting up 1 thread(s)
[2005/May/6|10:30:45]       Threads completed
[2005/May/6|10:30:45]       Light cache contains 0 samples.
[2005/May/6|10:30:45]       Light cache samples collected; compiling lookup tree.
[2005/May/6|10:30:45]       Prefiltering light map.
[2005/May/6|10:30:45]       1 interpolation maps registered
[2005/May/6|10:30:45]       Rendering image.
[2005/May/6|10:30:45]       Setting up 1 thread(s)
[2005/May/6|10:30:45]       Threads completed
[2005/May/6|10:30:45]     Calling endFrame().
[2005/May/6|10:30:45]       Entering endFrame().
[2005/May/6|10:30:45]         Broadcasting NOTIFY_POST_RENDERFRAME.
[2005/May/6|10:30:45]         Applying render effects.
[2005/May/6|10:30:45]         Number of raycasts: 0
[2005/May/6|10:30:45]         Clearing global light manager.
[2005/May/6|10:30:45]         Clearing direct light manager.
[2005/May/6|10:30:45]         Clearing ray server.
[2005/May/6|10:30:45]         Clearing geometry.
[2005/May/6|10:30:45]         Calling frameEnd() on render instances.
[2005/May/6|10:30:45]         Calling frameEnd() on render objects.
[2005/May/6|10:30:45]         Clearing camera image sampler.
[2005/May/6|10:30:45]         Clearing camera sampler.
[2005/May/6|10:30:45]         Region rendering: 2.1 s
[2005/May/6|10:30:45]       endFrame() done
[2005/May/6|10:30:45]     Updating irradinace map info.
[2005/May/6|10:30:45]   RenderAtTime() done.
[2005/May/6|10:30:45]   Stopping frame timer.
[2005/May/6|10:30:45]   Total frame time: 3.5 s
[2005/May/6|10:30:45] Render() done.

This does not seem to be machine specific on my farm, and tends to occur randomly. Please advise.

[Dynedain]

I need to figure this out ASAP, a week-long render that needed to start yesteday is being held up by this problem

[RErender]

Is there drive space on the failing boxes, scanline will do the same thing if the drives are full.

[Dynedain]

Plenty of drive space - 20GB + drives with nothing installed but windows and max. Its fairly random as to which nodes it happens to, has hit both the single proc win2k machines (1GB ram) and the dual-proc winXP (2GB ram) machines

[vlado]

It looks like a geometry poblem; how many faces do you have in the scene? Do you have RPC or SpeedTree objects? Ar you using motionblur?

Best regards,
Vlado

[Dynedain]

Total poly count is probably around 4million or so when counting the vray proxy objects. The speedtrees are stripped out to a seperate file (which renders fine). Mblur is on. There are no RPCs.

[Dynedain]

I did some more tweaking to the scene (more vray proxies and deleting unecessary objects) and I have it limping along (knock on wood) on less than half my renderfarm (afraid of adding more machines lest the problem shows up again). This wouldn't have been nearly such a problerm if Vray had returned an error to backburner instead of trying to render the scene anyways, which is why I've submitted the bug. Tracking down the messed up frames, deleting them, adjusting the job settings to skip existing frames and restarting it so that it can try to rerender the messed up frames is a pain and shouldn't be necessary.

[vlado]

Yes, I know it's a pain; will be looking at this.

Best regards,
Vlado

[Dynedain]

I haven't been able to move to 1.47.03 yet (not switching mid-render) but the problem is still occuring on other renders based on variations of this scene:

[2005/May/18|11:51:38]             Updating 'water'
[2005/May/18|11:51:38]             Loading bitmaps for 'Map #1'
[2005/May/18|11:51:38]             Updating 'Map #1'
[2005/May/18|11:51:38]             Loading bitmaps for 'Terrain'
[2005/May/18|11:51:38]             Updating 'Terrain'
[2005/May/18|11:51:38]             Loading bitmaps for 'Map #109'
[2005/May/18|11:51:38]             Updating 'Map #109'
[2005/May/18|11:51:38]             Loading bitmaps for '08 - Default'
[2005/May/18|11:51:38]             Updating '08 - Default'
[2005/May/18|11:51:38]             Loading bitmaps for 'Garage Floor'
[2005/May/18|11:51:38]             Updating 'Garage Floor'
[2005/May/18|11:51:38]           Preparing objects
[2005/May/18|11:51:39]           error: UNHANDLED EXCEPTION: Calling GetRenderMesh() for node "Walls" at time 13360
Last marker is at P:\Vlado\SVN\vray\vray\src\vrayrenderer.cpp, line 1334: Preparing camera sampler

[2005/May/18|11:51:39]           error: UNHANDLED EXCEPTION: Preparing objects
Last marker is at P:\Vlado\SVN\vray\vray\src\vrayrenderer.cpp, line 1334: Preparing camera sampler

[2005/May/18|11:51:39]           Preparing instances
[2005/May/18|11:51:39]           Creating render mesh for 'First Floor 366'	': fixed mesh
[2005/May/18|11:51:39]           Creating render mesh for 'First Floor Ceiling': fixed mesh
[2005/May/18|11:51:39]           Creating render mesh for 'First Floor Ceiling': fixed mesh
[2005/May/18|11:51:39]           Creating render mesh for 'First Floor Ceiling': fixed mesh
[2005/May/18|11:51:39]           Creating render mesh for 'First Floor Ceiling - Garage': fixed mesh
[2005/May/18|11:51:39]           Creating render mesh for 'First Floor Garage': fixed mesh
[2005/May/18|11:51:39]           Creating render mesh for 'First Floor Garage01': fixed mesh
[2005/May/18|11:51:39]           Creating render mesh for 'FixedWindow99': fixed mesh
[2005/May/18|11:51:39]           Creating render mesh for 'FixedWindow100': fixed mesh
[2005/May/18|11:51:39]           Creating render mesh for 'FixedWindow113': fixed mesh
[2005/May/18|11:51:39]           Creating render mesh for 'Ground Floor 368'': fixed mesh
[2005/May/18|11:51:39]           Creating render mesh for 'PivotDoor11': fixed mesh
[2005/May/18|11:51:39]           Creating render mesh for 'Second Floor 378'	': fixed mesh
[2005/May/18|11:51:39]           Creating render mesh for 'Second Floor 379.5'	': fixed mesh
[2005/May/18|11:51:39]           Creating render mesh for 'Second Floor Ceiling': fixed mesh
[2005/May/18|11:51:39]           Creating render mesh for 'Second Floor Ceiling': fixed mesh
[2005/May/18|11:51:39]           Creating render mesh for 'Box18': fixed mesh
[2005/May/18|11:51:39]           Creating render mesh for 'FixedWindow70': fixed mesh

------------ Content removed for brevity -------------------

[2005/May/18|11:51:40]           Creating render mesh for 'Curtain35': fixed mesh
[2005/May/18|11:51:40]           Creating render mesh for 'FixedWindow218': fixed mesh
[2005/May/18|11:51:40]           Creating render mesh for 'FixedWindow219': fixed mesh
[2005/May/18|11:51:40]           Creating render mesh for 'Window Trim01': fixed mesh
[2005/May/18|11:51:40]           Creating render mesh for 'Roof Pool House': fixed mesh
[2005/May/18|11:51:40]           Creating render mesh for 'Roof 09': fixed mesh
[2005/May/18|11:51:40]           Creating render mesh for 'SpeedTree01[EnglishOak_MD-mod]': fixed mesh
[2005/May/18|11:51:40]           Creating render mesh for 'SpeedTree01[PeachTree_MD_Fruit-modified]04': fixed mesh
[2005/May/18|11:51:40]           Creating render mesh for 'Hair MR Object01': fixed mesh
[2005/May/18|11:51:40]           Creating render mesh for 'SpeedTree01[PeachTree_MD_Fruit-modified]05': fixed mesh
[2005/May/18|11:51:40]           Sampling displacement maps.
[2005/May/18|11:51:40]           Updating render lights
[2005/May/18|11:51:40]           Updating atmospheric effects.
[2005/May/18|11:51:40]           Updating render effects.
[2005/May/18|11:51:40]           Updating the tone operator.
[2005/May/18|11:51:40]         Scene setup: 8.5 s
[2005/May/18|11:51:40]         Compiling render instances geometry.
[2005/May/18|11:51:40]         error: UNHANDLED EXCEPTION: Compiling geometry
Last marker is at P:\Vlado\SVN\vray\vrender\src\renderlight.cpp, line 185: Preparing lights (VRayLight Fireplace)

[2005/May/18|11:51:40]         Broadcasting NOTIFY_PRE_RENDERFRAME.
[2005/May/18|11:51:40]       beginFrame() done.
[2005/May/18|11:51:40]       Broadcasting NOTIFY_BEGIN_RENDERING_ACTUAL_FRAME.
[2005/May/18|11:51:40]     Setting up DoneCallback.
[2005/May/18|11:51:40]     Rendering frame region (0,0)-(800,450).
[2005/May/18|11:51:40]       Lightmap sample size is 72 bytes.
[2005/May/18|11:51:40]       Building light cache.
[2005/May/18|11:51:40]       Tracing 1000000 image samples for light map in 4 passes.
[2005/May/18|11:51:40]       Setting up 1 thread(s)
[2005/May/18|11:51:47]       Threads completed
[2005/May/18|11:51:47]       Light cache contains 0 samples.
[2005/May/18|11:51:47]       Light cache samples collected; compiling lookup tree.
[2005/May/18|11:51:47]       Prefiltering light map.
[2005/May/18|11:51:47]       Loading irradiance map from file "\\VDCNTS2\Projects\Current Projects\2004\KAA Quail Meadows\Quail Meadows 3dsmax\textures\lighting calcs\poolcam.vrmap"...
[2005/May/18|11:53:22]       Irradiance map loaded successfully
[2005/May/18|11:53:22]       1 interpolation maps registered
[2005/May/18|11:53:22]       Rendering image.
[2005/May/18|11:53:22]       Setting up 1 thread(s)
[2005/May/18|11:53:22]       Threads completed
[2005/May/18|11:53:22]     Calling endFrame().
[2005/May/18|11:53:22]       Entering endFrame().
[2005/May/18|11:53:22]         Broadcasting NOTIFY_POST_RENDERFRAME.
[2005/May/18|11:53:22]         Applying render effects.
[2005/May/18|11:53:22]         Number of raycasts: 0
[2005/May/18|11:53:22]         Clearing global light manager.
[2005/May/18|11:53:22]         Clearing direct light manager.
[2005/May/18|11:53:22]         Clearing ray server.
[2005/May/18|11:53:22]         Clearing geometry.
[2005/May/18|11:53:22]         Calling frameEnd() on render instances.
[2005/May/18|11:53:22]         Calling frameEnd() on render objects.
[2005/May/18|11:53:22]         Clearing camera image sampler.
[2005/May/18|11:53:22]         Clearing camera sampler.
[2005/May/18|11:53:22]         Region rendering: 102.2 s
[2005/May/18|11:53:22]       endFrame() done
[2005/May/18|11:53:22]     Updating irradinace map info.
[2005/May/18|11:53:22]   RenderAtTime() done.
[2005/May/18|11:53:22]   Stopping frame timer.
[2005/May/18|11:53:22]   Total frame time: 110.8 s
[2005/May/18|11:53:22] Render() done.

[Dynedain]

Ok, I appear to have found a work-around.

If I copy the irmap, and resubmit the job to half the machines using the copied bitmap, then I don't get the errors. Which tells me part of my problem is too many machines accessing a file simultaneously (which has been posted as a max problem before). So at least this job is rendering.

But, as I understand it, usually when a machine can't get the bitmap, it fails the frame and just trys again - so my problem with Vray not returning errors properly to backburner hasnt been solved by this kludgy fix.

[Dynedain]

Looks like I spoke too soon...problem is still occuring...