Distributed Render problem

[Luke72]

Hi all,
i have purchased some computers to make a small renderfarm, but we have a problem if the machine used for the process of DR are superior of 2.
these machine are a dual xeon (8 cores each).

Everything is ok in a very very smalls scenes, not the same in a ordinary project, like a ordinary "medium" scene file like 80mb of data.


Any suggestion?
thanks.

[joe_b]

Re: Distributed Render problem

What is the problem you have when you have more than 2 machines working on DR? Does it crash, or not render properly?

One thing to check would be to make sure that all of the machines are running the same version of the DRSpawner. Different versions of the spawner working together can be a problem.

[Luke72]

Re: Distributed Render problem

if i try a render with 2 hosts, the render starts.
if i try with 4 hosts, only my "client" machine start to render.


here is the info:



Beginning render sequence
[RenderView] startCameraTime=0.000000, endCameraTime=0.033333
[RenderView] numCameraTMs=2, numFrames=1, frameSamples=2
Initializing built-in VFB.
[RenderView] startCameraTime=0.000000, endCameraTime=0.033333
[RenderView] numCameraTMs=2, numFrames=1, frameSamples=2
Starting DR
Connected to render host 192.168.1.6
Connected to render host 192.168.1.16
Connected to render host 192.168.1.19
Connected to render host 192.168.1.20
Using 4 hosts for distributed rendering.
Preparing camera sampler.
Preparing ray server.
Creating and initializing 4 thread(s).
Allocating memory for build data of 5697261 faces (159523308 bytes, 152.1 MB).
Initializing face build data.
Creating 'done' event.
Starting first thread.
Waiting for thread completion.
Releasing thread resources.
Preparing faces for intersection tests.
SDTree statistics:
Total number of faces stored: 5697261
Max tree depth: 47
Average tree depth: 26.857
Number of tree nodes: 591108
Number of tree faces: 11184008
Number of tree leafs: 268978
Average faces/leaf: 41.5796
Memory usage: 334.64 MB
Scene bounding box is [0,2600.68,-405.264]-[3027.45,5007.31,510.526]
Preparing direct light manager.
Preparing global light manager.
Irradiance sample size is 84 bytes
Photon size is 56 bytes.
Light cache sample size is 112 bytes.
Building light cache.
Tracing 1000000 image samples for light cache in 4 passes.
Setting up 4 thread(s)
Light cache time interval is (0.000000, 0.000000)
Light cache time interval is (0.000000, 0.000000)
Light cache time interval is (0.000000, 0.000000)
Light cache time interval is (0.000000, 0.000000)
Bitmap file "T:\3d_resource\Textures\Clienti\Grafica_london\QU INTA_1 copy.jpg" loaded.
Bitmap file "T:\3d_resource\Textures\Clienti\Grafica_london\QU INTA_1 trasp2.jpg" loaded.
Bitmap file "T:\3d_resource\Textures\Clienti\OGGETTI PATCH\DOGHE_SPLIT.bmp" loaded.
Bitmap file "T:\3d_resource\Textures\Clienti\OK_for rhino\Body mirror.bmp" loaded.
Bitmap file "T:\3d_resource\Textures\Clienti\Grafica_london\fa ns55.jpg" loaded.
Bitmap file "T:\3d_resource\Textures\Clienti\MARLB_01.bmp" loaded.
Bitmap file "T:\3d_resource\Textures\Clienti\RED.bmp" loaded.
Bitmap file "T:\3d_resource\Textures\Background\HDRI\DOSH EXTREME HI_RES\1\SN\DH211SN.jpg" loaded.
Bitmap file "T:\3d_resource\Textures\Clienti\ALERONDER.bmp " loaded.
Bitmap file "T:\3d_resource\Textures\Clienti\BODY01.bmp" loaded.
Bitmap file "T:\3d_resource\Textures\Clienti\SIDEALERMALB------.bmp" loaded.
Bitmap file "T:\3d_resource\Textures\Clienti\ALETTONE.bmp" loaded.
Bitmap file "T:\3d_resource\Textures\Clienti\A.bmp" loaded.
Bitmap file "T:\3d_resource\Textures\Background\HDRI\DOSH EXTREME HI_RES\1\SN\DH211SN.jpg" loaded.

Bitmap file "T:\3d_resource\Textures\Clienti\OGGETTI PATCH\glasses.jpg" loaded.
Render host turbo04 (192.168.1.19) is not responding
Render host turbo05 (192.168.1.20) is not responding
Render host turbo01 (192.168.1.6) is not responding
Scene transferred to turbo02
Scene transferred to turbo01
Scene transferred to turbo04
Scene transferred to turbo05
Render host turbo02 (192.168.1.16) is not responding
Threads completed
Merging light cache passes.
Light cache contains 32399 samples.
Light cache samples collected; compiling lookup tree.
Prefiltering light cache.
1 interpolation maps registered
Rendering interpolation maps with minRate=-4 and maxRate=-2
Setting up 4 thread(s)
Threads completed
Calling endPass() on irradiance maps
Calling prePassDone() on irradiance maps
Image prepass completed.
Rendering image.
Setting up 4 thread(s)
Threads completed
Waiting for DR to finish
Clearing global light manager.
Clearing direct light manager.
Clearing ray server.
Clearing geometry.
Clearing camera image sampler.
Clearing camera sampler.
Clearing QMC sampler.
Clearing path sampler.
Clearing color mapper.
Closing DR
Render cancelled

[Luke72]

Re: Distributed Render problem

the spawner are the same on all machine.

[joe_b]

Re: Distributed Render problem

Interesting - it seems to transfer the information to all of the machines...So none of the DRSpawner applications crash when you start the render?

I am wondering if it is a synchronization issue with 32 cores trying to get up and running.
Does the render finish pretty quickly with just the host doing it? If so it might be that they are taking a while to all get going.

If you want to send me your scene so we can test it on our end you should make a support ticket through our Support section and either attach the 3dm file ( if it is under 10MB ) or post it on a 3rd party file hosting website ( like http://www.mediafire.com ) and send me the link.

[Luke72]

Re: Distributed Render problem

yeah , i try to send you the link of a scene, trough a mail in the support section.
what is the mode to obtain the support tickets?
i have a regular licence, but not info about the tickets...


Anyway, the spawners of the host seems not "receive" nothing than superior of "xx mb"
it waits....
Ok, sometime it crash, but in a extreme situation like stop an relaunch heavy scenes from my render client.

I try also this thing like a DR "host to host" but we have the same problem, only two machine works...
(note: in this specific scene)

I suppose is a memory bottleneck, or similar, cause everything is fine in small 3d scenes.
I don't think is a hardware problem,
my nodes are equipped with XP64 4gig of ram

[joe_b]

Re: Distributed Render problem

I received your ticket and we'll be taking a look at it as soon as possible. I'll let you know when we make headway on the ticket. Thanks for sending us the scene

[pickinboy]

Re: Distributed Render problem

I received your ticket and we'll be taking a look at it as soon as possible. I'll let you know when we make headway on the ticket. Thanks for sending us the scene

Joe VRay team,

this is an enormous issue.

The users are using DR to render large files - to get the speed up. I figure that is what DR is for.

Is there anyway to increase the wait time to get the 3D model? It varies time to time...

Also does teh Spawner have a 'Let Go' command? Post render cancel?

Thanks, Glenn Johnson > ;D

[dkendig]

Re: Distributed Render problem

this is an enormous issue.

This forum thread was made in April, so it's quite old. This was indeed an enormous issue. As far as we know, this issue was resolved as of an internal build version on May 1st 2008 at 4:52PM EST. This fix would have been included in the SP1 release. We don't use this forum to track or respond to bugs, we have a support ticket system. If you have found that this issue is still happening, please submit a support ticket so that we can work to repair this issue:

http://asgvis.com/index.php?option=c...d=3&Itemid=142

Is there anyway to increase the wait time to get the 3D model? It varies time to time...

I'm afraid I don't quite understand the question. Could you elaborate a bit more?

Also does teh Spawner have a 'Let Go' command? Post render cancel?

When a render is canceled or completed, the DR Spawner will become available for new render jobs. I don't know if that's what you meant by this question though.

[pickinboy]

Re: Distributed Render problem

Hi all,
i have purchased some computers to make a small renderfarm, but we have a problem if the machine used for the process of DR are superior of 2.
these machine are a dual xeon (8 cores each).

Everything is ok in a very very smalls scenes, not the same in a ordinary project, like a ordinary "medium" scene file like 80mb of data.


Any suggestion?
thanks.

Luke, put the most complex stuff on layers.

Then try turning each layer to a mesh.

At some point it goes through.

Big NURBS seems to be the problem....

Hope this helps.. Glenn

[Gijs]

Re: Distributed Render problem

I think it could be related to an issue I ran into lately. If one of your images is a progressive jpg, then DR will choke on it. Try to turn off maps and see if DR still fails. If not then there is a fair chance I'm on the right track...