Distributed Rendering - Experience From the Trenches

[cgrant3d]

In replying to another post in this forum about DR I realized there was a lot of disparate information I had to gather from all kinds of different forums / posts when I setup DR here... There really wasn't a post that let me know what worked / what didn't and just how the overall system worked. So, here's a thread I think would benefit new users, where those of us who have successfully setup DR can post our suggestions / thoughts in general...

So, here is a copy of my thoughts on DR in its current 1.46.08 beta state... Maybe some of the best suggestions from this thread can be compiled by a moderator and set up as a sticky FAQ or something...

--------------------------------------------------------------------------

Technically, DR (distributed rendering) works but there are some things to keep in mind, especially if you are spoiled by backburner...

1. DR currently works only with the beta version of vray. As of today, the current beta is 1.46.08 with the final version being 1.5. Thus, it's getting close to the final rev, but it's not without its faults... The key is to expect problems , small or large, and don't be surprised when something goes wrong - it's BETA.

2. The vray spawner (equivalent to backburner server) must be restarted a lot. A lot. Just do yourself a favor and expect it and you'll be fine.

3. Don't hit cancel. The max pause/cancel buttons that appear after you hit render. Those are mystery buttons. I think they work like this: every other time you click cancel vray (much like max used to do) decides you've spent too much time working on your project and you should take a break, thus quickly crashing max regardless if you saved recently or not.

4. DR is awesome. It's enabled me to render things across several stations that I would have needed an entire weekend to process on one computer.

5. The host computer (the one you're working on) MUST participate in the DR process. So, in the middle of the day when you hit render you have to wait to use your computer until the DR is done. Works great for previews. Great. BUT if you have a hi-res print due at 5pm don't expect to use your computer much while it is rendering. Vray does a great job of using every CPU cycle available, unlike the MAX Scanline renderer, whose inefficiencies make it easy to keep on doing photoshop work / light max stuff even while rendering...

6. Your VRay license is tied up while waiting for DR to finish. Don't expect that you can switch to another workstation and work on your project while a render is going. Not going to happen. Vray uses a network license server, when DR is going, that license is used up. You could buy another license of course though. In which case this problem isn't a problem at all...

7. DR is absolutely unbelievable. I use it every day. When I have to use the scanline and can't use DR, it just seems so wasteful...

8. Don't get any ideas about backburning your DR render so you can work on your workstation... It would be a great idea if it worked. But no. So, I repeat, when you start a DR, your license is tied up. There isn't some slick way to bypass it... You'll have to wait it out.

9. There is no way to change / schedule DR renders. The computers that are available to render when you hit render are the ones that render. They don't automatically join if you start them after the render has started. You can't remove them (although truth be told I never tried manually killing one with task manager because I've never had a project that I could risk it)

10. If you successfully hit cancel and nothing crashes, you must wait for the currently rendering buckets to finish. So, if you set up the render as super duper high quality uber cool, you might wait half an hour or whatever for all the rendering buckets to finish before you cancel. Or, you could kill the process with task manager, restart all your vray spawners / 3dsmax.exe processes on your rendering stations and reload your max file... Unless your farm is huge you can probably do all that before vray would otherwise have finished. It's what I do everyday nearly. Annoying but it's offset by #11.

11. Distributed Rendering, IMHO, is the best thing as a max user that I've ever seen. I know it's not technically "new" as other renderers have had it, but for those of us who waited and waited before selecting a 3rd party renderer, it's absolutely indispensible!

SUMMARY: Get Vray. Play with it, get comfortable with it. It's very different from scanline. Set up DR, it's pretty easy to do. Play with it, get comfortable with it. IT'S WORTH IT.

Christopher Grant
3D Artist, Fleetwood
Webmaster, ScriptSpot

[egz]

Great initiative Chris!

I've added the post to VRay.info, and I'll try to keep it updated as the thread progresses, and as VRay changes.

http://vray.info/topics/t0116.asp

[dbuchhofer]

egz, a thought for you, and for us in general... in terms of the way we contribute, and have to re-say or try and compile helpful info often into one spot?

Would it benefit us to have a WIKI set up whereby we can all edit and update and tweak tips.. an open version of vray.info almost

[dbuchhofer]

A little more on topic

12. In 1.46.08 if the DR servers (Vrayspawners) are currently rendering a job, and you second another job that overlaps / specifies the same render slaves, you crash the first render.. the second sent render finishes, but only using the local processors.. So Be careful !

13. you WILL be restarting the vrayspawners often.. so if you're lazy like me, and dont want to walk over... heres a couple tips on how to do it remotely from your own desk.. (Disclaimer.. some setups or network administrators may have these disabled.. but for the majority of us it will work fine.)

13a. Tools to download: the PSTools utilities by Mark Russinovich, a set of DOS programs to help with administering remote systems..

13b. Stopping the Vrayspawner via a Batchfile.
Putting this in a batch file (A text file with the extension renamed to .bat) will kill the vrayspawner process on the computer rendernode3

pskill \\rendernode3 vrayspawner60

13c. Starting the Vrayspawner via a Batchfile.
this will execute the process vrayspawner on the remote machine rendernode3, let it interact with the desktop (So you can USE it), and detach the process (so you can continue with the rest of your batchfile)

psexec \\rendernode3 -i -d C:\3dsmax7\vrayspawner60.exe

13d. So.. to automate that a little.. assuming you have the hard drive of rendernode3 shared with the name C_Drive.. this would end the process.. clear the network cache, and then restart it...

pskill \\rendernode3 vrayspawner60
del /q \\rendernode3\C_drive\3dsmax7\Network*.*
psexec \\rendernode3 -i -d C:\3dsmax7\vrayspawner60.exe

(You can do similar things with services for backburner and much more.. but you can experiment on your own )

[dennis]

lovely man. Been thinking it every time i had to do it. finally somebody did it for me hehe.

cheers mate.

/edit

i kill the 3dsmax process aswell since it can be 'stuck' and take a while to shut properly down so added

pskill \renderX 3dsmax.exe

[plastic_]

here is a quick way to start multiple boxes with a loop:

set vraynodes=BOX1 BOX2 BOX3 BOX4 BOX5

FOR %%A IN (%vraynodes%) DO psexec //%%A -i -d C:/3dsmax7/vrayspawner60.exe

(i used / instead of backslash because it seems to disappear in the post.)

it's a good idea to use a loop pskill before, to avoid startig multiple instances of the process.

[vize]

if psexec (pskill) dont work on remote winXP.

go to start
> control panel
> administrative tools
> Local security policy
> Security Options
> Network access: Sharing and security model for local acconts = Classic...

[artykris]

Re: Distributed Rendering - Experience From the Trenches

5. The host computer (the one you're working on) MUST participate in the DR process. So, in the middle of the day when you hit render you have to wait to use your computer until the DR is done. Works great for previews. Great. BUT if you have a hi-res print due at 5pm don't expect to use your computer much while it is rendering. Vray does a great job of using every CPU cycle available, unlike the MAX Scanline renderer, whose inefficiencies make it easy to keep on doing photoshop work / light max stuff even while rendering...

I`m resolving this problem like this:
For instance - I`m using 8 PCs for rendering VrayDR.
I started the work from my host machine, and when I want to work in the PS at the same time I open the task manager - processes tab, find 3dmax.exe, hit RMB and there you have 'set priority' - I set it to 'BelowNormal' and v`uala !
Your host machine bucket gets slower, but you can work on the host machine parallel to DR rendering. Your PS application gets priority over 3dmax. (just be careful about the memory usage)


PS I`m re-posting this answer, because the cgrant3d`s DR tips have been re-posted here, and I thought that my trick might be also useful

[plastic_]

artykris,
this can also be archieved by turning on the "low priority thread" option in the vray/system rollout.

[artykris]

plastic:
artykris,
this can also be archieved by turning on the "low priority thread" option in the vray/system rollout.

I didn`t know that, thx.

[silver]

hey guys, come on, i know Dr is everything you just said that it is, but when it works. And if it works for you , it doesnt mean that works for everybody, here in my office for instances my life has been really hard and disgusting. i've bought vray because of DR and it works only when its wanna do that, i can start my day using DR and just at next render it stops , after all, doesnt matter what i do, its impossible to me get it working again. So i think you should take care when you say that it is the most amazing thing in the world, because there are some people like me and many others wich are in troubles with it. I know off course that vlado and his team are working on to fix the problems but we cant just say " Set up DR, it's preatty easy to do... fill confortable with it... " it would be indeed if it works fine with everybody and not in few computers. I am also saying this because this is a way to show that there are still lot of people as me wich are waiting for a good solution but until now we continue alone

[RErender]

hey guys, come on, i know Dr is everything you just said that it is, but when it works. And if it works for you , it doesnt mean that works for everybody, here in my office for instances my life has been really hard and disgusting. i've bought vray because of DR and it works only when its wanna do that, i can start my day using DR and just at next render it stops , after all, doesnt matter what i do, its impossible to me get it working again. So i think you should take care when you say that it is the most amazing thing in the world, because there are some people like me and many others wich are in troubles with it. I know off course that vlado and his team are working on to fix the problems but we cant just say " Set up DR, it's preatty easy to do... fill confortable with it... " it would be indeed if it works fine with everybody and not in few computers. I am also saying this because this is a way to show that there are still lot of people as me wich are waiting for a good solution but until now we continue alone

Maybe take a good look at your network setup? I have had very few problems with DR...

[silver]

hi RE render, maybe you're right, but i dont think so, because if sometimes it works and i dont change nothing in my network setuo, i dont know if this is the point. but if you could tell me what is your network setup, perhaps this point me a way , do you use hub or router, how you handle with ip's, this sort of thing

[Ramsay]

hey guys, come on, i know Dr is everything you just said that it is, but when it works. And if it works for you , it doesnt mean that works for everybody, here in my office for instances my life has been really hard and disgusting. i've bought vray because of DR and it works only when its wanna do that, i can start my day using DR and just at next render it stops , after all, doesnt matter what i do, its impossible to me get it working again. So i think you should take care when you say that it is the most amazing thing in the world, because there are some people like me and many others wich are in troubles with it. I know off course that vlado and his team are working on to fix the problems but we cant just say " Set up DR, it's preatty easy to do... fill confortable with it... " it would be indeed if it works fine with everybody and not in few computers. I am also saying this because this is a way to show that there are still lot of people as me wich are waiting for a good solution but until now we continue alone

Sounds like you are not using fixed IP addresses... try giving each server and workstation a fixed IP address, it may solve your problem

[silver]

i have two machines here with fixed ip's, 192.168.0.1 and 192.168.0.2 , doesnt matter if i put name machine or even number ip's at DR settings, both options can or or not, depending of the day, time, whether... well, actually i cant see what makes this process to be so instable. for exemple, yesterday i tried again to reinstall vray at both machines, after that, i create a very simple scene with a plane and some teapots... when i hit render, DR works pretty well, so i thought it was fine, but, after that, i just opened a real job and hit render on it, and it stoped again to works. this inconsistent caractheristc is the most ood IMHO, how could it worked in few minutes before and now just stoped without a reazon?