distributed rendering vfr4

[Gijs]

Re: distributed rendering vfr4

Does the DR process not automatic upload the textures to the slaves first and than start the rendering?

No I don't do that now. I can - but if you have a lot of textures you are going to drastically reduce the speed of the DR process.

Many times in max, I whish that it would automatically detect local textures and upload them... so couldn't it be done that only local textures are distibuted to the slaves?

[Micha_cg]

Re: distributed rendering vfr4

Does the DR process not automatic upload the textures to the slaves first and than start the rendering?

No I don't do that now. I can - but if you have a lot of textures you are going to drastically reduce the speed of the DR process.

I think the best soultion would be to implement a command ala "upload textures to slaves". A second command could be "update textures at slaves". So, the user could load all textures to the slaves first time rendering and update modified ones later. And final, a command "delete textures at slaves" could clean up at the end.

[Gijs]

Re: distributed rendering vfr4

I'd prefer an automated process for the first two. I presume that it is possible to check automatically:

-if the rendering is going to use textures that are stored locally
-if the textures at the slave side are different from the textures used at the master.

[joe_b]

Re: distributed rendering vfr4

The main reason I opted against transferring the textures is because all the communication back and forth is handled by the V-Ray SDK. So its not exactly a trivial thing to implement something that checks all the machines for local textures, etc (it would require a lot of network code that we don't need to worry about at all right now). The quick way to implement this is to just always distribute all the textures to the DR slaves. So I'm not sure if people want to be transferring their textures on every render. But I guess I can do that by default and if people have a problem with that bogging down the DR render time they can put the files somewhere locally and opt out of the texture transferral.

[Micha_cg]

Re: distributed rendering vfr4

If I understand right, it's no problem to implement, that all textures are send to the slaves. Fine. The problem is to check the textures for update issue, right? If yes, this is not so important. I think, the user can send textures again after modification or manualy bring the single, modified textures to the slaves. So, the workflow would be, that first all textures are send to the slaves, than renderings can be done, but without resend the textures allways. Sounds good.

[dalomar_cg]

Re: distributed rendering vfr4

I think I may have another possible solution. If at first render a temporary folder is made on both the host and slave machines in a set file path (c:\program files\asgvis\distributed rendering\temp...for instance) to store all of the textures in. Now at each render vray checks that folder on the host machine and if a new texture is needed that isn't in that folder the image gets added from the true location. This added texture is then transfered to the temp folder on all machines. This will prevent the need for any network communication between the machines since the temp folder will be the same for all machines. When the drspawner is closed it can clear the contents of the temp folder. This has a minimal amount of textures to be transfered as well as a minimal amount of network communication needed.

[joe_b]

Re: distributed rendering vfr4

I think I may have another possible solution. If at first render a temporary folder is made on both the host and slave machines in a set file path (c:\program files\asgvis\distributed rendering\temp...for instance) to store all of the textures in. Now at each render vray checks that folder on the host machine and if a new texture is needed that isn't in that folder the image gets added from the true location. This added texture is then transfered to the temp folder on all machines. This will prevent the need for any network communication between the machines since the temp folder will be the same for all machines. When the drspawner is closed it can clear the contents of the temp folder. This has a minimal amount of textures to be transfered as well as a minimal amount of network communication needed.

Thats a good idea - but what happens when a new dr slave is introduced? The host will have those files in the temp folder, but the new dr slave will not. Thats kinda why I'm saying all-or-nothing at the moment.

[joe_b]

Re: distributed rendering vfr4

If I understand right, it's no problem to implement, that all textures are send to the slaves. Fine. The problem is to check the textures for update issue, right? If yes, this is not so important. I think, the user can send textures again after modification or manualy bring the single, modified textures to the slaves. So, the workflow would be, that first all textures are send to the slaves, than renderings can be done, but without resend the textures allways. Sounds good.

Again - what happens when a new dr slave is introduced. This send "at first" will not send anything to the newly added slave.

[Travis Serio_cg]

Re: distributed rendering vfr4

The next set of problems you will run into is sharing and permissions. I personally think that if you were localize the textures to a directory like "Hey where do you want to put all of these textures?" C:\textures you could then make sure that C:\textures is shared on the network, then VFR should only have to pass \\server\c\textures to the slaves. Granted it might be slow to pull the maps across the network if they are very large resolution maps but it would work and you would only have to worry about file sharing permissions on 1 machine vs all of the nodes.

well now that i think about it, youll have to transfer the maps no matter what, so why not localize them to the server and just pass network paths to the nodes. seems like a lot less work. Change C:\Textures to \\servername\c\textures

[Travis Serio_cg]

Re: distributed rendering vfr4

the only other thing i can think of is some sort of node transport file that contains all of the scene info and maps in some compressed format, but that goes far beyond whats possible with the current SDK. Perhaps its something for discussion with chaos group.

[Micha_cg]

Re: distributed rendering vfr4

Is it very often, that a new render slave will be introduced? And if a new slave is introduced, why it is not possible to start the "send textures to all slaves first" again?

The quick way to implement this is to just always distribute all the textures to the DR slaves. So I'm not sure if people want to be transferring their textures on every render. But I guess I can do that by default and if people have a problem with that bogging down the DR render time they can put the files somewhere locally and opt out of the texture transferral.

From my view, I would be happy, if I could send the files per "send textures" button to the slaves. The default load from the server is good for scenes with less and small textures or a long final rendering without many tests. So, both methods, the default load from the server and the custom send button, could be nice.

[Travis Serio_cg]

Re: distributed rendering vfr4

hey micha thats a great plan, the only problem is that people will have issues with network share permissions and now with vista even more read/write restrictions.

[smail]

Re: distributed rendering vfr4

Hey, does anybody have got the same problem as me - I can't run the program for distributed rendering ...Spawner.exe at the slave machine :'(?(what happens is better described in "bug" forum for Beta2) Slave machine runs on Core 2 Duo 1,6 GH and XP Home 32bit...
Thanks. Smail

[joe_b]

Re: distributed rendering vfr4

Hey, does anybody have got the same problem as me - I can't run the program for distributed rendering ...Spawner.exe at the slave machine :'(?(what happens is better described in "bug" forum for Beta2) Slave machine runs on Core 2 Duo 1,6 GH and XP Home 32bit...
Thanks. Smail

I'm not sure - a few people have had this issue. We're still looking into it. I'm guessing its some kind of missing dependency, but I've not been able to reproduce the problem on my machines.

[smail]

Re: distributed rendering vfr4

Hey, does anybody have got the same problem as me - I can't run the program for distributed rendering ...Spawner.exe at the slave machine :'(?(what happens is better described in "bug" forum for Beta2) Slave machine runs on Core 2 Duo 1,6 GH and XP Home 32bit...
Thanks. Smail

I'm not sure - a few people have had this issue. We're still looking into it. I'm guessing its some kind of missing dependency, but I've not been able to reproduce the problem on my machines.

Hi Joe, thanks for your reply, I tryed to use dependencywalker.exe and it show me that I haven't got these two .dll > DWMAPI.DLL and EFSADU.DLL. You can download the .txt file Depency Walker report created from dependencywalker.exe with all report.
Thanks, smail