NAS Bottleneck Question

I tried rendering an animation with all my files linked to a NAS. About 10 computers tried accessing saved IRMaps and LC files.

The machines just hung.

So my question is, is that normal? Would you expect the same behavior using Samba or Server 2003?

I ended up getting around this problem by locally saving all my files to each machine and it rendered fine.

at the office i have my paths saved into an windows server 2003 dedicated server and have no problem when using DR over the network.

DR works fine.

However, I’m talking about a large animation file. 21 megs for the saved maps, 140 megs for the actual file, and loads of textures run over Backburner.

hum…last time i tried had no problem…
i might be able to run some tests tomorrow
:wink:

It depends on the NAS and the files i guess, but a maschine hanging is expected behaviour if it is too weak to keep up :stuck_out_tongue:

Seriously also a nice and performant samba maschine will bog down when under heavy enough fire of a decent sized farm. Not only networkwise but also CPU wise.

Regards,
Thorsten

So I guess I’m just wondering if it’s generally best practice to run larger files in local directories on each machine.

I do have a Gigabit network but it seems there are limits either from drive speed or processing.

Is there an easy way to drag files over to one directory than have that directory automatically copy the files over to all the other computers at once instead of having to do each one manually?

no problem on the NAS here.

No problem on NAS here… Lacie Ethernet 2TB…

@thatblanch + idesign : how many slaves on one job and scene/maps/irrmaps size ?

Just wondering.

Regards,
Thorsten

Test your network speed, and compare nas performance

http://www.smallnetbuilder.com/component/option,com\_nas/Itemid,190

probably it is network speed issue

Network Speed is at 1 gigabit on all adapters and switch.

I am not a network guru but not all nas drives give the same performance, even though they have all 1 gigabit.

For your question..
Is there an easy way to drag files over to one directory than have that directory automatically copy the files over to all the other computers at once instead of having to do each one manually?

You can just write a simple batch file and run it when you need it

copy *.* \\slave2\shared /Y
copy *.* \\slave3\shared /Y

/Y for overwriting existing files

Very handy thank you - how do you work this .bat file? You drag all the files you want to copy over to the .bat then it copies it all on over?

copy all of your files in a separate folder and put your batch file into that folder and run it

I usually put all of them to windows shared folder, by default all slaves have that folder

Thorsten, our renderfarm in the moment is:

In the Rack everything 1 Gbit with a 48port/1Gbit Dell switch

1x Dell 2950 Server Dual QuadCore 1.6 Ghz w/ Windows Server 2003 x64

- 2x250 Raid 0 for system
- 6x250 Raid 5 for Maps, Models, Proxy’s, etc

1 Lacie Ethernet Disk 2TB (with 2xLacie BigDisk 1TB via USB for Backup) for Work (dwg, max, cliente files, etc)

6x Dell 1900 Rendernode’s Dual Quadcore 3.0 Ghz, 8 Gb, 15k system disk (xppro x64)

APC UPS 3000

Out of the rack, comunication with workstation’s is 100 Mbps, but in near future, everyting go to 1 Gbit (new office).

This setup is new and in test.. normaly I render DR with 1 worksation and 4 slaves.. sometime’s, a file don’t work with DR (i have to do some test’s - object problems ou size problems)

K that’s about what i thought. The problems i was referring to earlier didnt appear as long as we didnt exceed 30-40 slaves if i recall right.

Regards,
Throsten

The batch file works like a charm - saves a TON of time thank you and also seems to be faster than using the Windows GUI LOL.

I take it that if I want to delete files I would add ‘delete’ instead of ‘copy.’ Is there any way during deletion that I could add a parameter to prevent the deletion of the .bat file itself?

You could specify the types of files you do want to delete

del *.jpg *.tga

nas are a little slow in my experience
in most of case nas can be the right way
but when you have many vray proxy
botltleneck can be awfull
testing a scene

all servers take 30min to render
first frame
(botltleneck of nas network i think)
max file archive was 350M
30 min for the first frame
10 min for folowing frames

in my experience
if you have 10 renderslaves or more
you must have w03 or 08 server
or some linux server running
backburner manager

just copy all proxy and bitmap
in local server

4min to render
first frame
4 min for folowing frames

my reflection about that experience is:
have your maps in your rendering server
max files in your W03 or W08 server or nas
frames can be on nas
maps in your local rendering server

Well, I’ve concluded a NAS is not efficient for handling large scenes with many proxies or large IRMap/LC saved calculations saved to it. It causes slaves to hang by a bottleneck until all the files are distributed. This could add 10 minutes + in some situations (for mine, it was actually 20) before the scenes start to move.

Moving all the files locally to each computer results in an almost instantaneous startup of the slaves.

It would be nice to have all the files in one centralized location. It seems a better production quality NAS, a dedicated Samba server, W3k, or W8 may work a lot better. I bought one that had stellar reviews - it’s great for working and DR, but bad for animations. But even so, I have a difficult time thinking these files are handled more easily when multiple machines and people are accessing/transferring files from one place at the same exact time.