Server hardware / editing speed when compositing 1080p

[cgrant3d]

We've been rendering everything @ 720p for the longest time so the past few months we've taken a couple projects and done em at 1080 to test the workflow / process. It's amazing how much slower everything moves when you're trying to comp EXR's @ 1080 vs 720. We've got full duplex gigabit to the workstations & a decent file server w/dual full duplex gigabit connections.

For those of you who have made the leap to 1080 (or beyond) what is your network / server / hardware setup? Any sort of special network info - ie 10gigabit ethernet, etc?

Thanks,

[joconnell]

For the comp stations on game of thrones everything was on fiber channel for the comp stations. Realistically you're not going to get anything resembling smooth unless you're working off local disk arrays for comping - their nuke scripts with multichannel exr files and a few paint nodes were responsive enough but no where near real time over the network. Rob Nederhorst was using a machine with either one of those memory chip disk cards or some kind of high speed local disk array on priest and he seemed fairly happy with it, but again nothing was being pulled from the network.

[Dariusz Makowski (Dadal)]

Im not working on 1080p but I'm preparing my network to deal with them.

The workflow is pretty much inverse approach to centralized servers...

Basically u mirror all data across all workstations that work on the project. So if there is 5x workstations, each of them will have a 4TB HDD that is live mirroring all data between all 5 workstations. This way each got updated project and all is based locally. All u have to do is to copy renders from network server to workstation, rest is easy.

As to hdds, 512gb/1024 SSD... Raids 0. U also need fast buss speed for CPU/Ram I'd sugest 1600 to 2500+, as much as u can ! And a lot of GHZ per core to maximize single core tasks...

[instinct]

All u have to do is to copy renders from network server to workstation, rest is easy.

Depending on the type of work and the amount of material this can be quite a hassle actually. And don't forget this can lead to all sorts of "Oh damn, didnt know there was a new version". Nuke 6.3+ supports built in localising that seems to work quite okay. It still is a hassle tho.

As for our setup, without going into too much detail we are using scale out storage, 10GBit in the backend (multiple dedicated 10GBit links into the farm and the artist networks). With the amount of rendering, updates and parallel projects localising seems kind of daunting. Tho i am always open to suggestions.

Regards,
Thorsten

[Dariusz Makowski (Dadal)]

Heya

Yea could be, then u could set up all outputfolders to just be atutomatically mirrored to ur workstations.

In any case... 10gbit connection is 1gb/s data transfer rate. If u have 10pcs ur server need to be able to trasnfer 10gb/s in order to keep up max data transfer... unless u have specivic swtich that can just split data to separated workstations by itself instead of transfering it from source...

In any case, does ur server HDD can actually outpuut 1gb/s + ? (I mean 1000 mb file/s)

[instinct]

Well it's obviously not a single HDD, not even a single node, but yes it does. The network is still the limiting factor.

Regards,
Thorsten

[Dariusz Makowski (Dadal)]

Maybe try doing a local test, on workstation see if workstation on its own can handle 1080p. If no then its not network issue.

[cgrant3d]

Some great suggestions so far. Got me thinking about picking up a SSD raid card that delivers 1500MB/s ... That'd fundamentally change our process and I didn't even realize how affordable it'd be.

[Dariusz Makowski (Dadal)]

Ur workstation should have build in RAID controller - if no dont buy from that distributor again, in any case just drop in SATA 6, SSD see if that helps, if no get another 1 raid them in 0 and see if that works

[robdeja]

We've got fibre attached storage connected to the compositing and edit workstations. These chuck out about 700MB/s sustained rates when delivering uncompressed TGA file sequences - the drives are optimised to work best with files of this size.
We have got SSDs in some of the other machines and although technically should achieve reads of 500MB/s, they fail to do so due to reasons such as the onboard software controllers getting bottlenecked (you want dedicated controllers really). And while the burst read/write rate is good, the sustained data rate isn't so good, so when editing it sometimes struggles to play a couple of 1080p streams back in realtime although on paper it should be fine.
For compositing, SSDs are not so bad since you generally cache short sequences to ram and it's just the fetching of the frames which takes time.