2 x Intel Xeon E5-2699v4

[sherridge101]

Hi all

We’re looking at adding to our rendering capacity and was wondering if anyone had any experience with the 2 x Intel Xeon E5-2699v4 configuration?

The Boxx workstation with these 2 monsters looks great, but we’re just wondering whether we go for a workstation or to put the chips in a rack mounted server.

Any experience would be much appreciated.

Cheers

[sherridge101]

Or any experience with the Apexx 4 or Apex 5 machine?

Cheers

[Morbid Angel]

Well that depends on what you need the server for. Is it for pure rendering like DR, or sequential rendering, then perhaps a rack server is better. But you have to consider that rack server is usually loud and hot, so you need to cool it and have it in a sound proof space or else it will be difficult to sit next to it the whole day.

Workstation is only good with powerful video card, so if you plan on expanding later or using the workstation for actual work besides just pure render then its better of course to have a workstation.

[smbell]

We have 10 dual xeon rack machines 4 dual xeon 2630-v3s and 6 dual xeon 2640-v3s and 4 workstations running an array of dual xeon models. The workstations are for the artists and render at night, the rack machines have no discrete video cards and are just straight up rendering whenever we need them.

Yeah, they are loud as hell. We have them in a separate room with a soundproof door and you can still hear them all very well.

The newer generation xeons run surprisingly cool compared to generations past. We have upgraded the racks probably three times now over the years and they get cooler and use less power with more cores each time. It's honestly pretty amazing! We actually had two external AC units installed for the first generation xeons as they would raise the temp in the room past 95 degrees if they were going for several days, now with the current gen, we only have one turned on and it's overkill really. I go in the server room to cool off after walking back from lunch

Those cpu's you are looking at are beasts, super jealous. The fastest we have is a dual 16 core setup and it's what I run for my standard workstation. It runs v-ray great!

[jstrob]

Hi all

We’re looking at adding to our rendering capacity and was wondering if anyone had any experience with the 2 x Intel Xeon E5-2699v4 configuration?

The Boxx workstation with these 2 monsters looks great, but we’re just wondering whether we go for a workstation or to put the chips in a rack mounted server.

Any experience would be much appreciated.

Cheers

My advice: avoid Boxx, Dell, Apple etc, Just buy directly a Supermicro board for a workstation and choose your own case it will cost half and will be better. I owned a Boxx one time and it was a supermicro board inside, a cheap and noisy case and fan for a exorbiting price. I now build my own with pieces from Newegg it's far better.

You can also buy Supermicro barebones, they are rackmounts servers and all you have to add is the ram and cpu. So it takes 30 minutes to install and they are good to render and surely cheaper than Apexx.

You can also get very cheap racks from middle atlantic. I got a slim 5 but I don't remember the size I paid maybe 300$ and I can't be more statisfied with it. https://www.amazon.com/s/ref=nb_sb_n...slim+5&x=0&y=0

And look here before buying any cpu: http://www.cpubenchmark.net/high_end_cpus.html Choose the fastest one just before they start to jump to twice the price for only 5% more speed!!! And just make sure the cpu will fit on your dual cpu board by reading the supermicro specs. There seem to ba a whole batch of cpu that are faster and way cheaper than the E5-2699 v4.

[jstrob]

To cool my server room I use this since many years: http://www.newegg.ca/Product/Product...-067-_-Product

and I forgot to say supermicro also make barebone that can fit 4 servers in 2U (so 8 cpu rendering inside of only 2U). SYS-2028TR-HTR it will be my next buy.

[jstrob]

Here is even an example of a cheaper build than a appex and really more computing power:

1 *** Supermicro Render Server ( 4 Servers in 2U )
32 Kingston - Memory 8Gb ECC REG 2400Mhz
4 Samsung - SSD 850 PRO 2.5in
1 Supermicro - 4 nodes in 2U ( SYS-2028TR-HTR )
8 Intel - CPU 2620 v4 2.1/3.0Ghz 8 Cores 25Mb 2133Mhz

You should be able to get all this for around 10 000 USD (it's a quote I saw that from a friend, it was from a few months ago and it included warranty and assembling).

[sherridge101]

Hi all

Thanks for all the advice.

We render high res prints and some 72 frame 360 spins for web.

Dmitry
Good advice on the GPU.
We are thinking of a GPU for monitor and a heavy duty GPU for look dev and some rendering.
As we will be handling large CAD data, HDR’s and textures, we’re looking at the P6000 and a case that allows to add in more if need be.

smbell
Good point on the AC and noise.
We could put any rack mounted machines in the server room and maybe pad out the walls with sound proofing
Are you using a render manager for submitting the images to the servers?

jstrob
Thanks for the advice on building a pc.
Would be great if building a pc for myself, but the machines will be for the company I work for
and so the reason I was thinking Boxx, was for the support etc. if and when things go wrong.

Cheers for the list. Thats surprising to see the speed/cost difference - the 2679 being faster than the 2699 - I guess the faster clock speed makes up for the fewer cores - 20 vs 22.

Crazy that the:
Intel Xeon E5-2679 v4 @ 2.50GHz is $2,702.00*
and the:
Intel Xeon E5-2699 v4 @ 2.20GHz is $4,694.18
and yet a little slower.

Boxx have the following chip options:

2 x Intel Xeon E5-2620v4 (16 Cores, 32 Threads) 2.1GHz
2 x Intel Xeon E5-2630v4 (20 Cores, 40 Threads) 2.2GHz £ 630.00
2 x Intel Xeon E5-2640v4 (20 Cores, 40 Threads) 2.4GHz £ 1,300.00
2 x Intel Xeon E5-2650v4 (24 Cores, 48 Threads) 2.2GHz £ 1,830.00
2 x Intel Xeon E5-2660v4 (28 Cores, 56 Threads) 2.0GHz £ 2,520.00
2 x Intel Xeon E5-2680v4 (28 Cores, 56 Threads) 2.4GHz £ 3,280.00
2 x Intel Xeon E5-2683v4 (32 Cores, 64 Threads) 2.1GHz £ 3,500.00
2 x Intel Xeon E5-2695v4 (36 Cores, 72 Threads) 2.1GHZ £ 4,970.00
2 x Intel Xeon E5-2698v4 (40 Cores, 80 Threads) 2.2GHz £ 6,870.00
2 x Intel Xeon E5-2699v4 (44 Cores, 88 Threads) 2.2GHz £ 9,140.00

As the list link, there’s a big jump in price toward the top end, so perhaps the 36 or 40 core machine is the way to go?

Nice little AC unit there too.

What was your thinking on the x4 Samsung - SSD 850 PRO 2.5in?
Are you using some drives for caching?

—

The plan is to add a high spec work station, so we can work on during the day and render overnight.
As we’re looking to bring more people in, we might go for a couple more pc’s, and then invest in the server side. It’s still a little up in the air. The main thing is, that we need more render power…as always

One of the other concerns we have is whether V-Ray can render on all 88 threads, should we go the 2699 route. I’ve read that V-Ray uses no more than 64 threads? Is this correct?

Thanks again guys.

[Paul Oblomov]

We have now huawei units. They are cheap (compared to our IBM) and quite good, tho - network abilities and web ui is quite not THAT good, but enough.

[smbell]

smbell
Good point on the AC and noise.
We could put any rack mounted machines in the server room and maybe pad out the walls with sound proofing
Are you using a render manager for submitting the images to the servers?

We use Muster. It's just the right balance of customization and utility for what we do and the size of our studio. We use custom python scripts to submit jobs from a batch file, so every render we submit has a record of the frame range and various things like camera and render layer. It makes for keeping things consistent when it gets to comp stage very easy. If there is a problem with a render, we can resubmit it easily and know that it's the exact same frame range as before etc. Want to submit a new render layer? just copy the line, un-comment it out and submit. You know that the correct settings will all be applied since the submission line is copied. All the render managers have a UI submission thing, but we've developed something over the years that works well and Muster makes is pretty easy to work with. When we started using v-ray i was able to modify our submission script in a few minutes and it instantly worked for the new renderer.