my new, quite unique i think, workstation :)

[super gnu]

hi all, well ive been rolling various options around for months about what spec new workstation to build.

in the end it came down to a dual xeon or 1 or more oc'ed i7's

the thing that bugged me was the lack of overclockability on the sandy bridge xeons. i like to oc my hardware and get better bang for buck.


i also like fiddling round with bits of wire and soldering irons.


i also like things in neat packages.

so i came up with this:



looks normal enough.

midi tower case, a 3930k with an H80 cooler, 32gb ram, a samsung 256 gb ssd, and a 4gb gtx 670.

works very nice in max and photoshop.

looks like this inside:



not the cleanest install i agree, but certain hacking had to be done.

open the front and it looks like this:




so far so normal, but.... wait.. whats this?



an extra button?


you see, there is always a bit of extra space inside these pc cases, if you use micro atx quite a lot, and if you are neat with your wiring, really a lot of extra space. also, if you get a big enough power supply, you have plenty of juice left. mine is 1475w.


so, when im working in max or photoshop or reading the chaos forums, i use my workstation like normal, but when i want to render, i press THE RED BUTTON.


that activates this:




a second 3930k, mounted opposite the first. also with 32 gb ram and a samsung ssd. it shares the psu with the first one (custom wiring ) and is accessed with remote desktop from the main system.


both are nicely overclocked to 4.4 ghz, and cpu temps running stress testing (IBT) peak around 72 degrees. ive still got to optimise things a bit.. hoping to get the temps down a bit.

i also intend to fit a compact gigabit switch in the case too, so i can connect it all up internally. at the mo, they are both connected to an external switch.

id love a 10gb ethernet conenection between them, or infiniband or something, but for those prices i could buy another rendernode. - anyone got any suggestions?



basically means i have a compact, efficient workstation that breathes fire when i need it to.

cost about £3500 all in.


there is actually still space for a second gpu in there, but that may be pushing the thermals a touch far


main disadvantange over a dual cpu system, is i have 64 gb ram in there, but i can only ever use 32 in a scene.

[Alan Iglesias]

Nice, Gnu! Very innovative!

-Alan

[Dariusz Makowski (Dadal)]

Heya

Hehe nice rig! I like the micro addition !

[raymarcher]

Haha very nice, ace...

[super gnu]

well, since my thunderstorm related disaster, where i lost the ethernet ports on both motherboards, i decided to take advantage of the situation and improve the machine whilst fixing it. i returned one motherboard to the seller and got a replacement, so now i have one functional ethernet port for connecting my modem.

i then decided to take a risk on an experiment.

ive always wanted to try to get something faster than gigabit ethernet for DR and network rendering, since with ssd speeds aproaching 6 gbit, id imagine the network is always a bottleneck.

i looked at various options, including 10gig ethernet. way too costy.

then i found this page:

http://davidhunt.ie/wp/?p=232

i looked on ebay and got myself a pair of voltaire infiniband adapters secondhand for 30 euros each. cheaper than ethernet cards! pulled from some old HPC cluster somewhere.

i had no idea if windows drivers were available ( although i had a few hints that the drivers provided by openfabrics should work) and so it was an experiment.

the 1m cable to connect them cost almost as much as the 2 cards.

anyway, long story short, i plugged it all together, used a hacksaw to modify the plugs on the cable as it didnt quite fit, installed the drivers from openfabrics, and well.. it kinda worked out the box! i got approx double my gigabit transfer speeds. but no internet. after a few hours i had internet working but no remote desktop or backburner.

after a day of fiddling i had it all working, and then today discovered a config option in the driver that seemed to fix my bandwidth issues. now copying from one system ssd to the other, i get approx 368MB/sec. not too shabby. at least 5x what i was getting with my gigabit connection. still nowhere near the theoretical 10 gbit limit, but it seems you need to ditch the whole IP system to get the best results.. thats way beyond my knowledge level.

DR jobs start almost immediately on the few tests ive done. im not sure how to test any vray RT improvements, but im guessing there should be some improved responsiveness at least.

in theory, since my cards are dual port, i can double up the connections, but im not sure its worth it yet. i can also, in theory, connect up more rendernodes using the second ports on both cards, without needing an infiniband switch (available on ebay for £700 ish secondhand i think)

basically i now have possibly the worlds smallest HPC cluster.

[Dariusz Makowski (Dadal)]

Thats soooooooo cooooool. Nice1 ! I gotta try it at my place too. 10gb/s transfer mmmmmm sweet.

[JohnVK]

That is one SWEET machine you've built there!! Awesome!!

And thanks very much for the additional info on infiniband! Very interesting indeed to see it can be had for much less than I thought previously!! Cool!