Ghetto build

[biochemical_animations]

So, I've had a bit of fun the last few weeks on my new ws build that I thought I'd share.

After doing research on other sites, and getting advice from members, this is the current result.






It's not the most sophisticated build, but it is a nice and fast gpu powered build.

The MB is a supermicro x9drx with 10 pcie slots at 8 x gen 3, and an 11th at 4x gen2 i think, they all actually run at gen 2, but are plenty fast.

There are 7 x 980 ti's classifieds, which I got for between $290-$380 each as they are evga b-stock.

Dual e5-2670 which were only $233 for the pair run the cpu computing. I was able to get 8x8Gb sticks of ddr3-1600 for $200 from natex (used server ram ecc).

Power was a bit of a challenge. The key, I believe from what I read and tested, is to isolate the graphics cards power not plugged into the MB. I have two cards in the MB running off a main evga g2 1000w psu powering the MB and CPUs. I went with powered pcie riser ribbons 8x-16x that are also powered by the main g2 psu. An extra evga g2 1000w psu powers 3 cards, an 750w psu powers 2 more cards.

The extra psu's are always on via the tester tool, basically just a jumper, then turn on the main psu to powerup the system. I got psus that have an eco mode smart fan, so the fans only turn on when they need to. I measured the extra psu's when on and the system off via a kill-a-watt meter pull only 15 watts...so I leave them on all tne time.

Everything runs great, and the gpus run between 60-74 degrees C.

I'm sharing this because I really believe gpu rendering is the next wave of the future, at least for animation, and I'd like to see it continue to catch attention and advance in development. Granted, I'm still pretty new to learning vray, but if those that knew hadn't shared with me, I would be where I am now...so thanks.

[mitviz]

sweet, looks like you know what your doing there, nice work

[biochemical_animations]

sweet, looks like you know what your doing there, nice work

Yeah, thanks to your help

[savage309]

Very nicely packed

Best,
Blago.

[Syclone1]

Nice work, reminds me of my first farm I built to sit between the wall studs in the basement of my old house. It worked pretty well too as the studs were mounted against the concrete basement wall and the wall stayed cool all year which helped in the cooling of the slaves especially in the winter. Just need to remember to keep all components an inch or two away from the concrete or put a vapor barrier between in case any moisture seeps through

[JezUK]

I have a newbie question as I go about learning about the benefits of GPU rendering.

And I'm new to this type of building.

My motherboard has 7 x pci-e slots (16x) and I've currently taken up all the slot space with my 4 x 1080 ti's (each 1080ti takes up the space of 2 slots, even though a card only uses one pci slot, it blocks another one).

So, is it a question of just moving the GPUs away from the MB (by using Risers) that I'd be able to theoretically use all 7 slots with 7 x 1080ti's ? (my PSU is 1500w and I'd believe that'd be powerful enough for 7 cards).

Dang, I need to know where I can buy a stand for 7 gpus....... (I don't have the facilities to make my own).....

[super gnu]

1500w psu would unfortunately be nowhere near enough for 7 x 1080ti's under load each pulls between 250 and 300w.. so thats 2.1kw just for the gpus, not including the rest of the system. obviously those are max figures, but you should always leave a healthy amount of headroom in your power supply calculations (as psu's age they handle max load less well) . id not feel relaxed with less than 2.6kw for 7x gpus and a system to run them.

[biochemical_animations]

I use these
https://www.moddiy.com/products/PCI%...FQOUfgodrDwMSg

one thing about power, use MB psu for gpus plugged into MB...use extra psu's for gpus plugged into risers...use MB psu to power riser.

use kill-a-watt meter to measure wattage.

check gpu power usage beforehand...for example if gpu uses 250 watt, plan that for psu, either extra psu or MB psu.

mb psu will use cpu, memory, hdd, so check https://outervision.com/power-supply...latcalculators for psu calc.

you need to get the power correct or you could fry your cards and system. Having more power on hand is recommended. I have three 1000w psu's now, one for MB, cpu, hdd, risers, and a gpu, the other gpus run off the other psu's.

[biochemical_animations]

I have a newbie question as I go about learning about the benefits of GPU rendering.

And I'm new to this type of building.

My motherboard has 7 x pci-e slots (16x) and I've currently taken up all the slot space with my 4 x 1080 ti's (each 1080ti takes up the space of 2 slots, even though a card only uses one pci slot, it blocks another one).

So, is it a question of just moving the GPUs away from the MB (by using Risers) that I'd be able to theoretically use all 7 slots with 7 x 1080ti's ? (my PSU is 1500w and I'd believe that'd be powerful enough for 7 cards).

Dang, I need to know where I can buy a stand for 7 gpus....... (I don't have the facilities to make my own).....

btw, yes, if you move the gpus you could use all of them with risers...if your MB and os can support it. I had to enable 4G decoding in bios after like 5 cards...windows 7 was able to handle all 7.

[JezUK]

1500w psu would unfortunately be nowhere near enough for 7 x 1080ti's under load each pulls between 250 and 300w.. so thats 2.1kw just for the gpus, not including the rest of the system. obviously those are max figures, but you should always leave a healthy amount of headroom in your power supply calculations (as psu's age they handle max load less well) . id not feel relaxed with less than 2.6kw for 7x gpus and a system to run them.

Hmmm, I'm not so sure about that (though I have seen a lot of videos since I posted last night and yes, people are using 2 PSU's so maybe it'd be the right thing to do, I do have another spare lying around ).....

But my Corsair Link tells me my 4 x 1080ti's under full load pull something like 850w out of the wall and a little less into the system (circa 830w). From memory, when I just had 1 x 1080ti it was around 350w under full load. So I believe adding another 3 would take me around the 1350w mark.

I think it'd be under 1500w by a suffficient margin but I could watch it carefully during first tests and assess it from there.

Either way I'm committing going for this as my tests on my first ever GPU project last week showed that using RT Production my renders were taking just 6 minutes versus 25 minutes when tested under Adv CPU.

So I totally concur with biochemical_animation that "gpu rendering is the next wave of the future"

[super gnu]

http://www.tomshardware.com/reviews/...ti,4972-6.html


power consumption is a tricky thing.. id always plan for the absolute peak power your card *can* draw. (and then add a healthy chunk for luck) . not what you see when using it. otherwise you might have unexpected crashes.

id also definitely not want to run a psu at its maximum rating.

you may of course find that on *average* your system might well support 7 cards, although running flat out, your psu could only supply 217 watts to each card, and thats ignoring the rest of the system. .

grab an extra psu for a few of the cards, it cant hurt.

[JezUK]

http://www.tomshardware.com/reviews/...ti,4972-6.html


power consumption is a tricky thing.. id always plan for the absolute peak power your card *can* draw. (and then add a healthy chunk for luck) . not what you see when using it. otherwise you might have unexpected crashes.

id also definitely not want to run a psu at its maximum rating.

you may of course find that on *average* your system might well support 7 cards, although running flat out, your psu could only supply 217 watts to each card, and thats ignoring the rest of the system. .

grab an extra psu for a few of the cards, it cant hurt.

I think you're probably right - I think PSU's tend to be at their most efficient at circa 50% max, so it'd make sense to have two PSU's.

Would one 1500w for the 7 x 1080ti's and one 750w for the MB and Proc do the trick ?

I'm currently looking up Frames for rigs (I don't have the facility to make my own) - here is the kind of rigs I'm seeing (they all seem 'mining' related);

http://www.ebay.co.uk/itm/Open-Air-M...3D322559966792

[super gnu]

id do 5 cards on your 1500w psu and get another of 1000 to power the mb and two cards... gives you a bit of headroom, that way the absolute peak draw of the cards will only be the max of your 1500w psu, and on average it would only be putting out around about 1000 - 1200w..

[JezUK]

Thanks guys - your information is all a great help.

I basically want to move as many of the GPUs away from the MB and physically spread them out a bit. (keeping the current computer parts in the case I have, which does have an opening window).

I currently own a 1500w and 750w psu (same make, model type, Corsair AXi range) - 5 + 2. I'm pretty certain the 750w would be fine with running two cards and stuff.

I need to learn more about all the components needed to make this work.

Any ideas on a shopping list ?

I like this open rig

[super gnu]

that open rig appears to have either pcie1x or pcie 4x connectors on the risers.. youd want at least pcie 8x