new system sanity check

[squintnic]

About to order this build as my home machine with future expansion to 4 x 2080 Ti when I can afford them to make more use of Vray GPU
I mainly use max, vray, fusion, phoenix, unity and houdini

Was toying with a CPU with less cores as I use Phoenix a lot ...

Anything jump out as weird or not gonna work

CPU	AMD Ryzen Threadripper 2990WX (Base: 3.00GHz, Boost: 4.20GHz / 80MB Cache / TR4 / 32 Core / 250 Watt / Without Heatsink/Fan)	1
CPU Cooling	Cooler Master Wraith Ripper CPU Cooler, Addressable RGB Wraith Armor, Compatible with AMD Socket TR4 only, Official Cooler for 2nd Gen Threadripper	1
Motherboard	ASUS ROG ZENITH EXTREME MB, X399, 8x DDR4, 4x PCI-E 3.0 x16, 1x DIMM.2, 1x M.2, 1x U.2, 6x SATA3, 15x USB3.1, ATX, SupremeFX S1220, AREION 10Gb, Wi-Fi	1
Graphics	ASUS Turbo GeForce RTX 2080 Ti (Base:1350MHz,Boost:1560MHz), 11GB GDDR6 (14000MHz), PCI-E 3.0, 2x DisplayPort 1.4, HDMI 2.0b, USB Type-C	2
Graphics Link	nVidia GeForce RTX NVLink Bridge - 3 Slot,	1
Memory	G.Skill Trident Z F4-3200C16Q-64GTZ 64GB (4x16GB) PC4-25600 (3200MHz) DDR4, 16-18-18-38-2N, 1.35v, Quad Channel Kit, Silver Red	2
SSD	Intel 1TB SSD, 760P Series, M.2(PCIE), Read up to 3200MB/s, Write up to 1600MB/s, 3D TLC NAND, Type 2280	1
HDD	Seagate 10TB BarraCuda Pro 3.5" Hard Drive, SATA III, 7200RPM, 256MB, Includes 2yr Seagate Recovery Services	1
SSD #2	Intel 2TB SSD, 660P, M.2-2280 PCIe NVMe, Read/Write (Max) up to 1800MB/s, 3D QLC NAND	1
Case	Corsair Obsidian 750D Black Airflow Edition ATX Case, Side Panel Window, No PSU, 2x USB 3.0, 2x USB 2.0, 3x 140mm fans, Aluminum Faceplate, Steel Stru	1
Power Supply	EVGA 1000W SuperNOVA P2 Modular Power Supply, 80 PLUS PLATINUM, 6 x PCI-E, 10x SATA, 5x Molex	1
Operating System	Microsoft Windows 10 Pro, 64 Bit	1

[Muhammed_Hamed]

Really solid build as it is! it will work nicely with 2 2080tis
For 4 cards I would change few things,
-You will need 1600 Watts at least..
-Even if you are using blower style cards, stacking 4 2080tis will not be easy. You should go for an AIO liquid cooler like the H150i Pro, and make sure it is compatible with TR4(old ones don't have the mounting bracket for TR4)
You shouldn't go with an Air cooler, because you don't want to deal with any more heat inside your case. Even if you have blower style cards (which helps a lot), your cards and Mobo will produce a lot of heat inside your case. which can be an issue.
-The Obsidan 750D is a massive case, I have used it before for few builds and it does the job nicely. For 4 cards, I would recommend the Lian li 011 air for many reasons. The airflow in this case is by far the best in the industry
You will mount your radiator to the top of the case, with exhaust fans ,And you can still use up to 9 more fans!. So maybe 3 intake fans at bottom, 3 intake fans at front and 3 Exhaust fans at side. This way, you will get very good thermals and very quite machine
And you can use 2 power supplies in this case, so you can start with 1000W with 2 cards..
Also I would recommend the Samsung Pro 970 NVMe, you can use up to 3 with your Mobo and it has better write speeds compared the one your listed above.

[squintnic]

Thanks for the insight Muhammed appreciate it, finding it quote hard to find decent info about how people set up these 4 x GPU machines.

Will probably be easier to just got to a 1600w PSU now rather than have to replace stuff when I get more GPUs. This box will be in a stable air con environment but Ill look at the liquid cooling given its a small investment on a much larger one. Have also added a 10gbps network card to allow it to talk to my NAS at a reasonable speed

[glorybound]

Build me one too, please

[gadzooks]

Curious to the cost of all that? I would venture to say at least $10,000 USD?

[Muhammed_Hamed]

Should be around 8k USD for 2 cards, if you are in US
The 10 GB card will block one slot, you will be able to run only 3 GPUs in future
A while back Asus used to ship this card for free with the Zenith extreme, see if you can get an offer on both
https://www.asus.com/Motherboard-Acc...OG-AREION-10G/
Having good airflow, AC should help, still would recommend an AIO liquid cooler for your CPU.. The topmost card will be hottest, and very close to the massive air cooler.
If you mean custom watercooling, this will be way more expensive and adding more cards in future, you will need to redo your loop... for 4 cards, you should be fine with air cooling for your GPUs
7, 8 or 11 cards on air/water, are possible as well, you will need a different case, Mobo and CPU

[squintnic]

Yeah I looked for that offer but couldn't get it, will have to go with a 3 x GPU machine and ill expand into a separate render machine if I need more. I was hoping to use a lot of online render farm or Vray Cloud this year.

It comes to about $12,600 AUD including build and Win10 Pro....I I expect it will last me 3 or so years. Will keep my older 6 core i7 and overclock that as a box for Phoenix sims with fast drive and RAM

Will put the deposit down tomorrow.

[Muhammed_Hamed]

Really good price, considering Hardware is usually more expensive in Australia.
Also, The 2990wx is very powerful, probably faster than a single 2080ti with Hyibrd rendering, so should be a very powerful machine
Expected life span according to Asus/AMD is 8-10 years, and you can swap the cards whenever needed along the way
Power supply warranty is 10 years, RAM warranty is life time and Mobo is 5 years

[squintnic]

Yeah interested to see what kind of times I get with hybrid rendering and lPR viewport for fast feedback.

Trying to work out if its worth keeping this 3.5 year old machine and OC the CPU for phoenix sims given it works better with less cores that are higher clocked... and a use it as a render node for Fusion ... or just sell it and make my new machine cheaper and save the fuss of maintaining 2 machines?

CPU	Intel Core i7 5930K (3.50GHz / 15MB / LGA2011-3 / 6 Core, Without Fan/Heatsink, Fully Unlocked)	1	1
CPU Cooling	Corsair Hydro Series H80i GT Performance Liquid CPU Cooler	1	1
Motherboard	Gigabyte GA-X99-UD4 MB, X99, 8x DDR4, 2x PCI-E 3.0 x16, 2x PCI-E 3.0 x8, 10x SATA III, 1x SATA Express, 2x M.2, 8x USB 3.0, ATX	1	1
Graphics	Gigabyte GeForce GTX 980 Ti (Base:1000MHz,Boost:1076MHz), 6GB GDDR5 , PCIE3.0, DVI, HDMI, 3x DisplayPort, 980Ti	1	1
Memory	Crucial 32GB (4x8GB) PC4-19200 (2400MHz) DDR4, Ballistix Sport, CL16, 1.2v, Quad Channel Kit	2	2
SSD	Samsung 512GB SSD, 850 Pro Series, SATA III, Read up to 550MB/s, Write up to 520MB/s	1	1
HDD	Samsung 1TB SSD, 850 EVO Series, SATA III, 7mm, Read up to 540MB/s, Write up to 520MB/s	1	1
Case	Fractal Design Define R5 Black ATX Case, No PSU, 2x USB 3.0, 2x USB 2.0, Audio In/Out, 8x 3.5" HDD Trays, 2x 5.25" Bays, 2x140mm Fans	1	1
Power Supply	Cooler Master 750W V750 Semi-Modular Power Supply, 80 PLUS Gold, Intelligent 120mm Fan, 4x PCIE 6+2, 8x SATA, 6x Molex, Unique 3D Circuit Design

[numerobis]

Yeah I looked for that offer but couldn't get it, will have to go with a 3 x GPU machine...

As far as i know memory pooling with NVLink works only for pairs. So if you need more memory you can only use 2 or 4 GPUs.

[Muhammed_Hamed]

Nah, don't sell it
the 5930k overclocks very nicely and beats any Ryzen CPU(including TR) in single threaded tasks. You can run it at 4.5 GHz at least, which is over 25% more performance
I've seen few samples that runs fine at 4.7/4.8 GHz .. Overall the X99 platform has been my favorite Intel platform of all time, it still holds nicely to 2019 standards
Even for mulithreaded loads, it is only 30% slower than the 8 core Ryzen 2700X.. but faster in single threaded tasks
to see how the AIO cooler is holding up, load the optimized defaults in bios, then set cpu ratio to 45 and Core voltage to 1.295
Save changes and restart, then use Aida 64 to stress test, for like 10 minutes.. See if you get any errors and see what kind of temps you are getting
Best case scenario is temps under 70-80 degrees, meaning the airflow is good, and the cooler is working nicely
if it crashes or if you get any errors in Aida, it is still fine.. you can overclock through bclk frequency and changing few other settings accordingly
let me know how it goes then you can take it from there to see the max potential of your CPU.. maybe 4.7 or 4.8 could be possible, I will help you with voltages
And btw, this board can handle 4 2080tis just fine with a higher PSU.. So you can use it as a render slave if needed later
with 750 Watts, it can handle 2 2080tis (5930k draws way less power than a 2990wx)
you can even supercharge it with an m.2 NVMe

For hybird rendering, you should get an additional 2080ti for free, using your CPU .. This is the beauty of Vray compared to any other GPU renderer.

[Muhammed_Hamed]

As far as i know memory pooling with NVLink works only for pairs. So if you need more memory you can only use 2 or 4 GPUs.

He only needs memory pooling if his scenes are very big, and would not fit in 11 GB of VRAM
other than that, he will be using 3 cards for rendering without issues
for 11 GB of VRAM, I could fit up to 60 million polygons.. I barely ran out of VRAM in any project

[squintnic]

Great thanks for the insight Muhammed, Ill give a mild overclock a go once I get new one up and running.
I would love to turn the old box into a GPU box....(once I get more money these 2080Ti are expensive )

yeah i know nvlink only works for 2 cards, will assess once i get a few projects going. I just assumed 11GB was not anywhere near enough, so only became interested in GPU when I saw it was able to do 22gb

[numerobis]

He only needs memory pooling if his scenes are very big, and would not fit in 11 GB of VRAM
other than that, he will be using 3 cards for rendering without issues

Really? I thought it would also be needed for less than 11GB and card selection would be disabled immediately after a third card is plugged in...

[numerobis]

squintnic here are two boards with 4x PCIe 3.0 x16 and 10GbE
https://www.asrock.com/mb/AMD/Fatal1...onal%20Gaming/
https://www.gigabyte.com/Motherboard...REME-rev-10#kf