Which 7980xe motherboard supports the most GPU's ?

[JezUK]

I'm considering my upgrade options - earlier last year I was creating an Open Rig to hold and house 7 x GPU's as my mobo (an Asus X99-E WS can utilise that many).

However, time has marched on and I haven't gotten around completing that project, and I'm currently considering to upgrade my computer CPU, mobo etc to either a Threadripper or a i9 7980xe with as many GPU's as I can use.

I see that the benchmark results shows 7980xe processors with 4 x 1080 ti's but can't find any mobo that supports this via Google. Any ideas ?

Thanks.

[bildform]

I found only one intel mainboard with 7 pcie slots: ASUS WS X299 SAGE (90SW0070-M0EAY0)

None for threadripper.
Here is a nice link:
http://tomglimps.com/7_gpu_workstati...0_octanebench/

[JezUK]

I found only one intel mainboard with 7 pcie slots: ASUS WS X299 SAGE (90SW0070-M0EAY0)

None for threadripper.
Here is a nice link:
http://tomglimps.com/7_gpu_workstati...0_octanebench/

Thanks for that - this looks perfect !

[delineator]

Do pci-e lanes matter though? Without looking at the specs, I'm fairly confident TR has a ton more, however, does it matter? Do pci-e lanes matter for gpu rendering? I know that they don't so much for other gpu type rendering (I think I remember running across some octane benches that showed as much). For vray? I would also guess not, but curious if anyone has anything definitive.

[jstrob]

I don't think you should use more than 4 gpu for rendering on the same workstation. Just look at the benchmark:
8 x 1080 ti: 17s.
4 x 1080 ti: 20s
2 x 1080 ti: 32s.

To fit more than 4 gpu on most motherboard you will need to use risers and set above 4g to enabled and lower the pcie speed to gen 1. you will get very low framerate flipping frames sequences (like 10 fps). You wiil 2 power supplies and a 20 or 30 amp circuit instead of the regular 15 amp. So it's really not worth it. You better build 2 workstation with a max of 4 gpu in each one.

and most of all with more than 4 gpu your system qill be unstable as hell when doing 3D work.

[jstrob]

I would be curious to try an 8 gpu supermicro motherboard on linux. Maybe they could work better. but on windows with any other motherboard I don't think so.
https://www.supermicro.com/products/...028GR-TVRT.cfm

[allemyr]

Hi,

I might be wrong but I think its one thing to check before putting the idea of more then 4 GPUs on one motherboard on the shelf.

I've read here on the forum that the Benchmark scene doesn't scale well, don't know why that should happend but I think I've read it maybe someone can confirm or otherwise tell that I?am totally wrong here, but there is a chance.
I also think that the benchmark scene only should run on a single GPU (not the viewport GPU) because that's whats most interesting comparing different GPU's with each other, overclock and specific card should also be stated in the benchmark setup.

If you already have the PC availible I would test on a scene that takes a bit longer to render, maybe 10 minutes or similar, and a scene that transfers really fast to GPU, so transfering times isn't really included in your benchmark.

About stating overclock and model. To take the 1080ti for example some people run them at 1500-1600mhz, while they with good aircooling could run at between 2000-2100mhz without to much problem at all. I mean the difference on rendering time here is almost 35% from the same GPU.

Sorry for OT, but do really those PLX chips on WS motherboards really matter? Isn't it the CPUs PCI lanes that matter? Sounds a bit strange with infinite PCI lanes from a motherboard when theres the CPU that do the processing with a i9 44 PCI lanes.
Edit, for less confusion, PCI-lanes only affect transfering times to GPU what I know of, which should only be in interested for large scenes. I know that the PCI lanes doesn't affect the rendering on GPU after the scene is transferred to GPU.

But yes first check the Benchmark program so it scales well is my advice.

[JezUK]

I just purchased an x299 Sage by Asus - and will be building over the next few weeks.

For me, that one seemed the best because of the 7 pci slots.

I've already discussed the Benchmark issue on here with someone from Chaos - the benchmark test doesn't reflect that indeed the linear speed increase expected for more GPU's. Apparently a better benchtest will solve this.

So, yes, my intention is to go for 7 gpus (in both my machines) and across the network.

The *main* problem right now however, isn't any of that, it's sourcing GTX 1080 Ti's - NVidia is constantly sold out and I refuse to pay 1.5 times for a card due to this Bitmining craze etc....

[allemyr]

I think the Nvidia Titan Xp is a great choice now when a 1080ti Founders Edition doesn't cost 700 euros anymore, here in Sweden the Xp costs 1300 euros now and with its 3840 cuda cores compared to 1080ti 3585 cuda cores its much faster. Pascal based Titan X is more like a 1080ti, the Titan Xp is something else, much better.

Have to say that I think that Nvidia Titan Xp is a underrated GPU for rendering. I think that the memory handling is better on all the Titan X or Xp cards then 1080ti card, the differnce is in my experience, 2gb on large scenes, not the 1gb that specs says.

Nvidia have the Xp in stock which is also good. Max 2 per customer ofcourse, but has to be a way around that.

[JezUK]

I think the Nvidia Titan Xp is a great choice now when a 1080ti Founders Edition doesn't cost 700 euros anymore, here in Sweden the Xp costs 1300 euros now and with its 3840 cuda cores compared to 1080ti 3585 cuda cores its much faster. Pascal based Titan X is more like a 1080ti, the Titan Xp is something else, much better.

Have to say that I think that Nvidia Titan Xp is a underrated GPU for rendering. I think that the memory handling is better on all the Titan X or Xp cards then 1080ti card, the differnce is in my experience, 2gb on large scenes, not the 1gb that specs says.

Nvidia have the Xp in stock which is also good. Max 2 per customer ofcourse, but has to be a way around that.

Yes, I've been tempted to go for the Xp's because my new build is currently just parts with no GPU to speak of - I watch NVidia's site several times a day, but have now given up - for my budget, if I go the Xp route, I'll be effectively paying double the price over the price I paid for my 1080ti's (and is still advertised) via their site at that price. And for 4 cards (which I may even go to 7 one day), that's a lot of money to try to find from somewhere....

But it's now become a moral thing for me, I'm simply not prepared to line their pockets with excessive cash (so I'll take one or maybe two out from my other machine until this madness subsides).

Also, I notice that the Xp's do appear on their sites as available and then go, but not for long...... the 1080ti's however never appear as available...... I've been watching now around 2 months and feel people are jumping to the Xp in frustration (which may of course be exactly what NVidia want).

[Ivan1982]

Hi guys, since i have pretty much lot's of experience regarding this matter i will try to share my two cents.

Today i sold nine 1080, probably will dump another 30(thirty) 1080ti in following days/weeks. Right now as you guys know prices are insane, money you need for GPUs in today's market makes you think that reverting on CPU is not such bad idea as i would easily scale up with GPU if needed.

Just to put things in perspective and reasoning for this decision.

9x 1080(only GPU) is around 6K second hand(that is what i got and what is worse people start bidding and outbidding themselves) + you need another 3k to make 2 decent machines that can realistically fit only 8, but let's pretend they can fit 9 gpu's.

That is more than 9k, if you are buying new than it's even more and it's impossible to obtain such number of cards at once. What else can you buy for that money?

AMD TR 1950x that is no slouch, 5 of them cost around 5k, that is more than 15k in cinebench with some moderate OC you can push even 17k CB.
Asus prime x399-a MB around 1.3k for five of them
32gb x 5 around 1.6k
250gb ssd x 5 around 400
PSU around 500 for five of them

Total of 8.8k for components and you are left with 1.2k to buy cases, coolers...

Since i know what can 15k-17k cb do with vray advanced and what 9 x 1080 or 4x titanXp or 5x 1080ti can do... it's not even close.

GPU was very viable option 2 years ago, now with that ridiculous rule of GPU purchase limit combined with sky high prices that dream started to turn into small nightmare.

Switching to GPU is way harder then switching to CPU, you realize that soon enough, however there is no gain anymore that will give you will to do such thing.

This is just how i see it, yes there are rendering node lic costs, software, etc. but 5x 1950x are more than double the speed of 9x 1080.

Until some day when gpu becomes reasonable option again i am out as JezUK nicely said "I'm simply not prepared to line their pockets with excessive cash"

[JezUK]

What else can you buy for that money?

AMD TR 1950x that is no slouch, 5 of them cost around 5k, that is more than 15k in cinebench with some moderate OC you can push even 17k CB.
Asus prime x399-a MB around 1.3k for five of them
32gb x 5 around 1.6k
250gb ssd x 5 around 400
PSU around 500 for five of them

Total of 8.8k for components and you are left with 1.2k to buy cases, coolers...

And how much is left to buy the 5 x GPUs

[Ivan1982]

None, as you do not need to buy gpus

[delineator]

None, as you do not need to buy gpus

well, you need SOMETHING, unless your running true headless on linux....

appreciate the input though, thats definitely an interesting observation. I think that for most studios for mixed work loads, its both gpu and cpu and not dropping 100% of your money into just 1 bucket. gpu is great for denoising or quick hybrid scenes or just a quick RT for various reasons. I have somewhat of a mixed farm and I appreciate some have some 980ti's, some are just intel cpus, and a few have some 1050ti's. And I can cater my output to their strengths usually, though yes, cpu horsepower is the great commonality between them.

Let's check our numbers though and compare MSRP to MSRP - according to the benchmark, 1 1950x is equal to 1 Titan Xp (which in turn is roughly equal to 1 1080ti). Of course MSRP (which yes, can still be had) on a titan xp is $1200 and price of a 1950x machine....well, is much higher (maybe $3k for a built out machine, depending on ram). You can have upwards of 4x titans in a 1950x based machine, all fall under 1 license (for CG and most likely others, sometimes not). So I'm curious how you land at "Since i know what can 15k-17k cb do with vray advanced and what 9 x 1080 or 4x titanXp or 5x 1080ti can do... it's not even close." and if you had any numbers that brought you to that conclusion? Based on benchmarks, I just don't see it

This is a great resource where they actually show numbers in hybrid mode, and when you combine gpu and cpu, the results are about half of just the cpu alone or gpu alone.
https://www.pugetsystems.com/labs/ar...Titan-Xp-1062/

[Ivan1982]

Ok, are you for real?