Mobo for multiple GPUs

[super gnu]

yeah i think that if you were speccing a system, youd go for a dual one just to make sure 4+ cards will work fine, since its hard to find anyone using more than 4 cards on a consumer board ( since games dont work with 4+) so youd be taking a bit of a gamble on it working.

there may well be single cpu boards out there that will work fine with more than 4 gpus, and as i understand it, you only get issues in windows with the 8th card added, and it can apparently be fixed by a registry mod:

https://devtalk.nvidia.com/default/t...08565/#4208565


this motherboard (random search result) has 7 pcie slots for example:

http://www.newegg.com/Product/Produc...82E16813157538

[super gnu]

this one would likely work with 7x gpus, but you would need alternate ones on risers if they were dual slot. custom case needed obviously.

http://www.asus.com/Motherboards/P9X79E_WS/

stands to reason if it has 7 pcie 16x 3.0 slots ( 8x pcie if all are populated) then it should have no problem with 7 pcie devices in bios.

since all the windows issues ive seen seem to revolve around using 8+ cards, there should be no issue with windows either.

ive previously had a chat with nvidia, and their help guy ( no idea of the competence) stated there was no hard limit in their drivers, but 8+cards are "unsupported"


oh. and you would need 2kw+ of psu. .either one enormous, or more realistically, a couple of smaller ones.


if i was speccing out a machine to do this, id likely go for a dual cpu machine anyway.. more headroom for ram upgrades, and the extra cost is minimal compared to the 7x gpus..

[auke]

Thanks for the links and info! And I agree that it would probably be more trouble free to build a custom with 4+ GPU's on a multi-CPU serverboard than on a normal workstation grade single socket board. I'm currently looking at a X99e-WS based station with 4 GPU's as it will be fine for our needs, though learning more about the subject is always a good thing.

[super gnu]

yep.. ive just got my first titan x. but ive built all my own systems so far, and its quite good fun to plan out £20k workstations even if its unlikely youll build them any time soon ( well if that one big job comes along eh?)

[cray-zard]

@super gnu if your just working with interior projects is it okey to buy 980ti instead of the titan x? im about to buy 2 980 ti this week,

[super gnu]

all depends on your ram requirements. performance is similar. knowing how much ram i typically use in a project, and knowing that with rt gpu, proxies dont provide any ram saving benefit (they are all loaded in full to the gpu) i wasnt even considering gpu rendering till the TX came out. ive still had to abandon my first attempt at doing a job on it, since im maxing out the 32gb in my machine.

(aged warehouse modelled entirely with actual bricks, all with procedural displacement)


having said that some people find it easy to keep their jobs within 6gb, so you might be fine!

[3LP]

Having worked with GPU on GTX970 with 4Gb and with few Titan X now I can confirm I wouldn't by the 6Gb cards.
I was also tempted to buy the 980Ti but when I see how you just down't care anymore when using Titan X, it's really worth it.
I use 8k maps without even thinking of it, and I slap high ress geo and maps everywhere with FPP and railclone finger in the nose

Stan

[auke]

I agree as well, I ran some tests with past projects we did (fairly large architectural projects with lots of proxies and such). Usually the memory use is nowhere near 12 gb, typical use is between 2 and 4. Though every now and then you have a scene which goes over 6 anyway, and not needing to worry about it saves alot of hassle. Depending on your projects the 6gb 980ti might be enough though, I would run some tests with your past projects to see what you typically need.

[super gnu]

But did you not find that scenes with lots of proxies/displacement etc. use more ram on your gpu using RT, that on cpu when using normal vray? im not 100% clear on this, as ive not used RT much yet in production, but as i understand it, normal vray gets massive memory usage benefits from proxies and also displacement (when bucket rendering) since it only needs to load/calculate the dynamic geometry for the bucket its on.

with RT gpu the whole scene, proxies, displaced meshes, fur, etc. etc. must fit in the gpu ram in their entirety. there is no "dynamic geometry" as such (beyond plain instancing) . somebody please correct me if im wrong. id like to be.

[auke]

I always thought the same as well but I am not seeing this in the actual Vram usage. I can easily add alot of trees without the Vram usage going up much, no idea how this works though as I thought GPU's have to load in the full geometry instead of just references to proxies as well. This is something which I see in both Vray RT as in Iray. Maybe I'm just not throwing in enough proxies although I doubt that . I'm not sure about displacement, as I'm having trouble getting it to work correctly. Scenes with more than a useless tiny amount of displacement do not even survive the translation phase as Vray RT swamps my memory to 100% and then goes to a crawl/standstill. The only situations in which I went over 6 gb was with alot of very large textures and non optimized scenes (One scene of a large office building was filled with high-poly furniture, with most not using proxies or instances).

[cray-zard]

thanks for the answer, i tried opening some of my past project and rendering it in vray rt production my memory usage is just under 1.5 to 2 gb, ill be buying 2 zotac 980 ti ref this week, is it possible just incase (hopefully not) i needed more gpu memory, buy 2 titan x and use the 2 980 ti for distributed rendering? will the slave gpu (980 ti's) render the scene if your scene is above 8gb?

[super gnu]

no, if one of your gpu's has insufficient ram, the whole job will fall over in my experience. rt gpu is sometimes still a bit rough round the edges like that.

[auke]

Correct, the GPU will render if the scene will fit on your Vram and it will simply not render if it doesn't, even if you go over it just 1 kb. Good thing today though is that graphics cards seem to hold their value for much longer than they used to, so if you would decide to switch to Titan X's (or their successor) in a year or so, you will most likely still get a decent price of off those 980ti's.

[cray-zard]

just received my first 980 ti, i thought that from 970 to 980 ti the render time will be twice as fast, i think i just gained 30% speed coming from 970 to 980ti.

[super gnu]

why did you think going from 970 to 980 would be twice as fast? a titan x is only twice as fast as a 670!