Need clarification on using mixed GPU architectures with V-Ray

[Booska]

After looking at the benchmark page ( https://docs.google.com/spreadsheets...A0k/edit#gid=0 ), I notice that it's possible to use Maxwell based GTX cards with Kepler based Quadro cards. I just picked up two snazzy EVGA 980 Tis for a new system I'm putting together, and I also have a Quadro K5000 from the system I'm using now. I'm wondering how these cards should be set up to work with V-Ray? Is there anything special that I need to know?

I've heard that all the GPUs must be running the same driver, so does that mean these two kinds of cards can use the same driver? Do both of them normally use the same driver, or is a special setup required to make one of them run a driver that it normally wouldn't in order to be compatible with the other?

Since the K5000 runs the Max and Maya viewports very nicely, I'm thinking that I would like to use it to run my monitors and then use the 980 Tis for rendering. Does this sound like a good way to go? And lastly, can I expect much of a performance increase by adding the K5000 to the already speedy 980 Tis?

Thanks!

[savage309]

After looking at the benchmark page ( https://docs.google.com/spreadsheets...A0k/edit#gid=0 ), I notice that it's possible to use Maxwell based GTX cards with Kepler based Quadro cards. I just picked up two snazzy EVGA 980 Tis for a new system I'm putting together, and I also have a Quadro K5000 from the system I'm using now. I'm wondering how these cards should be set up to work with V-Ray? Is there anything special that I need to know?

I've heard that all the GPUs must be running the same driver, so does that mean these two kinds of cards can use the same driver? Do both of them normally use the same driver, or is a special setup required to make one of them run a driver that it normally wouldn't in order to be compatible with the other?

Since the K5000 runs the Max and Maya viewports very nicely, I'm thinking that I would like to use it to run my monitors and then use the 980 Tis for rendering. Does this sound like a good way to go? And lastly, can I expect much of a performance increase by adding the K5000 to the already speedy 980 Tis?

Thanks!

You have to put a driver that can recognise both the GPUs. Usually what I do is download latest & greatest from the nVidia web page, check if the OS can see both of them and if so - everything is okay. If it is not the case - I start trying different ones until the OS manages to find them. For me it usually works straight from the first try (almost every time).

To use the K5000 for viewport - just attach to monitor to it (I also prefer to put closer to the CPU the one with the monitor attached, but I don't know if this matters). Using the tool named ocldeviceselect.exe (comes with V-Ray installation) you can check what kind of GPU devices the OS can see, and you can choose which of them to be used by V-Ray RT for rendering (you have to restart Max/Maya/etc when doing changes from the tool). So if you want to use only the 980ti, just uncheck the Quadro and click "okay" (or whatever the button is named).

V-Ray scales linearly. Meaning that it will get as faster, as the the Quadro K5000 is. Choosing which GPUs to render is quite easy (just use ocldeviceselect), so you can test to see what works best for you.

Keep in mind that when you use GPU for rendering and monitor, the UI of the OS might become sluggish. You can fix that be reducing "Ray Bundle Size" and "Rays Per Pixel" from the RT Settings Tab until it is okay. More hints for RT here: http://docs.chaosgroup.com/display/VRAY3/GPU+Rendering#

[Booska]

Thanks again, Blago!

I was also just told not to connect the two 980 Tis via SLI because it can cause problems with RT.

I guess V-Ray can handle different architectures just fine, but have you noticed this config causing problems for any other software or with system functions? The system I got will supposedly play games really well, but I don't care about that since I don't play them. Playing with V-Ray and Max is plenty fun enough.

[savage309]

Thanks again, Blago!

I was also just told not to connect the two 980 Tis via SLI because it can cause problems with RT.

I guess V-Ray can handle different architectures just fine, but have you noticed this config causing problems for any other software or with system functions? The system I got will supposedly play games really well, but I don't care about that since I don't play them. Playing with V-Ray and Max is plenty fun enough.

Yes, I forgot. Don't use SLI and leave the multi-gpu support to the RT engine.
Mixture of different architectures works just fine for V-Ray. When you have your monitor plugged in one of the GPUs and SLI off, most of the apps use only it (like Windows, Max, Maya, Games), so I haven't noticed any problems with that, too.

[ThomasHorsley]

Thanks for that useful info - I have a related question:

I am using a single Fermi card for viewports & some render tests using Vray RT - Quadro 4000 with Max 2014 Design and Vray 3.20.03

Just bought a GTX 980Ti and was wondering about driver installation.

Shall I just update to the latest Quadro driver, plug in the GTX card into another PCIe slot a then use the ocldeviceselect.exe tool to unselect the Quadro?

I am wondering about the support for Fermi + Maxwell using Max 2014 - I am on subscription so could install Max 2015 or 2016 but that's always a can of worms so have resisted until now.

Anyone else using this combination? - If so, which driver are you using?

Many thanks, Thomas

[savage309]

I am not sure I quite understand the question well.
What you have to do is to choose which GPU you want to use for display and plug the monitor in it (probably on the PCI slot that is closes to the CPU, just in case). If you want to use that GPU only for monitor/viewport you have just to deselect it in ocldeviceselect.exe.

V-Ray can handle different architectures just fine, but I can't give you any advice about the driver and/or Max. Usually I am using the latest GTX driver and it works fine with my Titan X & 660ti.

In my PC I have a Fury X & Titan X. The first used for monitor/viewport/OpenCL and the second for CUDA.
It works great (even with 3 devices using V-Ray RT OpenCL - Intel CPU, AMD GPU & nVidia GPU).

[ThomasHorsley]

Thanks for your quick reply Blago.

I think you have pretty well understood my question -

I want to use the Quadro 4000 for viewport only (already am with monitor plugged in), and use the new 980 Ti for number crunching vray RT only....then maybe add more 980Ti s in future depending on what luck I have.

The main question was really about Fermi + Maxwell working together nicely.

Think I will just have a go at installing the 980 Ti, leaving everything as is (other than deselecting the Quadro in ocldeviceselect.exe.), and see if it works!

Thanks again, Thomas

[ThomasHorsley]

Quick update for anyone who's interested.

I installed the 980 Ti today alongside the Quadro 4000 without too much trouble:

Latest Quadro driver (353.82) didn't work with both cards installed - only the Quadro card was visible to windows.

I am now using the latest Geforce driver ver 355.82 - Did a 'clean install' option

Quadro is plugged into the monitor, 980Ti does all the cuda work.

Finally, I turned cuda OFF on the Quadro using 'NVidia Control Panel' as the vray ocl device select would only allow all gpus to be selected. (Think that has to do with environment variable but didnt want to mess with that).

It works

Marvelous

[chriserskine]

do you find that the quadro works just as fast with the geforce drivers as it did with the quadro drivers?

or dose non of that really matter anymore now that 3dsmax is running nitrous

[auke]

Working on a similar configuration, quadro K4000 as graphics driver and a Titan X as Cuda device. Both running on the latest geforce driver. The quadro seems just as fast as before although it does give some occasional issues. These issues manifest themselves as blurry/checkered viewports which last a couple of seconds and BSOD's in autocad, the viewport issues are hardly a problem and the BSOD's are so rare I'm not that concerned about them as long as you save often (which you should anyway )

[ThomasHorsley]

My setup - quadro 4000 with 980 ti is actually my second pc which is currently being using for rendering only.
I use teamviewer to connect to it and havent had any problems yet at all.
I am planning to move my 3ds max licence over to the second pc soon - just wanted to test the setup before doing so as my licence is standalone so not as flexible a setup as network licence.

In case youre interested, its pretty fast when using active shade from my main pc DR to this second pc using vray rt cuda
Ill be interested to see the difference (how much faster it is) when I dont render dr over the network.

[eyepiz]

Working on a similar configuration, quadro K4000 as graphics driver and a Titan X as Cuda device. Both running on the latest geforce driver. The quadro seems just as fast as before although it does give some occasional issues. These issues manifest themselves as blurry/checkered viewports which last a couple of seconds and BSOD's in autocad, the viewport issues are hardly a problem and the BSOD's are so rare I'm not that concerned about them as long as you save often (which you should anyway )

So the Quadro is running on GeForce Drivers? Wouldn't it would defeat the propose of the Quadro?

[auke]

Yes, in a new built it probably would, but I already was using this setup and added the Titan later for experimental use mostly. If we will decide to go for gpu rendering the Titan X will go out again and we will setup a dedicated station with multiple GPU's as a slave for our 3ds Max workstations. Please understand though that the Quadro K4000 does sporadically have issues when running on GTX drivers. I've seen some people of forums running Quadro's and GTX's together fine but I have no idea about specific models and drivers.

[ThomasHorsley]

Yes, this setup (for me) is a temporary stepping stone upgrade to allow testing of the workflow using VRay RT on the GPU.
At some point I will probably sell the Quadros and buy another GTX card.