Vray RT for maya Nvidia 32G Memory card, fp64, fp32 or fp16

[MarceloViana]

I found this answer on a 2014 post:

Originally Posted by abowen does anyone know if double precision (FP64) memory makes a noticeable difference on the titan black ? not sure its worth the extra coin verses getting an extra card maybe to add to your system.

Double-precision calculations make no difference to V-Ray RT (and most other GPU renderers out there).
Best regards,
Vlado

nvidia has this Quadro GP100 that have 5.2Tflops FP64, 10.3Tflops FP32 and 20.7 Tflops FP16
1) So i'm wondering, does Vray-RT use FP64, FP32 or FP16?

Also about memory, Quadro GP100 have only 16G hbm2.
I know it's really expensive at $9,000 where you can have a 24G, 12Tflops fp32 P6000 at $6,000. But here is the thing: It has a nvlink able to connect 2 cards. while pci is able to transfer up to 32G/s the nvlink run up to 80G/s, also Nvidia state that with 2 cards on this link a software can see not 16G x 2 but a single 32G card.
2) My question is: Vray-RT is one of this softwares? If so means one can load a 32G scene right?

Thanks to anyone that can answer.

[savage309]

I found this answer on a 2014 post:

Originally Posted by abowen does anyone know if double precision (FP64) memory makes a noticeable difference on the titan black ? not sure its worth the extra coin verses getting an extra card maybe to add to your system.

Double-precision calculations make no difference to V-Ray RT (and most other GPU renderers out there).
Best regards,
Vlado

nvidia has this Quadro GP100 that have 5.2Tflops FP64, 10.3Tflops FP32 and 20.7 Tflops FP16
1) So i'm wondering, does Vray-RT use FP64, FP32 or FP16?

Also about memory, Quadro GP100 have only 16G hbm2.
I know it's really expensive at $9,000 where you can have a 24G, 12Tflops fp32 P6000 at $6,000. But here is the thing: It has a nvlink able to connect 2 cards. while pci is able to transfer up to 32G/s the nvlink run up to 80G/s, also Nvidia state that with 2 cards on this link a software can see not 16G x 2 but a single 32G card.
2) My question is: Vray-RT is one of this softwares? If so means one can load a 32G scene right?

Thanks to anyone that can answer.

1. V-Ray GPU uses FP32 only. We plan to do some R&D on uses of FP16 for some calculations (but only for small parts of the code - no way to make everything FP16) when the FP16 becomes more massively adopted.
2. Yes, V-Ray GPU is the first comercial product outside NVIDIA that supports NVLINK. For that you have to use the nightlies. Also, we still make copies of data that is not very large in size but used often on each GPU. We do that to reduce the NVLINK trafic. So usable memory will be below 32GB.
Also, GP100 is about 25 to 65 percent faster than P6000 with V-Ray GPU. On paper it should be slower, but it is different architecture and because of that comapring just TFLOPS doesn't tell the whole story.

Btw, you might find this useful.

Best,
Blago.

[MarceloViana]

1. V-Ray GPU uses FP32 only. We plan to do some R&D on uses of FP16 for some calculations (but only for small parts of the code - no way to make everything FP16) when the FP16 becomes more massively adopted.

That's ok since you already let us know that this thing is even faster than P6000.

Also, we still make copies of data that is not very large in size but used often on each GPU. We do that to reduce the NVLINK trafic. So usable memory will be below 32GB.

Well, do you think that this "not very large in size but used often on each GPU" overcome 2G? because i'll be pretty happy with 30G for my scene(15G on each card).

Also Blago, since you guys already working on this nvlink, do you think that is possible to build a machine with 8 GP100 or better saying 4 nvlinked GP100s? Just like a nvidia VCA. Will vray-rt support it? I mean will be 4 cluster of cards in one machine, i don't know if you see that way, maybe it isn't a problem at all(one can hope).

Anyway thank you very much for the answer, and the link.

[savage309]

That's ok since you already let us know that this thing is even faster than P6000.



Well, do you think that this "not very large in size but used often on each GPU" overcome 2G? because i'll be pretty happy with 30G for my scene(15G on each card).

Also Blago, since you guys already working on this nvlink, do you think that is possible to build a machine with 8 GP100 or better saying 4 nvlinked GP100s? Just like a nvidia VCA. Will vray-rt support it? I mean will be 4 cluster of cards in one machine, i don't know if you see that way, maybe it isn't a problem at all(one can hope).

Anyway thank you very much for the answer, and the link.

How much less than 32GB is scene specific and although we have tested the NVLINK I can't tell exactly how much less.
We can also controll precisely which memory we want to have copies on each GPU. At some point I guess too much memory sharing would result in slowing down the render (because NVLINK transfer will become a bottleneck). I don't really know since we haven't tested that much, but the good thing is that we have controll on which memory gets allocated where, so if it gets too slow, or if it needs the opposite - sharing more stuff between the GPUs, it is quite easy to do. Answer to those questions will come as more people gets to use and test those.

GP100 allows having only 2 GPUs connected. There are special non-PCIe based GPUs in the NVIDIA DGX-1 that allow to connect 4 at full speed, and they also have 8 GPUs in them. This makes 2 x 4 GPUs connected. Those two groups of 4 GPUs have some (slower) NVLINK betwen them, so in theory you have 8 GPUs connected, but we haven't still tested DGX-1 to know if it works well with 8 connected, or with only 4 ... Also, DGX-1 is a bit more pricey.

Best,
Blago.

[MarceloViana]

Wow, your reply is very fast, thank you for that.
Yes DGX-1 is the way to go, but as you said $130,000? really? not for everyone. But i think you didn't get what i'm saying. I'm talk about... imagine a nvidia VCA with 8 GP100, gpus 1 and 2 are nvlinked, gpus 3 and 4 are also nvlinked, as 5 and 6 and finally 7 and 8. the 8 gpus are not nvlinked like DGX-1, but one can say(if assume that the cards that are actually linked is one card) that this VCA has 4 32G cards.
So i know that Vray already run great managing the memory on this one 32G card(2 GP100). the thing is: will vray work adding 3 more 32G cards(6 GP100) on the same machine?
I know is earlly to ask this questions, i don't expect you already have done all the tests, just want to know if you are work in this direction, to allow any group of 2(nvlinked) GP100 on the same machine.

[savage309]

will vray work adding 3 more 32G cards(6 GP100) on the same machine?

As far as V-Ray is concerned, I don't think there should be any troubles.

Best,
Blago.

[MarceloViana]

very good.
You make my day. thank you Blago very informative, and made me very exited about this tech.
keep good work guys.