Evaluating GPU Memory Size of Scene (What Numbers to Trust?)

[krysalis]

Hi All,

I am running some tests to determine if we might use Vray GPU for final renders and, if so, evaluate VRAM requirements for our use case. We currently only use Vray CPU for final renders.

Using a representative test scene, I get different numbers from live monitoring of stats vs. the memory tracking report. So I am left wondering which numbers to believe?

MSI Afterburner reports ~18/22GB per card, respectively, whilst Vray Stats in the Maya VFB shows ~14GB each (Image 1, below). That makes sense to me, as I can chalk up the discrepancy to other non-Vray loads (Maya, Windows...) that are contributing to the total.

Where I get lost is when I look at the Vray Memory Tracking Log (Image 2, below), which shows a Frame Size of ~30GB. Why the difference? The only logical leap I could make is if the log is adding up the totals of both cards combined? If so, that makes it less useful for assessing VRAM requirements, unless simply dividing by number of cards is considered sufficient? I'm not sure if I'm barking up the right tree with that explanation anyway...

This particular test I ran in RTX, but I get similar discrepancies if I run CUDA (just a bit less memory usage, as to be expected).

Also, is it weird that CUDA takes ~38 minutes per frame, while RTX does it in ~14 minutes? I didn't expect such a large delta...?

FYI, System Config:
AMD Ryzen Threadripper PRO 3996WX 64-Core
512GB RAM
2 x NVIDIA RTX A6000 48GB NVLink
Windows 10
Maya 2024

Thanks,

Tyler



IMAGE 1


IMAGE2

[vladimir_krastev]

Hi, thanks for posting.
The VFB stats shows how much memory is taken by the rendering process itself. The MSI afterburner shows all the memory taken by all the processes passing trough the GPU at the given moment.
If you are using it only for rendering the stats should be closer. Deleting the memory in the log by the number of GPUs is considered sufficient.

Also, is it weird that CUDA takes ~38 minutes per frame, while RTX does it in ~14 minutes? I didn't expect such a large delta...?

RTX video cards have different architecture and when you switch V-Ray to RTX mode it is expected to be much faster than CUDA.

You may also find interesting this thread and this article.

[krysalis]

Thanks for the clarification and links.

I do find it odd that I am getting faster times from Vray than Vray GPU on test frames. Any thoughts on that?

[Muhammed_Hamed]

Hi Tyler, and welcome to the forums,

Using a representative test scene, I get different numbers from live monitoring of stats vs. the memory tracking report. So I am left wondering which numbers to believe?

This is normal to see, in your case you should stick to what MSI AfterBurner reports
One of your GPUs is at 22 GB of VRAM and the other one is 18 GB of VRAM, I think the first GPU is where you connected your monitors so Maya(and other open apps) will take a chunk of VRAM from this GPU. And the other card is exclusively used for GPU rendering, so the scene is taking 18 GB of GPU memory to render in reality
It is possible to change which card is used by Maya viewport(and other OpenGL apps) see the last section in my post here
It should always be the card through which you connect your monitor(s)

So why is V-Ray reporting 14 GB of VRAM instead of 18 GB if this GPU is used exclusively for GPU rendering, the VFB stats could be inaccurate in some cases or that there are other instances of Maya or other apps running in the background. for the Quadro RTX A6000, you will be getting around 45 GB of usable GPU memory not 48 GB because of Windows
We have future plans on improving GPU memory reporting

Where I get lost is when I look at the Vray Memory Tracking Log (Image 2, below), which shows a Frame Size of ~30GB. Why the difference? The only logical leap I could make is if the log is adding up the totals of both cards combined?

We don't report on the NVlink combined memory anywhere, you only have a message in the log that V-Ray GPU recognized your NVlink setup
Keep in mind that some data needs to be shared between the 2 cards, so combining 2x A6000 you don't get exactly 96 GB of usable GPU memory, it will be slightly lower
This Frame total doesn't represent anything, I will investigate what it does

This particular test I ran in RTX, but I get similar discrepancies if I run CUDA (just a bit less memory usage, as to be expected)

This is expected to happen, RTX mode uses slightly more VRAM than CUDA mode
And Progressive mode uses slightly more VRAM than Bucket mode, around 3-4 GB

Also, is it weird that CUDA takes ~38 minutes per frame, while RTX does it in ~14 minutes? I didn't expect such a large delta...?

RTX mode uses dedicated Ray Tracing hardware to speed up rendering, your GPUs are equipped with the second generation RT cores. For some scenes it could be the case that RTX is a lot faster than CUDA mode(usually a scene that contains a lot of hair, fur, leaves, trees where big surfaces of geometry will intersect with your lighting)
That being said the gap is not usually as mush as you noticed, is there a chance to share your sample scene with us under NDA? we will be able to investigate and tell you more about his difference in render time between CUDA and RTX modes

One more note, is that V-Ray GPU will be slower with Render Elements you add. You need to use the least amount of Render Elements to be as fast as possible

Another note like what my colleague Vladimir linked, we support Out of Core Textures with the latest release of V-Ray for Maya. It reduces GPU memory usage drastically in case you don't have enough GPU memory for your scene.

Let us know if you have any questions, you can also Email me at muhammed.hamed@chaos.com

Best,
Muhammed

[krysalis]

Thanks for the help! I read all the links referenced. FWIW I did follow the instructions in the announcements for GPU on how to configure Windows and Nvidia Control Panel options for GPU rendering.

This test scene is indeed one with a lot of hair. It's a creature with full body of dense fur using hair shader. So from what you said, that seems to be why RTX is doing so much better.

I would be glad to share the scene provided there's an NDA. It is confidential work, but I could really use a few pointers as I'm trying to see if GPU rendering could possibly be useful to us moving forward, and this particular asset is one we'll be using a lot so is our best torture test. Right now, the most curious thing is CPU rendering is actually giving me faster results than GPU rendering, and looks a little better too. I feel like I must be missing something .

[Muhammed_Hamed]

Hello krysalis

I would love to take a look at your asset, we will investigate why RTX is so much faster than CUDA. I talked to the devs about this, 3x faster is not realistic in their view
We also have access to 64-core Threadrippers at the office, I will compare the performance and quality with the standard V-Ray renderer.
Please message me at muhammed.hamed@chaos.com to discuss sharing the assets, and any questions or help needed
Thanks!

Best,
Muhammed