V-Ray error: Failed to allocate...

[tobiasroesli]

Hi,

I got some Random Errors when Rendering with Deadline.

Its not with every Frame, sometimes it works for 3 or more Frames.
Its also with other Scenes, it doesn't matter how complex it is, already a simple scene returns errors

"V-Ray error: Failed to allocate 803208960 bytes, device 2, buffer default bufname" or "V-Ray error: Failed to allocate 787401216 bytes, device 3, buffer default bufname"


It is not specific for a GPU and its only for the RTX 2080TI Render Node.
My two other GPU Rendernodes with GTX 1080TI dont have this Problem of Crashing.

I'm using Vray Next 4.30.00 from Nov 22 2019
Nvidia Studio Driver: 441.28
Maya 2018.6

DEADLINE LOG:
https://we.tl/t-7tU4gO2j2V

[salvadoresz]

I have the same issue, i get similar errors on both rtx and non rtx pcs, what seems like after a random number of frames, but usually after 20-30 frames.

[tobiasroesli]

Nice to know, I only got them on my System with the RTX, the GTX's are fine. and I have it after 6-8 Frames

[Martin.Minev]

Hello,
We will try to reproduce the issue here and will get back to you when we have any news.

[NorthVisualStudio]

I have the same issue, an email has been sent to Martin.

[Martin.Minev]

Hi,

The following error is related to memory usage. The scene did not fit in the VRam of the specific machine.

tobiasroesli RTX mode and Deadline are working here as expected. Could you send us one of the scenes, that is causing the issue to support@chaosgroup.com, so we can investigate it. Does the issue occur only when using RTX mode?
We could also arrange a remote session if needed.
salvadoresz The issue here could be related to a specific animation and memory usage. Could you also send us a scene, so we can check more deeply? Please mention this thread in your email.
NorthVisualStudio Currently investigating your scene and will post an update.

[impact_ant]

I have had same errors on 2 pc's that have external gpu's connected. Martin already aware of my problems.

When it does crashes, always specif amount of frames, crashes on frame 50, restart 3ds and start from frame 50, crashes on 100, then 150, then 200. Vram does not run out, and I am using EVGA precision x/msi after burner to monitor vram usage, usually there 1GB+ remaining on the 1080's and 3GB on the 2080ti's. If I change the resolution, or texture setting, then it may crash on frame 20, 40, 60 , 80,100. Seems to be very consistent.

The workstations containing one internal gpu seem not to crash. If I uncheck the external gpu's on render settings on the pc's containing the external gpu's, unfortunately I still get frame crash after so many frames, but the number of frames before it crashes changes.

The current way I got around this problem on the workstations with external gpu machines is to use backburner and server, and when 3ds max crashes, backburner sends the job again. But the first frame to be rendered on the new backburner job might have a render glitch (lighting looks wrong or a texture missing) then the following frames appear fine.

The 2 external gpu boxes are different devices, one is a netstor na255a with multiple gpu's, the other is a razor core.

In my case, does look like external gpu issue, but error is far too consistent frame count wise to be running out of memory.

I have had this problem since October on 3ds max 2019 and 2020, windows is always up to date as well as nvidia drivers.

[damnengine]

Running into the same problem here but in 3dsmax. It's just a simple flag animation and every 200-300 frames it seems to crash. I have to restart max to get it going again. The system has 2 x RTX 2080ti's and 2x GTX1080 ti's. Already tried turning CPU off or only render on the RTX cards. It doesn't seem to matter whether I use CUDA or RTX.

[GregCooper]

This sounds like the same issue I’ve been having which I mentioned under the VRay for maya issues thread. The problem for me turned out to be the latest recommended nvidia driver. Rolling back to 419 has fixed the issue. Although I’m rendering with 1080tis so this might not be an option on RTX cards.

batch rendering would stop after a certain number of frames, sometimes 2-3 frames or sometimes up to 20. It all depended on the scene but led me to believe that maybe the memory wasn’t being fully cleared after each frame

[salvadoresz]

Yeah the problem is in 3ds max for me too, i'm monitoring resource usage (ram, vram etc) with all of my renders, and i always make sure i optimize the scenes to have plenty of resources left. The ram and vram usage goes up and down, not very noticeable if i only render a handful of frames, but depending on the scene, after 20-30-60 frames, i see that the ram usage sometimes doubles and finally crashes 3ds max. I also have to note that this happens with some of my VERY simple scenes too.

[pixelwerk]

Sounds like what is happening here to me as well on non-RTX cards, using MODO but rendering on GPU using the standalone

[salvadoresz]

Did anyone find a potential cause for this issue? I have 2 vray licenses, i've been rendering all night. One of my computers already crashed, the second one is about to crash because ram usage is getting close to the limit. When i started the scene, ram usage was about 14GB, now after 55 frames ram usage is 26GB.

[salvadoresz]

out of curiosity i went on octane pages, and it looks like some of their users have similar issues, someone mentioning that installing game drivers instead of studio drivers fixed it for them, so i will try that.

[GregCooper]

salvadoresz I mentioned above that rolling back to an earlier driver fixed it for me. I’m pretty sure it’s an nvidia driver issue. If you try the game driver let us know if you have any luck

[HowardDay]

Please let us know if this fixes it - I'm running into the same issue.