The Device Kernel Took Too Long to Execute

[Joelaff]

I keep getting these errors on a render node that has a 1080Ti. I tried the latest drivers. I tried the recommended driver. Same thing. The scene is pretty heavy, and does use a lot of memory, but it is rendering fine on lesser hardware; so not really sure what the issue is.

Any tips are how to troubleshoot this further? I am render CUDA Hybrid - Max 2021.1 hotfix1. Guess I should bump to hotfix 2 as a start.

[a7az0th]

Seems like a bug. You hit some piece of code that for some reason is slow to execute. Windows has an embedded "heartbeat" that monitors that the GPU is not stuck and if it decides it is stuck (because it did not respond in time) Windows will kill it and restart it.
This is when you get the "Device Kernel took too long" error.
If possible I would like to take a look at this scene so I can see what's taking so long and fix it

[Joelaff]

OK. Can you use a scene with PRTLoader (Krakatoa) and Frost? Or could I export it to some static format? This scene has a lot of particles that are meshed with Frost. This actually came out faster than using the PHX ParticleShader.

Note that this scene renders fine on other, even lesser (980 or even lower) hardware.

[a7az0th]

I am sure we'll manage. If you have it as a .vrscene I can use that

[Joelaff]

I am sure we'll manage. If you have it as a .vrscene I can use that

OK. I may try the vrscene because otherwise we have to deal with PRT files and AUR files and everything. Would you still need those with a vrscene? I thought those were pretty self contained.

[a7az0th]

Not sure Maybe I will. Bitmaps/Proxies have to be provided separately as the VRscene file itself only contains references to them. I am not sure for AUR and PRT, but I would suspect the same

[Joelaff]

OK. I will try to send one frame of everything and leave it as a .max file, and we can see if that works for you guys. It might not be for a couple days, but it might be today. Have to see.

Thanks again.