Vray Live render crash with VrayDisplacementMod

[simona.b]

I get a crash using Vray 5 GPU hotfix 1 with 3ds Max 2021, when in live rendering mode and using VrayDisplacementMod - it happens sporadically but also when changing parameter Type (3D mapping to Subdivision, 2D mapping) etc.
It end's up freezing rendering window, whilst 3dsMax remains functional, so am able to save file but cannot render again until I force close Max.

Rendering on GTX 1080 X 2 (driver version 442.23)

Errors vary, usually unexpected handling one that forces render to stop and freeze but currently trying to reproduce error and getting "error 700" that GTX 1080 won't be used for rendering. "initDevices()failed"
Now even with displacement modifier turned off, I can't use "live" render mode, needs another Max restart.

Don't think this is to do with graphics card memory as geometry is relatively small (isolated currently) and I was rendering whole scene earlier ok...

[hermit.crab]

Does this happen on a specific scene? Could you send it to support@chaosgroup.com so we can take a look?

[simona.b]

I can't send the scene I'm afraid due to it being a military project. I tried to make it crash in the same way by creating lot's of teapots in a fresh scene with vrayDisplacementMod and random texture bump. If I switch between Isolate and Unisolate mode views of a teapot vs lots of other teapots and play around with "amount" of height and rotate around the scene, eventually it seems to crash or freeze up. I got "unhandled exception occurred, while applying changes to scene" right now, but earlier on a test it simply froze and after a while I managed to press stop live rendering and eventually it returned to normal.

The teapot scene is 1mill polys compared to 3 mil of my actual project.
I can prevent this from happening by simply not use "live" render mode excessively to play around with amount settings and try not to switch from isolate and unisolate modes, but stop in between then kickstart previewing again.
Maybe the scenes are just too heavy with data and GPU and vray together struggle a little bit but if I slow down previewing it's ok.

[hermit.crab]

From what you described, you may be running out of memory due to the excess amount of objects with displacement. Dynamic geometry can take sufficient amounts of RAM/VRAM and time to subdivide and compile. You may try monitoring your workstation's memory usage to see if that is indeed the case. To fasten things up, you may optimize the displacement (i.e. increasing the edge length, lower resolution textures, activating view-dependent).