Speaking from experience, it’s super helpful to post up a demo scene that shows the problem. I’d be delighted to go through and see if I can replicate the result you’re seeing, and also to propose any work arounds that I might find. Be sure to send the scene to support@chaosgroup.com as well as posting it here.
This is an old problem, but I’m having the same issue with current version of 3ds Max (2021-2022) and V-Ray (5.1-5.2).
This is simple scene with only one HDRI+OSL map loaded in a dome light: https://www.dropbox.com/s/e4g9ijvp90…L_GPU.zip?dl=1
Rendering on CPU - no problems, rendering on GPU - after light cache is calculated 3ds max will crash and quit. I’m having the issue with any OSLMap or VRayOSLTex.
Both CPU and GPU scenes are uploaded int the file above.
It looks like there is a problem with the color values from VRayBitmap going to OSL map. If I use VRayCompTex (with only SourceA) in between the GPU render will not crash 3ds max.
This is the updated max scene: https://www.dropbox.com/s/t99j7eg40r…U_fix.zip?dl=1