Can you post the scene or a version of it that will reproduce the problem?
It would also help to know your VRay/Maya/OS versions + nvidia hardware and driver versions.
My specs:
OpenCL 1.2 CUDA 7.5.15.
Driver version: 10.18.13.5382
Quadro K4200 & Geforce GTX 660Ti
Vray Nightly build 26.5.2016
Win10 Pro
I will prepare the scene later the day. But does CUDA vray support vector displacement?
So i tried again with fresh scene and I come to same conclusion:
Check out the picture with comparison I made.
Oh, btw. I am using 26.5.2016 release of vray
Vector displacement is a supported feature on the GPU, however the GPU handles displacement a bit differently than the CPU.
At the moment the Bump map that is applied to the mesh after displacement that you can see when rendering on the CPU is not a supported feature on the GPU.
Also, the GPU requires normals to be calculated before rendering and cached (this is the Cache Expanded option so V-Ray will always consider it ON when rendering on the GPU).
This option relies on the max number of subdivisions to produce good results. You can see how the CPU and GPU results are quite similar if the Cache Expanded option is ticked.
So I will advise to crank the max number of subdivisions (which by default is set to 4) and see how this works for you.
Unfortunately I don’t think I can give you a better solution at the moment.
We will update the help pages to include this information as soon as possible.