What value for GPU texture resize have you tried? There must be a max value which will help you overcome the CUDA error 2, you simply need to experiement a few times until you to find it.
The NVidia settings do not really affect V-Ray GPU. Just make sure you’ve set your device to max performance.
Regarding the render performance of different GPU models, as already mentioned, we can only advise result comparisons of the benchmark results at V-Ray Benchmark – test your CPU/GPU rendering power | Chaos