Help with AntiAliasing geometry

[tioem]

I enabled Sub-pixel mapping + clamping and tested different clamp values but the issue didn't go away.

Thanks for the image 1funk! Octane and the native render of Modo really seems to be doing a better job there.

-t

[1funk]

I don't suggest using subpixel mapping + clamping anyway, but if you do, be aware it's not compatible with adaptive lights.

You have to change "light evaluation mode" on the render item.

[tioem]

Yeah I will probably not be using subpixel+clamping since it didn't solve the issue. I'm going to continue investigating and use the native renderer meanwhile.

[vladimir.nedev]

You can try increasing the size of the AA filter, you have it set to 1.8, which is lower than the default 2.0. Try 2.5 and see if it looks ok.

Also note that if you zoom out of the image (i.e. make the image smaller in the VFB), there is no filtering applied, so the aliasing effect will look worse than it actually is.

Greetings,
Vladimir Nedev

[1funk]

Thanks Vladimir. I actually reset tiom's settings to defaults in my last test, and also tried different filter types. BUT I didnt try playing with the filter size!

You're right. Setting it to 2.5 (or 3) does look much better. I'd have to try this on another scene just to see how much it blurs other details though.

Here is progressive sampler with filter size = 2, 2.5 and 3 (everything else reset to defaults)

[^Lele^]

Or you could simply switch to a gaussian 2px to (likely) match the other render engines.
One thing i noticed in the comparison, though, is that the aliased edge in V-Ray isn't nearly as intense as in the other two, which could pose a problem (that low grey you see is under sRGB gamma, its true value is likely very close to 0).
Choice of filter ofc will play a potentially big role in shaping sub-pixel detail, and Lanczos, as we have it implemented without the negative component, is known to retain sharp edges (all too) well (hence the need for a broader filter size: the graph remains peaky, so with 2px it's considering one pixel and a wee bit, in reality.).

Edit: i attached a wiki image of a gaussian filter curve ("pixel coordinates" for us, on the horizontal axis: the pixel we are filtering crossing the vertical axis, and the nearby on the ticks.), and two examples of the *full* Lanczos kernel shape (we don't do the negative bit).
Notice how different they are from 0 to +-2.
Notice, further, that at size 3px, the positive part of Lanczos (if interpolated removing the negative bit between 1 and 2) encompasses entirely the two adjacent pixels, providing for a (much) smoother weighted average than 2px could (and likely ending up looking much like a ~2px Gaussian).

[1funk]

^Lele^ yeah I did try Gaussian 2.0 on tioem's scene and it's closer to the others. It still looks a bit jaggy for my tastes. A slight increase to 2.25 seems to help though.

[tioem]

Hey. After hours of experimenting without any success I decided to try out the GPU renderer. The images are both rendered with exactly the same settings but the GPU gives much better AA. Is this some kind of a bug on the CPU side?

[vladimir.nedev]

Can you share this second scene ?

Greetings,
Vladimir Nedev

[tioem]

The second test scene can be downloaded here:
https://www.dropbox.com/s/zpgb5gfyju...asing.zip?dl=0

-t

[1funk]

tioem I was hoping someone would step in with a solution. I tried your new scene and no matter what I did, CPU always renders jaggy AA.

[tioem]

I gave up as well

The only trick that seems to work is to start rendering with GPU RT and head to V-ray preferences and activate the CPU under CUDA devices. This has worked fine so far with product rendering scenes but it would be nice have the option to rely on CPU in case of compatibility issues.

[vladimir.nedev]

I gave up as well

The only trick that seems to work is to start rendering with GPU RT and head to V-ray preferences and activate the CPU under CUDA devices. This has worked fine so far with product rendering scenes but it would be nice have the option to rely on CPU in case of compatibility issues.

Oh, we actually have this one logged.

It's only RT CPU that's working incorrectly, if you use Production Adaptive or Production Progressive, you will get the same result as RT CUDA.
So, you can use the CPU renderer and have the good AA.

Now, if you want RT CPU to match the other modes, you will have to set the CPU Rays Per Pixel to 1.
This will work around the performance optimization that RT CPU is doing and that leads to the incorrect AA.
We have this logged but it's not an easy fix since it affects DR performance.

Greetings,
Vladimir Nedev

[1funk]

Thanks Vladimir. You're right. It looks like we were both testing this in RT CPU. I thought I tried a production render, but obviously not.