Anti-aliasing of masked render outputs doesn't work with new default sampling method

[janko22]

Hi, turning on "Anti-alias masked render outputs" doesn't work if adaptive method is set to Default. Legacy works fine.
You can easily check this if you create two adjacent materials and make them 0 defuse and both pure white in self-illumination. Create alpha outputs for both materials. The border between them will not be anti-aliased in alphas even with this option checked with Default sampling method. The same border is ok if you switch to Legacy method.

[janko22]

This still hasn't been fixed. Just tried it with nightly from 26.6.
Any word on this?

[vlado]

I still haven't figured out how to implement it...

Best regards,
Vlado

[janko22]

Ok, but please don't remove Legacy option untill it's fixed.

[vladimir.nedev]

This still hasn't been fixed. Just tried it with nightly from 26.6.
Any word on this?

I am not sure we should fix this option, I am actually thinking about removing it. It was a hack to begin with.
With the new default value for "max subdivs" set to 24, this hack can cause a significant slowdown of the render actually.

This kind of anti-aliasing problem can turn up in other matte elements, like the Surface ID, or the multi-matte or material ID, when I eventually add them to V-Ray for MODO.

You can easily render out the alpha masks, Surface ID, etc, in a separate render with a black material overriding all materials in the scene, Image Sampler set to Fixed and subdivs set to something like 3 or 4.
You can also increase the Min Subdivs of your normal render to something like 2 or 3, and this might actually have less impact on render times than checking the " Anti-aliasing of masked render outputs", although I haven't tested this, so not sure about this.

Greetings,
Vladimir Nedev

[vladimir.nedev]

Ok, but please don't remove Legacy option untill it's fixed.

Yes, this forum thread is the reason I haven't removed the "Legacy" option yet.

Can you check if rendering out the alpha masks separately with the Fixed sampler and a simple material works for you ?

Greetings,
Vladimir Nedev

[janko22]

Just checked, and it works fine with both Adaptive subdivision and also Fixed.
From my tests now Adaptive subdivision is a bit faster then Adaptive and much much faster then Fixed. This is all with rendering only material alphas (no GI, lights, reflections...).
One thing also is that for Adaptive subdivision you have to check "Anti-alias masked render outputs", but for Fixed it renders fine checked or unchecked (subdivs at 4).

Anyway, I guess you can remove the old Legacy method now, thanks for leaving it in for me

[janko22]

I think Fixed at 6 is closest to Adaptive at 1-24 for alpha edges

[janko22]

I tested this further and it looks like Adaptive 4-24 is the best way to go for alphas. That way main alpha matches perfectly other material alphas that are rendered later and there is no guessing for subdivs in Fixed method

[vladimir.nedev]

Using Adaptive at 4-24 with all black materials and "Anti-aliasing of masked render outputs" OFF, should basically be the same as using Fixed at 4 subdivs.
The adaptive image sampler won't see any differences between neighboring pixels (because all materials are black) and will take 4 subdivs everywhere, just like the Fixed sampler.

Ideally, we should find a better solution. Maybe render the matte elements (masked alphas, multimatte, material ID, surface ID, etc) in separate passes after the main rendering is done,
one pass for each such render element, where the adaptive image sampler is considering only the specified render element and taking enough samples to anti-alias it well.
Or maybe this can be merged with the main rendering, where only the matte elements will receive additional subdivs to those allocated for the main image.

These solutions will require quite a bit of changes to the code though, and since the work-around is not that complex, they will be low priority for now.

Greetings,
Vladimir Nedev