Very rough edges when using the Render Mask feature

[Alex_M]

I've been trying to use the Render Mask feature but most of the time it produces nasty rough edges. Please check the image below to see what I mean. Notice the ugly edges around the blue hull of the boat. In this image I've used Adaptive sampler, antialiasing filter VrayLanczosFilter with size of 2.0 and BF+LC for GI. Is there something I can do for the edges to appear smooth?

[vlado]

Can you get me some way to reproduce this? A part of the scene will be helpful. This generally works fine on simpler scenes, but maybe there is something going on.

Best regards,
Vlado

[Alex_M]

Sure. Please let me know where I can upload the file securely and I'll send the archived scene.

[vlado]

Sure. Please let me know where I can upload the file securely and I'll send the archived scene.

If you email our support guys to support@chaosgroup.com they will provide you with FTP info where you can upload the archive.

Best regards,
Vlado

[Alex_M]

Actually, no need to send this particular scene. I managed to figure out the exact steps to reproduce the problem which can be tested even with a simple scene. Please download the attached scene and try the following:

1. Render the whole image with no Render Mask (don't adjust any settings)
2. Enable Render Mask with the option Selected
3. In the VFB enable Glare Effect from Lens Effects
4. Select the box and render again

The result is rough black outline around the object that appears only after the render finishes completely. So looks like this happens when Render Mask and Lens Effects are used in conjunction. I hope you can manage to reproduce it. If not, let me know and I will try to make a video of the steps.

[Zdravko.Keremidchiev]

We reproduced the issue and logged in our bug taking system. As soon as we get any update we will let you know.

[Alex_M]

I get the same problem again. I'm not using glow and lens flares but I still get weird fringing/edge around the objects that are rendered with Render Mask. Any way to avoid this or is it a bug? I'm using the progressive sampler and BF+BF. Tried bucket too but still the same.

[Svetlozar Draganov]

Any chance to send us that scene as well, it might be the same issue but it might be something else?

[Alex_M]

Sure, I just sent you a stripped down version at support@chaosgroup.com Everything is set, you just need to make a small region render around the slatted ceiling. I've also provided a vrimg file to load into the VFB before making the region render so you have a point of reference.

Let me know if you need anything from me or if any questions. Thanks for looking into it!

[Alex_M]

Not trying to push you guys, I know you've got a lot of work around the launch of the new SP but I just wanted to know if there is any progress made on fixing the problem with using Render Mask with Lens Effects? You know, the black outlines that appear around the selected objects as seen in my first post in the thread. Is there a workaround that maybe I can use temporarily?

Thanks in advance!

[Zdravko.Keremidchiev]

The developers are currently working on it. It should be fixed soon.

[Alex_M]

Neat!
....

[tashko.zashev]

Hi Alex,

We managed to fix this bug for the next SP release.

[Alex_M]

Thanks so much for looking into this and fixing it!

[mitviz]

seems like this was not fixed