I am using a variety of trees and plants that have opacity mapped leaves. I am getting a lot of anti-aliasing on the leaves which is creating problems when i want to composite backgrounds in. Can anyone recommend some render settings or changes to minimize this effect? For now i am using photoshop to clean it up but that seems like a ‘hack’. It seems that there could be some settings to help but I am not sure where to start.
Thanks a bunch!
How do the opacity maps look? Is there a nice even multi-pixel blend (number depending on the map resolution) in the transition between the white and black? If not, that may be causing your problem and you could do a little blurring in photoshop to smooth out the transition.
Also, I’ve seen folks try to use compressed images like jpg for intensity maps like this, and it is usually a bad idea. Make sure all intensity maps are pristine and uncompressed as compression artifacts will often cause all kinds of problems.
-Alan
So if I open a blank drawing and render a few of the same trees, there are no anti-alias issues. So i am not sure if because the trees are in the background that it creates antialias problems? Any tips other than adjusting each of the trees’ opacity leaf bitmaps? Are there settings in Vray or Max to jack up to minimize this? Or other settings and methods?
U need to premultiply alpha if I’m not wrong…
I should have mentioned that I am using PSDmanager with default settings. One of the default settings is Premultiplied Alpha (box checked). Any other ideas? Is it sampling? Or maybe settings within the Xref materials for the trees?
You’re best off rendering against a black background but even then that won’t completely get rid of the fringing as Photoshop doesn’t properly handle premultiplied images.
EDIT: You can also try “Remove Black Matting” under Layer-->Matting (you have to apply the layer mask first though).
Is that coming from refraction pass? Or just transparency, just put 4x group and apply same alpha to each image, each group will decrease the edge ever so slightly…
ALSO: make sure that the opacity map for the leaves is ‘treated’ as a gamma=1.0 image through the whole process. If there is only one phase (import, export/save, PSD-pass) in the process where the texture is seen as a gamma=2.2 image the greyscale anti-aliased edges are brighter as they are supposed to be and therefore the alpha-mask is slightly larger too !!!
For the following I’m assuming your scene has a background color/image or an environment map set (this wasn’t 100% clear to me from your description). So I’m sorry if the following is already basic knowledge to you.
Problem:
That you are using psd-manger and Photoshop doesn’t matter. The problem is that you are rendering against a blue sky background. So due to anti-aliasing (and also transparency) many leaves get a blue tint as expected. This works nice as long as you don’t change the background dramatically or have large transparent areas (like with the leaves here). There is no simple way to get rid of the blue tint. (In theory it is possible to reconstruct the original leave colors by using the alpha channel and the original background image. - but it’s not the best and easiest way to do it).
Solution:
You need to render against a pure black background. So set the Background Color/ Environment Mapping to black. That will later allow us to undo the anti-aliasing against the black background and compose any other background properly.
Preserving the reflections of the original background:
If you now lost the reflections of your background e.g. in windows/pool etc. then:
psd-manager Photoshop export + Adding a new Background:
3 ways to do it:
Best Regards,
Daniel
Daniel,
Thanks alot. That is definitely the issue. I have been using a Vray Sky in the environment map slot. Then in photoshop i would place a backplate behind my render. As long as the backplate was mostly blue sky then the antialias was not that bad. But sometimes the antialias would be very noticeable against a dark background. I just did a test by rendering the same scene with the Vray Sky map removed (black background) and the problem disappeared! Hooray.
Now i have to compensate my overall lighting since I am losing the positive lighting effects of the Vray Sky in the slot…
put the vray sky in the vray/environment override slots… put it in both the gi override and reflection override, then you can render against black but keep the sky’s effects on the scene.
put the vray sky in the vray/environment override slots… put it in both the gi override and reflection override, then you can render against black but keep the sky’s effects on the scene.
Awesome! Thanks. All
out of curiosity: could you just put the Vraysky in the tex slot of a Vraylight, then toggle it to invisible to get the same effect? I don’t use the environment much at all, so more curious than anything, but I was under the impression the sampling was better/faster with the Vraylight rather than the environment - but maybe it’s all the same…
tks /b
i could be wrong, since i dont use the domelight too much (when doing flythoughs, a really detailed imap / fast rendering frames beats a quick imap and slow rendering frames imho), but i think if you had it set to invisible, youd still need to have it in the reflection override slot for it to show up in refs.
Yep Brett you can do exactly that with the vray dome light.
I do this all the time to take advantage of importance sampling with the vray dome light.
If you set it to invisible you will render over black and can use any sky you like in post. If you forgot to make it invisible the dome light turns up in your alpha i.e. you alpha will be pure white which can be very annoying so make it invisible.
No super gnu you do not need to still have a sky in the reflection override slot. If you make the light invisible it will still show up in reflections. If for some reason you don’t want your dome light to turn up in reflections you can tick affect reflections off in the dome light options.
