AA white edge

…without anyone biting my head off, could someone please direct me to a thread that deals with the white edge created when compositing seperate elements of an image using alpha channels.

i am pretty sure it is caused by certain AA filters, but i am not sure which to use to avoid the white edge.

i am rendering an animation, and doing a separate pass for the trees, then compositing them back together in post. i am getting the white halo around the edge of the alpha channel.

i have searched the forum, and checked a lot of the topics, but i can’t seem to find exactly what i am looking for, but i am sure a thread exists. i would keep searching, but i need to send this to the render farm tomorrow, so i am running out of time.

thanks. ts.

if the halo is around the edges then it doesnt sound like an AA problem, rather like a premultiplying issue.

Thorsten

I think what instinct said sounds like its on the right track.

If you can’t get it fixed, this is what I have done to correct it in Combustion - but I’m sure AfterEffects is similar too.

Select the inverse alpha channel so everything but the trees is selected, then expand and feather that selection a few pixels to cover the white edges. Then just darken that selection until it blends in nicely with the background.

Probably not the best way to do it, but it works fine if need be.

what software are you using?
combustion should have no problem with transparencies, not sure about AE, but for some annoying reason PS needs to do a ‘remove white matte’

there’s a thread here:

http://www.chaosgroup.com/forum/phpBB2/viewtopic.php?t=17251&highlight=

It might happen also if you are multiplying render elements which have big contrast along the same edge. There is a thread discussing this somewhere too.

Best regards,
Vlado

attached is the problem i am running into. i am rendering the building, and the trees in separate passes. when i use the alpha channel created by the tree pass, it leaves a line around the image. currently i am using Adaptive QMC at 1 min 4 max, with a secondary AA filter using the VrayLanczosFilter set at a size of 2.0.

the attached image is comp’d in Photoshop, by the final animation will be comp’d in AfterEffects. i have Combustion 2.0, but i have never used it, and given my tight deadline of Monday first thing AM, i don’t think it would be wise for me to dive in head first.

given the resolution of the animation, i am not sure it will be as noticeable, but i am not sure. the attached image was 4000 pixels wide for a poster. i guess i would rather figure out why it is doing it, than to just ignore it, and pretend it does not exist. i can always do the selection expand like Tim suggested.

i am saving the image in TIF format.

i will read the attached thread and see if it solves the problem.

thanks for the help so far.

Hey Travis,

The image doesn’t work - asking me to login to karslberger.

thanks Tim. i rehosted the image on my own site, and edited my post above. hopefully it shows up now.

Or, what you could do is this.

Load the main frames as a background in your environment slot and then render the tree pass. So then, for each frame, there is the corresponding base rendering behind it and the edges should blend in perfectly. Make sense?

Or, you could pick a median green tone for the environment color and that would match the color of the trees better than the white or blue that is currently there. But I think the background frames is the best option.

that should be a good work around. thanks again.

…ya, it was blue not white. i am setting the bldg to have a alpha contribution of -1.0. it creates a nice mask, so the images fit perfectly of each other. the blue was the sky beyond.

Nice.

when rendering out alphas you should ALWAYS set your BG to black (well in 99% of the cases). That makes compositing things later alot easier.

…but wouldn’t that just cause a black outline instead of a blue?

Most programs deal with black as part of the alpha and blend it in properly.. I think

There is also something in PS that can fix the edges… cant recall what is was tho…

Whenever I do things like object & tree passes, black never seems to work out. Always leaves that ugly little fringe.

In PS it is Layer/matting/remove black matte to remove the black edges.

In Combustion in the footage controls there is a “premultiply” button with a color swatch. The color swatch controls what color pixels remove at the edge of the alpha, in your case setting it to the background blue color would fix it. I don’t know what it is in AE but it must have something similar.

Thats the one.. thanks Eric

PS (v 8.0) layer/matting/defringe/ 2 pixels

works very very good.

but best idea was first render is b/ground to tree pass.

i dont go for pre-ultiplied that often, i usually prefer straight.