skies - render in model or add in later.

[stef.thomas]

So why not just use a colour or gradient that matches the sky you wish to use. Or the vray sky, then sustitute using the alpha. That way the fringe will pretty much be the same as your background image anyway. If you're lighting a sunset scene using the vray sun/sky then the vray sky will take on a more orange hue. Once you've loaded the alpha as a selection you could always use the refine edge tool in photoshop to feather the edge slightly if you wish before applying it as a mask.

[Pixelcon]

All of your suggestions are valid and the method(s) that I use are the same or simmilar.

All I'm after is a perfect alpha channel without having to employ these types of workarounds to make post-production more flexible.

I think currently the defringe tool in photoshop produces the best results regardless of what is rendered out of max. I tried defringe even with a hot pink background out of max and it completely removed it without noticable damage to the objects in the scene.

[rivoli]

one of the way to do this kind comping in ps, is setting the alpha channel as layer mask. that's about it, if you rendered against black that is.
when using ps, which is not capable of handling premultiplication and can't remove colour matting as ae can, you shouldn't be rendering with any other solid colour as background.

edit:
as an aside, the alpha channel in itself will always be "perfect". it's sampled as the rgb beauty pass is, so it should always match, regardless of resolution or quality of the sampling.

[joconnell]

Isn't this the whole issue tho?

Using the alpha channel which inevitably has to be pre-multiplied in reality will cause some type of fringing, so if one renders to a solid background, say black, will create a nice black outline, albeit small, around your scene when said alpha channel is used?

You're quite right - it's strange that photoshop is actually quite shit for dealing with alpha channels. All of the other comping packages have a control to pick what your bg colour is and by using the semi transparent pixels at the edges of objects in the alpha channel, find out how much belongs to the object and how much to the bg colour - then it's able to separate the two perfectly. I'm sure you can do something with blending modes in a really convoluted way to deal with this in photoshop.

[Codi]

Rendering against black BG, selecting/removing alpha and using "Remove Black Mate" in PS works quite well for me, better than defringe.

[joconnell]

Yep - or render straight.

[Pixelcon]

Rendering against black BG, selecting/removing alpha and using "Remove Black Mate" in PS works quite well for me, better than defringe.

I tried that but I didn't work for me. I'll give it another shot.

You're quite right - it's strange that photoshop is actually quite shit for dealing with alpha channels. All of the other comping packages have a control to pick what your bg colour is and by using the semi transparent pixels at the edges of objects in the alpha channel, find out how much belongs to the object and how much to the bg colour - then it's able to separate the two perfectly. I'm sure you can do something with blending modes in a really convoluted way to deal with this in photoshop.

Interesting. I wasn't aware that alpha channels had any "play." I assumed they were exact and should work in exactly the same fashion between multiple products.

I guess this explains my problem!

I'll try the matte function again.

Thanks all!

[joconnell]

Yep the alpha channels themselves are fine - it's photoshop being shit at making use of them. For example it's got a filter to remove a black or white fringe, or a defringe control which effectively eats the edge back - it doesn't have any controls to deal with images rendered against other colours however.

[Pixelcon]

I see. Thanks for the clarification.

I tried out the remove matte options and they didn't work out as well as I'd hoped. I think I'm going to stick to the defringe option for now. considering my renders are usually 4000px wide, the defringe should work to an acceptable level.

Cheers jc

[mresailor]

Rendering against black BG, selecting/removing alpha and using "Remove Black Mate" in PS works quite well for me, better than defringe.

Can you explain how to do this exactly? (step by step as if you were talking to a first grader..) I am not really an expert in PS - or Vray for that matter...
I rendered with a black background (using Vray Sky in the Vray environment map) - but I was never able to put in a "background image" in the MAX environment... based on the readings in here - seems like the post editing would be better anyway...

However...there are trees in the model... with leaves... do I have to zoom in and select every little friggin black spot between the trees? Seems like there is an easier way to do this...
(see attached images)

Also, thinking of animating a fly through of this model.... how can all this be done while animating - I am sure there is a way to do this without having to render out frame by frame then going back in and inserting a pic of a sky... (and to think about it, the sky would not "rotate" around the model like it should - using the frame by frame thing..) Seems like using a dome environment would be better no?

Big submission that was due yesterday - had to submit with black backgrounds like this..
however, I have to present in person on Tuesday.. would love to be able to beef these up and put in some skies and possibly fly around with animation..

THANKS!


By mresailor at 2009-07-23


By mresailor at 2009-07-23

also - for the second image.. when rendering for the animation - or even a still for that matter.. I would love to just make the background white.. no sky or anything...

[Codi]

Sure.

First of all, you have to save your render using an image format that has alpha channel. I recommend TGA if you are just starting compositing images.

There are many ways of removing the alpha, I'll try to explain the easiest to follow ...

- Open the file in Photoshop.
- Choose: Select/Load Selection
- From the load selection menu in the Channel dropdown choose "Alpha 01" and click OK
- Choose: Layer/new/Layer via copy and click OK

(Thats the same as CTRL+click Alpha, CTRL+J)

You have now a copy of your foreground in a new layer. Assuming you rendered against a black background, you'll have a black halo all around the edges. To remove that you have to choose Layer/Matting/Remove Black Mate.

Now put any image you want in a layer below the foreground, to have a white BG:

- Create a new layer (Layer/new/Layer...)
- Drag it below your foreground.
- Choose a white color in the color picker.
- Edit/Fill.../OK

For animations we usually render the sky in a separate pass from spherical maps as you say, and composite it later on. I have not much experience comping animations, maybe someone else will give you advice if you tell which package you can use (AE, Combustion, ...).

Regards.

[rivoli]

either that or you can just use the image's alpha as channel mask, and that's it. which is basically what happens in any compositing/editing application when you import footage with an alpha, the information in this channel will define which parts of the image are opaque and which ones are transparent. white being fully opaque, and black totally transparent.
you can either output a sequence of files which store the alpha channel, such as tga, tif, exr.. or you can render the alphas as separate files and then use them as mattes.
depending on the application you'll use, the workflow may be a bit different, but at the end of the day they all do the same thing.

[Codi]

Correct me if I'm wrong, but if the render is done with a pre-multiplied alpha over black BG you'll have to do the remove black mate step, which is not possible when you use PS channel masks.

If you store the image without pre-multiplying the alpha then I guess it would work.

[rivoli]

it definitely works with straight outputs, which I guess ps handles better. but for very simple comping like these, it's not a problem with premultiplied images either, even without remove black matte.
here's a quick tree&sky:






if I remember correctly things start to fall apart when you have premultiplied outputs with dof or mblur, then ps shows how much it sucks in handling them. you can work around it, though it's not quick. by any means.

[Codi]

Yep, straight output = not premultipliying.

You only need to have it into account when saving the render and set the image options acordingly.