Antialiased isolated reflections

[tricky]

I *think* I have asked a question like this before, but I can't find it.

What I am after is *just* the reflections as a selection. For example, I want to render a pane of glass in a building with pure, 100% reflections (white in the reflection slot, black in the refraction slot, black in the diffuse slot, environment set to black) and then have a selection of that within Photoshop so I can muck around with JUST the reflections.

The problem at the moment is that if I open the render in Photoshop and use the colour selection picker to select just the 100% black parts (so that I can delete them) I get a nasty aliased edge where it meets the reflections. I can use feather, grow, etc but I still get a nasty edge when what I really (really, REALLY) want is a lovely clean selection of the reflections all by themselves.

Does anyone have a method for this? (I hope I am clear)

[ene.xis]

multichanelmatte!

[RErender]

Oh, that's just mean :P

[ene.xis]

sorry...but its true!
u could render only the objects u want no gi with a non transparent material in another render pass...right?

[joconnell]

Yeah I'd probably do a render pass with the windows / reflective bits white and everything else black as a scanline pass presuming there's no motion blur - dimo's lovely multimatte thingie looks like your best bet.

[tricky]

I'm not sure how this method would work (and you obviously know I can't get the multimatte thing working ). Doesn't the multimatte thing just seperate out different object selections - not just the reflection bits?

Maybe these images will explain better:

Here's the render of the building with 100% reflection on the glass:


Here's the render of a mask for just the glass (using our own methods - not the multimatte thing):


Now here's the render of the glass with the rest of the image removed using the mask from above (this is now a floating layer in photoshop, so you should see the chequerboard pattern behind, but saving as a jpeg makes this white):


And here is a section from this layer once I have selected all the black and deleted it. Again, this is a 'floating' layer so you should see the chequerboard behind:


Notice the dirty edge around the edge of the reflections. I want this to be properly antialiased. I could add the reflections to my image as a screen layer, but I definately do not want to be using that method - don't ask why.

Is this clearer?

[instinct]

AFAIK this will never happen. As the Alpha and mask channels of a reflective object will be opaque and there's no way to trace reflected IDs. I think that was discussed before, but i might be off the track here. a workaround would be adding vraycolor to the environment-slot of the reflection set it to pure white and set all the other objects to pitchblack. render that as a pass and you should get a mask layer that you can use in photoshop.

Regards,
Thorsten

[tricky]

Ahhhhh - I see where you are coming from. It seems so obvious now! Thanks instinct!

[m_hinks]

Just out of curiosity, we use a similar method to render out a base render, no glass, and a 100% reflect glass and split out the self reflect and glass to comp together with better control.

Id love to use the multimatte element, but it doesn't seem to be as clean as using psd manager. multimatte would require you to colour pick the objects to generate your mask. With the psd generated but psd manager, you get objects or materials on there own layers that you can simple right click and ctl click apply mask and done. There is no anti aliasing problems with this either, where I think the multimatte would.

I am thinking that multimatte would prob be best for animation work or low quality images, but for large print would, fringing would be critical.

[instinct]

Well you misunderstood how to use it it seems. First of all you dont color pick the MultiMatte. For a good reason the 3 included elements are pure red, pure green and pure blue. You can simply copy the r, g, b channels to be a layermask to get rid of AA problems. Another thing you have to keep in mind is that the MuliMatte uses the sampling of the RGB rendering. Hence when using undersampling that will be reflected in the elements too of course. It might not be noticable in the RGB because there is shadow or low contrast, but in the matte it will be appearent.

And btw using elements (VrayMtlSelect with a VRayBlend to be specific) you wouldnt have to render twice at all to comp reflections in post together with the MultiMatte

Regards,
Thorsten

[m_hinks]

Yes it seems I did misunderstand!! But it still remains that using PSD manager would give you better control having all object/material type masks in one hit. Im def going to have a play with multimatte tho.

[instinct]

i never really used PSD manager, and we wouldnt really have any use for it, as we're working in a completly exr based pipeline and do mostly animation. Basically what needs adjustment is the import of the layers in your post tool rather then the output of vray i'd say, but that's a diff topic

Regards,
Thorsten

[panthon]

in your post tool

Nuke?

[instinct]

well it works fine with nuke :P

But for PS it's rather the import that fails then the output of VRay (=loading multichannel EXRs doesnt work out of the box, besides it doesnt handle linear too good)

Regards,
Thorsten