create mask for window glass

[OPEN_RANGE]

I am lost in how to accomplish this. I have an interior rendering. I have a plane mapped with an image positioned outside and it shows the important mountain views. I have the window glass material adjusted to where it is showing some subtle reflection. I have fresnel 2.0 reflection 30,30,30 and a refraction amount set to 245,245,245.

I render the scene and it looks good. Having the background photo visible is important because it helps with all the reflections in the various materials like floor, cabinets etc.

what i would like to accomplish is having all of the benefit of showing the background plate but then be able to mask it out in photoshop and drop in the background photo so I can edit/adjust it independently from the rest of the rendering. I do not want to loose the subtle reflections that are in the windows while doing that.

I've tried hiding the background image and making a separate render. This is time consuming (having to render twice) but if that was the best way to achieve this I could do that. I just can't seem to generate an alpha that I can use effectively in photoshop. I can't figure out how to accomplish this. I am rendering as .exr to start with.

I'm pretty sure this has been discussed before. Maybe there is a "best practice" work flow to accomplish this? TIA for any help or pointing to a tutorial or something. I don't have the compositing knowledge of how to get what I want. Below is the image/render I'm working with.

[glorybound]

I use PSD Manager and my glass is on its own layer; everything is already masked for me. Have you tried material id's, or using the refraction element? In your case, you can just render with a black background you would use your alpha channel, no?

[OPEN_RANGE]

Bobby, thanks for weighing in. I tried Material ID, the problem is if I make the glass the material that has the corresponding ID, then everything on the outside of the glass is wiped out. Railings, steel beams, etc. get covered up and aren’t masked out.

If I render with black background and use alpha channel then I loose the reflections that are in the glass.

I got lost I trying different settings. Like the reflect and refract Affect Channels setting. I’m not sure if that should be set to “affect all channels” or one of the other choices.

I’m unsure if the fact that I have refraction set to 245 is maybe complicating this. Having that bit of refraction seems to help,with the reflections in the glass.

I don’t remeber what I did but I did up with a layer that was sort of the inverse of what I want. The background photo appeared on the geometry outside the widows instead of in the spaces around all the geometry (steel railings, braces, horizontal beam etc.)

[glorybound]

If you do a reflection pass then you would have access to the reflection.

[LarsSonparsson]

What if you set backplate properties to "not visible to camera"?
Then it will still affect reflections etc.

[OPEN_RANGE]

ALERT to Forum Moderator. I am not receiving email notice of replies anymore.

I'm having some real trouble getting this figured out. I would like to try some of the suggestions above but first I seem to have a more fundamental issue to work out. No doubt user caused. Check out the images below. One is a wip version of the final. The other is the same scene trying to render just the model without any backplate so as to get a simple alpha channel. You can see that all the geometry outside of the window glass is semi-transparent. I've tried all kinds of stuff but can't seem to get that to work as it should. I have no idea what is causing it.

Vray glass is Reflect= 27,27,27. w/ glossiness.99. Fresnel Reflections 2.0. Refract= 255,255,255 Both Reflect and Refract set to Affect Channels= Affect all Channels

Vray Environment GI Environment Enabled using vray sky as map. Reflection/refraction environment = turned off/un-checked.

Saving as .exr. Opening is photoshop with EXR-IO plug in using default options.

What I would ultimately like to achieve is to have a renderings with the reflections in the glass (right corner windows) and the rest of the scene affected by the backlplate photo. The window openings would all be black/alpha masked and then in Photoshop I would be able to place the background photo as a layer underneath. that way I could position the backplate independently and more importantly be able to adjust the exposure, contrast etc of the rendering and the backplate independently form each other.

[OPEN_RANGE]

Just now got 1 notice of reply made yesterday at 3:30 PM. Have not received notice of any of the other replies.

[glorybound]

Just now got 1 notice of reply made yesterday at 3:30 PM. Have not received notice of any of the other replies.

Yeah, I get them a day late, which is annoying.

[OPEN_RANGE]

I'm remembering now going down this rabbit hole before. What causes the trouble is that If there is glass in the windows, using basic vray material and regardless of refraction or reflection amount including zero. In that case having a black environment outside and rendering an image with an alpha channel will work BUT, and this is the gotcha, any geometry like railings, roof overhangs or other parts of the building that are on the outside and beyond the glass, will not appear in the image. It will be gone sort of left out by the alpha calc. SO the thing to do is you have to hide the glass.

I'm thinking the workflow would be - make one rendering with backplate showing. Capture a reflection render element if possible. Render again with backplate hidden and all glass hidden. Combine together these renderings and elements in layers in pshop. Use the hidden glass alpha to mask out the windows. Paste backplate into that mask. Add back the reflection element that has now been lost. I haven't tried this but may some day. Lot's of effort required. For now I'mt accepting not being able to have the separate back plate and rendered image post editing capability for these situations. It would be super helpful if some easier way was available, like having a glass material that doesn't wipe out everything on the outside of it in the alpha channel, Wish list.

[Morne]

1)
Untick "visible to camera" for plane with mountain in max object properties, untick "Visible in refractions" in VRay object properties, then render against black. You'll still get the reflections of the mountain etc.
Then you can just stick in whatever you want as background in PS

2)
Alternatively make sure your glass have "affect all channels" in refraction, then right click the mountain and change the object ID g buffer to whatever number, 15 for example. Then add a multimatte render element and make sure the mode is objectid (not materialID), and in either red, blue, or green slot, stick in the number 15 (the number that corresponds to the g buffer number you gave it earlier. You will then get a mask that you can use in PS to replace the background, although you may get some fringing on the outside railings, window frame etc in this case as you didn't render against black.

[OPEN_RANGE]

Morne thank you for your reply. What you describe in the #1) procedure above is exactly what I would like to achieve I have now tried that and I get a sweet render with all black beyond the windows and cool reflections in the glass. BUT no alpha channel.If I check the alpha in the frame buffer it's all solid white across the entire image, like alpha is not being calculated. If I try to load the alpha selection in pshop I get a "warning no pixels are selected" message.

Not sure what I'm doing wrong. I am rendering as .exr and I have 1 render element - vray de-noiser. also have lens effect from VFB enabled. Using 1 vray dome light with hdri. I tried with dome light "affect alpha" ticked (default) and un-ticked. Makes no difference

If I could get this to work as you describe I would be very psyched. Just can't seem to get it set up correctly.

Edit> I believe i may have it now!. I needed to also change refractive to "affect all channels" in procedure #1 above.

Morne! Thank you x100! The magic formula you provided has been so elusive to me. I have a very extensive post in this forum from last year I could link to where this same issue was discussed by many and this solution was not arrived at. you rock!