hi. just wondering what you guys do for the sky in a typical exterior shot. there are so many options - do you use vray sky, an image in 3ds max or post it in after the render. if you post it in after the render how do you guys get matching reflections? does anybody use physical sky domes mapped with hemi images mapped to it? does anybody use hdri? or hdri combined with vray sun? does anybody use vray noise for clouds. any advice or ides would be very much appreciated. thanks.
i use vraysky, but then composite the render with a backdrop photo. this way i have (more or less) the brightness distribution on the sky (from vraysky), which helps with aligning the backdrop.
also this way one can - by mixing layers - achieve pretty nice results. i know myself (and my clients), and prefer the flexibility of adding the sky in post over fixed sky from raw rendering.
We pretty much use the same method here. VRay Sky or a custom picked HDRI in the render, which is then replaced later in photoshop/after effects etc
What would be awesome would be if some software existed that generates a completely realistic HDRI sky directly in Max that VRay can use as an HDRI source, complete with clouds/atmosphere etc. Something like Ozone that was VRay compatible, basically
Here’s hoping…
Flexibility wise you’re best off rendering against black with whatever sky / hdri used in the scene you want - if you render against a sky image or a vray sky it’s very hard to get a clean edge on your model unless it’s very straight. Most compositors will give you an option to separate a model rendered against a solid colour but with the variation in the vray sky or an image it makes it harder to do. As regards swapping in a sky later if the colour / rough look of the sky is similar you should be fine - ILM got by for quite a few years using nothing but reflection maps (not even raytracing) that looked roughly similar to the environments they were supposed to represent. for example if you had a simple, gradient type sky then it may look a little odd to have reflections in your models windows with lots of little clouds in it. If you’re using curvy objects where the reflections are more warped and distorted though, you’ve got a lot more room to play around.
it’s easier when you use alpha channel / mask 
well, that’s exactly what john is talking about 
since almost everybody tends to render premultiplied outputs, it’s always a good idea to render against a black/solid background.
Yes of course - the issue is that in compositing you’ve got two types of images, premultiplied and straight. This means that the soft pixels at the edge of objects might be made up of 50% of the object, and 50% of whatever the background is. This is fine for something like after effects to deal with if the background is a single flat colour (like black or white for example) since it has an option to pick the colour that the edges of objects are blended with. If you’ve got an image or a vray sky in the background however, it’s far more complicated since each edge pixel from your objects might be blended with a slightly different colour. The other option is to render your images straight which will give you quite ugly looking edges - that’s because they haven’t been blended with your background which is what’s causing the compositing problem anyway!
Ok, I get the whole, rendering with the a solid background.. but when using Vray Sky, it gives you a gradient of blue… I can turn that off, use a black background for example… but things get darker…
I’ve been trying to use Lele’s workflow with the Vray Sky/Camera, .255 exposure situation… works great when you want to use the default gradient blue… but how can I use a black background in this situation… (then put in an image of a sunset for example in post..)???
I know it seems like a simple question, but how do you use an alpha channel map for the back ground? And how does something like this effect Lele’s workflow?
I’ve got a big presentation due on Weds - any tips (very soon) would be great! Thanks all!
Can you put the vraysky in the vray environment, reflection, and refraction slots but put a solid background in the max environment settings?
yep, that’s the way to do it, as pixelcon suggests. leave max’s background as it is by default, and override it using vray’s environment channels.
what are peoples methods in say photoshop, to remove this solid colour? Select by colour range? Or are there far better suited options as ‘select by colour range’ isn’t a very exacting method…
just select the alpha channel…
got it.. thanks! That seems to speed things up a bit too - right?
Isn’t this the whole issue tho?
Using the alpha channel which inevitably has to be pre-multiplied in reality will cause some type of fringing, so if one renders to a solid background, say black, will create a nice black outline, albeit small, around your scene when said alpha channel is used?
I’ve just done some simple tests which confirm my thoughts.
Any solid colour in max’s background slot will still be accounted for in the alpha channel. It comes down to the alpha map not being accurate enough. The lower the resolution the lower the accuracy.
The finging effect is relative to the amount of pixels it covers so with higher resolutions the finging effect is reduced but never totally abolished. It also seems with higher AA settings, there is no reduction in the fringing.
A comparison between no filter and a catmul-rom filter shows that the edge-hardening of the catmul filter produces even less desirable results…
I don’t really know what to try next…
So why not just use a colour or gradient that matches the sky you wish to use. Or the vray sky, then sustitute using the alpha. That way the fringe will pretty much be the same as your background image anyway. If you’re lighting a sunset scene using the vray sun/sky then the vray sky will take on a more orange hue. Once you’ve loaded the alpha as a selection you could always use the refine edge tool in photoshop to feather the edge slightly if you wish before applying it as a mask.
All of your suggestions are valid and the method(s) that I use are the same or simmilar.
All I’m after is a perfect alpha channel without having to employ these types of workarounds to make post-production more flexible.
I think currently the defringe tool in photoshop produces the best results regardless of what is rendered out of max. I tried defringe even with a hot pink background out of max and it completely removed it without noticable damage to the objects in the scene.
one of the way to do this kind comping in ps, is setting the alpha channel as layer mask. that’s about it, if you rendered against black that is.
when using ps, which is not capable of handling premultiplication and can’t remove colour matting as ae can, you shouldn’t be rendering with any other solid colour as background.
edit:
as an aside, the alpha channel in itself will always be “perfect”. it’s sampled as the rgb beauty pass is, so it should always match, regardless of resolution or quality of the sampling.
You’re quite right - it’s strange that photoshop is actually quite shit for dealing with alpha channels. All of the other comping packages have a control to pick what your bg colour is and by using the semi transparent pixels at the edges of objects in the alpha channel, find out how much belongs to the object and how much to the bg colour - then it’s able to separate the two perfectly. I’m sure you can do something with blending modes in a really convoluted way to deal with this in photoshop.
Rendering against black BG, selecting/removing alpha and using “Remove Black Mate” in PS works quite well for me, better than defringe.