Isolating shadows - Perhaps a noob questions?

[9k_JHT]

So I've been attempting to figure this out for some time with very little luck. Any help would be appreciated.

What I'm trying to figure out is how to isolate ground shadows as a render element that would mimic the appearance of an alpha (for the ground shadows only). There are a couple resources that describe how to do this through alpha manipulation. (https://vimeo.com/79655969)

The catch is that I need my hero object to also have an alpha of it's own (separate from the ground shadow alpha). If I manipulate the alpha through the Vray Object Properties of the floor plan I can either get the ground shadow alpha, but no hero object alpha or the hero object and ground shadow alpha together (I want them separated).

I've attempted to do this through the Vray Element Shadow pass which does not accurately represent the ground shadows (see below)-

Beauty-

Shadow Pass-

Beauty with alpha mask of hero object applied and shadow pass using to mask a solid black layer to get shadows-


Notice how the shadows in the last example have some issues and do not match the beauty.

My end game is to be able to have a PSD comp with the hero object on one layer, the shadows (with proper transparency) on another layer which would allow me to place a background color on the bottom layer.

Any input on this would be greatly appreciated. I'm not a noob Vray user, but trying to do this left me scratching my head. The only way around this would be to mimic the object alpha by using a multimatte pass for the hero object, but I'd like to avoid doing this to keep the setup as light as possible.

Thank you in advance.

Best,
Clayton

[Morbid Angel]

You need to render them in two passes. One with your object and the floor invisible to camera, second with your floor on with your object invisible to camera. This way you will get clean shadow which you can layer in comp correctly. You can use object properties to set the shadow matte options to alpha -1 and affect shadow, this will put the shadow into the alpha so its easier to comp with.

[9k_JHT]

Thank you! I was hoping to do this with a single pass and render elements, but if it takes 2 passes it takes 2 passes.

Thanks much.

[Morbid Angel]

Thank you! I was hoping to do this with a single pass and render elements, but if it takes 2 passes it takes 2 passes.

Thanks much.

Well if you just set the ground plane obj properties to alpha -1 and affect shadow, and leave the plane visibility on it will bake in the shadow with the objects alpha together. The issue is that you often correct the elements, in comp but it affects the alpha too and might have adverse effect on shadow, so splitting the two is common practice.

[9k_JHT]

Well if you just set the ground plane obj properties to alpha -1 and affect shadow, and leave the plane visibility on it will bake in the shadow with the objects alpha together. The issue is that you often correct the elements, in comp but it affects the alpha too and might have adverse effect on shadow, so splitting the two is common practice.

Alright.. Back at this again. I've started splitting up my shadows and the primary render via layers in Maya. Works great in concept, but in practice not so much.

Here's what I'm doing-

Hero object with floor. Everything normal here except that the floor does not affect the Alpha. So I get a clean alpha.

Second render layer with everything on it again, however, the hero object has primary visibility turned off (so it still casts shadows, but is not visible). The floor is in a VOP (and I tried mat wrapper) with -1.0 for affect alpha, but shadows ticked as well as shadows affect alpha ticked.

For whatever reason however, the alpha from the shadows is coming out like garbage, not matching the actual shadows of the render... Images below (of second render layer with hero object primary visibility off, but still casting shadows)-

Alpha from matte floor object (same regardless of doing this through Vray Object Properties or Wrapper Mtl)-


Shadows using Alpha as a mask on a black solid-


Actual shadows without VOP or Wrapper mtl-


So what am I doing wrong to get this inconsistency? The actual shadows do not match the shadow alpha. The shadow alpha produces odd halos. If someone could please put me right here I'd be very grateful. I must be missing something.

Thank you,
Clayton

[Morbid Angel]

You would have to post example of your scene so we can have a look.