Shadow Problems

[Da_elf]

http://home.caribsurf.com/elfpro/problem.zip

In the zip should show some of the problems. The box should be a:blocking GI Environment light from the left teapot. and it should also be casting a shadow on the teapot from the sun. However the teapot shouldnt be casting shadows like that.

(pretend the image being used as a background image was a photograph and i recreated the scene in 3d to match the background image)

[natty]

i think you just need to Exclude Tea pot 1 from the sun light

[Da_elf]

would work. however suppose you have an object thats half in, half out of the shadow of the box.

also...
matte shadow objects should have an affect on all normal 3d objects. and recieve GI info from those objects. however they shouldnt be contributing GI information into the alpha channel. im working on a project where i was walking next to a cliff face with the camera then come a spot where the cliff face isnt casting a shadow on the ground. so the 3d character needs to be walking in the shadow of the cliff and will only get skylight from one side. then when he walks into the sun he should be fully lit. the cliff is blocking the environment light from the character like it should but its also casting a GI skylight shadow onto the ground plane making my alpha darker where that shadow is

[natty]

mmmm ... not sure how you would overcome that ... probably do a shadow pass and edit in Photoshop?

[Da_elf]

maybe richard rosenman might have a solution? hehe. or anyone else who has done heavy compositing?

[RErender]

I think you have found a bug, the direct light should not cast a shadow through the box matte in your file. If you apply a wrapper to the box with the same settings the result is as expected (no direct shadow from the teapot). If you try it make sure to uncheck matte in the properties because the object settings overide the wrapper.

[Da_elf]

its a bug really? cool. i didnt try matte as a wrapper since the wrapper doesnt have the "dont cast gi on other mattes" option thats to be found in the properties

[Seraph135]

Unless I'm misunderstanding...This doesn't look like a bug to me. This looks like a typical case of double shadows. Its simple to fix if your doing it all CG, but i'm not sure how to do it if your compositing into a photo.

I'm sure there are other ways to do it, but for this example heres how. For the sake of simplicity I've decided not to render with GI, but its basically just adding another raw gi pass.

top 2 are the individual layers. The box was set to "invisible to camera" for the teapot pass so that I would get shadows from it. The middle two are the shadow passes for each. The lower left is the two shadows combined. In Digital Fusion this is done by using a "maximum" blend mode with the two passes acting as masks on a solid color layer. In photoshop its a "lighten" blending mode. The lower right is the final comp.

[Da_elf]

here is one of my main problems however. in compositing over a real photograph using your scene as an example. the box would be a real box and the teapot is 3d. the real box is already casting a shadow on the ground. so we cant have the 3d box casting shaows on the ground. but i need to to cast shadows on the teapot. i think i know what ill do though. ill do some tests and get back to you with the results hehe

[RErender]

Seraph, did you look at Da_elf's file? It is set up with a matte object between the sun and the teapot. There should not be a shadow cast by the teapot....

I think you misunderstood , my point was not that it can't be done, just that I do not think Da_elf's file was exhibiting correct behavior, give it a try and let us know.

[Seraph135]

I couldn't get his file to work. It kept crashing on me. Was it a max 6 file? If he has a matte material on the box (which I also did) and I'm getting shadows from the box on the teapot. Then I'm doing a shadows pass for the teapot. I still don't see a bug. I'm just not sure how to composite a CG gernerated shadow seamlessly into a photographs shadow. I did experience some funny things happening using the matte properties for an object versus using a material for a matte. Let me know if you have more trouble guys. Now I'm really curious to figure out the issue.

[RErender]

It is a max 6 file, In my test using vraymtlwrapper as the matte gave the results I would expect as you got. The problem is with the matte controls in the object settings and v1.45, in this case the direct light cast shadows through the matte causing the double shadow of the teapot. It would be interesting if you could reproduce it.

[Seraph135]

I'll give it another try later today. I didn't experience some strange behavior with object properties. I can't recall exactly what all those things were, but I do remember than I had trouble getting objects with a matte setting to NOT show up in the alpha. Reguardless of the alpha contribution settings I gave them. Positive, zero and negitive values. Anyways...I'll get back to you on this one.

[Seraph135]

I had a couple minutes so I tried it. I'm not getting any problems. I'm on Max 5. I used a box with its object properties set to matte with -1 alpha contribution. It rendered correctly.

[Da_elf]

actually its not a bug. i spoke with vlado. its the setting "GI doesnt affect other mattes" that causes the shadows to mess up. when you turn it off things are fine. however now the box casts a shadow on the ground plane. which when compositing would double up the shadow on the ground in intensity. ive got a work around to solve my problems will post a tut later.