Vray shadows not working usably in AE compositing

[dre4mer]

The flag shadow issue is solved it seems but i'm hitting another issue that might be related. I am rendering out just two passes to a single multipass 16 bit half float EXR for the moment.

Only Two Passes: GI and I have the diffuse from a rim light attached to light_select rendering to a RIM_DIF pass.

In After Effects I bring my EXR into a 32 bit comp. Set the EXR to no color management. Place GI at the bottom of the comp set to "normal" RIM_DIF above that set to "add" then an adjustment layer at the top with an exposure effect gamma 2.2. From my understanding this is the correct workflow for using EXR's in AE.

The problem is apparently vray holds the shadows in the alpha channel and the gamma change doesn't effect the shadows. Let me show you what I mean.

This one is how I presume it SHOULD look and is what it looks like if I place a grey solid under neath my layers inside of my EXR precomp. However that grey solid could never stay there as it needs to instead be composited into the live action stadium.


This is with the gamma 2.2 shift (the shadows are unaffected)


This is without any gamma 2.2 (everything is dark)


Here are my material wrapper settings on the stadium.


As you can see the shadows are unusable for compositing in after effects because of this. Does anyone understanding what is going on and if there is some way to fix this problem? I have a couple hundred shots waiting to get started on!

Thank you for any help!

[Svetlozar Draganov]

Would you please re-attach the images since they are invalid.
The forum doesn't support drag and drop functionality, please use Insert Image button instead.

[dre4mer]

Hmmm sorry about that it looks like they work for me here. But here are the images again.

This is as the composite should look (when I place a grey solid inside of the precomp under the plate. The shadows are gentle)


This is how it looks with gamma 2.2 applied to the crowd layer (as you can see it only effects the crowd, not their shadow, and the shadow is very black.)


This is with no gamma 2.2 applied


This is my stadium material wrapper material.

[dre4mer]

simple comp test

This zip contains a simple comp test to show what is happening,and the srouce files associated with it. It would seem this is just basic compositing with vray so I don't understand why the shadows are having issues.

SIMPLE_COMP_TEST.zip

Here is a still frame from the comp. As you can see the shadows are too black.


Thank you for any assistance! This is kind of driving me crazy becuase I don't have a way to render out my plates in a fashion that is usable.

[kagemaru]

I've some difficulties trying to understand what is your goal and what problems you are having.
Having said so, first thing i would say is that whenever you are trying to replicate in post something happening in the render engine, you must work in linear fashion as vray does.
For after effects you then need to set up a working space (usually sRGB) and a linearize working space



After doing so, when composited, your image should look like this



And to my eyes, colors are fine. The clear problem is how after effect handles alphas when micro values are in there, and you get those very harsh pixel in the shadow, while for example the same composite in nuke gives this



But that's a different problem from the one you are having i think.
One may argue about using a workflow that involves exporting a beauty with an "alphaed" shadow baked in, but i guess it depends on your job and requests you get.

[dre4mer]

First of all thank you for replying and trying to help!

I suppose all i'm really trying to do is have the ability to effect the shadows in post in After Effects. I have tried various combinations of passes and I have NOT been able to get a good method that allows me to make the shadows more pale. I have tried to modify the shadow as described in the passes document http://help.chaosgroup.com/vray/help...erelements.htm

"the Shadow element is not, strictly speaking, a part of the image; however, it can be added to the Raw Lighting element to adjust the shadow intensity. Simply adding the two elements produces lighting as though no shadows were computed."

But this does NOT work as described, at least not in After Effects. Are you able to point me to the passes I need to be able to make something that looks like the beauty pass, AND allows me to change the shadows in compositing? It seems like it shoudl be rather simple. But i've been trying stuff for days.


I am using 32 bit projects in AE with working space set to "none". Then using a 2.2 gamma adjustment layer on top of my entire EXR comp. If I can instead use 16 bit, with working space set to srgb that would save alot of computing power. It does change how the images look though.

[kagemaru]

Hi dre4mer, let's see if i can be of any help.
First thing, about AE project settings, the screen i posted is a random image i found on the internet: what i was suggesting is using a linearize working space instead (and thus compositing in a linear workflow). In this way you will do your math as vray does, and i strongly suggest that, especially if you want to recreate the renderer output in post.
Using 16bit instead of 32bit depth is a different matter and will give you more color depth, not a more correct composite per se.
Apart from that, i would suggest to render the beauty with its own alpha in one pass, and then in a second render just the shadows in order to better use tools that a compositing package will give. There are different ways to do so: for example i wouldn't use the shadow pass, i would make the hero object not visible to camera (but casting shadow, gi, reflection, etc) and the resulting beauty would be my shadow/gi/refl pass to be composited, usually in multiply/add.
The problem you are having with the shadow "baked in" is that they are merely a premultiplication of the alpha and thus every color correction won't affect them since they're black from the start. Since they live in the alpha, it's in there you want to have your color correction.
I know it's kind of hard to explain (sorry for my english btw), i'm attaching a zip with an example i did in vray for max but it's exactly the same. As you will see, i put in the shadow layer a fill and a level filters that demonstrate how you can correctly modify it.

test_shadow_comp.zip

[beenyweenies]

I am using 32 bit projects in AE with working space set to "none". Then using a 2.2 gamma adjustment layer on top of my entire EXR comp. If I can instead use 16 bit, with working space set to srgb that would save alot of computing power. It does change how the images look though.

Weighing in with some additional info.

- I always set my AE project to 32 bit, even though my renders are 16 bit (half float). I do this because AE does not support 16 bit float, only 16 bit int. This info may be outdated but I still do it. I also set it to use sRGB Working Space, with Linearize Working Space checked. This creates the correct environment for linear compositing.

- Any 16 bit sequences/images that I import, I go into Interpret Footage, to the second tab, and set linear to ON. You have to do this with 16 bit images, because the default setting is "32 Bit Only" so 16 bit half-float images won't be properly interpreted as linear.

Once these steps are done, everything is exactly as it appears in Vray frame buffer with sRGB turned on. No need for gamma adjustment layers etc.

[kagemaru]

- Any 16 bit sequences/images that I import, I go into Interpret Footage, to the second tab, and set linear to ON. You have to do this with 16 bit images, because the default setting is "32 Bit Only" so 16 bit half-float images won't be properly interpreted as linear.

Actually if you render out in openEXR, ae interpret them as floating point 32bit even if they are rendered out in 16bit half float, you can see it in the project bin when selected.

[dre4mer]

Thank you for the additional information! So Also just to clarify, when you are rendering the plates from vray do you set gamma to 2.2 for the renders or leave it at 1 using this workflow? I assume 2.2 right so that you don't have to do a gamma adjustment in AE?

As for my scene i'm splitting it into two plates, one for the shadow pass with the crowd hidden and one with only the shadows. I guess I was expecting to much ease from being able to just generate one EXR that made everything work beautifully. but two separate plates like this this seems to be working in AE fine.

[beenyweenies]

Thank you for the additional information! So Also just to clarify, when you are rendering the plates from vray do you set gamma to 2.2 for the renders or leave it at 1 using this workflow? I assume 2.2 right so that you don't have to do a gamma adjustment in AE?

For color mapping, I personally use Linear Multiply, all multipliers left at 1, gamma 2.2 and the mode set to "don't affect colors, adaptation only."

This is a fully linear, straight forward setup, so the rendered frame sequence does NOT have any color mapping or 2.2 gamma baked in. The 2.2 gamma adjustment to your frame sequence happens when you set After Effects to 32 bit and tell it to use an sRGB working space and to linearize the workspace. Those settings maintain the linear benefits of your frames under the hood, but displays them with an sRGB profile, aka 2.2 gamma. With this setup, you do not need to do anything manually to get things looking correct.

If you tend to use other mapping types like Reinhard, you would use the "colors only, no gamma" setting in Vray mapping setup.

[beenyweenies]

Actually if you render out in openEXR, ae interpret them as floating point 32bit even if they are rendered out in 16bit half float, you can see it in the project bin when selected.

Thanks for the info, I've been exclusively working with 16 bit TIFFs up to this point so I had no idea. We're finally moving to EXRs in the coming weeks so that's good to know!