Live Action and CG integration

[wim.mutsaers]

I just dont understand why is so difficult to do. Probably there is some technical limitations for the render engines make this effect.

[vlado]

I just dont understand why is so difficult to do. Probably there is some technical limitations for the render engines make this effect.

In theory, the only perfectly correct way to do this is to render three separate images as in the above mentioned tutorial on spot3d and there is no way around that. In practice, render engines try to achieve the matte result in one render by using various tricks or partial solutions, f.e. by making the simplifying assumption that matte shadows can only be a result of illumination from an actual light source.

So to answer your question, this is not difficult - there is a working straightforward solution. It just needs three renders. (Of course, as explained in the tutorial, there are ways to speed up these renders).

In any case, we are looking for possible ways to implement a more correct solution for V-Ray (we need it for V-Ray RT anyways).

Best regards,
Vlado

[wim.mutsaers]

Well, Tnks a lot Vlado. That tutorial works perfect. I seted up everything in 3 render layers(normal, black and pure) and add other render layer to composite all the rendered images. So it works great everytime I render. It has to go in 3 passes but in the end works perfect.

Thanks again Vlado for the patience. If someone is interested I can post some images.


Cheers. This is closed and solved

[RobPhoboS]

Please posts some images, always useful !

I'm still stuck on this:

make sure that you override the GI environment with a black color.

Sorry to sound stupid but I want to ensure I understand correctly. Do you mean in the 'Environment' section of the Render Settings window, or do you mean in the material override for the matte ground ?
I've tried popping the same black checker texture in both but it doesn't appear to work for me.


griml0ck kindly sent me his approximate workflow in a PM, hugely helpful I just now need to sit back and try out what he mentioned, as he wasn't using the domelight like I was in my post.

[wim.mutsaers]

Ok, I will make some images to post.

[wim.mutsaers]

So, I have these 3 render layers:



Then in a fourth render layer I have an imageplane(texture option) comped with the rendered images like the tutorial that Vlado pointed.
It works great in any renderer, and perfect with vray.




Any questions, feel free. I hope that this can help.

[wim.mutsaers]

btw Rob, the environment in the render options is set all to black. I dont think it has any effect.

What's griml0ck solution? Tnks

[RobPhoboS]

Awesome, thanks for posting that, struggling with time at the moment, I should probably bring my laptop to work !
I'll just double check it's cool to post what he mentioned

[jasonhuang1115]

I did some tests over the weekend. I was able to get a 1-to-1 matched results between composite image and the beauty render by rendering 3 images following the spot3d tutorial. BUT, with this approach I seem to lose the ability of independently controlling the shadow, reflection, and GI caught by the matte surfaces (as all these are part of or baked into the background plate).

EDIT: I take this back. I found that by setting up a multimatte passes for matte surfaces and color correct the "Mask" (refer to the tutorial), I was able to control the shadows, reflection, and GI for the matte surfaces in post. Still it is not a typical workflow for live-action integration shots as we usually don't render the CG against the plate for final renders.

So, currently in Vray, is it possible to set up a render layer to extract matte surfaces received shadows, reflections, and GI effects (as a whole or as individual render elements) and comp them back to perfectly match the beauty?

[vlado]

Still it is not a typical workflow for live-action integration shots as we usually don't render the CG against the plate for final renders.

You don't have to render against the plate. In the tutorial, we never render against the original background. All the rendering is done with the proxy scene and the original background is only used in the final comp.

So, currently in Vray, is it possible to set up a render layer to extract matte surfaces received shadows, reflections, and GI effects (as a whole or as individual render elements) and comp them back to perfectly match the beauty?

You can get separate render elements with the shadows, GI, reflections etc, even without ever resorting to matte objects.

Best regards,
Vlado

[jasonhuang1115]

You don't have to render against the plate. In the tutorial, we never render against the original background. All the rendering is done with the proxy scene and the original background is only used in the final comp.
You can get separate render elements with the shadows, GI, reflections etc, even without ever resorting to matte objects.

I am trying to wrap my head around this. True in the tutorial the background is a proxy scene to match the actual background. But if the background is a photographed plate, I assume I should for example create a shadow catching plane, project the plate onto it, and set that as matte surface with vrayobjectproperties or vraymtlwrapper accordingly.

That means I cannot have a perfectly matched comp result to beauty by having the two renders below and the original plate:
1. a cg character pass that is against black background
2. a shadow pass that has all the information (shadows, reflection, GI) the matte surface plane received with the CG character not visible to the camera. (reflection and GI in RGB, shadows in Alpha)
3. original plate

Based on your explanation in post #17 in this thread, it's currently not doable. Or am I understand incorrectly?

[vlado]

I am trying to wrap my head around this. True in the tutorial the background is a proxy scene to match the actual background. But if the background is a photographed plate, I assume I should for example create a shadow catching plane, project the plate onto it, and set that as matte surface with vrayobjectproperties or vraymtlwrapper accordingly.

The tutorial does not use matte objects. You will have to create a ground plane, and you could project the background on it as the diffuse texture of a V-Ray material, although you could also just make the ground plane of the general color and reflectivity that you need.

That means I cannot have a perfectly matched comp result to beauty by having the two renders below and the original plate:
1. a cg character pass that is against black background
2. a shadow pass that has all the information (shadows, reflection, GI) the matte surface plane received with the CG character not visible to the camera. (reflection and GI in RGB, shadows in Alpha)
3. original plate

You can do that of course, if a character on a ground plane is all you need. In that case the matte surface option in either the V-Ray object properties or the VRayWrapperMtl material will work just fine. The original post, and my reply to it, refer to the much more difficult case of inserting a CG object into a closed CG environment where the interaction between the objects is more complicated than just a shadow, a reflection, and some minor GI.

Best regards,
Vlado

[jasonhuang1115]

Vlado,

It did not work in my test posted below in terms of perfectly matching the beauty output. Hope you or guys here can point out if I set up incorrectly. I set up three spheres and a plane set to matte surface via vrayobjectproperties in the scene. I project the plate onto the ground plane as you mentioned. An environment override is set in the Background texture slot with Mapping type of "Screen" for the plate.

To get the shadow pass, I set the plane's vrayobjectproperties to:
Matte Surface: on
Alpha Contribution: -1
Shadows: on
Affect Alpha: on
Reflection Amount: 0.3
GI Amount: 1.0
No GI on other mattes: on

There is something missing in the shadows. I post the comp tree below.
comp tree
difference

[vlado]

Can you post your scene?

Best regards,
Vlado

[jasonhuang1115]

There you go! The maya file built in 2012 and Vray 2.20.01 as well as the Nuke comp script.
CGLA_in_vray.zip

- Jason