Recommended workflow when projecting with image based lighting

[Fredrik Averpil]

Hello, I am coming from a mental ray and renderman background, now learning V-Ray and trying to get my head around camera projection as well as fast image based lighting/reflections.

Short version; this is what I am trying to do:




Long version:

I have:
* An image sequence (filmed material), locked off camera of an environment
* A 3D object, made in Maya, which I want to place on the ground of the filmed material and integrate it into the environment

Now, what I want is to place the 3D object in the filmed material and have it integrated. I'm now trying to figure out how to:
1. Project the image sequence onto dummy geometry
2. Create a environment dome around my scene

Clarification:
- I want to project the image sequence onto dummy geometry so that I can have the 3D object (e.g. a car) cast shadows/reflections on the dummy geometry (e.g. the ground). I'd also like the dummy geometry to be casting shadows and be reflected in the 3D geometry as well as be included in occlusion and GI calculations.
- I would map an HDRI image onto the environment dome in order to cast reflections from it onto the 3D object with image based lighting as an option.

Problems/Questions:
I have now tried to create this workflow but I really can't figure out what the proper projection setup would be in V-Ray for Maya. Which shader should be used in conjunction with Maya's "projection" node?
What node should ideally be used when creating a reflection dome which could also be used for lighting?


Thanks in advance,

[sv]

Fredrik,
I'm no expert in vray for maya but have been using a particular flavour of PRman for maya for over a year and used vray for 3dsmax for many years before. The VrayDomeLight is probably what you are after for a reflection dome and can also been used for (very impressive) image based lighting too.
Ambient Occlusion I used to calculate via a shader in vray - the equivalent of trace distance you set in the shader - not sure how they are doing it now in vray 2.0 - you might be able to output occlusion as an AOV / render element... sure someone can confirm...

[jacksc02]

I feel for you SV.... Really

[aweidenhammer]

Correct me if I'm slightly wrong but here's my method...

Once you have your camera and proxy objects created/aligned...

create a vrayLight material. Assign a projection (Maya node) to the color slot. Assign the plate image to the projection node.
Set the projection node to perspective, and choose your camera. Once you assign your material to your proxy objects you can see the texture projected in the viewport to check any issues.

Create vrayDomeLight. Assign the matching hdr to it and use the placement node to align it close to your plate. You'll need to adjust the intensity by eye to match the hdri to your plate.

Now the trick. Apply a single vrayObjectProperty to the proxy objects. set that VOP to:
matte surface on
alpha contribution -1
Shadows on
Affect Alpha on

This VOP will override the shader's inability to receive shadows in alpha.

Make your domelight invisible, and render. I find you have to up the sampling quite a bit to get clean shadows in this method.

If anyone has a better method, please let me know. But this is pretty straightforward, and has worked for me. I plan to do a training video on this subject soon, so I'd be happy to hear any other tips, or feedback on this before I record something that is incorrect.

Andrew

[Yannick]

I usually do like Andrew described, but I use image plane that I then switch off for my final render. The backplate in add then in post.
Vray Dome light for HDR lighting and VrWarpper material to catch the shadow, camera projection for reflection.

yannick

[Fredrik Averpil]

Hey guys and thanks for the tips!
I've been busy for quite a while but I am now back up on the horse and ready to give this one another go.

I've done just as you guys suggested and it seems to work nicely; I'm projecting my backplate onto dummy geo using the VRayLightMtl, and wrapping all of that into the VRayMtlWrapper, which is applied onto the surfaces. Then I use the VRayLightDome to cast reflections as suggested as well. By adding in an additional light source(s), such as the VRayLightSphere, I get really nice shadows from the sun's direction.


Imageplane as reference
When shading and test rendering like this inside of Maya, it kind of gets a little tedious to save the render out and comp it onto the background all the time. Instead, I'm thinking of just adding in the backplate as an imageplane, which can be turned off whenever I want to render out for comp. I've mapped the backplate as a texture (allowing for linearizing an sRGB backplate by using a gamma node). This should work without unexpected behaviour and won't contribute to GI or any other stuff, right?


Collecting the reflections on dummy geo
One question though; as I use the VRayLightMaterial in the wrapper, the dummy geo won't be able to receive any reflections. Let's say for example that the ground on which these primitives are sitting on is wet rock. There are no reflection settings in the VRayMtlWrapper.
How would you guys go about such a case?
I'd like to avoid setting up a separate layer/pass for this.


Possible RT bug (or RT limitation)?
I also stumbled upon a problem when working with RT using this "workflow". It seems that RT can't handle the VRayMtlWrapper and/or VRayLightMtl correctly. Please see the attached image below. It's the same scene rendered first in RT and then using the standard V-Ray renderer. Build 14204 on Mac OS X Snow Leopard, Maya 2011 64-bit.



Both renders above are from the same scene, I just switched between RT and the standard V-Ray renderer.
As you can see, the alpha is wrong and the ground plane is visible in the rgb. Another issue I get is that the VRayLightSphere's shadows are not being shown at all.
I tried to map the VRayLightMtl directly onto the ground plane and the shadows did still not show up on the ground from the VRayLightSphere. When I mapped a standard VRayMtl onto the ground, the shadows caused from the VRayLightSphere appeared correctly though.
Could this be a bug or is it a limitation of the VRayLightMtl and VRayMtlWrapper?

[vlado]

One question though; as I use the VRayLightMaterial in the wrapper, the dummy geo won't be able to receive any reflections. Let's say for example that the ground on which these primitives are sitting on is wet rock. There are no reflection settings in the VRayMtlWrapper.

In fact, the VRayMtlWrapper can show reflections from the base material (there is a reflection slider to control their strength as well). So as a base material you can use a VRayBlendMtl material in additive mode where the base is your VRayLightMtl material, and the coat is a reflective material.

I also stumbled upon a problem when working with RT using this "workflow". It seems that RT can't handle the VRayMtlWrapper and/or VRayLightMtl correctly. Please see the attached image below. It's the same scene rendered first in RT and then using the standard V-Ray renderer. Build 14204 on Mac OS X Snow Leopard, Maya 2011 64-bit.

In general the RT engine does not provide a very accurate preview of matte materials. Unfortunately this is not something that is easy to fix.

Best regards,
Vlado

[vlado]

I tried to map the VRayLightMtl directly onto the ground plane and the shadows did still not show up on the ground from the VRayLightSphere.

The VRayLightMtl material does not compute any lighting, so it will not be able to show you any shadows.

Best regards,
Vlado

[Fredrik Averpil]

In fact, the VRayMtlWrapper can show reflections from the base material (there is a reflection slider to control their strength as well). So as a base material you can use a VRayBlendMtl material in additive mode where the base is your VRayLightMtl material, and the coat is a reflective material.

I actually just tried this – didn't expect it to work – but it did!
A little slower to render ... but it works!