has anyone had success using render layers to do a full composite in post, similar to the tutorial here:
i’ve tried to follow this but cannot get the image in PS to look anything close to a regular render in VfR - i feel like i’m missing a step, possibly involving linear workflow. any advice appreciated!
thanks!
so does anyone use this workflow with VfR?
Can you show us what is the difference between the final-composite and the beauty-pass from V-Ray ?
Compositing Render Layers procedure is the same regardless of tje 3D-platform - so if this workflow works fine for Max it will work for Rhino as well.
The linked example is much to complex for starting with layers. I tried a simple test - a plane with a sphere above, only a HDRI env. Output channel: RGB color and GI. The two channel should be same looking, but it didn’t get it, the GI channel looks like without gamma correction. How it should be working? I tried different output channels, but nothing shows the right result for the simple test.
A simplified tutorial (HDRI env + light) could be great.
… I found this basic tutorial:
http://www.jlynson.com/basic-renderpass-compositing-in-vray/
But I don’t get the right composited image. Could be great if some one could show an example done per VfR. For example I don’t found the right gamma settings.
An other problem is that Photoshop doesn’t support layers in 32 bit mode. Is there a free alternative?
Here are some notes I took during a weebinar on a subject:
Channels to save in V-Ray (some may not work correctly depending on the V-Ray version you’re using)
-Contribute to image pixels:
--Diffuse
--Reflection
--Lighting
--Etc.
-Describe an aspect of the scene:
--Normals
--Z-Depth
--Velocity
--Etc.
-Help in selection:
--Render I.D.
--Material I.D.
--Node I.D.
The image is made up of three components: illumination + reflections + refractions
For the first example we will use these channels:
-Reflection
-Refraction
-Lighting
-GI
No gamma correction on output and no color mapping:
-Global switches – Gamma correction – Unlock – Output = 1 (input still 2.2)
-Image sampler = Adaptive subdivision – Min = 0, Max = 2
-Remove AA filters
-Color mapping – Linear multiply – 1 – 1 – Clamp output and sub-pixel deselected
Rectangular lights must not have “Store with IR enabled”
Render appears dark because of no gamma correction, click on “sRGB” in the V-Ray frame buffer.
Save render in EXR.
To quickly save all the channels:
-_SaveRenderWindowAs “path/image.exr”
In Photoshop, new image in RGB 32 bits.
We use two blending modes in Photoshop:
-Addition = Linear Dodge (add)
-Multiplication = Multiply
Layers layout:
Reflection Linear dodge (add)
Refraction Linear dodge (add)
Lighting Linear dodge (add)
GI Linear dodge (add)
At this point the order doesn’t matter.
→ Try to adjust exposure of the reflection channel (you can make a selection to get local control on the strength of the reflections).
To change the materials appearance, we’ll have to “split” our lighting passes.
RAW GI and RAW Lighting are the same as the GI and Lighting passes we used before, but without the Diffuse information,
Make a new render with:
Neutral color on the object(s) we want to change the material.
Channels to use:
-Diffuse
-GI
-RAW GI
-Lighting
-RAW Light
-Reflection
-Refraction
Re-render with different materials but with only the Diffuse channel active.
In Photoshop organize the layers this way:
-Reflection Linear dodge (add)
-Refraction Linear dodge (add)
-Light (folder) Linear dodge (add)
----RAW Light Multiply
----Diffuse Normal
-GI (folder) Linear dodge (add)
----RAW GI Multiply
----Diffuse Normal
You can now change the Diffuse layers with the (very quickly computed) alternate renders or alter this layer in Photoshop.
By default, the lighting contains all the lighting information. But we can split each light to a separate pass (MultiLight) so we can adjust the strength of the light as well as its color.
For first pass, in VRay Global switches – uncheck Lights under Lighting. Open the GI channel from this pass in Photoshop.
Make sure “Hidden Lights” is unchecked.
Under Environment, uncheck GI (Skylight).
Make one render for each light, with only this light active, and save only the channels Lighting and GI.
Enable all lights and make a render pass to get the channels Reflection and Refraction.
In Photoshop:
-Reflection Linear dodge (add)
-Refraction Linear dodge (add)
-Light 01 (folder) Linear dodge (add)
----GI Light 01 Linear dodge (add) – (adjust exposure here)
----Lighting Light 01 Linear dodge (add) – (adjust exposure here)
-Light 02 (folder) Linear dodge (add)
----GI Light 02 Linear dodge (add) – (adjust exposure here)
----Lighting Light 02 Linear dodge (add) – (adjust exposure here)
-Light 03 (folder) Linear dodge (add)
----GI Light 03 Linear dodge (add) – (adjust exposure here)
----Lighting Light 03 Linear dodge (add) – (adjust exposure here)
-GI Env.Only Linear dodge (add) – (adjust exposure here)
-Light (folder) Linear dodge (add)
----Raw light Multiply
----Diffuse Normal
-GI (folder) Linear dodge (add)
----Raw GI Multiply
----Diffuse copy Normal
To try different materials, change “Diffuse” and “Diffuse copy” (in Photoshop or re-rendering only these channels with new materials)
Micha - thanks for confirming that this is not obvious. Svetlovar, I just got swamped but will try to post by end of the week. Mgibeault, that looks pretty much like what I was doing but ill try it again following your notes. Have you had any success with it?
if we get a solution ill post a tutorial as ill be making one for my students.
Yes, of course it works! I’m using this from time to time and it works fine.
Thank you for the notes mgibeault. I get a quite good result now. 
My current problem is that PS CS4 doesn’t allow to use layers in 32bit mode. Doe’s somebody know a free alternative?
Hi Micha,
If you have After Effects CS4 as well, try that, it allows for advanced compositing.
If I remember correctly from the V-Ray for Max forums (you could search there), it was even better than Photoshop, or less confusing, as it was adding/multiplying layers with an appropriate naming convention (instead of “linear dodge” for example).
This is great, can’t wait to try it! Thanks mgibeault!
Thank you for the hints.
I found, that Gimp should do it, but the EXR plugin crashs here at the moment.
An other solution for little money could be Photoline
At Photoline I got the layers working.
this worked perfectly! i’m sure i had the color mapping and image sampler settings wrong before. thanks so much for the help!
one nagging question though … with AA turned off, i get aliasing, all over the place. how is this supposed to be removed? presumably in PS, but how?
it’s really noticeable on the lights and in the stripes on the bed…
Yes I know it’s a problem. I think that if you enable AA you’ll have slightly different results for each layer and they won’t play well together.
In Photoshop, you can try filter->gaussian blur or edit->Fade Gaussian Blur and choose Fade Mode to Color.
Maybe an Unsharp Mask filter could work in some situations.
An other problem is, that the diffuse color channel shows no pure colors if a fresnel effect is used:
Hm, that’s kind of disappointing. Is the same AA problem happening in 3DSMax or is this a VFR issue only? Seems like so much added hand work it’s not really worth the trouble, yet seems to be a popular workflow in max. Micha, that’s also weird, but if you turn off trace reflections in the material (under “options” not reflection layer) then it seems to render correctly for diffuse layer. Not so helpful I guess as you’d have to do it manually for each but I guess it’s better than having to substitute all the reflective materials. A little.
Did you try to render with AA? I see no problem with AA. If I understand right, the AA sampling is determinated during the final pass and it’s the same for all channels.
An other workaround to get the right diffuse color could be rerender the scene with global disabled reflections. But this isn’t a good solution. I hope the support team can help.
i was just following instructions =) i’ll try it with AA - mgibeault did reply above that it could cause trouble, but i’ll give it a shot.
i tried to disable reflections in global switches, it actually didn’t change the diffuse layer for me (i’m not using the nightlies)
It may work with AA on but in some cases you’ll get strange outlines around bright objects when you compose the layers.
Since the aliasing problems are usually on lights or specular highlights, I cover this with a white blurry halo.
