VRay Render Elements jagged edges - The color mapping option INSIDE elements

[Morne]

Hi

With the exception of bright jagged edges, I'm getting an excact match to my VFB when comping these elements in photoshop:
VRayDiffuseFilter
VRayRawReflection
VRayRawReflectionFilter
VRayRawLighting
VRayRawGlobalIllumination

It seems to be something with the Lighting or GI pass. Almost asif the geometry is 1 pixel bigger in these passes.
Also I've noticed there is a check box inside the various elements for "color mapping" and by default some of the color mapping is on and others is off. So which should it be?
Default?
On?
Off?

Is the default some on some off the reason for the jagged egdes?

[vlado]

Can you show some image example?

Best regards,
Vlado

[Morne]

Not yet
But I think I've track it down to either 1 or 2 problems (or both)

1: The 1st Problem is the diffuse element sees to be the culprit being almost 1 pixel larger than everything (creating a halo effect after comping)

2: I only have photoshop CS3 Standard (not extended) So when I open my exr channels I have to first bring it down to 16bit from 32bit before I can add layers etc.

Will try in Max's Compositor 2011 (Toxic) and report back. Lets hope Toxic can handle full float 32bit files

[Morne]

Same result with CS3 Standard and Compositor 2011 (Toxic 2011)

So the comped version it not 100% same as the vfb in colours. But 1 thing at a time. I 1st want to get rid of the halo effect around my objects. In the attached images you can see the original vfb image on the right and also the comped version left (quality my be lower due to jpg compressions)
I save the vfb as vrimg then extract from this file single channel exr's from the various elements.

Then I comp it like so: (from bottom up)
Group - Add
--VRayRawRefraction - Multiply
--VRayRefractionFilter - Normal
Group - Add
--VRayRawReflection - Multiply
--VRayReflectionFilter - Normal
Group - Add
--VRayRawGlobalIllumination - Multiply
--VRayDiffuseFilter - Normal
VRayRawLighting - Multiply
VRayDiffuseFilter - Normal

[vlado]

You will get similar issues whenever you multiply elements together.

Rather than using the raw elements from V-Ray, you can try calculating them from the regular element and the corresponding filter (e.g. rawRefraction=refraction/refractionFilter; rawReflection=reflection/reflectionFilter; rawGlobalIllumination=globalIllumination/diffuseFilter).

Best regards,
Vlado

[Morne]

huh? is this / devide with each other? or was this just quick to type and you mean multiply with each other?

I'm getting the same result whether I use RAW or straight. I get jagged halo edges and also the colour doesn't match. I get the same result using either Photoshop CS3 standard or Toxic 2011 (Composite 2011)

All I need is a workflow that gives me diffuse, gi and light, reflection and how to put it together.
Now I'm using:
diffuse multiplied with light
then add on top of this diffuse multiplied with GI
then add on top of this reflection

The colour DOES NOT match (ie it comes out about 10 or 20 % brighter) and also the jagged edges story.

I'm using pretty much the workflow from Chris Nichols's tut on comping and also other similar tuts.
When adding each element in max I leave each element's default setting for color mapping either on or off depending on the default for that particular element
I'm rendering Linear with gamma 2.2 set in max and in color mapping I have "Don't effect colors (adaptation only)" ticked

Any thoughts anybody please?

[vlado]

I meant divide, yes. Once you get the raw element, you can use it in the same way as the one from V-Ray (e.g. multiply it by the respective filter element). The advantage is that now you can do any color corrections that you like, and you won't get edge issues.

Best regards,
Vlado

[Morne]

I think I was busy editing my message when you replied.

So now I'm confused which one gives the advantage for color correction without edges? The raw element or the straight?
Also please check this way of comping:

"straight"light/ diffuse
then add "straight"GI/diffuse
then add "straight"reflection/"straight"reflectionfilter

and this will give same as vfb without edges?

any ideas how I can / in photoshop or Composite 2011? Neither seem to have "divide" blending mode

[Donald2B]

Rather than using the raw elements from V-Ray, you can try calculating them from the regular element and the corresponding filter (e.g. rawRefraction=refraction/refractionFilter; rawReflection=reflection/reflectionFilter; rawGlobalIllumination=globalIllumination/diffuseFilter).

Use the formula that Vlado posted earlier in this thread. I am pretty sure that Photoshop does NOT have a divide option. I use Nuke for all my compositing, so I haven't really taken a look at Composite 2011. I will try to do it later on this weekend to see if I can get it to work in there. I would think it should be possible, but I haven't tested it out.

[Morne]

anybody have any clues?
I'm still not getting the correct result

[sandykoufax]

I've heard that photoshop cs5 has divide blending mode.
In case of Composite 2011, how about try 'Math Ops' node.
It has 'Div' blending mode.

[Morne]

MAX 2011 scene - VRay 1.5SP5

OK Seems I've been doing it correct all along. Toxic DOES NOT Work for me. CS3 Standard DOES NOT Work for me.

What does work is CS3 Extended - go figure.

[pg1]

it is not possible to do this in photoshop
if you want to understand the process better
http://www.fxguide.com/forums/showthread.php?t=6838
the workflow is in nuke but the theory is the same for every image