Still white edges around the trees

[Crayox13]

Hi guys,
I used defringe in PS but still these strange white edges remain. On the far right top corner you can see one leaf is half white, so it seams it's more than edges of the objects. I think this started when I started to use LWF and/or *.exr.

[jozef_chvojka]

HI,

well it looks like you have transparent leaves and rendering on a bright background... maybe just try to render with this sky picture as a background plate, or set your background color similar to that you have on your BG image

[3DMK]

bitmap issues

Hi,

One thing to be aware of is the way bitmaps get averaged at a distance. Objects that are far from camera and appear small will have their bitmaps averaged to gain the colour. If your tree is only covering 25% of the bitmap and the rest is white [background] your maps will end up white in the distance because 75% of the bitmap is white!

Does your tree map have a white background??? Make sure your bitmap background is an averaged colour of the tree.
To do this simply duplicate the tree layer in photoshop then apply '''Filter/blur/average. This will average the colour of your tree into a solid colour.

Also on using bitmaps with alpha channels

Set the bitmap 'BLUR' parameter from the default of [1.0] to [0.1]
This will give you nice crisp edges to your leaves.

Hope this helps

[Crayox13]

Guys,
Thanks for help.
@3DMK These trees are from Evermotion, and they really did a great job matching their textures. When I was using different trees before, I saw this what you mentioned, bad textures that showed weird edges immediately in VFB. When I render this image it blends perfectly against vray sky which is gone when I save *.exr and load in PS.

When I defringe, those straight edges, like that of building, are fixed, but trees get 'broken'.

I'll try blurring alpha, and see if that helps.

@jozef
That would work I think, since, as I said, in VFB everything is perfect, but it's just not practical to use background in 3D. Since I have several cameras and views batch rendering.

[jozef_chvojka]

I agree.
Try to match your render background colour with the average of the sky then.
I think that you do use much brighter render background color comparing with the clouds that are just behind your trees.

[3DMK]

Check Leaf material Refraction-causes your defringe problem

When I defringe, those straight edges, like that of building, are fixed, but trees get 'broken'.

I've noticed that evermotion often use Refraction on the leaf material, Which causes your defringing problems
Check to see if your Leaf material has any refraction as photoshop will not defringe full transparency properly. Defringe is only designed for fixing Edge transparency and you will see this more than ever if you try to defringe an image with a glass material...it just doesn't work.

You are much better off using Vray 2-Sided material instead of refraction for tree leaves as it will give a much better look in a fraction of the render time & you get a perfect composite every time.

Hope this helps

[3DMK]

Photoshop CS3 & .EXR problems

A friend of mine mentioned to me recently having some wierd artifacts when changing from CS2 to CS3 & using .exr files.
He said that he never got this in CS2 [needless to say he kept CS2 installed just to convert the .exr to 16bit photoshop files as he never found out why it was doing it yet]
Apparently the problem was that Often you would get white patches such as in your image and the wierd thing was that those areas were almost black in the render in max??? Very strange.

Cheers

[simmsimaging]

I have also had some problems with .exr and white fringing on objects, but it was only happening when rendering with RPM and backburner. It did not show in the VFB, only when I opened in Photoshop so I think it had something to do with pre-multiplication, but I could not figure it out for sure. Rendering to .hdr instead got rid of the problem for now. Did you try another format to see if the problem persists?

b

[3DMK]

I am using CS2 & have never had any problems using .EXR files. My friend says he is convinced it is some bug in photoshop CS3 not the render that is causing problems.

Sorry I can't help more.

[Crayox13]

Guys, still problems.

If I use hdr, it doesn't save alpha channel. Never worked with hdrs before. I go to MAX frame buffer, select alpha, save, and it saves rgb, am I doing something wrong?

Also, since I read exr will now be able to store elements in its layers, I would REALLY like to use it. Any ideas that CS4 will have that problem fixed?

[trick]

One should also check whether the leaves on the Evermotion trees have both an opacity map and a hybrid translucency map. If you show the leaf material in the material editor you will notice strange fringing around the opacity mapped edges when you switch the translucencymap on/off. As alreay said, remove the translucency mapping and use a VRay2SidedMtl.

[Crayox13]

Yes, I believe that was one of the problems, but now I have an exterior where I use vrayfur for grass, I render this in a separate pass, and then compose. Since grass is normal diffuse material, no refl or refr the problem remains, I guess, still in CS4.

[trick]

Can you show the material settings and if you use any sampling on the textures ? Also an enlarged material preview ?

[Crayox13]

What you see is caused by the grass layer, not the bush. Maybe that wasn't clear.

Grass - vrayfur is the simplest vray material possible, it only has diffuse grass texture. Nothing else.

I only use VraySincFilter for antialiasing, size 2.0

[Crayox13]

What about hdr? Why is it that when I clicked alpha channel in MAX frame buffer and saved it, it saved rgb, and not the black and white alpha image that was displayed?