Still white edges around the trees

[trick]

Maybe not the solution you're seeking, but have you tried TGA (32bit/pixel, premultiplied alpha) ?

[Crayox13]

Hm, no, but if it works, it IS a solution, right now I'll take anything

I switched to exr only because of the color depth of 32 bit, but come to think of it, 32 bit tga should have the same behavior in PS when I adjust exposure (if the render was too bright or too dim).

Edit: forgot to mention, that I would like to keep exr in the future since it should be able to save elements all in one file - layers. That's what I read, now I have to check if it's true. Any info on that?

[Crayox13]

Tga is a bit better, trees are ok now, but the pass with grass is still not "perferct" It's a very complex pass in terms of its edges, very tiny pixels, etc.

Should I not use antialiasing? I read before some of you are not using it.

Also, just realized that tga is 32 bit because of alpha I guess, it doesn't store data like exr, I can't get details from burned areas. So this is really a temporary solution.

[trick]

Just some idea's, don't know if they work:

Can't you render to VRIMG, convert to EXR for normal and mask layers, and convert to tga-32 for the mask ?
Try out PDPlayer with VRIMG/EXR to see if compositing is better ?
Render the grass seperate but straight on the previous render as background ?

TGA-32bit/pixel has a more precise color depth which may be of help with soft colortransitions/gradients, when using printers or monitors that support more then 8bit/pixel. It's not the same as HDRI ! But why do you need the grass to be HDRI: can't imagine that you would have exposure problems with the grass !

EDIT: not TGA but TIF-16bit has a more precise colour depth. TGA32 is just 8bit color + alpha !

[rivoli]

TGA-32bit/pixel has a more precise color depth

actually tgas don't have any more precise colour depth than any other ldr format. 32bit, in this case, is misleading. doesn't mean actual colour depth, it just means it saves an output of four 8bit channels: rgb and a.
if you need a full float output you'll have to save as exr, hdr, tif..

as long as exrs and channels go, you can certainly save multichannels exrs from vray, the only problem is not all applications are capable of dealing with them. actually only few of them are. photoshop and after effects, for example, can't read any channel apart from rgba. in order to do that, you need a plugin such as proexr. which I'm not sure it's been compiled to work with cs4 yet.

the compositing problem may depend on premultiplication, which ps doesn't handle properly. you may try rendering to tga with premultiplication unchecked, it'll give you an awful output, but it should work correctly once you mask it.

edit:
I forgot, if you're going to save to a premultiplied output, remember to render the grass against a black background.

[Crayox13]

First, yes, no need to have grass exr.

Second, never used PDplayer, just never had time to try it. I'm gonna try it now though, hope it's not too complicated

I use vraysky for ambiance, and with vraysun. When I render the grass pass, vraysky is the background. How do I put a black background but keep the sky?

[rivoli]

uncheck "use map" in max's environment, and drag the vraysky map in the vray environment. that's about it.

[trick]

actually tgas don't have any more precise colour depth than any other ldr format. 32bit, in this case, is misleading....

Yes, sorry. Was confused !! Indeed 16bit Tif has a more precise colour depth.

[Crayox13]

Thanks for all your help, it's solved. It was a combination of problems.

Yes, Evermotion did use refraction, and sometimes translucency materials also. I've removed those.

Also, I now render against a black background while keeping vraysky for GI Thanks rivoli