Vray Unreal Quality Issues

[ionut_nicolae]

Hi, just started using Vray with Unreal and have some questions/problems with Baking maps, I can get better results and faster in other baking software right now, but maybe I am doing something wrong!

I see others on the forum complaining of toon like black outlines, as the lightmap padding or dilation is failing, and it looks to me like its failed to ignore back faces/interior surfaces where the rays are effefctively only hitting back faces, normally ( in other bakers ) this gets optimised out by back face tolerance percentage I believe. Example a large plane with a cube sat on top, where the cube intersects the floor would be black in the lightmap as no light hits it and can be judged to be interior and not visible to user as any rays hitting it will bounce and immediately hit a back face which should invalidate it and terminate any further bounce calculations and more importantly you can then discard the lighting on that pixel and perhaps instead dilate a nearby pixel that is safely within bounds and lit, so discard what would have been solid black and pull in a nearby lit pixel, this cures the problem of black lines around the edges. The alternative is having to render lightmaps at increasingly high resolutions, or being very meticulous and hand crafting meshes with no overlapping/hidden surfaces ( no thanks, life is too short ). Though I've seen even at high res it has black edges that look a bit like AO but might be this same bleeding/dilation issue.

2nd to this issue, the denoising isnt very effective, it looks a bit like compression artifacting and ends up leaving splots and crud all over the place, I would hope to see an AI denoise being used instead however that has to be carefully done as the denoising or AI can get confused around edges or close to invalidated/interior texels and either leave you with incorrect guess work or an ugly border where inside texture is nice and smooth and around the edge is noisy as the denoise is just masked away poorly.

I would like an option to bake without any denoising also or to control that so after a bake is done the denoise can be run seperately and differing strength can be tried without rebaking everything-or ideally it just always works well

I'll also add I'm looking forward to seeing the Directional Map and Light volume for dynamic items added.

I'm curious if it handles out of core memory with the GPU so if theres more data/large meshes than can't fit in memory it doesnt just crash?

thanks, Alex

Before and after denoise example:



Example of padding/dilation black edge issue:

[Alexander.Atanasov]

Hi Alex,

Sorry for the late reply.

I see others on the forum complaining of toon like black outlines, as the lightmap padding or dilation is failing, and it looks to me like its failed to ignore back faces/interior surfaces where the rays are effefctively only hitting back faces

The dilation parameter in the V-Ray Bake Settings tab refers to the amount of pixels to expand around geometry UV islands and not around shadowed areas or backfaces.

normally ( in other bakers ) this gets optimised out by back face tolerance percentage I believe. Example a large plane with a cube sat on top, where the cube intersects the floor would be black in the lightmap as no light hits it and can be judged to be interior and not visible to user as any rays hitting it will bounce and immediately hit a back face which should invalidate it and terminate any further bounce calculations and more importantly you can then discard the lighting on that pixel and perhaps instead dilate a nearby pixel that is safely within bounds and lit, so discard what would have been solid black and pull in a nearby lit pixel, this cures the problem of black lines around the edges. The alternative is having to render lightmaps at increasingly high resolutions, or being very meticulous and hand crafting meshes with no overlapping/hidden surfaces ( no thanks, life is too short ).

Can you elaborate on which "other bakers" are you referring to? In UE in internal and overlapped shadowed areas there is still "blackness" but they offset that shadow a couple of pixels inward and thus eliminating the dark edge seen from outside.

Though I've seen even at high res it has black edges that look a bit like AO but might be this same bleeding/dilation issue.

Yes, currently we recommend using high resolution for lightmaps as a workaround the problem as we are looking into this how we can achieve an inwards offset like the one UE does but no ETA at the moment.

I'll also add I'm looking forward to seeing the Directional Map and Light volume for dynamic items added.

Unfortunately I cannot give you an ETA on that but it is logged.

2nd to this issue, the denoising isnt very effective, it looks a bit like compression artifacting and ends up leaving splots and crud all over the place, I would hope to see an AI denoise being used instead however that has to be carefully done as the denoising or AI can get confused around edges or close to invalidated/interior texels and either leave you with incorrect guess work or an ugly border where inside texture is nice and smooth and around the edge is noisy as the denoise is just masked away poorly.I would like an option to bake without any denoising also or to control that so after a bake is done the denoise can be run seperately and differing strength can be tried without rebaking everything-or ideally it just always works well

We do not have that as an option for the moment but are looking into a way to expose more control of render setting during baking. As a workaround for the moment you can export light bake .vrscenes, that you can then open in Notepad++ for example, and modify parameters in it and render them in standalone as EXRs and then import them using the Process Atlases workflow. Or export the EXRs from UE using the Export EXRs option and then edit them in Nuke(in an upcoming version you will be able to modify them in Photoshop with the Exr-IO plugin) to your liking.

I'm curious if it handles out of core memory with the GPU so if theres more data/large meshes than can't fit in memory it doesnt just crash?

That will be an upcoming feature somewhere either later this year or around the start of the next.

Best regards,
Alexander