Intel Denoiser + alpha awareness!

[HowardDay]

Currently if you have a semi-transparent object and are using the intel denoiser, it doesn't seem to be aware or denoise anything *behind* the semi-transparent object. This leads to windows having very noisy interiors, when if the glass is removed, it works perfectly well. I hope this is something easily resolved. (for what it's worth, the Nvidia denoiser seems to handle this well, but I much prefer the end result/quality of the Intel denoiser overall)
Thanks!

[zero-13]

Not to derail but I've been trying to find a comparison or just an overall opinion regarding the default V-Ray denoiser and the more recently added Intel option?

I tend to only use the Nvidia denoiser for IPR and the default denoiser works so amazingly well for final output.

So just curious besides this deal-breaker issue with Intel denoiser, how do you feel about the default denoiser vs the Intel implementation in overall quality, specifically in regards to animation?

Thanks!

[^Lele^]

Currently if you have a semi-transparent object and are using the intel denoiser, it doesn't seem to be aware or denoise anything *behind* the semi-transparent object. This leads to windows having very noisy interiors, when if the glass is removed, it works perfectly well. I hope this is something easily resolved. (for what it's worth, the Nvidia denoiser seems to handle this well, but I much prefer the end result/quality of the Intel denoiser overall)
Thanks!

You can fix this if you set the glass shader to "affect alll" channels for its refraction.
This is because then the data channels will show through (f.e. normals, diffuse filter, and so on).

This is the same as with the V-Ray denoiser.

[HowardDay]

I'm pretty certain I have already checked that option. :/ Let me do some tests and get back to you!

[HowardDay]

Okay, I definitely have the "affect all" channels set on the glass materials. Unfortunately I'm still seeing the same behavior. :/
Here's some examples:
The glass area raw and then denoised:

You can see that *maybe* some of the edges of the dome are slightly denoised, the overall effect is missing almost entirely.
Just to show that I'm not crazy, here's another area of the render showing the denoiser clearly working great:

That light *also* has a glass dome over it, so there's a small amount of noise still present at the center.
If it helps, I'm using the latest nightly build, in GPU/RTX rendering mode.
Thanks!

[^Lele^]

Umh, what do the other render element look like for those bits?
It could be because the stuff behind glass is black in some of the REs, or maybe an OIDN issue, either with the version, or us: i haven't tested it extensively enough yet to know for sure.

[HowardDay]

Well I have a bump/normals, diffuse, global illum, and alpha pass available. Here they are:

The end result is this:

As you can see, the circled areas remain noisy - the albedo of the glass is set to white, but it is rendering as black in the diffuse pass.
I hope this helps.

[^Lele^]

Umh, it'll make rendering slower and noisier, but perhaps try using a diffuse+selfillum vraymtl (same color in both) instead of a lightmtl.
That should carry across to the diffuse.
Should you not need the lighting (either because you can fake it with an actual light, or because you can do without), use a vrayMtlWrapper to not emit GI from the self-illuminated material (EDIT: No need for this anymore, it's enough to *not* tick "GI" in the vrayMtl), which would speed up rendering.

[HowardDay]

..for the glass? Or for the things the glass occludes? Because this is entirely universal - it doesn't matter what type of mtl is on the objects *behind* the glass, the non-denoising behavior is the same. I've had like trees and walls and stuff back there and it wont denoise it. :/
The glass already a full vraymtl, and adding self-illum to it will just blow it out...

[^Lele^]

Eh, i missed the reply, and edited above.
For the light materials behind the glass, it's them being black in the diffuse, that i can see.

[^Lele^]

Umh, it's truly odd, as i can't seem to repro the behaviour for OIDN.
It looks nigh identical to me wether i set the glass to affect all channels or not, wether i use a lightMtl or a vrayMtl for the emissive parts: it always attempts a denoise and there is no discontinuity behind or around glass, that i can see.
I'd ask you to send me one little part of the offending scene to test locally, if at all possible.

[piotrus3333]

this is rtx, not cpu. classic issue with alpha behind refractive obj. do your glass with blend mtl. separate refractive and reflective components. helps with alpha, might fix OIDN as well.

[HowardDay]

Thanks for the idea Piotrus - but unfortunately, no dice. It still has the exact same lack of denoising on the background objects. Lele: Here's the part of the scene - I tried to trim down the unneeded textures, so you might get a warning on that, just ignore it.
https://drive.google.com/file/d/1wlr...ew?usp=sharing
I am using the RTX renderer as Piotrus mentioned.

[^Lele^]

Allright, i think this isn't solvable via REs.
The fact that the glass returns a black diffuse is unavoidable (refraction will always have precedence over the diffuse.): you use it to scatter the light behind it, so you can't quite hide it, or override it in any meaningful way.
The V-Ray denoiser does a slightly better job at it, but then the eddges of the frame tend to have noise left pretty much intact (perhaps manually changing kernel size and strenght may help, perhaps not.).
You're likely better off using the NVidia one, or other non-RE sensitive denoiser (i.e. one that works on the beauty pixels only.).

This said, if you abandoned physicality, you'd get much quicker renders and a properly denoised image.
For example, make a standard V-Ray material, set the diffuse to the blue you need. In the self-illum channel, use a comptex with the blue emissive color in one slot (using a vraycolor map) and an AO (VrayDirt) in the other, set the comptex to multiply.
Turn on GI in the emissive part of the shader.
Assign the shader to the *glass*.
Fiddle with the AO parameters until it looks decent enough.

It isn't the same as seeing the light and underlying structure through a frosted glass, but it'll look quite similar (add more complex maps if you feel you really need the various darkenings due to the underlying structure), and denoise just fine.
See the two attached samples, original, and modified as explained above (it's a quickie, you'll get much better results spending a bit more time on it.).

Lastly: Is there a particular reason why you have your max and scene set up like you have? You're technically not under LWF, but not using ACES either, that i can see.
Not being under LWF will lead to max making a mess when saving stuff out (from the VFB, or directly on disk.), among other things.

[HowardDay]

Allright, i think this isn't solvable via REs.
The fact that the glass returns a black diffuse is unavoidable (refraction will always have precedence over the diffuse.): you use it to scatter the light behind it, so you can't quite hide it, or override it in any meaningful way.
The V-Ray denoiser does a slightly better job at it, but then the eddges of the frame tend to have noise left pretty much intact (perhaps manually changing kernel size and strenght may help, perhaps not.).
You're likely better off using the NVidia one, or other non-RE sensitive denoiser (i.e. one that works on the beauty pixels only.).

This seems really odd to me, since you have access to a mask of where the refractive areas are *as* a RE. Can't you use that knowledge to make sure the denoiser runs on those areas? Otherwise you're pretty much just leaving a pretty big exception to the use of any sort of denoiser pass in renders - IE "Denoise only works if you have no refraction, otherwise it's broken". That doesn't seem like the best option to me, at least.

I know I can do a non-physical version of this. I could also just render it in scanline, and it wouldn't even need the denoiser. The point of having VRay and using it is to *do* stuff like this, and not have to work around a problem to fake it.

Lastly: Is there a particular reason why you have your max and scene set up like you have? You're technically not under LWF, but not using ACES either, that i can see.
Not being under LWF will lead to max making a mess when saving stuff out (from the VFB, or directly on disk.), among other things.

The Gamma settings in 3dsmax and Vray have always been broken for me. I get hugely inconsistent results, and cannot get the renderer to just save out an EXR with the proper settings. I can save each frame out of the VFB with some success, but that's not a solution for animations. The only reliable overall solution I've found is to turn all that stuff off, and just work in AE to Color Correct the result. If you can set it up to have the proper linear workflow and send the max file back I'd love to see it! I have lost weeks of time fiddling and following tutorials to have it not actually work, though. :/

Thanks.