Grainy HDRI interiors with Global subdiv 3.3

[Pixelab]

Before 3.3, my HDRI domes were up to 300/400 subdiv to get smooth shadows (with no real increase in render time because less grain / less antialias) but now with 3.3 global subdiv, HDRI dome light are systemically undersampled for interior scenes.

I have to enable local subdiv & put my dome to 400 to get a nice result, but AFAIK this is not adivised with the new method (+ I have to increase mat subdivs to 100+)... So I was wondering if that's not a bug ? Ideally I'd like to keep the global subdiv DMC but increase only the dome subdivs x10.

You an clearly see the ugly fringe in the RGB, the RAWlight pass shows the culprit and the useless AA time spent on the undersampled direct light...



DMC Noise thresold 0.005 / AA 0.1 (enough for my taste, I go lower for final images)

The scene (HDRI gets trough the window only)



I tried modify resultion & adaptativeness of the light texture slot, but it's not changing anything (maybe I should try extreme values?)

Does using an EXR version instead of a HDR file would help ?

[joconnell]

Up your shade rate instead or if you've only got 1 thing in the scene that's definitely causing the issue, leave all at default so the shade rate is used for the majority of things, turn on use local subdivs and up the offending light / material!

[Pixelab]

Up your shade rate instead or if you've only got 1 thing in the scene that's definitely causing the issue, leave all at default so the shade rate is used for the majority of things, turn on use local subdivs and up the offending light / material!

How much would you try ? I'm already at 16

(thanks for your VIMEO tutorial, really useful !)

[joconnell]

Go for large values for the fun of it! Vray does some internal calculation things to reduce the shade rate on objects that don't need it so you can try some big numbers and it won't kill you - do a small section and try 24 / 48 and so on - it'll get you away from the aa sampler trying to clean up that light.

[joconnell]

As an aside - the vray guys came up with the defaults of 1/24, colour threshold of 0.005 and a shade rate of 6 by profiling a lot of scenes. It might be worth getting them to look at yours as they're constantly trying to make vray more automatic and intelligent. Your scene is quite specific in terms of where the noise comes from so a general average setting probably won't be optimal but they can likely add in a few more calculations to recognize this type of scene and adjust accordingly!

[Pixelab]

As an aside - the vray guys came up with the defaults of 1/24, colour threshold of 0.005 and a shade rate of 6 by profiling a lot of scenes. It might be worth getting them to look at yours as they're constantly trying to make vray more automatic and intelligent. Your scene is quite specific in terms of where the noise comes from so a general average setting probably won't be optimal but they can likely add in a few more calculations to recognize this type of scene and adjust accordingly!

The defaults are catastrophic on my side. Huge render times & a lot of grain. Maybe because I'm still working with IM/LC and it's optimised for BF/LC ?

Best results for me are 400 on HDR, 200 on all lights and AA 1/12 + 0.05 (more or less the numbers you had in your video) I get clean results in 1h on our little 4 Xeon CPU farm for 4800 wide interiors

But I want to switch to automatic... with no luck.

Made a test with the MSR at 48 and it doesn't help, I'll try some extreme value like 200.(I guess that lowering the DMC thresold won't help?)

[Morbid Angel]

I think AA of 0.1 is way too high. You need to lower that or else the samples get terminated too soon.

[vlado]

You can always get me the scene to vlado@chaosgroup.com and I'll take a look at what's going on.

Best regards,
Vlado

[Pixelab]

I think AA of 0.1 is way too high. You need to lower that or else the samples get terminated too soon.

Actually not, those are DMC samples, and with local subdivs of 400 on the light, there is almost no grain left (even with AA still at 0.1)

You can always get me the scene to vlado@chaosgroup.com and I'll take a look at what's going on.

Best regards,
Vlado

File sent, thx for looking into this !

[Pixelab]

With local subdiv 400 on the light, you can clearly see the that the AA stop wasting time on the large flat surface directly illuminated (render time roughly the same)

[joconnell]

Oh duh, here's what happened.

If you have your aa set to 1/12, light subdivs to 400 then for every aa ray shot you're getting 42 light samples taken as per the vray ui. If you're using shading rate of the default 6 then for every aa rate shot you're getting 6 samples. The big difference here is when you're trying to use the shade rate, it'll only fire when aa rays fire and since you're using an aa colour threshold of 0.1, the aa is cutting off very early and as a result cutting off the shade rate sampling with it. If you want to go with something similar you can leave your aa 1/12 settings then go to your light and put in 400 samples. See what values it shows you under your subdivs for per aa sample. Now if you set the light back to 8, you can keep it selected so you can see the per aa value and instead raise your shade rate until the per aa sample text reads the same as you had when the light was set to 400 subdivs.

Here's a very simple test render of a teapot - glossy grey material on it and the floor, two area lights left and right.



From the top left we have: aa 1/12, colour thresh of 0.01, shade rate of 8 and light and material subdiv of 8. With the colour thresh of 0.01 we get plenty of aa and red in the sample rate since there isn't enough shading samples to clean it up, normal enough here. From the vray log window, we get 17 camera rays per pixel, 160 shadow rays and 72 reflection rays.

Next we drop the colour threshold to 0.1 as you're doing in your renders. Since the colour threshold controls how fine a level aa is working to and we've told it not to work as hard, we now get 1.6 camera rays per pixel. on top of that we get 8.4 shadow rays (that's our 8 shade rate rays per pixel for the lights) and 3.42 reflection rays. Since our shade rate rays only fire when we shoot aa / camera rays, our colour threshold of 0.1 brings down the aa rays so we get less chances to shoot shade rate rays. The sample rate shows nothing in the scene is getting our max 12 aa rays but we've a noisy render. The aa is working a little bit on our reflections too.

Next we keep our aa 1 / 12 and colour thresh 0.1 but put our light and material up to 200 subdivs. We're now at 1.28 camera rays per pixel since our lights and materials are cleaner, we've got 41.7 shadow rays per pixel and 27.9 reflection rays per pixel. the high light and material subdivs mean that even though we're shooting small numbers of aa rays, each aa ray is getting lots of light and material samples. The sample rate shows that the aa sampler isn't doing any work on the reflections any more either, it's not needed since the 200 subdivs take care of it.

Lastly for the fun of it I set the lights and material subdivs back to 8 and then turned up the shade rate until the info in my light subdiv settings read the same amount as when I had it set to 200 subdivs - I had to go to a shade rate of 257 to get the same value. Here we get 1.28 camera rays, 46.7 shadow rays and 30.4 material rays. Same thing with the reflections, aa sampler has nothing to do here.

In conclusion, you using a low colour threshold for your aa means that it's choking the shade rate's opportunities to sample your lights and materials. You'll have to turn up the shade rate amount much much higher to get enough material and light sampling done on the back of your small amount of aa rays to get enough quality. The only bad point might be that shade rate is global - you might be increasing the quality on things that don't need it as much (VLADO - How much does early termination kick in on shade rate?) so you might be better in cases like this where it's pretty much only one thing causing your noise to directly up the quality of that only.

What I'd also say is your workflow might be a slightly less general one - most people wouldn't be able to use edges with a colour threshold of 0.1, it might be okay if you're doing really high res print stuff but for film / tv / animation the edge quality you'd get at 0.1 would be far too ragged looking and cause a load of problems with any kind of movement! The default of 1/12, shade 8 and colour threshold of 0.01 is pretty good overall quality wise and similar to the vray defaults. They might not suit you quite as well but they were chosen as a general, cover all set of values. If you know that you can get away with less aa and put the effort into shading instead then go for it, you'll just have to break slightly from the standard shade rate workflow! What the shade rate workflow is supposed to address is scenes that have loads of lights, materials and objects where you don't want to set lots of individual subdiv values. In your case, if you only have one source causing all the noise but everything else is okay, you'll possibly be best off just upping that one thing - it'll be nice and fast for you to do and probably get the best combination of ease of use and render time.

[Pixelab]

Wow, thanks a lot for your explanation ! That's basically what I understood (thanks for the tests, that's helping me how extreme I should go to test values)

In conclusion, you using a low colour threshold for your aa means that it's choking the shade rate's opportunities to sample your lights and materials. You'll have to turn up the shade rate amount much much higher to get enough material and light sampling done on the back of your small amount of aa rays to get enough quality.

That's the part I don't get. Isn't it called Minimum shading rate ? What you explain would make sense if it was called Maximum shading rate. But you probably a better knowledge of the algortithm so I guess you're right. (Ill reread you post a third time )

What I'd also say is your workflow might be a slightly less general one - most people wouldn't be able to use edges with a colour threshold of 0.1, it might be okay if you're doing really high res print stuff but for film / tv / animation the edge quality you'd get at 0.1 would be far too ragged looking and cause a load of problems with any kind of movement!

Is HD interior stills for architecture such a specific worflow ? I agree that 0.1 might renders a bit jaggy (we often go to 0.05 for final images) At first I thought "it will look really ugly but lets try extreme values" and I was amazed how good enough (and fast) it was.

I don't want to invent an unorthodox workflow. If the new 3.3 was fast enough, I'd go with it. But since I've read that AA rays are slower than shade rays, I was trying to avoid unnecessary camera rays. And I didn't understood why you would have to shoot 17 camera rays per pixel where 2 is enough.

Luckily, for exteriors, the dome "grain" is obviously not an issue and we have beautiful and fast result with the new 3.3 global sudbiv method. That's why I asked if it was possible to have best of both world... I mean, just force an increase of the subdiv of a specific source of noise if it's is still present (eg. HDR domes in interiors) ?

[joconnell]

Wow, thanks a lot for your explanation ! That's basically what I understood (thanks for the tests, that's helping me how extreme I should go to test values)
That's the part I don't get. Isn't it called Minimum shading rate ? What you explain would make sense if it was called Maximum shading rate. But you probably a better knowledge of the algortithm so I guess you're right. (Ill reread you post a third time )

Okay so minimum shade rate means that even if your subdivs in your material and lights aren't set high enough after they've been divided by your aa values, you'll still get at very minimum, the number of shading samples that you choose with that value.

Is HD interior stills for architecture such a specific worflow ? I agree that 0.1 might renders a bit jaggy (we often go to 0.05 for final images) At first I thought "it will look really ugly but lets try extreme values" and I was amazed how good enough (and fast) it was.

Hd not so much, stills probably make the big difference though. For example grant warwick's workflow for print can get away with worse AA values than I can because it doesn't move, and if it doesn't move it doesn't shimmer or fizz. If you're doing stills then you're not going to run into this problem so if the edges look good enough to you then fine

I don't want to invent an unorthodox workflow. If the new 3.3 was fast enough, I'd go with it. But since I've read that AA rays are slower than shade rays, I was trying to avoid unnecessary camera rays. And I didn't understood why you would have to shoot 17 camera rays per pixel where 2 is enough.

You're totally right on this one. The only thing that was happening to you is if you go with the shade rate workflow, shade rate just dumbly piggy backs on top of your aa sampling. In your case you were using the colour threshold to turn down the aa heavily and the lack of aa rays led to the lack of shade rate rays too, hence you not getting clean results. Give it a go with massive numbers of shade rate rays - use the little sample feedback calculators near any subdiv settings in your lights or materials to get an idea how many shading rate rays you're getting per aa sample.

Luckily, for exteriors, the dome "grain" is obviously not an issue and we have beautiful and fast result with the new 3.3 global sudbiv method. That's why I asked if it was possible to have best of both world... I mean, just force an increase of the subdiv of a specific source of noise if it's is still present (eg. HDR domes in interiors) ?

Yep you're totally fine with this too. What'd probably be a good option for you is to set your aa, material and light subdivs as normal and shade rate around 8 until you get the edges you want and if the light solution from it is still dirty, make sure "use local subdivs" is on and then just increase your domelight settings until it starts to tidy up. Vlado's set the workflow so that you don't have to spend ages chasing sampling settings but if you know exactly where to go for your scenes and it's quick to change it I don't see much problem in it!