Matte shadow render element grain

[evanerichards]

I have a still image I'm trying to render out a matte shadow for. It is a very large image. 4992 x 3328

The problem is, I cannot figure out how to get the matte shadow to not be so grainy. And it is realllly grainy.

I don't know why that would be the case either. Every light in the scene has a subdivision of 32.
I am using no GI.
I am using adaptive DMC. With a minimum of 1 and a max of 128 subdivs.
My threshold is .001
My AA settings use Gaussian with a size of 2.
The adaptive amount of my dmc sampler is .85

Any idea how to clean that up? Is it just being grainy because the image is so big? Should that make any difference?

Any help would be greatly appreciated.
Thanks!

[joconnell]

Two things - your shadow could be really weak in the final image so vray could be undersampling it to save time, more likely though your really high max aa is reducing down the subdivisions of your lights a lot. Just for a quick test, render a small region of the shadow area with your current settings. Then turn up all your light's subdivs to something like 200 and re render - compare for time and quality.

[evanerichards]

The shadows pretty defined. I don't think its a matter of a weak shadow, but we'll see. I'll try cranking the subdivs on the lights and see what happens.

Cheers

[evanerichards]

Well, I don't know what the cause was, but cranking the subdivs on the lights solved the problem. I've never ever had to go over 32 before but I cranked them to 256 an it looked much cleaner. Perhaps it is because it is such a giant image.

Thanks for the help!

Cheers

[djorzgul]

Well, don't want to be rude or anything, but joconnell gave you a pretty clear answer what's wrong:

your really high max aa is reducing down the subdivisions of your lights a lot.

The question I'd like to ask is why this happen, and is it a kind of rule to avoid having global sampling much higher than light, shader... sampling?

[joconnell]

It's likely not because it's a giant image, it's probably something in the scene a tiny bit different to what you've done before causing the issue - there's a huuuuuge discussion over on the max forum about trying to find the "perfect" cover all settings that work for every scene and there aren't any so if you find you've never had to go over 32 before it's likely to using similar setups which have all worked out quite well.

What happens with vray is that as your AA sampling starts to get higher, vray starts to reduce the sample settings of your lights and materials to prevent you using really high AA with other high sample settings and giving you bonkers render times. So if you've got low aa like 1 / 4, as vray gets up towards using 4 aa samples, it'll divide your other sample settings by 4. Say you have a light with 24 light samples, your aa 1 parts of the image get the full 24 light samples, the aa 2 get 12 samples, 3 get 8 samples and 4 gets 6 samples. If you're using aa 1/128 and using 32 light samples, by the time vray is using 128 aa samples you're getting 0.25 of a sample from your light, so to counteract it you've to turn up your light samples a huge amount. It seems like suicide turning up your settings to such high values but it does actually work.

The main thing is to try and isolate what the toughest part of your image is - if you've simple geometry then your AA can likely be low, but if you use a lot of soft lights or lights using textures then it might be the light sampling causing problems so those subdivs can go up. Likewise if you've a lot of blurry reflections or refractions then you could look at using higher material samples but lower aa and lower light subdivs. If you've really finely detailed geometry but simple lighting and materials then you're turning down everything but leaving AA up.

A good bet is to try turning on the passes and using the frame buffer to see which part of your image is the noisiest. Raw GI and Raw lighting will show you your light sampling quality, reflection / refraction will show you your material quality (presuming you're using either in your materials) and sample rate will show you which parts of your geometry are getting your low (blue) and high (red) aa samples. If you see noise in one of those passes and not in the others, then you'll know where needs more sampling and where you can possibly save some time.

[evanerichards]

I'm confused. When you say max aa are you talking about Anti-Aliasing? Or about the Adaptive Amount of the DMC sampler?

[joconnell]

Yep - the max number in your adaptive DMC anti aliasing settings. Default is 1 / 4.

[evanerichards]

Ok. Cool. I just wanted to make sure we were on the same page. AA has always meant anti-aliasing to me but I'd hate to have a discussion for days only to find out that someone else was talking about AA as being something completely different
Also, when you say "max number in your adaptive DMC anti aliasing settings" do you just mean the number? Because I don't have the option of a max or minimum number. It's just a single number.

I have a standard settings preset that I always use that has always worked fine for me. I am using that now, but it was grainy. The only difference I can see from the time when it was rendering clean matte shadows to the time when it was rendering grainy matte shadows, is the size of the image. But if you say that's probably not the problem, then maybe there is something else I changed that I forgot about.
I doesn't seem like it has much to do with the AA settings. I could be wrong, but it doesn't look that way to me. Check out these renders I made and let me know if I am missing something.



From this render it seems that the AA is having a negligible effect on the graininess of the matte shadow.

That is very useful advice though about rendering out a Raw GI and lighting pass, a reflection/refraction pass and a sample rate pass to check where your grain is coming from. I hadn't thought of that before. In all of those other passes (raw light, reflection and refraction), the passes are extremely clean even with the lights at 32 subdivs. The Sample rate pass is completely blue though so that would seem to indicate...low AA samples? How is that possible?

Thanks for being patient. I'm kind of new at this and am just trying to figure out how all the pieces fit together.

Thanks!!

[joconnell]

Ah okay - what type of anti aliasing are you using? Fixed, adaptive dmc or adaptive subdivision?

[evanerichards]

I'm using Adaptive DMC. I see what you are talking about now when you are talking about the settings being 1/4. You mean the adaptive DMC subdivisions are at a min of 1 and a max of 4.

I see what you are getting at but when someone would ask me what type of AA I am using I would say "Gaussian"
Are adaptive DMC subdivisions and AA the same thing?

[joconnell]

Right - when you say gaussian, that's the "filter". And yep I'm talking about your min and max settings. AA is the overall term for anti aliasing but if someone on the vray forum says aa 1/4 it's likely shorthand for adaptive DMC anti aliasing with a minimum of 1 and a max of 4.

[evanerichards]

I understand. And you are correct; decreasing the max subdivs did smooth out the matte shadow a lot. Here are the renders at different subdiv levels.



What I don't understand though, is how this works. You say that upping the max subdivs decreases the subdivs of the light, but I was under the impression upping the subdivs increased the quality of the render not decreased it. When I do a full render (not just the matteshadow pass) it looks much better the higher I crank the max subdivs. See the pic below for a reference.



Does this mean you have to essentially do two renders with two differen max subdiv levels? One with high max subdivs for a good quality beauty pass and one with low max subdivs for a good quality matte shadow?

[joconnell]

Yep - in some ways vray is being too smart for it's own good. In your dinosaur example there's a bare minimum number of samples you need for the image and that could be anti aliasing, lights or material samples, depending on what the complex part of the image is. 1/1 isn't good enough for your image as there's likely a lot of geometric detail in it with the mesh or even the bump / texture maps so in this specific case more AA will make the image better since you're putting the sampling in the part of the image that needs it.

If however you use settings that are too high, vray will start trying to optimize things for you. The adaptive DMC anti aliasing ties in with every other sampler in vray which is what gives it it's speed advantage over mental ray - the AA sampler is aware what the light and material sampling is doing and it tries to adjust the two accordingly. Anti aliasing sampling can in theory clean up any kind of noise in a vray render given enough quality, as all the other parts of sampling are fed back through it before it arrives at your final pixel, so it's like the AA sampler is the boss of the others. If the AA sampler knows it's going to be using a really high number of AA samples in part of your image, it'll ask for less samples from your materials and lights to try and keep things fast. The annoying thing though is that sometimes this isn't the right option. For example if you've a really glossy material, the AA sampler might get a noisy result with low AA samples. Then when this isn't good enough, it starts using higher AA but lowers your material samples to compensate. The problem is it was the material samples causing the problem, but the AA sampler is being too bossy and trying to do it using it's samples, rather than upping the material sampling.

In this regard vray still needs input from the artist - if you can identify where your noise is coming from, it means you can up your sampling to fix that problem and your image will clean up quicker. The nice things is there's only so many variables possible in each setup - geometry / texture complexity (aa sampling), Material complexity (material subdivs) or shading / lighting complexity (light subdivs) - if you look through your passes (try out the light select element too) you can often see the grain and then you're on your way to clearing things up.

[evanerichards]

Awesome reply. This explains a lot.

Thanks again for taking the time to explain it all.

Cheers!