How do I test the quality of an HDRI?

[VKNT]

Hi everyone,

as I mentioned in my spherical sky thread, I found a way to make my own HDRI's. Hurray

Today I've made a first test. Came out ok I think.

To create the HDRI I made 6 2EV brackets. From almost entirely white to almost entirely black. The .hdr file was made straight from PTGui, my panorama stitcher.
I then boosted the sun a bit in ps to match up the shadow intensity.
The file is 5000px wide. I can make it 15000 px wide, but to me it's useless cause I haven't got X64. Renders larger than 800px makes max9 crash. Plus vray needs more than 60s to prepare the light. Using 5000px only 10s.

But I was wondering what's the best way to test the quality of an HDRI? Is there some sort of testscene I can throw it at?

This is what I thought could test it, Here's a quick render with some different materials. It's a vrayHDRI in a skydome. The visible seam comes from a vrayplane that I used to create the shadows for the objects. How can I get rid of it, not losing the other shadows?
I'll make some further tests 2morrow, must get some sleep now..

[vlado]

About the V-Ray plane, you should go to the environment section of V-Ray and use a black GI override.

Best regards,
Vlado

[jujubee]

I then boosted the sun a bit in ps to match up the shadow intensity.

You shouldn't have to do that as the HDR no longer becomes accurate. But ignoring accuracy and focusing on desired results, this may be necessary for you.

As for testing quality, there's no specific benchmark scene out there you can test against.

As a general rule of thumb, HDR quality can be defined by the following:

1) HDR file size: The greater the file size generally the greater the resolution. The greater a resolution, generally the lesser noise you will have in an image. The more samples that would have been taken.

2) Final exposure adjustments: How high can you visibly boost the exposure in the HDR? How low can you lower the exposure in the HDR? HDRs that are only bracketed a few F-Stops do not contain enough information for a HDR editing program to extrapolate proper floating point information from. That's why Debevec originally suggested increments of 8 F-stop brackets in total (I think this figure is correct - it's been quite a few years.) They may work, but the results will often be crappy.

3) Chromatic abberation: You'll start to see a rainbow type of effect along the corners/outer perimeter of a spherical lens or a reflective ball. Some camera lenses handle this a lot better than others. Also, you would obtain better results with a 180 degree fisheye lens versus photographing a highly reflective metal ball.

4) Stitching Noise/Blur: When you look at the HDR, is it blurry? This is a result of adding one exposed image on top of another up to an averag of eight times. Since they are taken at varying times of minor increments, noise can be introduced here. This is probably the biggest factor. If your camera moves a fraction of a centimeter during pressing a shutter, if a cloud moves in the sky, or the camera motor itself moves a fraction - multiply this times how many brackets you are compositing on top of each other and that will most likely result in your noise/blur.

The best results comes from a Spheron camera which is designed to stitch together an image seamlessly and takes successive brackets in very quick succession with little shake introduction.

The second best results comes from a fish-eye lens as this means you don't have to stitch 32+ images together by hand. You can accomplish this perhaps by 1/2 or even 1/4 of the steps a regular lens would take.

5) Camera lens: are you using a proper MM lens to film the environment you want?

6) You can look at the background or in a reflection of your final render. If the image itself is super-crisp, then you know you have a pretty good HDR. The second factor effecting this would be scale by increasing the resolution
and seeing how it looks.

7) Camera megapixels: Self-explanatory relating to resolution. When I first created HDRs, digital cameras were at a maximum of 3 megapixels. They're at probably an average of 10-12 now for most prosumer models.

Camera/Tripod centering: A beauty of DSLRs is that they generally have their optics centered above their tripod hole. Non-DSLRs are typically offset. So when you turn the camera on a tripod to photograph the other direction, the images will technically not be 180 degrees parallel to the other side due to this imbalance. This will create some level of noise/blur at the stitched seams.

So to summarize:

1) Quality can be determined largely by the image noise/blur itself, resolution/file size (although a large file size doesn't necessarily mean that it's better), the exposure/floating-point range, and how you took the photos.

2) From a Vray quality standpoint, the VrayDomelight optimizes the sampling of a HDR better than the native Max HDR I/O handler. Always use a Vraydomelight in conjunction with HDRs for optimal quality. Also adjust the sampling and resolution of the Vraydomelight to higher values for cleaner results.

3) To compare two HDRs to each other, you would need another HDR that is comparable in file size to the one you are testing. Use the same light and Vray render parameters to render out your image. You would go into the final image and have to look at background quality as well as noise in shadows and splotches. Unfortunately, the natural intensity of the compared HDRs will always differ at your default settings.

[VKNT]

Thanks Vlado, that worked.

Many thanks Jujubee for taking time to write this long explanation.

So let's see

1. HDR File size: I can make it 15000px wide. > 600mb file. That's even more than a spheron..

2. Exposure adjustments. I have an Canon S2. It's not a dSLR. Max F8 but I use F4.5 as this is the sweet spot of the (fixed) lens. I go from 1/10 to 1/3200 in 6 steps. Providing 12 EV (according to my camera). I could try to override the shutterspeed, but not sure how far I could go.
So I have to compensate the sun by painting in a sun in ps. Until I find a better way this is guesswork

3. CA: A little. My photos have some more in them, but it seems like my stitch software gets rid of most of it using the multiple exposures.

4. Stitching Noise/Blur: On non-moving objects: none. Moving trees, a little halo. It takes 2s for 6 exposures. When the buffer is running out, 4s.
Ok, a spheron is better, I admit.. But I'm working with a 1K rig vs. 70K here to put things in perspective
I do have some flare issues, I'll try to make a lens cap fix that.

I hear you on the fish-eye. But using a fisheye means low resolution. And until I get my dSLR, I can't change lenses.

5. Camera lens: Euh, the fixed lens of my amateur camera
I have a camera head, so having no parallax errors.

6. I'll check this out.

7. I have 5mp..

I might as well share it, so you can all have a look at it.
There are some slight misalignments. But I didn't spent more time on it because it's just a test to find a good workflow. The environment is not the most inspiring to use on projects I guess

Below are the six exposures I made. You can download the .hdr on my website.
www.vknt.be/hdri/VKNT-SKY-0027_SunAdded2.hdr

[VKNT]

[jujubee]

The brackets look good at first sight.

But I'm working with a 1K rig vs. 70K here to put things in perspective

I saw a used one on Ebay about 6 months back for 15k.