benchmark database **UPDATED**

[posterus]

hello,
i've got a little benchmark databse i'd like you all to test. i think is ready to go but there may be a bug that i dont know of. so if you have a minute and feel like inputing your benchmark stats i'd be greatful.
http://www.vrayelite.com/benchmarks.php
thanks,
-joe

**UPDATE**
Everyone,
The benchmark database has been update per your suggestions. You now can create an account/login. This will allow you to modify, delete and add new render times for different configurations. I have tried to accomodate for all scenarios but there are many variables to keep track of and its quite tricky.

In order to keep the data from those of you who have already submitted data I had to create an account for you and assign you a password, which you can change later. The following users will get a PM with their login info.

MiKaLeus
Lucetius
flavrsaver
Fermi Bertran
kalografik
DJ-Studd
oltarz
robslegtenhorst
domel31
SGrunger
chalek

Everyone else please feel free to signup and submit your time/configuration. I am pretty sure most of the bugs have been worked out but I wont be suprised if someone were to find a glitch...I hate PHP
thanks,
-joe

[dlparisi]

Looks cool. Always glad to see another vray site. A few comments and questions about the benchmark:

1. I've suggested before that we need to have some additional information put in the timestamp that indicates both the version of vray (already in there) and what version of the benchmark was used. Also,I think we need to get some type of a conversion factor so times can be compared between versions otherwise we'll be stuck constantly rerendering the benchmark whenever a new version (1.5 RC4, RC5, 2.0, etc, etc.) is released. Maybe percy wouldn't mind a slightly modified version of the file that includes this additional info. Also, even though it's become somewhat ubiquitous, you may want to give some credit to percy for his benchmark.
2. I think you need to add an entry for what the processor is (Core 2 Duo, x4200, etc). The ghz may be a little confusing when trying to compare machines.
3. Is there any way to verify the times via a submitted framestamp or something, I can somehow see this getting out of hand.
4. I'm not on MAX 9 so I can't submit my times, which is kind of a bummer too - should there be much diffence in the times on different versions of Max/Viz? I didn't think this was a problem before.

Looking forward to seeing more times listed.
Thanks, David

[jujubee]

Ditto on David's comments. And Percy definitely deserves the credit/permission for his scene.

[posterus]

thank david,
i guess i was going for simplicity thinking that people didn't want to sift though too much data. i think the conversion factor is a good idea though i have no idea how to implement that i guess once we figure out a conversion factor between the different versions i could somehow multiply it to the user entered rendertime on the fly... but that coding ability is currently beyond me.
i like the idea of different 3d software packages like 3ds max, maya, viz but i think having multiple versions of each will get to be too much.
i'll make sure to give credit to percy for the benchmark...good call
for now i plan to delete any suspicious rendertimes manually. ex. a 54 second rendertime on a pentium 2
as far as the rest of the site, i was hoping to wrap it up before i go on vacation this week but i wont be able to...still under construction.
i'll look into your suggestions.
-joe

[percydaman]

Ditto on David's comments. And Percy definitely deserves the credit/permission for his scene.

none of that needed, but thanks!

[jow]

I agree with David, you need to be able to add processor type. Other than that, great idea

@Percy, the black smoke, Nice one

[Dynedain]

I was thinking about doing this also. It should be setup so that it does a comparison against a "gold standard" image (using ImageMagik or GD Library) to make sure that people haven't played with the settings of the benchmark to get faster results.

[suurland]

Very good idea, but you defitnatly need info about cpu type, else it's totally pointless I think. My dual xeon is about 3 times as slow as the new dual xeon 51xx at the same clock speed.

Also it says nothing about wheter it's rendered on a 32 or 64bit machine.

/Thomas

[posterus]

good point suurland. i'll fix that.
@dynedian: could you elaborate on the gold standard a little more? it sounds interesting. i was thinking about having a link after the rendertime which would open the user's image. this way anyone can easily look at a suspicious image/rendertime if need be and report inconsistancies. with a link to the image you wont have to download each user's image over and over
thanks,
-joe

[Fermi]

And it would be nice to edit, or delete your own submissions where they to be wrong or inaccurate, I was trying to introduce Processor specs, max version vrayversion, and ended with 4 submissions instead of 2 and they are not correct .

Fermí

[Dynedain]

good point suurland. i'll fix that.
@dynedian: could you elaborate on the gold standard a little more? it sounds interesting. i was thinking about having a link after the rendertime which would open the user's image. this way anyone can easily look at a suspicious image/rendertime if need be and report inconsistancies. with a link to the image you wont have to download each user's image over and over
thanks,
-joe

ImageMagik and GD Library are image manipulation libraries that can be called from PHP (or other SSI languages). Most web hosting environments have at least one of them already installed.

My thought was that using calls to them, you could take the user uploaded file, crop off the Vray render strip and do a image subtraction calculation from a "gold standard" image. If any pixels after the subtraction aren't black, then that image is different than the "gold standard", meaning that either the image was rendered with a different version of VRay, or settings were changed from the original benchmark file.

For example, changing a material's glossy settings can dramatically reduce a rendering's time, without producing visually noticeable differences in the rendering. An image subtraction would find it though.

As long as the render setting are not changed, and the VRay version doesn't change, the image (minus the vray output bar) should be identical to the "gold standard" image. Of course you would need a "gold standard" image for each version of VRay to compare against. The comparison would need to happen during the submission process.

[DJ-Studd]

Asus W2JC Centrino Duo T2600. Just beat the Macbook Pro with the same processor, yay!

-- DJ

[posterus]

benchmarks updated. please read my 1st post above.
-joe

[jujubee]

You may want to remove ehsan from your list... he's had a spate of intentionally crashing and kiddie-scripting into the forum here.

[posterus]

You may want to remove ehsan from your list... he's had a spate of intentionally crashing and kiddie-scripting into the forum here.

sounds familiar now that u mention it. noted

Also to those who have data in there, you may need to fix some of your variables...like the vray version. It may default to something that is not your configuration but it is easy to update in your profile.
-joe