Announcement

Collapse
No announcement yet.

Bucket and Progressive Samplers' Measured Performance

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Bucket and Progressive Samplers' Measured Performance

    Thanks to Olli's scene we could profile a very detailed, production grade car, at very high resolutions (around 60 MPx.).
    So we now have a real-world measured performance chart for bucket versus progressive, with and without (many) Render Elements, and at increasing resolutions.

    Tests conducted in max 2017.
    The render was set up as a 1-100, 0.025 N.T.
    The reason for the max AA subdivs is to make sure the noise threshold was met, and we didn't hide from stuck buckets or long sampling.
    The higher Noise Threshold, instead, was chosen so to finish the profiling in a reasonable time, and should carry absolutely no conceptual difference from the same test carried with a lower Threshold.
    The image resolution was 8961 x 6723 Px (~ 60.2 MPx), while the 16 Render Elements (when on) consisted of the usual set of beauty elements, and nine multi-mattes.
    Sampler Type Render Elements Active Resolution Render Time (Seconds) Slower than Quickest Task Slower than Bucket Sampler
    Bucket No Quarter (~2k) 4195.27 0.00% 0.00%
    Bucket No Half (~4k) 16558.8 0.00% 0.00%
    Bucket No Full (~8k) 64265.5 0.00% 0.00%
    Bucket Yes Quarter (~2k) 4589.23 8.58% 0.00%
    Bucket Yes Half (~4k) 17591.5 5.87% 0.00%
    Bucket Yes Full (~8k) 68197.3 5.77% 0.00%
    Progressive No Quarter (~2k) 4289.12 2.19% 2.19%
    Progressive No Half (~4k) 18955.7 12.64% 12.64%
    Progressive No Full (~8k) 67315.7 4.53% 4.53%
    Progressive Yes Quarter (~2k) 4671.05 10.19% 1.75%
    Progressive Yes Half (~4k) 18301.9 9.52% 3.88%
    Progressive Yes Full (~8k) 85519.2 24.85% 20.25%

    So, the first three rows are our "zero", the fastest way to get to an RGBA image: bucket sampler, and no Render Elements.
    The next three rows are again the bucket sampler, but with render element active: we can see the slack is between 5 and 9 percent, give or take decimals.
    Now, the first thre progressive sampler rows are calculated without Render Elements, and the performance slack with the bucket sampler doing the same job is between 2 and 4%. The middle result, showing a 12% difference, should be currently treated as an outlier, as i believe windows updated mid-job (don't ask...) (EDIT: I think it was windows defender working on a schedule.).
    The last triplet of rows in the table above has the worst penalty for the progressive sampler, where compared to a bucket *without* REs, it's 10% or so slower, until the resolution is very big, and then it jumps to a 24%.
    Notice the differences between Samplers (instead of tasks and samplers) are *much* more contained, for all but the two outlier values.

    The test is being re-run to validate the results, so for now take with a pinch of salt.
    Last edited by ^Lele^; 11-03-2019, 07:33 AM.
    Lele
    Trouble Stirrer in RnD @ Chaos
    ----------------------
    emanuele.lecchi@chaos.com

    Disclaimer:
    The views and opinions expressed here are my own and do not represent those of Chaos Group, unless otherwise stated.

  • #2
    Lele, thanks for taking the time to profile these settings! It's good to see a comparison with tangible numbers. I think for the amount of given render elements and the resolution, the hit in render time has to be accepted. If it could be further improved though, I would be happy.
    Did you also compare the rendered output? I remember the progressive sampler handled the out of focus highlights much better, which I like. Bucket tended to produce a noisy result in these areas.
    Also, the settings for the ray bundle size were at default of 128? And one question regarding the noise threhsold: It really was 0.025? Did you miss a zero was that on purpose to make the test reasonably quick?
    https://www.behance.net/Oliver_Kossatz

    Comment


    • #3
      Mind you, i am really not convinced the two outliers can be taken at face value.
      Those would be the half res progressive without REs, and the full res progressive with the REs.
      The issue was that windows defender ran once a day on the -very full of data on disks- slave.
      So the jobs which took the longer may have incurred into a one or two system scans, skewing the results.
      I since deactivated the task in the task scheduler, and the new batch of tests is running as we speak, so we'll see then.
      Should we still have doubts after a rerun, i'll try and get hold of one of the many-cored machines in the office, so to rerun these tests a few times, but much more quickly.

      RBS was at the default, yes.
      I used 0.025 on purpose, yes, to finish in a decent time.
      For the resolution these are at, should they go to print, it'd be mighty fine i'd reckon.

      I'll do proper image analysis in a little bit, but there are things i learnt in the while about how V-Ray goes about noise threshold checking which are cool to share, i think.
      A) The progressive sampler's noise threshold checking is fully multithreaded, and done in a way which retains performance (and i'd say the numbers tend to confirm that. Rado's no slouch...).
      B) The bucket sampler isn't as naive in Noise Checking as i had assumed: it'll do a very competent job across buckets too, if of course not always as comprehensive as the progressive. If detail seems amiss, particularly with DoF or Moblur (but very fine geo would count as well), just raise min AA samples (2, 3 subdivs.), and there should be noticeably more parity between the two Samplers' results.
      Last edited by ^Lele^; 12-03-2019, 05:52 AM.
      Lele
      Trouble Stirrer in RnD @ Chaos
      ----------------------
      emanuele.lecchi@chaos.com

      Disclaimer:
      The views and opinions expressed here are my own and do not represent those of Chaos Group, unless otherwise stated.

      Comment


      • #4
        Thank you very much for doing these tests! very interesting results here


        -------------------------------------------------------------
        Simply, I love to put pixels together! Sounds easy right : ))
        Sketchbook-1 /Sketchbook-2 / Behance / Facebook

        Comment


        • #5
          I would be curious to see some results with notorious stuck buckets.
          http://www.jd3d.co.uk - Vray Mentor

          Comment


          • #6
            The render was set up as a 1-100, 0.025 N.T.
            The reason for the max AA subdivs is to make sure the noise threshold was met, and we didn't hide from stuck buckets or long sampling.
            There was plenty of "stuck" material in the scene, so you can consider the test comprehensive in that respect.
            If there was a stuck bucket somewhere, it eventually came unstuck (notice the render times...).
            Lele
            Trouble Stirrer in RnD @ Chaos
            ----------------------
            emanuele.lecchi@chaos.com

            Disclaimer:
            The views and opinions expressed here are my own and do not represent those of Chaos Group, unless otherwise stated.

            Comment


            • #7
              subscribed. Thanks for posting the tests, Lele.
              always curious...

              Comment

              Working...
              X