# Stuck buckets

**URL:** <https://forums.chaos.com/t/stuck-buckets/99514>\
**Category:** Issues\
**Created:** [July 8, 2020, 9:28pm UTC](https://forums.chaos.com/t/stuck-buckets/99514 "2020-07-08T21:28:49Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![kosso\_olli](https://avatars.discourse-cdn.com/v4/letter/k/dfb087/32.png) [@kosso\_olli](https://forums.chaos.com/u/kosso_olli)\
**Post date:** [July 8, 2020, 9:28pm UTC](https://forums.chaos.com/t/stuck-buckets/99514/1 "2020-07-08T21:28:49Z")

</div>

I know this has been discussed numerous times in the past, but is there anything that can be done against it?  
It is so frustrating to see your render basically being finished in an hour and then getting stuck on two single buckets for an additional four hours! This is happening so many times now, on nearly all our scenes.

---

<div class="post-metadata">

**Author:** ![3d-laboratory](https://dub1.discourse-cdn.com/chaosprd/user_avatar/forums.chaos.com/3d-laboratory/32/72830_2.png) [@3d-laboratory](https://forums.chaos.com/u/3d-laboratory)\
**Post date:** [July 8, 2020, 10:03pm UTC](https://forums.chaos.com/t/stuck-buckets/99514/2 "2020-07-08T22:03:00Z")

</div>

Hello Oliver, as I saw on your WEB-site you often work with car paint. I have the same problem sometimes. When two object sare illuminated by a sunlight ore a light dome and the two objects are reflecting each other I have the seme problem. For example; I have a facade with some elements who reflect cars wich are located on the ground and get sun from a sunlight or a HDRI. If the sun makes a light spot on the cars which are reflected by the parts of the facade these buckets renders a long, long time. When I remove the cars all is fine and I have normal render times again. May be that your problem it is caused by the reflections…(as you are German you can write an email to me in German language [info@3d-labor.de](mailto:info@3d-labor.de) if you didn´t understand what I mean)

---

<div class="post-metadata">

**Author:** ![\_Lele](https://dub1.discourse-cdn.com/chaosprd/user_avatar/forums.chaos.com/_lele/32/49986_2.png) [@\_Lele](https://forums.chaos.com/u/_Lele)\
**Post date:** [July 9, 2020, 12:01am UTC](https://forums.chaos.com/t/stuck-buckets/99514/3 "2020-07-09T00:01:55Z")

</div>

> [@kosso\_olli](#):
>
> I know this has been discussed numerous times in the past, but is there anything that can be done against it?

Well, what is it that is taking long?  
If it can be sample-stopped (f.e. a strong highlight where noise won’t be visible), then cap the max AA.  
V-Ray will sit on those highlights until either they are within noise threshold range (often quite difficult indeed.) or max AA is reached (much more common.).

Otherwise, if you don’t plan to do loads of post, clamp the renderer to, f.e. 13 float.  
It’ll leave you a few stops of play, but maybe it’ll bring those highlights home in more reasonable times.

But if it’s not highlights, then strategies may be different.

---

<div class="post-metadata">

**Author:** ![Joelaff](https://dub1.discourse-cdn.com/chaosprd/user_avatar/forums.chaos.com/joelaff/32/71647_2.png) [@Joelaff](https://forums.chaos.com/u/Joelaff)\
**Post date:** [July 9, 2020, 1:49am UTC](https://forums.chaos.com/t/stuck-buckets/99514/4 "2020-07-09T01:49:47Z")

</div>

Actually, there is a very simple strategy that Backburner (the original Backburner, not Max stolen name version) used in 1994 or so. Backburner was the network rendering software with Specular International’s Infini-D software.

Backburner did in 1994 what I have been asking every other renderer to do since. Once the number of remaining remaining blocks is less than the number of free machines/cores /4 you chop the remaining blocks up into fourths and put four processes on it. But you keep the existing core or machine rendering it as well. Whichever processes finishes first wins and the job is done.

In other words every core works on the remaining blocks by subdividing them while they are still being rendered by the original core or machine. I keep saying core or machine because this should happen with net rendering because some render nodes are an order of magnitude faster than others. Backburner was for network rendering, similar to VRay distributed rendering. It worked so well so long ago. With distributed render you could subdivide and put multiple machines on the subdivided blocks.

Last Block Syndrome, or LBS as it is more commonly known, has been an insidious issue for decades, but it has a solution. Of course for now we just have to use a smaller block size. VRay5 has better dynamic subdivision than 3 did, but I still think the dynamic subdivision should start faster still. Any speed loss due to smaller blocks is more than made up for by time saved from less severe LBS.

---

<div class="post-metadata">

**Author:** ![vlado](https://avatars.discourse-cdn.com/v4/letter/v/779978/32.png) [@vlado](https://forums.chaos.com/u/vlado)\
**Post date:** [July 9, 2020, 7:04am UTC](https://forums.chaos.com/t/stuck-buckets/99514/5 "2020-07-09T07:04:44Z")

</div>

It’s on the “to do” list to finally try and tackle this.

Best regards,  
Vlado

---

<div class="post-metadata">

**Author:** ![Joelaff](https://dub1.discourse-cdn.com/chaosprd/user_avatar/forums.chaos.com/joelaff/32/71647_2.png) [@Joelaff](https://forums.chaos.com/u/Joelaff)\
**Post date:** [July 9, 2020, 7:12am UTC](https://forums.chaos.com/t/stuck-buckets/99514/6 "2020-07-09T07:12:38Z")

</div>

Thanks, Vlado.

I am sure in most cases it is really not that severe. However, it does drive the users insane to see a single core working on a block trapped in LBS (Last Block Syndrome). About 10% of the time this a real issue. I would love to see the brute force approach applied again in 2020. Throw all the resources at it, and whichever finishes first wins! And the user wins!

I love the idea of the dynamic splitting. I just think it needs to be even MORE aggressive. With most image filters (says the guy who likes the Soften filter at 3.98-- not lumped in here) the smaller block size makes very little difference because it resolves LBS much faster.

I was impressed the Specular International guys got it done so well in 1994. A shame they sold out to Macromedia who then killed Infini-D in favor of their own RayDream Designer (which was nice, but also got killed.)

Thanks.

---

<div class="post-metadata">

**Author:** ![kosso\_olli](https://avatars.discourse-cdn.com/v4/letter/k/dfb087/32.png) [@kosso\_olli](https://forums.chaos.com/u/kosso_olli)\
**Post date:** [July 9, 2020, 7:42am UTC](https://forums.chaos.com/t/stuck-buckets/99514/7 "2020-07-09T07:42:57Z")

</div>

> [@\_Lele](#):
>
> Well, what is it that is taking long?

Hey Lele,  
it’s the headlights of course, in all the cases. Sure, with all the bright light, refractions and reflections they take longer to render, but for some reason, there is an inconsistency. Out of let’s say 100 buckets rendering the headlights, two or three always get stuck in different places. In the case of the image I started last night, the car itself was finished in under an hour, but it took a whopping 7 hours to finish the last two buckets.  
In the past, I often canceled the render and rendered the remaining small area with progressive, but this is not an option while rendering over BB.

---

<div class="post-metadata">

**Author:** ![Joelaff](https://dub1.discourse-cdn.com/chaosprd/user_avatar/forums.chaos.com/joelaff/32/71647_2.png) [@Joelaff](https://forums.chaos.com/u/Joelaff)\
**Post date:** [July 9, 2020, 7:49am UTC](https://forums.chaos.com/t/stuck-buckets/99514/8 "2020-07-09T07:49:00Z")

</div>

> [@kosso\_olli](#):
>
> Hey Lele,  
> it’s the headlights of course, in all the cases. Sure, with all the bright light, refractions and reflections they take longer to render, but for some reason, there is an inconsistency. Out of let’s say 100 buckets rendering the headlights, two or three always get stuck in different places. In the case of the image I started last night, the car itself was finished in under an hour, but it took a whopping 7 hours to finish the last two buckets.  
> In the past, I often canceled the render and rendered the remaining small area with progressive, but this is not an option while rendering over BB.

Wow! What bucket size? Have you experimented with smaller buckets? We usually end up going that route in these cases. Makes very little difference in render time vs. larger buckets even without LBS (Last Bucket Syndrome).

I know headlights are notoriously slow to render here too, and for us is is an HD or 4K shot where the headlight isn’t seen very long. I can only imagine for a high res still.

---

<div class="post-metadata">

**Author:** ![kosso\_olli](https://avatars.discourse-cdn.com/v4/letter/k/dfb087/32.png) [@kosso\_olli](https://forums.chaos.com/u/kosso_olli)\
**Post date:** [July 9, 2020, 8:14am UTC](https://forums.chaos.com/t/stuck-buckets/99514/9 "2020-07-09T08:14:18Z")

</div>

> [@vlado](#):
>
> It’s on the “to do” list to finally try and tackle this.

Okay, great. If it comes to the point where this will be tackled, please don’t hesitate to ask for examples. There are quite a few scenes illustrating the problem.

---

<div class="post-metadata">

**Author:** ![kosso\_olli](https://avatars.discourse-cdn.com/v4/letter/k/dfb087/32.png) [@kosso\_olli](https://forums.chaos.com/u/kosso_olli)\
**Post date:** [July 9, 2020, 8:22am UTC](https://forums.chaos.com/t/stuck-buckets/99514/10 "2020-07-09T08:22:59Z")

</div>

> [@Joelaff](#):
>
> Have you experimented with smaller buckets?

Yes. It does help a little, but smaller buckets just means more buckets getting stuck. If it’s just two or three buckets from a render node, I can still render on the other ones, but more stuck buckets mean more blocked nodes. Bucket size right now is 16.

> [@Joelaff](#):
>
> I know headlights are notoriously slow to render here too, and for us it is an HD or 4K shot where the headlight isn’t seen very long. I can only imagine for a high res still.

Imagine rendering a closeup of the front fascia of a car, full-frame at print res with the headlight out of focus somewhere in the frame. The horror.

---

<div class="post-metadata">

**Author:** ![Joelaff](https://dub1.discourse-cdn.com/chaosprd/user_avatar/forums.chaos.com/joelaff/32/71647_2.png) [@Joelaff](https://forums.chaos.com/u/Joelaff)\
**Post date:** [July 9, 2020, 8:31am UTC](https://forums.chaos.com/t/stuck-buckets/99514/11 "2020-07-09T08:31:48Z")

</div>

> [@kosso\_olli](#):
>
> Yes. It does help a little, but smaller buckets just means more buckets getting stuck. If it’s just two or three buckets from a render node, I can still render on the other ones, but more stuck buckets mean more blocked nodes.
> 
> Imagine rendering a closeup of the front fascia of a car, full-frame at print res with the headlight out of focus somewhere in the frame. The horror.

LOL @ the horror, Colonel Kurtz!

But yeah, that sucks, man. Good luck.

---

<div class="post-metadata">

**Author:** ![fixeighted](https://dub1.discourse-cdn.com/chaosprd/user_avatar/forums.chaos.com/fixeighted/32/45854_2.png) [@fixeighted](https://forums.chaos.com/u/fixeighted)\
**Post date:** [July 9, 2020, 3:44pm UTC](https://forums.chaos.com/t/stuck-buckets/99514/12 "2020-07-09T15:44:44Z")

</div>

This is probably a dumb idea but I’ll voice it anyway; wouldn’t it be possible, when getting to a problem bucket, to search the pixels in the bucket and if they are of a similar brightness then simply stop that bucket at an average of the pixel values?  
If this was set to a maximum time per bucket, or something similar, then wouldn’t that be sort of like clamping but on a very bucket-specific level and without having to clamp anything per se?  
Apologies if that is just plain nonsense 😛

---

<div class="post-metadata">

**Author:** ![joconnell](https://dub1.discourse-cdn.com/chaosprd/user_avatar/forums.chaos.com/joconnell/32/36063_2.png) [@joconnell](https://forums.chaos.com/u/joconnell)\
**Post date:** [July 9, 2020, 6:46pm UTC](https://forums.chaos.com/t/stuck-buckets/99514/13 "2020-07-09T18:46:44Z")

</div>

Worth your while doing the headlights as a second pass with clamping / progressive and invisible to camera in the main one for now? Have you got a good automated rendering solution for all your stuff? Changsoo Eun from fuseFx is writing his own logic based render manager with lots of passes / tokens and so on that could be good for this type of thing.

---

<div class="post-metadata">

**Author:** ![\_Lele](https://dub1.discourse-cdn.com/chaosprd/user_avatar/forums.chaos.com/_lele/32/49986_2.png) [@\_Lele](https://forums.chaos.com/u/_Lele)\
**Post date:** [July 10, 2020, 9:24am UTC](https://forums.chaos.com/t/stuck-buckets/99514/14 "2020-07-10T09:24:34Z")

</div>

> [@fixeighted](#):
>
> This is probably a dumb idea but I’ll voice it anyway; wouldn’t it be possible, when getting to a problem bucket, to search the pixels in the bucket and if they are of a similar brightness then simply stop that bucket at an average of the pixel values?  
> If this was set to a maximum time per bucket, or something similar, then wouldn’t that be sort of like clamping but on a very bucket-specific level and without having to clamp anything per se?  
> Apologies if that is just plain nonsense 😛

That’s super easy to do in post, for individual pixels (that’s how fireflies removers, like the simple one i built, work.).  
It is a good idea.

Until, that is, one has a strong specular covering an area.  
Or the area in question doesn’t lend itself to naive blending (like Olli’s headlights.).

The problem with time alone is that we don’t really know what is being computed (that’s, technically, why we render: to find out.) : it may need that time to clean up or look bad beyond repair.  
In other words, some areas take a long time even if they aren’t Olli’s headlights, purely through shading complexity.

That’s why i suggested manual user intervention: a user knows things a renderer doesn’t until it’s done, however long that may take. 🙂

---

<div class="post-metadata">

**Author:** ![fixeighted](https://dub1.discourse-cdn.com/chaosprd/user_avatar/forums.chaos.com/fixeighted/32/45854_2.png) [@fixeighted](https://forums.chaos.com/u/fixeighted)\
**Post date:** [July 10, 2020, 9:45am UTC](https://forums.chaos.com/t/stuck-buckets/99514/15 "2020-07-10T09:45:36Z")

</div>

Good points Lele; I was thinking after I posted that fireflies were actually the best application of what I was thinking of.  
Back to looking around for that special wand thingy then 🙂

---

<div class="post-metadata">

**Author:** ![kosso\_olli](https://avatars.discourse-cdn.com/v4/letter/k/dfb087/32.png) [@kosso\_olli](https://forums.chaos.com/u/kosso_olli)\
**Post date:** [July 10, 2020, 9:57am UTC](https://forums.chaos.com/t/stuck-buckets/99514/16 "2020-07-10T09:57:40Z")

</div>

The headlights are just an example. We have the same issue in a lot of other situations as well.

---

<div class="post-metadata">

**Author:** ![\_Lele](https://dub1.discourse-cdn.com/chaosprd/user_avatar/forums.chaos.com/_lele/32/49986_2.png) [@\_Lele](https://forums.chaos.com/u/_Lele)\
**Post date:** [July 10, 2020, 10:29am UTC](https://forums.chaos.com/t/stuck-buckets/99514/17 "2020-07-10T10:29:43Z")

</div>

Yep.  
Headlights are stereotypical. They are slow by design. They are the impossible “chrome spheres” of raytracing.  
The potential issues are very many, and there is no engine out there which doesn’t suffer from some of them (unless, ofc, they entirely forego adaptive sampling, eh.).

I’m too old school, perhaps, but even with the “magic” coming, i’d say a user’s best bet is to spend half that stuck-bucket time in scene optimisation.  
They used to crop up when i was in production, and in a movie production they used to crop up a LOT.  
At some point, for all the pipelining and formalisation of the process, a human (often me.) had to stay late to \*manually\* ensure the sequence rendered in due time, and to justify to supervisors the -hopefully minor- change in looks.

Notice that not all issues can be cheated.  
Take fast moving, tiny highlights (CFR. fast spinning wheel rims.), for example.  
There may be tech researched which could help, but they are inherently difficult for the statistical rendering we (all!) do.  
Very unlikely to be found, very intense when they are.  
One either suppresses them (and goodbye highlight) or one bites the bullet and samples them properly (ooooh, boy. that can be very very very long.).  
If a human knows the shot will have those, then it can be planned for better (passes, pre-rendering compared to the rest of the scene, and so on), which may mitigate its impact on the process as a whole.

I’m being rain on the parade, i get it, but i still would not rely on the fix: it may be it’ll only go so far, and that the results won’t be good enough for everyone and all cases.  
A stronger pipeline, and the artisan’s love to one’s task, will take one farther, and quicker, i’d wager.
