Adaptable bucket size at render's end

[zoranm]

Hi all,

Here is scenario:

I'm rendering a large image on 10 PCs via DR. Some of the PC are new but, inevitable, some are old and much slower. Everything is fine until the render's end. By the rule of Murfy's Low the slowest PC will pickup the last bucket and the last bucket is a glass bowl with caustics on a stainless steel table with glossies etc... Result is that I'm waiting next 30 minutes for the last bucket to finish.

Even if the last bucket is on good PC we still have heaps of other buckets idle ( which is just plain sad to watch)

Is it possible to tell Vray (in DR dialog box) to subdivide last 20 buckets into smaller ones by a factor of 2, 4,8.. and finish the render


In reality: I set it up to render 64 pixel buckets and tell it to subdivide by factor of 4. When it reaches last 20 buckets it'd divide them into 16x16 buckets and the render'd finished faster

Is it possible to impalement something like that?

regards

Zoran

[simmsimaging]

Wow - that's a great idea if it's possible. Why wait for the very last bucket though? As soon as there are less buckets than the total cores there may be some benefit to subdividing them down, depending what the overhead for that may be. Cool idea anyway


b

[Morbid Angel]

thats an interesting idea indeed...Vlado?
problem is, what if the problem bucket is not the last, or not even in the last 10-20 buckets. What if lets say your most complex part of the scene is in the beginning of the bucket order. So then you will still get some buckets stuck there no matter what. I think if this formula can be weighted towards beginning, middle, or end of the frame that would be a better flexibility.

[simmsimaging]

I was wondering about a more adaptive bucket solution as well, but not sure if it was feasible, or even useful. Some kind of adaptive bucket size based on an initial random sample of bucket times, or a running average etc.

Maybe the gains would be too small on typical shots, but for problematic areas it might be very handy. As someone who does a lot of glossy reflections through glossy refractions with depth of field I can appreciate the stuck bucket problem very well


b

[jrandom]

Sounds like a great idea

Heck do some sort of a quick pre-calculation to determine the heaviest buckets and send those to the fastest machines, if there is a way to tell vray what the fastest machines are.

[Dave_Wortley]

It would be fantastic if when render buckets became free they could help out the last remaining buckets. I'm guessing not easy though

[^Lele^]

Dmitry has the issue right.
The only issue with small buckets is the AA of the edges, which adds some computational time when the buckets are too small.
If an image is taking a really long time in a particular area however, the tradeoff with small buckets borders' AA versus total render time leans towards smaller buckets being way faster.
All this to say that if an image is taking a lot to render, don't be afraid of having it render with 16px buckets.
That will lead to the same 64px bucket being split into four, and having four cores on it rather than one will just about wipe out the overhead from bucket edges' AA.

[Morbid Angel]

Lele correct me if im wrong but isn't edges of the buckets aliased twice to ensure there is no lines between buckets?

[^Lele^]

indeed it is.

[jonahhawk]

edit: nevermind... misread earlier posts.

[renderfarmer]

This doesn't only happen on setups with one slow CPU but with any Hyper Threaded CPU as well.

I recently picked up an automotive client and my hexa-core 3.33GHz cpus are for naught when I end up waiting disproportionately long for that last Hyper Threaded bucket to render a portion of a headlight with caustics and dispersion turned on...

Vlado; is there any way for Vray to differentiate between Logical cores and Physical ones? If so, then would it be possible for Vray to re-assign that last bucket to a physical core that is idling?

If not, then would it be possible to automatically divide up the last bucket of a given rendering into buckets of sufficient size to occupy all of the available CPUs? Even if it had to render some extra pixels for AA I recon this would result i a much faster final rendered image than otherwise leaving that slow logical cpu to finish the job.

Thanks.

[vlado]

We have done some work to improve hyperthreading performance in the latest builds; it is in principle possible to find out what processor a thread is running on, but ideally the OS would be smart enough to handle the load balancing and move threads from an over-occupied core to an idle one.

In any case, we are working on dynamic re-assigning of buckets towards the render end.

Best regards,
Vlado

[renderfarmer]

Thanks Vlado. I don't know if intel or Microsoft are to blame but the management of HT load balancing has been quite poor in my experience. Maybe the Sandy Bridge cpus are better, I'd have to do some head-to-head comparisons at home in my free time (free time, haha!!).

[zoranm]

If not, then would it be possible to automatically divide up the last bucket of a given rendering into buckets of sufficient size to occupy all of the available CPUs? Even if it had to render some extra pixels for AA I recon this would result i a much faster final rendered image than otherwise leaving that slow logical cpu to finish the job.

Thanks.

I have asked for this before. We have some 40 PCs for servers/DR and it's always that dreaded last bucket that's been asigned to the slowest machine in the office. The other thing is that closer to the end of render you get, less and less contribution from slaves you have. It'd be nice if we can specify a "percentage of remaning buckets" that should be subdivided and re-distibuted to other machines.
For example:

percentage of remaning buckets = 10%
subdivision multiplyer= 2


After 90% of rendering is done, remaining buckets would be divided into 4 new buckets each and re-destributed to other PCs
That'd save some time for sure, even if some additional time is requared for smoothing inbetween the buckets.

regards
Zoran

[vlado]

Yes, that's what we were planning anyways; it makes no sense to do it for the last bucket only

Best regards,
Vlado