Render farm Management

[robusto92]

I have set up our 10 blade render farm it works well except when multiple people try to send renderings. I have set it up with back burner but is there a better software to use to actually que multiple renders or can back burner actually do that? Currently we wit until the farm is clear to send a new rendering. I probably did not setup things right, any help would be appreciated

Thanks

[caypiranha]

for instance "Deadline" is quite common. Maybe you have to invest a little, but this is a reliable tool to count on.
There is also a Tool called "RenderPal" I used for small farms, ages ago, but it´s still under development. Probably a bit cheaper than Deadline. But there might be many others as well.

Back Burner, is one example of an Autodesk Tool, which has existed for ages and never got a real cool update. Not ever to mention about getting better features. Typical Autodesk. What should we say?

[Morbid Angel]

I have set up our 10 blade render farm it works well except when multiple people try to send renderings. I have set it up with back burner but is there a better software to use to actually que multiple renders or can back burner actually do that? Currently we wit until the farm is clear to send a new rendering. I probably did not setup things right, any help would be appreciated

Thanks

there should be no issue to just submit multiple jobs at once to backbruner? did you encounter some problem? bb is limited as its free. I would also recommend deadline, though it costs money but well worth it.

[Morne]

+1 on Backburner should be able to handle things just fine. If you're constantly having issues, maybe setup the manager on a non rendering pc and see if that helps

otherwise, there's also this:
https://www.pulze.io/products/render-manager

...which is apparently very good

[kosso_olli]

We tried Deadline and Pulze recently. Deadline is a nightmare to set up and we could not get jobs with DR to work. Pulze is a breeze. Very easy, very clean. There is not decision made yet, but so far Pulze is ahead by miles.

[Alex_M]

I have had the same experience with Deadline. Last time I tried it (a few years ago, I think) it was indeed a nightmare to set up so I just gave up on using it.

[robusto92]

You are correct I can submit multiple jobs but the first job is the only one that gets the benefit of the 10 blades. Also the 2nd job will start on a blade but it will not have as many cores to render because the first rendering is going also. Ideally I was hoping Back Burner could be setup to accept many jobs but only run 1 at a time, so all the renders get the 10 blade benefit

[Morne]

You are correct I can submit multiple jobs but the first job is the only one that gets the benefit of the 10 blades. Also the 2nd job will start on a blade but it will not have as many cores to render because the first rendering is going also. Ideally I was hoping Back Burner could be setup to accept many jobs but only run 1 at a time, so all the renders get the 10 blade benefit

You can easily do that with backburner
The workflow is a bit different per whatever solution you go for, for example:
1 still image per server
2 still image, using DR, split over all servers
3 animation, regular, 1 frame per server (or if the frames renders fast, it will typically assign a batch of frame, for example 10 frames at a time, to a single server)

Also, you can set priorities. So if a job has higher priority, it will skip the waiting list
Also you can add/remove servers to a particular job, while other jobs are going

For example if you have all 10 servers busy with 1 job, you can remove 5 of them, and assign them to the next job, etc

Use the BB Monitor for all this

[robusto92]

I did forget to mention all of the blades are vray render nodes. So when I send a rendering, it only shows up on 1 blade in the back burner monitor, but once the rendering starts, it is utilizing all the blades but still only shows one in the monitor. So if someone else sends one it goes to another blade then tries to DR but is blocked because the other render nodes are busy

[robusto92]

Ok I think I have it figured out just need to test. I am going to stop the render server software on all except 1. Then when we submit jobs there is only 1 possibility so it should utilize all the blade via the Vray DR

Thanks for responses

[Morne]

Well if you're rendering stills, that changes things.

You then designate 1 BB node as the "main render" and only submit the network render job to that single node. However, in turn, if you have DR ticked, then in turn that node, will use VRaySpawner (not backburner), to distribute to the other nodes. Backburner has nothing to do with VRaySpawner, ie DR rendering. So there is no way for Backburner to see that the other nodes are busy rendering a DR job

So then when submitting a new DR job via backburner, the job will be picked up very slowly, because there are no resources since all nodes are already busy.


So in other words, when submitting a new DR job to a single node, that node will act in a similar way as to when you just submit the DR job without backburner from your own machine. So if in the DR server list, you have ALL nodes listed here, then all nodes will use the spawner to render

The thing to remember is that BackBurner and DR are 2 totally different systems and the 1 has nothing to do with the other.

So have to understand this fact 1st then manage things accordingly

For example, if you have 3 artists for example, you could assign 3 nodes to each artist and when they send backburner jobs, they are not allowed to use nodes from the other users. That means all 3 artists will be happy as they can DR at any time without having to worry that somebody else are using "their" nodes.
The next thing to keep in mind with this scenario is that you should not submit a backburner job to "all available nodes". But you should send the BB job to a specific node
For example if you have 3 artists and each has 3 nodes to their disposal, then:
Artist1:
Node1, Node2, Node3 (Submit BB jobs to Node1, and make sure in DR, node1,2 and 3 are ticked)

Artist2:
Node4, Node5, Node6 (Submit BB jobs to Node4, and make sure in DR, node4,5 and 6 are ticked)

Artist3:
Node7, Node8, Node9 (Submit BB jobs to Node7, and make sure in DR, node7,8 and 9 are ticked)

The downside of this workflow is that often you will have nodes sitting idle. Note, that is specific for still rendering.


For animation rendering, its a totally different story and then you should not use DR, but submit to all nodes via backburner. Keep in mind though that if artist 2 is busy with a still render, then backburner will say that Node4 is busy, but it will seem that nodes 5 and 6 are ready (which is a lie), since DR does not utilise BB, there is no way for BB to know those machines are busy. What will happen then is that BB will attempt to send the jobs to nodes 5 and 6, but will fail since there are hardly resources available as its already busy with a DR job (via VRaySpanwer)