Crowds and renderfarm worklflow

[LL_Will_J]

Hi all,

We are investigating moving our crowds pipeline from Gollaem to Houdini. We have a maya renderfarm running vray and also the possibility to render standalone. However we do have limited Houdini licenses. So the recommended workflow is to write out to a vrscene and then to render with vray standalone. So now we are testing the workflow with a stadium with about 2000 agents. For 10 frames this allready creates a vrscene 40gb big. Same for caching first Alembic first. This might be somewhat doable for 2000 agents but for a full stadium with 50.000 people this would be quite impossible. By baking everything down to vrscenes you are missing the instancing advantage. Are there any other workflows we can investigate or maybe things we are missing.

Thanks alot!

[Gosho.Genchev]

Hey LL_Will_J ,

sorry for the delayed response.

V-Ray will bake the individual agents when exporting VRScene files, there is no way (that I am aware of) to circumvent this. Copies/Instances of agents using the same layer and on the same animation frame should be instantiated in the VRScene but that is about it.

A possible workaround is to bake the agents to alembic - for instance, a single alembic file per clip, with all layers inside of it. Once the crowd simulation is complete, it is possible to replace the geometry on the agent's points with the alembic files, using the "agentname", "state", "agentcurrentlayer" and "agentcliptimes" attributes to generate the correct alembic path. This should, in theory, significantly reduce the amount of data that needs to be transferred. However, I have no solution for blending between the agent's clips ... I'll try to figure something out and hopefully update you with a solution.

// as a side note - you don't need full HoudiniFX licenses to run Houdini on a farm in batch mode - IIRC an Engine license can also be utilized for this

All the best!

[LL_Will_J]

Hey Gosho,

Thanks for your response. I suspected there is no way to circumvent this. Unfortunately at this moment it is not really possible for us to have a Houdini engine license running for all the render nodes. Caching out everything to alembic would be an option but would add an extra step to manage data in a correct manner and would add other limitations like you mentioned. We are still evaluating if VFH would be a good addition to our pipeline but with this limitation of huge vrscenes the benefits would be too little unfortunately. I know you mentioned in other posts you are not planning to support the referencing of bgeo files in vrscenes but I really hope you will reconsider this. The native Houdini workflow where you write out a Packed Agent primitive to disk and then use that is very efficient. Especially when writing IFD's with Mantra, if all cached correctly, it barely takes time and disk space. Being able to reference bgeo's would truly expose the power of Houdini to vray and would users allow to use Vray with Houdini in an unrestricted way. So I hope you will reconsider.


Anyway thanks very much and I will keep an eye on the development considering this issue.

[GoodbyeKansas]

There are no plans to implement an Agent Procedural similar to the one mantra provides? We are already doing crowds in Houdini and would love to render everything with VRay but if we need to write out the geo for 10000+ crowds of pretty high res geos this will become crazy.

-- Erik

[bdancer]

> Caching out everything to alembic would be an option but would add an extra step to manage data in a correct manner and would add other limitations like you mentioned.

Not sure I got this - so baking bgeo is not an extra step, but baking alembic is?

> but I really hope you will reconsider this.

It's not about "reconsider", it's about time.
We have some subset of bgeo supported by ply2vrmesh utility, but current bgeo specs contain much more data.
I may implement some new types and make bgeo as a proxy input, but only for some limited data types (poly, hair, points) and only for one level of packed instancing. So, I'm really not sure how useful it may be. And i'll take a lot of time. And we already have all of this implemented for Alembic...

Fido Film, "Agent Procedural"? Like you want us to re-implement Houdini's agent animation system in V-Ray? Nop, this won't happen... Or I didn't get the question?

[GoodbyeKansas]

One could always dream

But no I am not meaning the whole system. I mean the part that somehow magically creates geometry at render time. Basically with mantra you don't need to write out all the stuff that is going to be rendered to the ifd file. It somehow does it when rendering. It makes it possible to render huge crowd systems fairly effort less.
I'll see if I can find some link describing it better.

-- Erik

[woytha]

I just come to this topic. When I am trying to find out, why my render is so slow.
I have just two quiet simpe agents (with several altering layers - like hair or moustache etc). I am instancing them throught crowdsource to number of 200-400... And preparation time of the render is starting to be really long.
It is like 1-2 minutes on Threadripper 1950x with 64gb ram. After the preparation phase it is rendered in seconds.

I just wonder - is it possible, that Vray is not handling the instancing? I know, that in the instance node I need to setup the Vray "raycaster instancing" on the node.
But it does not help with the crowds...

[woytha]

I just come to this topic. When I am trying to find out, why my render is so slow.
I have just two quiet simpe agents (with several altering layers - like hair or moustache etc). I am instancing them throught crowdsource to number of 200-400... And preparation time of the render is starting to be really long.
It is like 1-2 minutes on Threadripper 1950x with 64gb ram. After the preparation phase it is rendered in seconds.

I just wonder - is it possible, that Vray is not handling the instancing? I know, that in the instance node I need to setup the Vray "raycaster instancing" on the node.
But it does not help with the crowds...

Well, I got it. I was rendering the stuff direclty from crowdsource. If I do a crowd sim and then it feed into the new geometry node via dopIO then it is rendering literally in seconds!