I’ve implemented VRay render instancing in my plugin and it works fine to a degree, but it’s slower than expected and uses more memory than expected, so I want to know if I’m doing things right.
I’m doing things the same way that they’re done in the VrayInstancer sample project: at render time, new instances are created with calls to VRenderObject::newRenderInstance. Once those instances are created, they’re passed to the renderer with calls to RenderBegin and FrameBegin, and when a render finishes, they’re cleaned up with FrameEnd/RenderEnd.
However, when creating multiple millions of instances, these initialization/cleanup times can take extra-ordinarily long times. Several minutes. Testing other plugins (like Thinkbox’s Frost) using their VRay Instance mode takes only a few seconds for the same number of instances. Also RAM seems to bloat in my implementation…10 million instances of some lowres geospheres can require 50+gigs of RAM in some cases. sizeof(VR::VRenderInstance) returns 1320 bytes, so at minimum 10m instances would require about 13 gigs of RAM excluding any extra allocations during initialization…but the amount of usage I’m seeing is still much higher than what’s returned when using Frost for the same particle count, so I’m not sure what I’m doing wrong.
Since all of my instances use the same geometry, my thought is that maybe I’m doing things wrong. Do I need a new VRenderInstance for each actual instance? Or is there a way to pass VRay a single VRenderInstance object along with a list of transforms for it that represent the total number of desired instances or something? There’s not much documentation on how these things work under the hood, so I’m hoping a developer can shed some light.
Thanks!
This conversation makes me happy 
As an aside Tyson, how did you go about learning c++ or did you have a background in computer science before vfx?
No background in it, went from MAXScript to C# for Unity to C++ for MaxSDK. Took many years of trial and error 
I’ve been talking about learning c for years and kept waiting for the best way to learn it, I’d be making stuff at this stage if I’d just jumped in
Congrats on TyFlow, it’s looking seriously impressive!
Hey thanks! It’s almost ready for release…just a couple more features and a few more pesky details (like this) to iron out.
Oh there’ll be a lot of people in here very excited about that, we love having very elegant quick and dirty tools available to us for when the fx department gets a bit too tied up! What’s the oldest release of max it’s running in? We’re still on 2016 here for a while longer and no doubt there’s a few of the thinking particles folks that’d love to have some power back! I hope you’re selling it to recoup all of the development time!
Unfortunately it’s 2017 and above…is Thinking Particles no longer supported by 2016 either?
It’s more the rental idea that’s annoying, pflow / tyflow is far more accessible for people that haven’t used max much either!
Hello,
First of all some questions:
- what type of geometry do you want to instance - is it some fixed list of preset geometries like box,sphere, etc or do you allow users to specify custom objects
- do you plan some material variation options - like different instances to use different materials
- same for the various object properties/V-Ray properties - will all instances have the same options or there will be variation
Best regards,
Yavor
Hi Yavor,
1) I allow users to specify custom objects, but I also have a list of preset objects that they can choose from. However, for the sake of discussion, let’s say I limit users’ choices to a single node that will be instanced.
2) By default all instances will get the same material, but I also allow users to choose override materials for groups of instances. But let’s say users will only be able to assign the same material to all instances.
3) All instances get the same object properties.
The full VRenderInstance for each instance is probably the easiest way for custom objects - it will handle regular geometry but also special ones like VRayProxy, hairs etc.. And also you can easily do different materials or properties per instance. But as you’ve seen - it requires quite some memory..
For the simpler case where you have some fixed preset geometry instanced many times you can go with only one custom instance and some special geometry in it. Something similar is done in the vraymeshgeom sample in the SDK. The sample creates some spheres at random locations. Internally it uses only one custom VRenderInstance.
By the way - if you prefer we can continue by email 
Hey Yavor,
I haven’t studied the VrayMeshGeom sample project too closely yet in order to figure out what it’s doing, but it still seems to have pretty high memory usage. For example, if I set num spheres to 1 million, radius to 0.1 and num segments to 100, memory usage during a render (with default Vray settings, except GI is disabled and render mode is set to bucket) peaks at over 80 gigs (before I cancelled the render to avoid stalling my machine). It also only got through about 2 buckets in 5 minutes. So it rendered extremely slowly.
The same settings on a VrayInstancer object (1 million spheres with 100 segments) has a peak RAM usage of 8 gigs and the whole frame rendered in 5 seconds (after about 5 minutes of ‘Preparing Instances’). So it seems like in VRayMeshGeom, it’s not actually instancing the spheres…merely generating them procedurally on the fly and combining them into a giant mesh at rendertime? I’m not sure…
For reference, the same settings on a Thinkbox Frost object (1 million spheres with 100 segments set to render with Vray Instancing turned on) renders in 6 seconds with peak RAM usage of 2 gigs (their ‘preparing instancs’ phase takes about a second), and in my own plugin the same settings render in 4 minutes with a peak RAM usage of 8 gigs (my plugin uses a method identical to the VRayInstancer, with some additional multithreading in a couple of places).
So in summary:
1 million spheres, .1 radius, 100 segments:
- VRayMeshGeom: 80+ gigs of RAM, potentially hours to render based on the number of buckets that completed in 5+ minutes.
- VRayInstancer: 8 gigs of RAM, 5 minutes to render.
- My plugin: 8 gigs of RAM, 4 minutes to render.
- Frost: 2 gigs of RAM, 6 seconds to render.
Obviously Frost’s code is not public so we can’t know what they’re doing differently…but surely there must be a faster way to go about creating instances than either the VRayMeshGeom or VRayInstancer way of doing things? Either that or the Frost devs managed to implement time travel…
For some further information, I noticed that in my own plugin, the “compiling geometry” phase of instance preparation takes a long time (this is where compileGeometry is called on all the instances)…but why is that phase even necessary for each instance? Shouldn’t they all be using the same geometry? The same phase exists in the VRayInstancer plugin (instance.cpp, line 364) and is probably contributing to its slow speed as well. Could you shed some light on why compileGeometry needs to be called on all instances?
Hello,
You are right about the VrayMeshGeom plugin not really instancing the geometry - it creates each sphere separately. I’ll play a bit with the sample to get it to use instancing and will share it.
As for the compiling geometry phase in the VRayInstancer plugin. The VRayInstancer has an option to instance different geometry for every particle of the source. Also it can instance various custom geometries like the VRayProxy etc.. To handle those custom stuff and not have to write a lot of code VRayInstancer creates a full VRenderInstance for each particle and calls all the required methods (compileGeometry). But if you need millions of simpler geometries - then there are better options
Thanks Yavor, I’m looking forward to it!
I’ve made some quick modifications to the code to create only one real sphere mesh and the rest are instances of it with some different transform.
Also a thing I didn’t notice initially in the example - for some variance it creates a Multi/Sub-Object material with different material for each sphere - for a few spheres that looks nice but for a million it takes a lot of time and consumes ~10 gigabytes of memory
With those changes now the rendering takes a lot less memory - close to the Frost figure.
Rendering is also much faster but I’ll check a few more things to see if can easily be optimized
I’m attaching the original source and the changed version - don’t know which Max and V-Ray are you using so that I can also attached a compiled version..
vraymeshgeom.zip (22.5 KB)
Thank you very much, Yavor! With your modified code I now get identical results to the Frost plugin…~2 gigs of ram and 6 seconds to render a million spheres.
Might take me a while to get this new setup integrated into my plugin, but I will post my results here once I get it done.
Also…passing only certain channels (like the transform) to instances in your new code is interesting…can I pass other channels that will act as overrides on those particular instances as well? For example, materialD…or even UVs? Or do I need to create unique source voxels for those mesh variations?
Great that works now
As for other channels - unfortunately the only other channel that is obtained from the original voxel is the INSTANCE_ID_CHANNEL - one integer with some ID for that instance. It is saved for example in StaticMeshVoxelPrimitive::instanceID. After it is obtained the voxel is swapped with the source voxel and the rest of the code works with the source.
You can obtain it in the various intersection/shading methods through something like this:
// rc is VRayContext - shading methods have an input param const VR::VRayContext &rc
GenericPrimitive *primitive=rc.rayresult.primitive;
GenericPrimitive *ownerPrim=static_cast<GenericPrimitive*>(primitive->owner);
uint32 primType = ownerPrim->type();
if(primType==PRIM_TYPE_STATIC_MESH_VOXEL) {
StaticMeshVoxelPrimitive *voxelPrimitive=static_cast<StaticMeshVoxelPrimitive*>(ownerPrim);
....
Thanks for the example Yavor, great to see some really light c++ files to start browsing through!
Hey Yavor,
So how do I go about getting motion blur on these instances? I figured out how to assign more than one TraceTransform to the TMS_CHANNEL, but when I assign different translations to the tms, I get no motion blur (I’ve confirmed that the translations are indeed being set, by checking their assignment with getTM). I see a timeflags int in the MeshVoxel class, but my attempts to set the highest 16 bits to 2 (for 2 tm indices) had no effect. I can’t find any relevant notes in the SDK describing how to set moblur up either.
Here I’ve added some more code:
There was a missing method VR::Box getVoxelBBox(int i, int timeFlags) in the sample - I’ve added it and made it call the regular getVoxelBBox. The time flags param is actually needed mainly for alembic proxy files.
I’ve also modified a bit the buildVoxel code:
- the source voxel (index=0) doesn’t generate vertex velocity - if it does it will be used on all instances and if the instances
- instead the instance voxels now provide two transforms in the TMS_CHANNEL
In addition - I’ve missed to modify the getVoxelBBox for voxel 0 and it was returning a wrong bounding box.
And one final change - in getVoxelFlags I’ve added the MVF_HIDDEN_VOXEL flag for the source voxel - that way it is not rendered at all. This is not strictly necessary - it’s an option if you don’t want to show the source, placed at (0,0,0).
vraymeshgeom2.zip (11.9 KB)