Particles to grids (SDK)

[tysonibele]

Hey guys,

I was working on a project where I need to scatter some PhoenixFDFire objects around the scene....same sim...just need to offset the frame timing for each volume and copy each grid to a specific location. Several thousand of them. Obviously scripting this is one option but it makes for a very heavy scene and updating or making changes to the grids is very clunky.

Was wondering if PhoenixFD has any API calls either in the works or already existing (I looked for them but didn't see them) that basically allow you to send grid info to VRay at rendertime...similar to the VRay proxy API that allows you to send VRay geometry voxels to the renderer. Imagine a scenario where the render begins and VRay calls FrameBegin on my object like usual, and in that method I can pass IAur objects to vray with a transform and frame offset. Using that method I could send thousands of grids to VRay at rendertime only, and they could all be instantly updated with any changes to the underlying particles.

I guess if there's no specific API for this type of thing I could just manually do the copying + updating in the RenderBegin/FrameBegin functions, but wanted to check to see if there's an official workflow for something like this first.

[Svetlin.Nikolov]

Hey Tyson,

Sounds like an internal interface we created for the V-Ray Instancer a while back. Let me check what it does and how we can expose it through the Phoenix SDK and will get back to you. Off the top of my head, a few things could require special attention:
1. The existing interface most likely does not support different times for each instance.
2. Phoenix cache data currently is not read in chunks from the hard disk in order to minimize memory usage. Instead, loading a cache channel (smoke, temperature, etc) loads it in memory and uncompresses it on access. So, having many frames of the same cache sequence means that all of them will occupy RAM, so instancing large caches might need good error checking until out cache loading system matures more.
3. If the existing volume instancing interface could be exposed, it would most likely be a Max-only thing, and when I expose functionality through the SDK, I try to wrap it and have it the same between Max and Maya, so this might slow me down; This might not be relevant though, let me first dig in...

Cheers!

[tysonibele]

Cool! Although in the meantime I've since found a workaround (copying containers to particle locations at render start and deleting on render end) so please set this investigation to the lowest priority on your list. In fact, my workaround is more flexible than I imagine some direct interface call would be since it allows users to customize the reference nodes however they like, rather than me needing a full set of API customization functions behind the scenes....so you can probably just skip the whole endeavor

BTW I had a hell of a time getting render-time Phoenix containers to render. Copying them within max's PRE_RENDER callback doesn't work, despite other object types rendering fine that way. I'm guessing you guys are initializing PHX even earlier in the render loop? When examining the callstack during render I can see that VRay is doing a bunch of stuff before broadcasting the PRE_RENDER notification, which is fairly counter intuitive. I'd imagine that PRE_RENDER should be called.....prior to any rendering. I had to ultimately create some nasty hooks into the max interface to initialize my container copies when the render progress window is initialized by the max API, which luckily seems to occur prior to PHX doing the necessary initializations and thus my containers then render.

Again, this is low priority since my workaround is fine for now, but if you get a chance to look into why container copy/creation during a PRE_RENDER callback is not an early enough time to do it in the VRay render loop...that would be great

[Svetlin.Nikolov]

Ah, roger that. So the 3ds Max pre-render callbacks are... very special Phoenix uses a ton of them. The original reason is our attempt to support Defscanline, V-Ray, Corona and Octane. It turned out that some of the callbacks are called in one order by some renderers, and in reverse order by others. Some callbacks are called twice, some are called one before rendering begins and then at each frame. And there are major differences even between the calling patterns of Defscanline and Arnold. A further limitation in that e.g. in V-Ray we can't ask for some stuff, such as motion blur parameters, earlier than some point in time, while some other of our exotic functionalities, such as creating lights to illuminate the scene, can't happen later than a particular point in time.

This are the callbacks where Phoenix does stuff, order the way V-Ray and Scanline call them:

// NOTIFY_PRE_RENDER
// Animatable::RenderBegin()
// SimpleObject::BuildMesh()
// VRenderInstance::renderBegin()
// ----- Each Frame:
// NOTIFY_RENDER_PREEVAL
// VRenderInstance::frameBegin() <- The VRayFrameData is available only from this point on
// Atmospheric::Update()
// VRenderInstance::compileGeometry()
// StaticBunchPrimitive::getBBox() <- If this is not set by now, we can't continue. InitVolume() or buildParticleTree() must be called before that
// NOTIFY_PRE_RENDERFRAME <- The scene must not be modified here or later - emissive lights must be created earlier
// SimpleObject::BuildMesh()
// ----- Rendering occurs here

However, we do motion blur preparations twice in order to accommodate the Corona order of callbacks.

In Volumetric mode of the Simulator and the Particle Shader, there is one more player - the Phoenix Atmosphere that we use to gather all volumetric effects so they can render together and blend correctly.

The bad thing about both the max notifications and the V-Ray virtuals is that they can be called in any order for scene nodes, so if e.g. one node does something in NOTIFY_PRE_RENDER that depends on another node doing something in NOTIFY_PRE_RENDER, there's a chance that it might look as if it's working at first, until it does not, so for pre-render steps that require preparing data incrementally, we have processing going into consecutive callbacks in order to make sure.

Particularly under NOTIFY_PRE_RENDER, all Phoenix nodes register themselves into the Atmosphere, and create fire lights - this should be the very first callback Phoenix uses when rendering starts.

I am hoping that with time we would be able to drop this concept of a global atmosphere object since even though it simplifies some stuff, it complicates other stuff as well...

[tysonibele]

Particularly under NOTIFY_PRE_RENDER, all Phoenix nodes register themselves into the Atmosphere, and create fire lights - this should be the very first callback Phoenix uses when rendering starts.

Ah yup, that explains the behavior I was seeing then.

Thanks for the explanation!