Fireball at cam simulation - resimulation optimizing question

[pikok]

HI.
I'm new here so I apologize for questions that might appear amateurish.
I've been working on a sim which is essentially a fireball that travels through the mine corridor and hits the camera. (see the attached photos)
The problem is resolution - when it's close to the camera, flames are splotchy and blurry - so obviously a resim is needed.
I'm on i9-10940X 14 core, 128 GB RAM, but at 1 billion voxels things got a bit slow , and the deadline, is tight, as it usually does.
I would be super greatful for any tips on streamlining the process or optimizing the time of the upresing. I've read through the docs, but reality is harder to grasp :/

The shot is roughly 50-60 frames total.

Here's my amateurish take on the problem and the route I'm considering:
1) Pass one - 120 000 000 sim (I'm using 2 SPF, 3 hrs)
2) Resim with wavelet export gives me 1 billion voxels approx. (about 8hrs) - hit render.
3) Further resim the previous resim, but only 10 last frames or so, where flames hit the camera and render that separately.

Further considerations and questions regarding my approach.
- is it gonna work? ) Is that approach something that has been tried before?
- Is it possible at point 2 or 3 of resim to reduce SPF to 1 to cut the resim time in half?
- Since the resim times are big, can I stop the resim and restore normally like I would with the standard sim? (it's hard to sacrifice hours of resim just to test that...)
- is there a way to add detail to flames close to the camera in any other way? Like modulating with a grid channel etc?
- is 128 GB ram gonna be enough to pull this off? At one billion I'm closing to 110 GB used...

and the last question - kind of off topic - Is it possible to add drag particles to an existing sim later on when I'm happy with the look? (without having simmed the particles in the first place)?

Much obliged for any help on the matters above!
Btw - love the software, my hardware just doesn't keep up :>

Best regards

Paul

[Svetlin.Nikolov]

Hey,

A lot of questions, hope I don't miss any:

- Seems like you want to resim over a resim. This works, but you need to redirect your cache paths and change the No-Export Channels option. In more details: Once you're done with the sim and then done with the resim, you gotta point to the _resim cache file sequence from the Resimulation rollout's Base Cache Files Path, and then set the Output and Render Path to the final caches - how you name them is up to you - _resim_resim, _resim_final_final, etc. HOWEVER, resimulation is a sequential process, like the simulation, and it won't be able to start from the middle - you gotta start from the beginning because Sources and obstacles are still working even during resimulation. If you decide to go with resim of resim - in the Resimulation rollout you must make sure to set No-Export Channels to None right from the start, otherwise it would skip the wavelet and UVW channels you need for resimulation.

If your sim is already taking 110 Gigs out of 128, I honestly don't think there is any way to fit that sim in memory - resimulation needs a lot of extra grid channels to run. An alternative would be to run a full blown simulation from start to end on the highest resolution you can fit in your memory, with as few Grid channels in the Output rollout as possible - ideally only the Smoke and Temperature channels, plus Grid Velocity if you need motion blur.

- When you resimulate, it's best to run the original sim with 1 SPF and then the resim with 1 SPF. If you use more SPF, the resimulation might start looking very differently. If the SPF between the original and resim don't match, it might also look very different. So sim 1, resim 1 is the best approach. Otherwise you might be lucky and get away with it, but there is no way to tell in advance until it's too late.

- Restoring a resimulation should work - we have test that regularly.

- Indeed you can add details close to the camera at render time. Check the "How to add more detail to the color and opacity of volumes using TexUVW" section here: https://docs.chaos.com/display/PHX4MAX/How+to+Videos. However, you must simulate with TexUVW enabled in the Output rollout, so this will increase the memory at sim time, but on the other hand you will be able to tweak the noise in the shader in any way during rendering.

- Aaand finally - yes, you can add drag particles with resimulation - you have to enable ONLY the Particle Resimulation checkbox under the Resimulaiton rollout and this should be all

Cheers!

[pikok]

Hi Svetlin. Thanks for a detailed answer.
I'd like to ask you a few more questions though:
- so lets say 1 billion grid sim from the start would take less memory than sim-resim approach?
- to render fire and smoke I don't need the fuel channel? Is that right? I have a render setup to use temperature for fire as I remember correctly.
- I have an issue with UVW mapping - dropping noise into modulating map slot dims the fire, but no additional detail shows up. Would it be due to the fact that the sim travels 12 meters within 2 seconds so?

[Svetlin.Nikolov]

Hey,

When you resim, you get more voxels based on the Amp.Resolution option. If it's 0, you get the same number of voxels, i.e. the same memory as with the base sim. If it's 1, you get 2x along each axis, so 8x more voxels and 8x more memory. So if you start off with 8x more voxels for the base sim, it will be the same as resim with Amp Resolution of 1, which is the default value of this option. However, it will be slower to simulate and iterate - this is the catch.

In order to resim, however, you need to run the base sim with Grid Velocity and Grid Wavelet, and each of them takes 3x the memory of the Temperature channel or Smoke channel. So Temperature + Smoke takes 2 numbers per voxel, but Temperature + Smoke + Wavelet + Velocity takes 8 numbers. This is for the base sim though, which has less voxels, but it will still be an overhead when you run the resimulation, and I can't say off the top of my head if there would be more memory used internally. So it's not so much after all - if I'm not forgetting something big, you should be able to pull off a resim as well.

The Fuel channel is needed when you need burning - from the Fuel rollout. If you don't need that, don't enable the Fuel channel at all. Phoenix's shader is very flexible and you can render the Fuel channel as Fire or as Smoke, but by default the fire comes from Temperature and the Smoke comes from the Grid Smoke channel, so unless you want to do some special shading effect, you won't need Fuel for the render.

However, I can't really say what's happening with the UVW without more info - it should not be connected to the velocity of the fluid though. If the noise is between black and white and does not go brighter than 255, then it's expected that it will decrease the strength of the fire - this is what modulation does. However, could it be that the scale of the noise is very large or very small? Can you try extreme changes, like increasing the scale 10 and then 100 times and also decreasing it like that, in order to see if some sharper detail shows up?

[pikok]

Thanks Svetlin. I really was close to hitting the ceiling with RAM on this one. At one point I had to start killing processes so the PHX wouldn't stop simming as the usage was about 126GB :>
When I have time i'll prepare the scene file, but meanwhile I think I'm gonna pass with UVW solution, sim holds up ok even in 4k, movement makes up for the resolution. I'll have to further investigate the UVW mapping - it's a hit and miss for me, even following the smoke tutorial - probably. I don't know if anyone else would dig it, but a way for previewing and assesment of UVWs would be nice :> Maybe on a mesh surface? IDK

As for memory menagement - I could have easily crank the sim up to 256 GB RAM usage if I had that much memory - the question is how do sims that big or bigger than that? The RAM basically limits the sim resolution, there's no way around it?

[Svetlin.Nikolov]

Ah, I believe the Phoenix mesh would show the TexUVW mapping correctly... at least in rendering, but probably in the viewport too.

And indeed, the RAM limits the sims...