Controlling Memory Use when rendering PHX

[Joelaff]

Any tips on reducing memory footprint when rendering? My scene is a little odd with regard to memory. I will see one frame render with about 6Gigs, and the next needs 21Gigs. These are successive frames in an explosion. There is not much of a visual difference between the frames. Though I am am using the playback Anim Index, but all my frames are integers. I do a hold (freeze) of an explosion, but with a moving camera.

My AUR files are about 500megs to 900 megs at quality 12. Not sure how many cells that is. I did a 0.7 resimulation on a ~55million cell grid. So maybe 80million?

The big memory use comes during "Transforming vertices."

Any thoughts? This happens with our without motion blur.

Thanks.

[ivaylo.ivanov]

The adaptive grid algorithm is not perfect and it reacts on spikes in the velocity. In the nightlies, this algorithm was changed a bit, and more precise one is used. It improves the problem somewhat. If you are not using a recent build, I will recommend you to try a nightly and see if this helps your situation.
Your files seems huge, are you using particles of some sorts? The particles are currently saved uncompressed, because we don't have efficient compression for them yet. And we will slow the reading/writing of the files, without much gain...

[Ivaylo Katev]

6 to 21 gigs is too big difference, it's strange for frames that are almost equal. one possible reason can be the interpolation of the frames, i will take a look.

[jacksc02]

Hi Ivo,
I am having a similar problem but not with memory but with the size of simulation files.
I have modified the submarine sample file and the simulation files spike from 16mb up to 2.1gb!
I have attached a screen grab of part of the simulation output files to show the file sizes

[Ivaylo Katev]

very interesting!
there is an option for backup interval, that automatically includes the velocity channel once per given time, but this cant be the case, too big difference in the size.
can you compare the content of the frame 188 and 187 ? or better make screenshot of the input panel when these frames are loaded

[jacksc02]

Ivo,
If I render at frames 186 or 188 nothing renders but 187 renders as it should.
The backup interval is -1
Attached is a better list of the .aur files

[Ivaylo Katev]

the disk is full?
i noticed when there is no enough disk space, there are small aur files that are not renderable

[jacksc02]

That is it!
I was saving the files to my SSD but they got too big!

[Joelaff]

I will try one of the recent nightlies... This may actually be the issue. I think many of the render nodes have a nightly build from a week or two ago, while others have the release.

My files are so big presumably because they are pretty detailed. They also stretch out over a large area. I do not have particles simulated, but I do use particles to make the simulation. In other words I use a bunch of PFlows to controls where streaks of stuff explode outward (initially looks somewhat like a bunch of rocket trails radiating outward and upward from a central point). But the entire volume (grid bounding box) is not full. There are areas that I would think would compress down.

[Joelaff]

Was out of town on location the past few weeks. But I got back to it. I tried the Aug 9 nighlty build.. Still varies between about 5 gigs to up to 24 gigs per frame when rendering.

No time to re-simulate now, though.

[Ivaylo Katev]

can you share the scene and if possible two cache files with big difference in the memory consumption?

[Joelaff]

I should be able to once I get past some deadlines (like in a week or two).

[Joelaff]

Just wanted to get back to you on this. I think the memory usage was actually narrowed down to an instanced particle system that was also being used in the scene. Even when rendering Phoenix we had the particles on to act as a matte object.

Thanks.