Horrible ram issue with animation

[fixeighted]

I have this scene that is only 118mb but is using a load of ifl sequences.
I am using scanline for speed but it's the same thing with Vray (only worse).
What happens is that I can only render around 40 frames before having to restart Max due to it using up all 64g ram
and not releasing it even after the render ends.

Is there any way of solving this at all?
I tried freescenebitmaps but it does nothing

I have so many shots to do that this may break me

[Joelaff]

Have you tried VrayBitmap rather than the built in Bitmap mode?

[fixeighted]

Yes, actually, even though I am using scanline I had the mats already set as Vray, which I was surprised to find that the scanline renders just fine.
So even with Vraybitmap no change.
I've set the file to open in one window, wireframe, turned off the buffer, enabled bitmap paging....every last thing I can think of.

Admittedly I have so many maps that that just may be the issue. 41 materials, each with a 6500 file ifl. This loads with no issue at all, and even though
I have the sequence limited to only 200 out of the total 6500 for most shots, this makes zero difference, which is utterly frustrating.

So it is like although the ifl says to look at only 200 of 6500 files, it still loads all of them.....

[Joelaff]

Would be interesting to see if using tiled textures and vRayBitmap worked, but then you would have to use VRay to render I think.

So there are only 41 images that need to be loaded per frame? The rest are part of IFL sequences that change each frame, right?

[fixeighted]

Yeah I need to use scanline for pure speed, as the fake dof is perfectly adequate.
In any case with tx files being the mammoth size that they are, I would have to buy several terabytes more disk space just to store those, so not cost effective.

Unfortunately all 6500 frames of each of the 41 materials need to be used, so 1/4 of a million files LOL - it is a crowd of kids singing in a stadium, to give you the idea.
I could break them up into e.g. only that time segment needed for any one shot, but that issuch a load of extra work that it's not feasible.
Plus, the fact is that there are around 60 shots and they all feature different amounts of the total, some with them all in.

It's a monster and has been extremely stressful so far, trying to figure it out.
The rest is a breeze - running the animations of the actual stadium, inside and outside, with around 65k people audience, landscape etc, through Vantage at lightning speeds
with very few problems. Just these feckin' cards....grrrr

I have today managed to get roughly 70 frames done before pc say bye bye, then I reset and go again. So that is doable, as most shots are max 200 frames - and at least I can just leave the entire bulk of seqs in the project.

I can safely say that this is the one project in my career that has absolutely no chance of ever being repeated

You live=you learn

[Morbid Angel]

Hi fixeighted
Are you rendering the entire sequence on your single machine using 3ds max ui session? 3Ds max natively will cache the textures in ram (so that subsequent render will be faster). Your best bet is to render it using backburner or equivalent, where each of the frames will be loaded/unloaded entirely during render process, that way you won't run out of ram.

[Joelaff]

In the future you might consider making some large textures that look like contact sheets where you put multiple people in each texture and you randomize the UV offsets. That way fewer individual textures are needed.

Tiled textures from the outset would be good too, though that would possibly be a lot of I/O, but I guess less than loading all the textures full res.

Those are always fun projects. Good luck.

[fixeighted]

Morbid Angel Thanks for that suggestion. Yes I'm on one machine and didn't even consider Backburner for this, so will give that a go now....didn't even have it installed
Fingers crossed this will solve the problem.

Joelaff Already there with this...these are grids of 20 kids per, as that was the maximum possible for the required resolution. So this is 800 unique kids with them instanced to create a total 3000+
Simple enough to lay out...just getting them rendered is killing me...unless BB works of course

There are all sorts of extra hurdles also, with having to get cameras in from Vantage which do not match properly, which use different unit conventions for focal length, need to be translated to a regular ancient Max camera for the fake dof....yeah, a ton of fun this one

Oh yes, the cameraman/client is doing the moves, only has Max 18, refuses to upgrade, doesn't know Max so I have to both save everything for that also, teach him how to basically use parts, teach him Vantage...the list goes on

[fixeighted]

Now I remember why I hated BB in the past...the setup is a fucking nightmare.
Did anyone run into the 'missing message' error when submitting?
Fixes include modifying the registry, so if you guys needed to do that then were there problems anywhere else when 'preferring IP4 over IP6'?

[fixeighted]

So I did get BB running the file, however the ram is still going up on each frame.
Did I misunderstand Morbid Angel and maybe this doesn't work if using a single pc, or is there a setting I have overlooked that tells BB to do the unloading?

EDIT: One thing I just noticed that although the ram still goes up using BB, at least I can suspend the job, which immediately frees the ram of course, by closing the Max clone.
Means I don't have to reset Max to clear it. So a partial victory for which the credit goes completely to you for even suggesting its use in the first place

I just don't understand why Max cannot release memory as a matter of course.

[Joelaff]

I think BB has an option to control the number of crashes rendered before max gets restarted. But we don’t use BB, high most render controllers do.

You could also use a batch file to run various frames ranges via 3dsmaxcmd.exe. Do like 5 frames at a time, or whatever you cash get away with. That would be pretty easy.

[fixeighted]

I think I've decided to stick with Backburner for these, as at least I can let peak usage go to 98%, suspend the job for a couple of seconds and it resets the ram so I can just resume it to finish the sequence.
Given that most sequences are taking only 20 mins then it's bearable.

This method raises the question; why cannot Vray utilise some similar method to periodically 'flush' the ram as BB is clearly doing?
Hopefully one of the devs has seen this post, or I'll ask in a separate one. It seems to me that BB can't be doing anything particularly magical to accomplish this.
Or maybe there is some script that says to BB 'suspend after x # of frames then resume for x #'
I'm no coder so have no clue...could be wishful thinking

Thanks for all the input...it's been of great value.

[Morbid Angel]

To be honest I can't remember how bb works its been long time since I used it. You can probably install deadline or a similar render manager they are typically free to use if you have just 1 machine, in deadline there is certainly an option to run job once per frame, flush the render info then load it again. Its called reload plugin between tasks.

[fixeighted]

Hehe, well I spent the best part of 3 hours trying to get Deadline downloaded installed and going through its very convoluted process of registration etc. etc, only to find that
the only options available were cloud based subscription pricing for not only Max use but Vray use also.
So I gave up and deleted it all

I also tried Pulze but that can't be used on the same pc.

Given that BB just worked and with no real hassle it became the favoured option.

I just want this whole project to go away LOL