help! huge scene lags with large bitmaps.

[super gnu]

ive got a scene with two extremely large jpegs for the ground plane. one is 10kx20k, and the other is 16kx16k. i set them both to load via the vrayhdri loader.

ive got bitmap paging disabled.

there is nothing else heavy in the scene.

i have to wait 10 minutes after a render finishes for max to come back, and i get pauses of 5-10 minutes in the material editor when i try to edit those materials.

what can i do to fix this?! ive got a crapload of work to do, the scene is about to get 100X heavier, and with these delays im in real trouble.

[super gnu]

hm. so far seems to be an issue (partly) with the hdri loader.

ive tried converting to png (not compressed) no change.

ive tried converting to tiled exr. no change.


if i switch tot he bitmap loader, the lags in the material editor go away.. however i still have the 10 minute wait at the end of a render.

[joconnell]

Run the jpegs through the image to multires exr converter or since they're 8 bit use make tx to make a multi res tiff - you're getting the full whack of your 20k image whether you need it or not, the multi res stuff will make a stepped resolution version of the image that looks the exact same but only loads a res you actually need for the final size each time.

[super gnu]

yeah i thought that would help too. but i cant find maketx online, or a guide as to how to use it for vray, and with multires exr, i get 1.6GB files, and no noticeable improvement.. (ive not tried rendering, but its the interface lags that are killing me more, and only getting rid of the hdri loader seems to fix that) seems i have to go back to the max bitmap loader.

might try merging into a new scene..

itss also rapidly climbing up to 25GB memory usage when i work with this scene.. at the moment its just a landscape with a few roads and streetlamps in. ive still gotta bring in the 1000+ buildings and 40,000 trees.

eeek.

[joconnell]

Ah kk - yeah mulitires will definitely help in rendering and less so in the ui. I've found the same thing with the ui at times and I'm never sure if it's using rather high sampling settings for the shaderball previews or if it's network congestion pulling the textures :/

[super gnu]

its definitely not nework congestion.. and tbh i dont think its shaderball sampling.

im pulling the textures from a local ssd raid array which does almost 1.1GB/sec

plus i get the 5 minute lag

every.

single.

time.

i click on a checkbox or button in the offending material. i dont think it redraws the shaderball that frequently?

seems like either a screwed up max file, a max bug or a vray bug. the fact that i dont get the interface lags with the bitmap loader suggests its a problem with the hdri loader.

[super gnu]

ok this is wierd.


ive swapped all my big textures back to bitmap loader

saved file
reopened max and merged scene in
seemed to be working ok.

open environment tab and choose vraysky from map lister

3 minute lag before adding it.

wtf?!

[super gnu]

hm.. does the standard max bitmap loader work with multires exr's?

[joconnell]

Unfortunately not.

[super gnu]

ok so, ui lag and pause at render end appear to be unrelated. as mentioned i can fix the mateditor lag my chaning to the bitmap loader for my large textures.. however i still get the pause at render end *without* those large textures in the scene.

ive got displacement off in global switches (its not my model so you never know whats hiding on a screw head in a window frame somewhere)


next up is to go through the scene to find wahtvever is killing it.

all good fun.

[super gnu]

ok so its one of those..

i went through layer by layer, turning things off and noting wether the delay after rendering was present.


it was nonexistant with a single simple object on screen, but, frustratingly, seemed to increase linearly with each layer i enabled.

so, no one culprit.

damn.

tried disabling embree, no effect.

only thing that maybe made a difference was switching default geometry type to static. but thats no solution on a scene with (eventually) thousands of proxies in i think.

[radoslav.platikanov]

About the VRayHDRI lag - currently the VRayHDRI viewport preview seems to be very slow with large bitmaps (including tiled exr images), so this could be causing the lags in the UI. The problem is partly in 3ds Max, but we'll definitely improve it one way or another. Meanwhile you can try hiding the maps in the viewport if that's an option. Another workaround that should reduce the lag is to use the legacy Direct3D viewport with the "Match Bitmap Size as Closely as Possible" option in the viewport driver settings turned off.

[super gnu]

hm.. i did try turning the viewport textures off entirely, but it didnt help.

[super gnu]

interestingly (and worryingly)

when max freezes up after the render for 5 loong minutes, the ram usage goes gradually UP.?!

i opened the scene with 6 gb ram usage.. after messing around with terrible delays in the mateditor, trying to set up and assign the tiled exrs i created, it was sitting on about 9gb

during rendering it went up to 15 gb.

during the 5 minutes i was waiting after the end of the render, it climbed up to 24gb ram usage

its now sitting on 24 gb of ram and im back in max.



this is for a scene i would expect to render in 5 - 10gb max.


im not sure its all to do with the big textures either.

ive been through the scene, it was a bit heavier than necessary, about 6 million polys. ive optimised it down to 3 million. didnt seem to make much difference to ram usage. max file has gone from 600 meg down to 200 though.


id love to send the scene in for testing, but 200 meg max file, + 2gb of tiled exrs + another 2 gb of textures. will be problematic.

[super gnu]

ok its some other issue with the scene i think, or a combination of issues.

if i open a fresh max scene and create a plane, and add a vrayhdri with my big exr into the scene, its all instant. no lags.


plus the behaviour of my file is so wierd. it takes ages to open, and when it appears to have finished loading, you touch the mouse, and it imediately starts thinking again, and ram usage climbs (then drops, then climbs again etc.etc.) another few minutes of that before the scene becomes interactive.

add to that the unusually large ram usage overall for the scene, teh terrible mat editor lags, and the big delay and odd ram behaviour after rendering.. it all sounds really.. wrong.


ive got to get this scene super optimised in the next couple of days as i will be really pushing the limits on polycount and animated geometry. the thought of *starting* with a scene in this state is quite disturbing.

ive looked through the scene, only seem to be editable meshes in there. nothing unusual certainly.

im on 3.3.03 and ive tried in 2014 and 2016 with the same results.

ive tried merging into a new scene, ive tried with viewport textures off, ive tried turning layers progressively on or off.. cant pinpoint anything.


im going to try to package up the (enormous) scene. hopefully somebody at chaos can work out what the hell is going on.