help! huge scene lags with large bitmaps.

[radoslav.platikanov]

plus the behaviour of my file is so wierd. it takes ages to open, and when it appears to have finished loading, you touch the mouse, and it imediately starts thinking again, and ram usage climbs (then drops, then climbs again etc.etc.) another few minutes of that before the scene becomes interactive.

This seems very similar to my tests with large images. The memory usage behavior can be caused by the same problem with the viewport textures. Still, it could be something else in your case.

[Morbid Angel]

I suppose you have tried merging into new scene?

[super gnu]

hm ok you might well be right. it seems that with nitrous, setting the file to "display object colour" rather than "show scene materials" then reopening the scene is -not- enough. it still loads the textures for the viewport.



i tried in directx as you suggested, opening the file with viewport set to object colours, it loads in 10 seconds and takes 1.8gb ram. and is completely interactive.

however i cannot work with all the textures completely hidden. the moment i try to show textures in directx, (even with it set to a viewport res of 512 pixels in the directx settings, match bitmap res. disabled) it sits there for 15 minutes with the ram slowly climbing to 16 gb!! then its even worse than nitrous it seems, plus the textures look awful.


the model was usable before, bit ram hungry but not too bad, when it was all set up with max bitmaps, ive not changed much.. changed the 12kx12k background textures for 20kx14k and switched all the bitmap loaders to hdri.

suddenly its absolutely unusable.


unfortunately for me, i used the "convert bitmaps to vrayhdri" tool from vray on the scene, then continued editing the model. there does not appear to be a way ( without scripting of course, im not a scripter) to convert a scene -back- to bitmaps rather than the hdri loader.


assuming i find a way to get my scene back to using the bitmap loader, and assuming i really want to use the hdri loader for my huge bitmaps on the landscape ( tiled exr takes much less ram to render) can i use the shell material to show a low res bitmap loader in the viewport but render with the hdri loader?



i

[joconnell]

You can convert them fairly easily, the only thing that doesn't travel well is anything set in the output curves - maxscript doesn't get access to that easily so if you're using a lot of map modification you're best off leaving your bitmap untouched and nesting it inside a colour correct or an output map.

If someone doesn't have it done already, I've got a script to go from bitmap to hdri, I'll add in the opposite although it won't have much in the line of error checking or options - hopefully it'd be the type of thing that you'd run just before render though.

[super gnu]

that would be ace. in fact you would be a livesaver.

if there were any output curves on the textures, ive already busted them (i assume) by running the "convert bitmaps to hdri" script. so changing them back cant hurt eh?


quick update anyway, after using the shell material on my largest handful of textures, so they render with a tiled exr via vrayhdri, and show a standard bitmap loader in the viewport, ram usage has gone down considerably,scene loads much much faster, and the lags are much reduced.




im certainly not going to go through the other 250 materials in the scene setting up shell materials for them, so id like to just turn em back into bitmaps.

cant help thinking vray/max /vrayhdri loader should handle this kinda thing internally though? seems like wahtever you do in the viewport settings (short of turning off viewport textures entirely) it seems to parse the full res files and do..something, something very slow, and single threaded, literally every time you touch a checkbox on the material. and for some reason its -much- worse with the hdri loader than the max bitmap.

[super gnu]

hmph.. it just doesnt end!


now ive noticed that with the tiled exrs, it literally takes 10x longer to render a block model on a satellite image.

maybe i need to increase the cache size...

[vlado]

In addition to increasing the cache size, make sure you haven't disabled filtering for those bitmaps.

Best regards,
Vlado

[super gnu]

i tried increasing the cache size to 4GB (larger than the exrs im caching) and it was still dog slow.. on a render that took 7 minutes with a high res jpeg it had an eta of 1 hr using tiled exr. it did use less ram though.

i didnt have filtering disabled, im using elliptical, but with a blur of 0.1


anyway in the end ive gone back to using jpegs and just made them smaller. for the more distant shots it will be fine, and for the closeups ill swap out for the full resolution jpeg, and hide some other stuff to save the ram.

[vlado]

i didnt have filtering disabled, im using elliptical, but with a blur of 0.1

It's nearly the same thing - blur 0.1 probably always loads the most detailed mip level anyways.

Best regards,
Vlado

[super gnu]

just for anyone following this, who maybe wanted to make TX files of large 8 bit textures:

http://forums.chaosgroup.com/showthr...2tiledexr-quot


the lovely Vlado has provided a 64 bit recompile of maketx that wont crash on textures larger than 8k.

what a guy

- im still back to using smaller jpegs though in the end. even closeup, ive found the visual differnce between the half-res jpeg and the full res exr/tx is so minimal, its not worth the huge boost in rendertime.

i do have the option to switch at any time though, should i need to, since im using the shell material to avoid viewport lag from my biggest hdri loaders.

now if joconnel comes up with the goods and makes a hdri to bitmap scene converter ill be back where i started, and free of lags

[joconnell]

Save your scene first, run this:

for i in (getclassinstances vrayHDRI) do 
(
	theNewMap = bitmaptexture filename:i.HDRIMapName
	theNewMap.coords.MapChannel = i.coords.mapchannel
	replaceInstances i theNewMap
)

Also if you give it ten minutes, Dave wortley will pop in with something far more efficient and stable

[super gnu]

that worked great! thanks


while ive got you on the line, VMC changes almost everything, apart from bitmap blur amount (and tiling/offset settings for some reason)


anyway i wish to change all my bitmaps to blur 0.1

i tried editing your script as follows:

for i in (getclassinstances bitmaptexture) do
(
texmap_diffuse.coords.blur = 0.1
)


but it gives the error

-- Unknown property: "coords" in undefined


it gives the same error if i just select all and type

texmap_diffuse.coords.blur = 0.1


despite this being the line that shows up in the listener when i change that parameter.


obviously it will be something to do with me knowing nothing about scripting.

[joconnell]

You're not far off, so the loop in this case is substituting the letter i for whichever bitmap it's currently looking at so rather than texmap_diffuse which is the name of the maxscript access point for the diffuse map channel in the vray material, you've to point it towards each of the bitmaps. The first line collects all the bitmaps in the scene and uses the letter i to represent each bitmap as it loops, use i.coords.blur = 0.1 and you should be good.

Dave wortley's done some very fast lessons at https://davewortley.wordpress.com/lessons/ - well worth looking at, even the basics give you a huge amount of control.

[Joelaff]

We found using tiled textures (EXR or .tx) causes terrible slowdowns if the textures are on a network drive. The solution was to sync the textures to all the machines local drive and reference the textures locally. This way each nodes pulls the texture from its own drive.

To do this we setup GoodSync on one machine (could be the server if you use a Windows server, or you could use rsync). The one machine syncs the textures from the server to itself. Then each node shares a folder and GoodSync is used to push the textures out to all those folders.

I have a thread about this someplace.

[super gnu]

ok well thats me pretty much decided against any tiled textures in this job.. ! apart from the scene lag issues with the hdri loader, and the slower rendering, i have to send this max file to be rendered at a client's renderfarm, so i dont want them to have to fanny around with local map paths !