sim suddenly starts writing 5kb cache frames

[ttdww]

Sorry if this is answered somewhere - I've searched around the forums.

Details
--remote-desktop'd into a network workstation
--license server/dongle is on another machine
--seems to access the license fine and start simming and creating cache frames normally
--I leave it simming and suddenly (in this case after frame 17) it starts writing 5kb empty cache frames. the sim continues and is taking expected time/frame
--it doesn't seem to have this issue on the workstation that the dongle is on. have simmed through the sequence several times and each time writing all the cache data.
--using v25058

Thanks!

[Ivaylo Katev]

in the most cases this is caused by overloaded hdd, but it can be also a calculation problem (nan values in the grid). however, in the second case the simulation speed drops rapidly, that is not true in your case, as i see the frame duration is still about 1 min/frame.

[ttdww]

i see. the hdd has about 200GB free. the slave workstation was writing frames through the network - not sure if that's a possible issue area. i'm currently simming under the same scenario however having it write to a drive local to it. we'll see how it goes.

[ttdww]

would it help if i posted one of the 5kb frames? let me know if the cache would contain NDA data such as pathing. thanks!

[ttdww]

problem repeats even when network slave writes to a local hard drive. i restored the sim at frame 14 (was saving every 5 frames from frame 0) and it immediately started writing 5kb cache frames. a resim also now starts writing 5kb cache frames at frame 16. this is an obvious problem since i need to use the primary workstation for mouse-time work.

Lowering grid resolution seems to resolve issue.

[Ivaylo Katev]

in this case the second scenario is more plausible. you can send one cache file and the log file will be also very helpful (C:\PhoenixFD)

[ttdww]

sure thing. where should i email it to? thanks!

i've also noticed while i raise and lower the grid that as soon as the cache file get to around 32MB, the frame after becomes 5kb. an indicator perhaps.

[Ivaylo Katev]

send them to phoenix@chaosgroup.com

[ttdww]

sent it! thx so much.

[Ivaylo Katev]

well, it looks like the hdd space problem (this files are just truncated), however you said that there is enough free space. can you try the same scene on different machine? or if you have antivirus software, can you switch it off for a while?

[ttdww]

sure, i'll give that a shot.. disabling antivirus. it happens on different hard drives. i'm also noticing that after i do a sim that keeps the cache files below 32mb, then once i do a particles resim, the resim caches start having the same issue as soon as they reach about 45mb. they then drop to about 15mb each and the particles vanish.

also, this is no longer happening on only the slave network machine, it's also happening on the primary machine with dongle and license server. i'm really dead in the water at this point.

another symptom: the moment the cache files start getting written truncated, the display showing the grid size, particle count, etc. goes blank and the sim just continues but with no info showing there.

[ttdww]

ok, a bit of good news, i went back to simming on primary dongled machine and to a local drive and it pushes past the 32mb-sim/45mb-resim point without issues. seems to be a network issue. this actually reminds me of a problem with release 3.5.1? of fumefx some time back discussed here. search for 'finalight' posts. Afterworks eventually resolved it in v3.5.2 i believe. may or may not be related.

[Ivaylo Katev]

actually you are the second one who has this issue, in the most cases the hdd space is the reason, but as i remember there is one unresolved case with the same complain. it would be really good if we find what is causing this behavior, it looks like a system issue, the file write operation just fails.

[ttdww]

will sending the scene help? yep, it'd be great to nail this down. i'm getting repeatable network errors on multiple workstations whenever they try to write to a network drive from PFD. i'm able to sim with other 3d apps through the same network with no errors and a lot of drive space available. thanks again!

[Ivaylo Katev]

yes, the scene may help, hopefully the problem can be reproduced