Is there any way to speed up ANIMA process, I'm frustrated

Anima Issues
1

Hi,

Although I like ANIMA in general I hate how slow it is. Just for simple scene with 20-30 characters on the street with couple of paths, I can easily spend half a day generating and fine tuning the scenes: waiting and waiting in every step on the way: when importing characters from Chaos to the purchases folder, when adding them to the scene, when adjusting them, baking, unblinking, importing to 3dsMax scene. Most of the time I see the rotating circle and message “updating the scene” or similar. And same again when I make a smallest change to the scene after test renderings. I start to wonder that I am making something wrong. Here is my workflow:

I initially place the project and the scenes to a Dropbox folder (Dropbox is on a fast local SSD drive, also the Anima purchases folder is defined to the same Dropbox, the reason for this is that my colleague need to access the same projects occasionally and he is sitting in an other town). When the scenes are ready I bake them, save them, sync the whole project folder to local network drive and import the scenes to 3dsmax and send the file to my local render farm by Deadline (the farm is in the same local network than my workstation). As the scenes are imported over the local 1Gbit LAN it may take 5-10min. This works quite reliably (which is the main thing) but it is slow. The only way have managed to get Deadline to work with anima is to have the ANIMA project on network drive that all render nodes can access. Deadline will upload the whole assets package to each render farm node, which takes time but that is not an issue because it will only affects the rendering of the first frame of the animation sequence, as long as it is reliable. As my scenes can have up to 2000 linked bitmaps (4GB total). My frustration is the part when I deal with Anima interface in one way or another as every action in Anima takes time, user-interface is lagging, I can spend ours to make small adjustment to the scene. Baking, unbaking saving, opening; all is frustrating. Importing to max can take 5 to 10 minutes as said (even this I can accept as I know it is 4GB package transferred to Max and deadline. I have a workstation that works well with 3ds Max (AMD Ryzen 9 5900X, ) 96 GB RAM, SSD 550 MB/s (C:) 3300MB/s SSD for local Dropbox; 1Gbit LAN.

Let me know if you have some improvement ideas to speed up the process.

PS. Strange thing is that althoug when I send the 3dsmax scene to renderin through Deadline and I define that all assets needs to be uploaded with the max scene to the Deadline repository, the clients still need to access them directly from the network drive, as if I remove the project from the network drive the Anima characters don’t have “clothes”.

Ok, my scenes can have 50+ characters maybe 50% of the 4D character, I know they are heavy, but isn’t this what ANIMA is for. What’s the point of having a crowd of 5 characters in a city.

Br Mikko

2

Hello Mikko,

Thanks for reaching out and sharing such detailed feedback.

First off, having 50+ characters (50% of them 4D models) should be okay for Anima to handle without spending half your time waiting for basic UI interactions or simple adjustments. Don’t get me wrong, there are definitely some steps that always take some time, but they should be done in a matter of seconds or a minute at most, rather than several minutes.

Your hardware is plenty powerful, so the bottleneck is almost certainly how Dropbox and your network drive are interacting with Anima’s asset generation and file streaming mechanisms.

Let me break apart the problems you exposed. I’ll try to describe below the possible causes of these issues and the steps you can take to speed up your workflow.

  • Move Working Projects and library (Purchases folder) Off Dropbox

Anima frequently reads and writes temporary cache files and metadata every time you add an asset to your scene, when you simulate, and/or when you bake/unbake that simulation. Continuous syncing services like Dropbox monitor every single file change in real time. This may lock the files momentarily, causing the lagging UI, freezing, and persistent “updating the scene” loading circles.

The best way to solve this is moving your active working project folder to a local SSD outside the synced Dropbox folder. As you do for the network rendering, only copy or sync the project to Dropbox once you are done editing or need to share it with your colleague.

  • Import or synchronize with 3dsMax using Alive only when you really need it

By default, the Alive sync feature will automatically import all assets from your scene into 3dsMax every time you save the scene on Anima. However, you can turn off that automatic sync and do it manually whenever you think it is worth waiting the time it will take to finish. More info about that here.

  • Addressing the Deadline / “Unclothed” Characters Issue

Anima uses dynamic file references for character textures and geometry caches. Even if Deadline uploads the core .max scene and assets to its repository, it does not support that same dynamic transfer on the imported Anima assets. Therefore, the Anima plugin still requires direct access to the Anima project folder structure at render time.

If you remove or rename the network drive, the paths defined in the imported Anima assets from your scene will not resolve properly, and your render nodes won’t find the textures and animated geometry sequences (you will still get the static geometries because those are stored into the .max file).

Moreover, by checking the Deadline option to send the assets to the nodes, you are transferring the Anima textures (those 4 Gb you mentioned) over the network twice, once by Deadline to prepare the scene in the nodes, and once again when the Anima plugin loads that project on render time.

Keep in mind that the Anima plugin is highly optimized for both loading speed and RAM usage. By default, it is only loaded what is used in a particular frame of animation. This is especially useful when rendering scenes with lots of 4D models, because the loading times are distributed along the frames of the sequence instead of all together on the first frame of the batch, and everytime only what is used for a given frame gets transferred across. Also, the RAM used does not include the whole 4D animated texture and mesh sequence of each model but just one single frame.

Please try the solutions suggested and let us know if they help to improve your user experience when using Anima.

3

Hi Albert, thanks for you detailed instruction I will test them asap.

Meanwhile I already found something interesting. As the scene I am working with is a high-rise city center with several blocks. I just started to re-build the city model from the ground up in 3dsmax just to identify the possible the bottle necks. I first made a simple plane in max, size approx 850m x850m to work as a ground plane. I exported it as fbx (quite small file only 47 KB). BUT when importing it to ANIMA it took 10min !!! No warnings, no errors, finally the plane was in ANIMA. I placed one character standing on the plane, and imported the scene back to 3ds max. Again 10min !. Again no warnings no errors. I repeated the process with a small plane of 5m x 5m. Now the fbx was imported to ANIMA without delay , also importing it back to 3ds max was a flash. Obviously the overall dimensions of the background fbx wil have a big impact on the transfer speed to both directions, even though the object is very simple. I will investigate this a bit further to see where are the practical limits for the fbx object and if the ground plane could be imported in smaller pieces. I will keep you updated.

Br Mikko

4

Ok, I made a bunch of tests with various size background objects, here are results of fbx file loading times to Anima and corresponding loading time of Anima scene to 3ds Max. What is a bit surprising to me is that background size had a big impact how long it takes to load the scene back to Max; my first thought was that the background should have no role to play there as the background is not transferred to max (why should it?), it is only the Anima characters that are needed in max, so this is strange. Here is the table of results, I used just a simple cube as a test object with different dimensions:

Cube size (m) fbx to Anima Anima scene to 3dsMax

50x50x50 0 0

200x200x200 10s 15s

300x300x300 15s 1min 10s

400x400x400 1min 10s 6min 15s

As you cans see the maximum practical object dimension should be below 200m as after that especially the scene loading time back to Max start to increase rapidly. In my tests I used only one Anima caharacter on the scene so the only factor is the background elemen(s). In also noticed that even though I did not make any changes to the scene the following load times to MAx were still about half of the first load, I assume this happens also every time max file is opened.

The workaround is to keep all background objects smaller than 200m rather closer to 100m for maximum dimension.

I tested my 400x400x400m cube by dividing it to 32 smaller cubes i.e 50x50m each, the fbx to Anima was 0s and anima to max 35s so that solves the problem almost completely.

By the way I made most of the test using the live link between Anima and 3dsMAx. One thing to note which was a bit annoying: as far as I understand the Anima scene should be baked before saving. If the scene is baked (i.e locked) the live link does not work from max to Anima; if you make any changes in max to background object and want to send them to Anima. Yes Anima is giving a warning that the scene is locked and should be unlocked but it also tends to freeze the link and I had to boot either max or anima couple of times to proceed. What I suggest to add on option to allow live link to unbake the scene in anima automatically or at least offer the option in the warning popup window when using the live link. Now you have remember to go back to Anima and manually unbake the scene every time before initiating the uplink transfer, unnecessary back and forth clicks and if you forget to do that (as happened to me many times) there is a danger of freezing the process.

I think this was one of the major root cause on my city project with 800mx800m ground plate being so slow. I will split the ground plate to smaller pieces now and then follow your instructions to see what impact they have.

Br Mikko

5

Ok, now I have changes the ground plate size to smaller overall dimensions and also divided it to separate smaller planes each less that 200mx200m in dimension in my city scene. BUT there is a new issue. When I place the new background file to Anima it is placed away from the original position so all anima character and paths are offset in relation to the new background. I red another discussion in the forum of how the background is placed based on it’s reference point and as the background is now different size the reference point is moved. Is there any way to fix the reference point to keep it unchanged when importing new background as the existing background is removed as soon as I import the new one (or is there a way to have more that one background attached to a scene). The only way I can see is to move all Anima nodes manually to relocate them in relation to the new battleground but that is a bit of work as I have more that 10 scenes with several paths and nodes in each with the same city background. If this change of reference point happens every time one imports a new background to existing scene, doesn’t that mean that one should never change the background to an existing scene or at least be ready to adjust the nodes manually.

Br Mikko

6

Hi again,

About your comments related to the use of Dropbox with Anima. I tend to disagree (correct me if I am wrong). If Dropbox is configured so that the the whole Dropbox is defined in “offline” mode, alll the files a physically located on a local disk, in my case a 4TB SSD disk with write/read speed of over 3000Mb/s. Dropbox is only syncin on background to cloud when there are any changes, but that happens with low priority.

What comes to deadline/anima relationship it is bit tricky. I am am not very familiar with deadline and the support for that is limited. After several tests I came to the conclusion that the most reliable workflow (perhaps not the fastest) between max and deadline (without Anima involved) is to let deadline to upload all the assets to deadline repository when launching the rendering. If I try to edit max scene with assets on network drive (to allow all render farm nodes to access them during render time), the editing max scenes are limited by the slow LAN speed. So in any cases fastest way of editing the max scenes is to keep all assets on a fast local drive. When it comes the time to launch the scene to rendering I let the deadline take care of assets uploading on the background and I can focus on the next project. Now the only additional step that comes in is if the Anima is used in max scene: I need to copy (or sync automatically) the Anima project folder to the NAS and re link the Anima scenes in max to be read from the the NAS drive (as Amima for some reason is not happy with Anima assets in the repository but needs to access them from the network drive visible to all render nodes. So It may take couple of minutes extra, but at least it is 99% reliable and easy to trouble shoot if something goes wrong. If someone has a better way to use 3dsmax/Corona/Anima/Deadline combo with render farm let me know.

Br Mikko

8

Hi Mikko,

Thanks for providing the additional context regarding your scene geometry and background updating process. I’ll answer below all the topics you raised.

Slow load on scenes that have big backgrounds

Thanks a lot for reporting this issue. I have been able to reproduce your tests on my side, and this is definitely something we will focus our effort on and try to improve as much as possible.

A preliminary analysis confirms this is a consequence of a new background parsing process we need to apply to the background to enable the usage of the new AI Path tools introduced version 7. We did notice a small delay in our testing process, but your tests highlighted a scenario we did not try, which is using a background with a massive horizontal surface. The vertical dimension is ignored in the parsing, so it takes exactly the same time regardless of the vertical extension of the geometry.

By massive I mean that the delay starts to be annoying on areas above the 50.000 square meters, which definitely is on the upper range of the projects usually done by Anima users. The image below represents that same area surface (500m x 100m), and takes 4 seconds to import it into Anima:

image
image1920×1357 133 KB

However, going beyond that scope, the delay is increased exponentially:

  • 500x100 → 4s
  • 500x200 → 16s
  • 500x500 → 120s

Moreover, even though it is not common to deal with projects of this size to simulate people, it is more common when simulating traffic. Therefore, we have reported this issue to the Anima developers (AN-1450), and they are already working on implementing different solutions that should help to deal with these scenarios more comfortably in the future.

Background Geometry Offsets

By default, we apply an offset to the geometry background imported that is based on where the centroid of all the mesh vertex is. The reason behind this is that CAD files sometimes have scene origins that are very far away from the actual position of the main geometry that you want to work with. Working at such a great distance from the origin may introduce stability issues in the animation engine we are using in Anima. This is a common limitation when working on high-performance environments like the one we use for the Anima viewport, and it happens due to precision errors in the position data that can be managed in realtime. You can’t have both, so CAD software (AutoCAD, Revit) prioritizes precision while visualization 3D software (3dsMax, Cinema4D, Unreal Engine) prioritizes performance.

To avoid this limitation, we reposition the mesh on every collision background import process so it ends up always close to the Anima scene origin. Then we store that origin offset, and we finally apply the inverse offset during the import of the Anima content back into the DCC application. This is not something new, it has always been this way in all Anima versions since the beginning, and should not have any impact on your workflow as long as you don’t remove that background node after the initial import.

The key idea to keep in mind is that the offset of your background geometry is stored in the node “Background Model” from your scene. You can find this node on the Anima node tree, in the scene you are working on. As long as that node is not removed, the offset will be kept the same, and any modified geometry you bring into your background will keep its relative position coherent.

At the same time, if that “Background Model” node is removed, whenever a new background is dropped into the scene, a new offset is calculated and adapted to that particular geometry. Then that offset is stored in a brand new “Background Model” node in your scene, along with the whole background geometry definition (all the vertex, polygons, and objects included in it).

Knowing that, you can use it to your advantage based on your needs. If you need to keep updating a geometry background over time, then you can export multiple versions of that geometry in different FBX, and all of them will match the coordinates perfectly as long as you don’t remove the “Background Model” node and just drop those backgrounds one over the other.

Please check the video below, it explains in more detail and with a few simple background samples how this offset concept works:

If you already removed the background node from your scene, a quick way to manually fix the problem would be by selecting the “Background Model” node and repositioning it manually inside the Anima viewport so it matches the position of your previous background. Then save the scene in Anima to sync with your 3D app, and from then on everything should work fine in further background updates (as long as you keep the same background node).

Can You Have More Than One Background Attached to a Scene?

No, each scene can only have 1 geometry background inside. The reason behind this is that the geometry backgrounds need to be parsed and pre-processed so Anima has an understanding of their geometry and properly adapt the animation of the assets accordingly.

That said, it is possible to have multiple objects inside each geometry background imported. Also, as Anima is a project based application, you can end up having multiple geometry backgrounds in your project and quickly switch from one to another depending on your needs.

On the other hand, notice that since Anima 7 it is possible to assign a “Non-walkable” definition to any of the objects included in your collision background. The documentation for this feature is still under development, but it basically means that you can keep some objects from your background in your Anima scene as a visual reference, but they will not affect the simulation in any way.

The process is the following:

  1. Click this icon to enable the selection of background objects directly in the viewport
  2. With 1 active, use the LMB to select the object you want to edit
  3. Once object is selected, use the RMB to open the contextual menu
  4. Click there to switch this object to “Non-walkable”. The icon will change in the node tree too
  5. To finish the process click on (1) again.

From then on, this object should be visible in the viewport but invisible to the simulation engine. Perhaps you can use this method to exclude sections from your background that you only want to keep as a visual reference but will actually never be used by the simulator.

image
image1920×1278 255 KB

This answer is getting very long, so I’ll leave it here for now. Whenever you have the time, please take a look at it and let me know what you think.

9

I can confirm this is an issue. As a workaround we save as, import a tiny box as a background and link that to 3dsmax..the file size goes down massively and speeds up the whole pipeline..not great!

10

Hi @gethin_hooper

Thanks for your feedback.

As mentioned before, the team is already working on several improvements on this issue.

I understand baking the Anima scene and then switching the background to a smaller one, or just removing it, can help solve the scene sync times and/or the Anima scene loading inside 3dsMax. However, perhaps I did not understand you properly, but I don’t think the 3dsMax file size will be reduced in any way by doing this.