Scene translation times - Open discussion

[daviddeacon]

Hi Everyone,

We have just come off a few projects that had some really heavy scenes, One of the major pains has been the scene translation times between Maya and VRay. Often taking just as long as the render takes on high end render nodes. We have been doing as much to optimise as possible, tiled EXR?s and VRay proxies etc.

We have a number of projects that are going to push up the level of complexity a lot so I think this problem is just going to get worse. So I thought it would be a good idea to open a discussion on scene translation times (Everything that happens before the first ray is fired) and what can be done to speed them up. I?ve seen a number of posts with similar problems, so I don?t think I?m the only one.

VRay?s raytracing and GPU rendering is getting so fast and so parallelised, scene translation is becoming a larger percentage of render time.

So in the interest of helping with the improvement of VRay. I?ve made some test scenes that demo some features that I know are slow to start. My testing method is to set the VRay?s max subdivisions to 1 so that basically no time is spent rendering and the results are all translation times.

I also want to point out, I don?t have any kind of rendering engine programming knowledge, or know how Maya sends data to VRay. So my suggestions may come across as naive and drastically over simplified. I?ve also never used VRay and Max so I don?t know if scene translation times are drastically different depending on the DCC app.

-

First up, Sub division surfaces. This scene contains 17 robotic arms that are all set up with there own subdivision settings. Nothing to crazy.

Global Subdivision?s disabled - 7 seconds
Global Subdivision?s enabled - 37 seconds

5x Slower to start, this problem seems to scale with the number of subdivision objects in the scene. The CPU usage is sat using one core for the whole of the translation time. Couldn?t this process be parallelised some how? In this case the models are completely separate, couldn?t each mesh be subdivided on a separate thread.

-

Instanced Objects - We have been using Maya?s MASH to scatter large numbers of objects more and more. As its flexible and much easier to use than XGen. Maya?s particle instancer is used to send the geo to VRay. This scene demonstrates some of the problems we have had using it to build large environments. The scene as a patch of terrain and had 500k points scattered on it. First I then instanced a basic sphere to eliminate any over head of instancing heavy geo. I then converted the sphere, to a VRMesh and an alembic and loaded via the VRay Proxy loader to compare. To eliminate Maya being the bottle neck I also included an Arnold based test.

500k instances -
sphere - Maya Geo - 51 second.
sphere - VRay Mesh - 65 seconds
sphere - Alembic - 110 seconds
sphere - Maya Geo - 24 seconds - Arnold

Interestingly using the proxies were slower to start rendering, and the alembic file is much slower than a native VRay Mesh. Arnold didn?t start instantly but did take half of the time to start. Again during this time the CPU usage is incredibly low, only using one core

Instancing sphere?s isn?t exactly what we would normally be scattering on a terrain, so I replaced them with a relatively high poly trees.

500k instances -
HighRes Tree - Maya Geo - 53 seconds
HighRes Tree - VRay Mesh - 65 seconds
HighRes Tree - Alembic - 3220 seconds + Got bored waiting so cancelled the render.
HighRes Tree - Arnold Stand-in - 24 seconds - Arnold

It?s great to see how adding high res geo doesn?t slow down the translation time, apart from when using the VRay proxies combined with an alembic file. This took so long I just gave up. This is good to know as we have often created animated trees using alembic files from speed tree, and will now convert them using ply2vrmesh command line.

I wanted to see how well the translation time scaled with more instances.

1000k instances -
HighRes Tree - VRay Mesh - 116 seconds
HighRes Tree - Arnold Stand-in - 58 seconds - Arnold

Arnold scaled basically linearly and VRay managed to close the gap.

-

Here is a link to the Maya project that contains the scenes from above.

https://www.dropbox.com/s/n4m2kuormr..._Maya.zip?dl=0

I plan to continue to add more test scenes, XGen archives and VRay Displacements etc, but I invite anyone to post their scenes that take a long time to translate.

I hope this post generates a discussion, and helps the development team improve VRay.

Best regards,
David.

[vlado]

Thanks so much for the Maya project files! They will be very helpful for our team to profile any bottlenecks.

Best regards,
Vlado

[Morbid Angel]

Sorry I am not clear is this for the GPU slowdown or CPU ?

[Morbid Angel]

Just a quick test of your subdiv scene. My render time including translation and compile time is 2.4 seconds from start to finish. I think this issue can be related to the particular Mac build you have.
Also I found that some of the surfaces like the cables had exploaded uvs which means the uv subdivision is insanely overloaded subdiving each polygon on its own.

[daviddeacon]

Sorry I am not clear is this for the GPU slowdown or CPU ?

Currently just testing the CPU.

Just a quick test of your subdiv scene. My render time including translation and compile time is 2.4 seconds from start to finish. I think this issue can be related to the particular Mac build you have.
Also I found that some of the surfaces like the cables had exploaded uvs which means the uv subdivision is insanely overloaded subdiving each polygon on its own.

Interesting that it could it be a platform issue. I'm guessing your on windows? We are running 3.60.03, but have had the slow subdivisions for as long as we used VRay.

I will take a look at the UVs, as I blew them away before baking the animation.

[Morbid Angel]

Yeah I am on windows 7 x64 to be exact, also running 3.6 nightly. Your scene contained uvs for all the robot arms so it may be related to that particular instance but I have no mac so I can't really make any comparison.

I would also suggest not to create individual subdiv set for each arm, rather one set for all of them and not control the objects from viewdistance subdivision that can result in some odd subdivs happening if the camera is moving through space. Instead set the edge length low and uncheck viewdependant, then set the subdiv value to something that you would be happy with in maya, like 2-3 (don't need to go 4 or higher) unless its displaced.

For the instance scene, I did not dig into it deep, but it rendered pretty fast too in a few seconds, I did get some MASH errors though.

[smbell]

Windows 10, maya 2018, vray 3.60.03 here. Dual xeon 2640 v3s, High res scatter scene

V-Ray: Total time updating frame 0h 1m 7.5s (67.5 s)

V-Ray: Total frame time 0h 1m 15.7s (75.7 s)
V-Ray: Total sequence time 0h 1m 16.1s (76.1 s)

just opened the scene and hit render.