Ability to recognize and export Alembic geometry directly from maya

[rainingwaffles]

In maya 2013 with viewport 2.0, maya reportedly has the ability to keep cache huge quantities of alembic data without major performance hits. However, I believe V-Ray will still view this as geometry, and an export to vrscene file would be potentially lengthy. Is it possible (or already in progress) to automatically recognize alembic data within maya and simply write it in the vrscene file a path to the alembic data rather than the whole chunk of geo?

If implemented, it feels like this option would need to be a vray extra attribute toggle on geometry since it's possible to put additional deformers on alembic geometry that wouldn't be represented in the raw alembic itself.

There are probably other drawbacks I'm not considering here, but to be most valuable it would need to support shader assignments, subd, displacements, mesh lights, all visibility flags, etc etc. However, this seems like a more intuitive workflow than using a single proxy node from a user standpoint.

Yeah, seems like a pretty tall order. ...but this is called "Wishlist" for a reason, right?

[vlado]

I'm not sure if we can recognize a piece of geometry as coming from an Alembic file, but, assuming that we can, you want to export that geometry automatically as a V-Ray proxy linked to the Alembic file, is this correct? For the record, proxies in the latest builds already support subdivision, displacement, mesh lights etc whether linked to .vrmesh files or .abc files (although they lose some of their memory-saving features in the process and become like regular meshes).

We might be able to recognize if the geometry has any deformers on it. If this is the case, would this be a global option, or still a per-object attribute?

Best regards,
Vlado

[rainingwaffles]

Ahh... if the exporter can automatically detect altered geometry, then it seems like a global option would work, although I could see certain situations where you might want to override per object in unexpected, unsupported cases (I was going to say per-face material assignments, but that would mean it's not unexpected).

Your point of this being the same as the existing proxy linked to the Alembic file totally makes sense and means the V-Ray core already has support for everything. That's awesome. Makes this seem like a real possibility!

[Zach Gray]

Is it possible (or already in progress) to automatically recognize alembic data within maya and simply write it in the vrscene file a path to the alembic data rather than the whole chunk of geo?

That sounds like a really really good idea. Our pipeline:

-'Rig' and Look files are separate.
-Export .abc files from rig. Import to lighting, assign shaders from 'Look' file.
-Export a .vrscene for the farm.

The .vrscene takes a long time, since it reads and translates all the .abc files. If I understand correctly, if we currently bring in our .abc files as vray proxies, they will be linked at render time? I'll need to rewrite our .abc importer, but that sounds great.

Checking for .abc files automatically, or setting an attr would be even better.

[emil.kirichev]

Just to clarify things a bit: what we can do is to automatically export the GPU cached alembic geometry as proxy (the one imported through Pipeline Cache->GPU Cach->Import...). The non-gpu cached gemetry, when imported (Pipeline Cache->Alembic Cache->Import Alembic...), becomes a regular mesh and we can't know it came from an alembic file.
Does that suit your needs?

Best,
Emil

[Zach Gray]

Emil,

You are correct, it becomes a regular mesh, and you can put additional deformers and modeling tweaks on top of it. Obviously, those would negate the .abc cache. It might be nice to have a custom attr that we could turn on that would identify that the mesh was linked to an .abc file - provided the user specified that it was unaltered.

[emil.kirichev]

It might be nice to have a custom attr that we could turn on that would identify that the mesh was linked to an .abc file - provided the user specified that it was unaltered.

But then you'd have to specify the path to alembic file too, which means you could 'override' any mesh on the scene this way. Isn't this tedious/error prone?

[Zach Gray]

I've looked into the GPU cache to see if that would work for us, but I'm not clear on how to assign shaders, activate subdiv/disp flags, etc.

The vrayProxy provides plugs to assign shaders to enable this. We didn't end up using vRay Proxies in our workflow because it didn't support subdiv/displacement. So we use .abc files so we can assign shaders/subdiv/disp. It's a hassle and slow to re-cache everything to the .vrscene.

But then you'd have to specify the path to alembic file too...

I think if the 'override' flag was set, it could trace the .inMesh connection to the .abc file, and retrieve the path that way.

Being able assign shaders/subdiv/disp to .gpu cache, or avoid re-caching the geometry into the .vrscene file with an extra vray attr.... either would be very helpful.

[paulDryzen]

I would love to have a direct alembic support too.
Something like a flag which tells VRay to export as cache .abc file would be nice.
Using VRay proxies is most of the time not possible if you for example have fur/cloth on some of the objects inside of the alembic file...

[vlado]

Using VRay proxies is most of the time not possible if you for example have fur/cloth on some of the objects inside of the alembic file...

I agree about the fur (although we do plan to support spline primitives in .abc files), but what's the issue with the cloth?

Best regards,
Vlado

[emil.kirichev]

I've looked into the GPU cache to see if that would work for us, but I'm not clear on how to assign shaders, activate subdiv/disp flags, etc.

Just as would assign to a regular mesh. Assign new material, select the new shading group, add displacement etc. What's missing currently is the additional attribute "displacement control" for the gpuCache node. I'm working on it and it'll be there very soon.

The vrayProxy provides plugs to assign shaders to enable this. We didn't end up using vRay Proxies in our workflow because it didn't support subdiv/displacement. So we use .abc files so we can assign shaders/subdiv/disp. It's a hassle and slow to re-cache everything to the .vrscene.

I believe vray proxies do support displ/subdivs.

I think if the 'override' flag was set, it could trace the .inMesh connection to the .abc file, and retrieve the path that way.

Unfortunately the inMesh attribute (or any other for that matter) does not hold source for the geometry (for 'regular'(not gpuCache) imported alembic).

[paulDryzen]

I agree about the fur (although we do plan to support spline primitives in .abc files), but what's the issue with the cloth?

Best regards,
Vlado

Using the Alembicmesh for collisions etc. This I think can be problematik with proxies, or am I wrong here?
Also in our case the fur was done with Yeti so everything was created at rendertime only.

[vlado]

This I think can be problematik with proxies, or am I wrong here?

Hm, I don't know, I haven't tried really. It might work if you preview the proxy as a full mesh in Maya.

Best regards,
Vlado

[Zach Gray]

Just as would assign to a regular mesh. Assign new material, select the new shading group, add displacement etc. What's missing currently is the additional attribute "displacement control" for the gpuCache node. I'm working on it and it'll be there very soon.

That makes sense. Where it gets tricky is when you have multi-surface caches, where each surface is attached to a different shader.

For example, in our caching workflow, we export a 'body' cache for a character; skin, teeth, eyes and any bound geometry. Then another 'cloth' cache; shirt, pants, coat. Then another for hair. The reason that we moved to this multi-mesh approach was that we reached the 256 OS limit for 'too many open files'. After working with you guys and Autodesk, we found a way to raise that to 1024 via the API. But having all those open handles caused high disk i/o overhead, and we were back to where we started for performance. That's why it's nice to have just 3 open files per character instead of 30. Of course we can make an extra single cache if we need to go back and make a correction.

The gpuCache is exposed as a single node. Unless we export each object separately, there would need to be a method for relating shaders, lights, overrides, etc... so then we are back to a ton of complexity. That's why I like the alembic cache option. It gives you a little bit more flexibility before sending to the renderer.

Unfortunately the inMesh attribute (or any other for that matter) does not hold source for the geometry (for 'regular'(not gpuCache) imported alembic).

I was talking about walking the dag for the .abc path on export, or post translate, or using MPlug if it's in the api. I'm probably missing something obvious, where it needs to exist on the node itself.

import pymel.core as pm

def abcPath(mesh_shape):
    abc_node = pm.listConnections((mesh_shape+'.inMesh'), type='AlembicNode')
    if abc_node:
        return abc_node[0].getAttr('abc_File')

Would this be possible using the post translate python? If the 'useAbc' extra attr exists, then don't evaluate any animation during .vrscene export. Then the post translate slots in the correct paths.

I believe vray proxies do support displ/subdivs.

When we evaluated which cache approach to write our exporter around, we passed on the vray proxy because of that... I'd like to find a way of using .abc if possible, but I suppose we could try to push it back to using proxy. I do really like the compatibility that alembic provides though.

Zach

[vlado]

The materials issue is one that we don't know how to solve yet. When you use a V-Ray proxy, either with an .abc of a .vrmesh file, we handle the material connections ourselves and all is ok. But with the Maya caches there are problems:

1. It looks like the GPU cache does not support multiple material assignments (although I would love to be proven wrong);
2. Regular Alembic import loses the connection between a given mesh and the .abc file and Alembic object that it was imported from.

So, for 2. not only would you have to specify which Alembic file generated a given mesh, but also *which* mesh it is in the file, so that we can do a proper multi/sub-object material.

Also, worth noting. Alembic files may have limitations on the number of open files, but .vrmesh files do not (at least on Windows)

Best regards,
Vlado