Ability to recognize and export Alembic geometry directly from maya

V-Ray for Maya V-Ray for Maya :: Wishlist
1

In maya 2013 with viewport 2.0, maya reportedly has the ability to keep cache huge quantities of alembic data without major performance hits. However, I believe V-Ray will still view this as geometry, and an export to vrscene file would be potentially lengthy. Is it possible (or already in progress) to automatically recognize alembic data within maya and simply write it in the vrscene file a path to the alembic data rather than the whole chunk of geo?

If implemented, it feels like this option would need to be a vray extra attribute toggle on geometry since it’s possible to put additional deformers on alembic geometry that wouldn’t be represented in the raw alembic itself.

There are probably other drawbacks I’m not considering here, but to be most valuable it would need to support shader assignments, subd, displacements, mesh lights, all visibility flags, etc etc. However, this seems like a more intuitive workflow than using a single proxy node from a user standpoint.

Yeah, seems like a pretty tall order. …but this is called “Wishlist” for a reason, right?

2

I’m not sure if we can recognize a piece of geometry as coming from an Alembic file, but, assuming that we can, you want to export that geometry automatically as a V-Ray proxy linked to the Alembic file, is this correct? For the record, proxies in the latest builds already support subdivision, displacement, mesh lights etc whether linked to .vrmesh files or .abc files (although they lose some of their memory-saving features in the process and become like regular meshes).

We might be able to recognize if the geometry has any deformers on it. If this is the case, would this be a global option, or still a per-object attribute?

Best regards,
Vlado

3

Ahh… if the exporter can automatically detect altered geometry, then it seems like a global option would work, although I could see certain situations where you might want to override per object in unexpected, unsupported cases (I was going to say per-face material assignments, but that would mean it’s not unexpected).

Your point of this being the same as the existing proxy linked to the Alembic file totally makes sense and means the V-Ray core already has support for everything. That’s awesome. Makes this seem like a real possibility!

4

That sounds like a really really good idea. Our pipeline:

-‘Rig’ and Look files are separate.
-Export .abc files from rig. Import to lighting, assign shaders from ‘Look’ file.
-Export a .vrscene for the farm.

The .vrscene takes a long time, since it reads and translates all the .abc files. If I understand correctly, if we currently bring in our .abc files as vray proxies, they will be linked at render time? I’ll need to rewrite our .abc importer, but that sounds great.

Checking for .abc files automatically, or setting an attr would be even better.

5

Just to clarify things a bit: what we can do is to automatically export the GPU cached alembic geometry as proxy (the one imported through Pipeline Cache->GPU Cach->Import…). The non-gpu cached gemetry, when imported (Pipeline Cache->Alembic Cache->Import Alembic…), becomes a regular mesh and we can’t know it came from an alembic file.
Does that suit your needs?

Best,
Emil

6

Emil,

You are correct, it becomes a regular mesh, and you can put additional deformers and modeling tweaks on top of it. Obviously, those would negate the .abc cache. It might be nice to have a custom attr that we could turn on that would identify that the mesh was linked to an .abc file - provided the user specified that it was unaltered.

7

But then you’d have to specify the path to alembic file too, which means you could ‘override’ any mesh on the scene this way. Isn’t this tedious/error prone?

8

I’ve looked into the GPU cache to see if that would work for us, but I’m not clear on how to assign shaders, activate subdiv/disp flags, etc.

The vrayProxy provides plugs to assign shaders to enable this. We didn’t end up using vRay Proxies in our workflow because it didn’t support subdiv/displacement. So we use .abc files so we can assign shaders/subdiv/disp. It’s a hassle and slow to re-cache everything to the .vrscene.

I think if the ‘override’ flag was set, it could trace the .inMesh connection to the .abc file, and retrieve the path that way.

Being able assign shaders/subdiv/disp to .gpu cache, or avoid re-caching the geometry into the .vrscene file with an extra vray attr… either would be very helpful.

9

I would love to have a direct alembic support too.
Something like a flag which tells VRay to export as cache .abc file would be nice.
Using VRay proxies is most of the time not possible if you for example have fur/cloth on some of the objects inside of the alembic file…

10

I agree about the fur (although we do plan to support spline primitives in .abc files), but what’s the issue with the cloth?

Best regards,
Vlado

11

Just as would assign to a regular mesh. Assign new material, select the new shading group, add displacement etc. What’s missing currently is the additional attribute “displacement control” for the gpuCache node. I’m working on it and it’ll be there very soon.

I believe vray proxies do support displ/subdivs.

Unfortunately the inMesh attribute (or any other for that matter) does not hold source for the geometry (for ‘regular’(not gpuCache) imported alembic).

12

Using the Alembicmesh for collisions etc. This I think can be problematik with proxies, or am I wrong here?
Also in our case the fur was done with Yeti so everything was created at rendertime only.

13

Hm, I don’t know, I haven’t tried really. It might work if you preview the proxy as a full mesh in Maya.

Best regards,
Vlado

14

That makes sense. Where it gets tricky is when you have multi-surface caches, where each surface is attached to a different shader.

For example, in our caching workflow, we export a ‘body’ cache for a character; skin, teeth, eyes and any bound geometry. Then another ‘cloth’ cache; shirt, pants, coat. Then another for hair. The reason that we moved to this multi-mesh approach was that we reached the 256 OS limit for ‘too many open files’. After working with you guys and Autodesk, we found a way to raise that to 1024 via the API. But having all those open handles caused high disk i/o overhead, and we were back to where we started for performance. That’s why it’s nice to have just 3 open files per character instead of 30. Of course we can make an extra single cache if we need to go back and make a correction.

The gpuCache is exposed as a single node. Unless we export each object separately, there would need to be a method for relating shaders, lights, overrides, etc… so then we are back to a ton of complexity. That’s why I like the alembic cache option. It gives you a little bit more flexibility before sending to the renderer.

I was talking about walking the dag for the .abc path on export, or post translate, or using MPlug if it’s in the api. I’m probably missing something obvious, where it needs to exist on the node itself.

import pymel.core as pm

def abcPath(mesh_shape):
    abc_node = pm.listConnections((mesh_shape+'.inMesh'), type='AlembicNode')
    if abc_node:
        return abc_node[0].getAttr('abc_File')

Would this be possible using the post translate python? If the ‘useAbc’ extra attr exists, then don’t evaluate any animation during .vrscene export. Then the post translate slots in the correct paths.

When we evaluated which cache approach to write our exporter around, we passed on the vray proxy because of that… I’d like to find a way of using .abc if possible, but I suppose we could try to push it back to using proxy. I do really like the compatibility that alembic provides though.

Zach

15

The materials issue is one that we don’t know how to solve yet. When you use a V-Ray proxy, either with an .abc of a .vrmesh file, we handle the material connections ourselves and all is ok. But with the Maya caches there are problems:

1. It looks like the GPU cache does not support multiple material assignments (although I would love to be proven wrong);
2. Regular Alembic import loses the connection between a given mesh and the .abc file and Alembic object that it was imported from.

So, for 2. not only would you have to specify which Alembic file generated a given mesh, but also *which* mesh it is in the file, so that we can do a proper multi/sub-object material.

Also, worth noting. Alembic files may have limitations on the number of open files, but .vrmesh files do not (at least on Windows) :slight_smile:

Best regards,
Vlado

16

Right. I see your point. I was thinking you only needed the .abc file, but the alembicNode only outputs an array of mesh data. It probably wouldn’t be safe to assume that the name of the mesh is the name of the mesh in the .abc file, but it does appear that the imported mesh name is generated from the .abc file.

Looking at the alembic source for the may plugin on the alembic site, it’s just populating the ‘mOutPolyArrayAttr’ numbered with the mesh data with their readPoly function. We’d need to add another linked string array, and modify the plugin, which brings it’s own set of problems.

What about the python bindings? Can we get the name by searching through the archive, as shown here?

And finally, would this be feasible to do in the post translate .py? I could check for an attr, and dig up the mesh names myself, if I knew how it needed to be formatted in the .vrscene.

17

You probably could, but I don’t think this has anything to do with V-Ray specifically. You should be able to load the Python module on its own.

And finally, would this be feasible to do in the post translate .py? I could check for an attr, and dig up the mesh names myself, if I knew how it needed to be formatted in the .vrscene.Certainly, you can do that. You’d have to create a MtlMulti plugin with the material for each mesh and that’s pretty much it.

Best regards,
Vlado

18

Here is what I’m seeing in the exported file. This alembic file has 15 meshes.

From my hacking on the .vrscene file, if I just attach a MtlMulti, it’s assigned by index. It looks like order for MtlMulti is the same as the mesh output array from the alembicNode.

So I think I could figure out the shader/mesh relationships, then build a MtlMulti in the post translate. Then vray could skip the full geo export for those files. I’m not sure how custom render stats for geo would be handled…that will take more testing.


GeomMeshFile latigoShape@mesh3 {
  file="P:/..../Finaling/Geocache/Alembic/Latigo_Rig.Latigo_cacheFlowSet/Latigo_Rig.Body_cacheFlowSet.abc";
  smooth_uv=1;
}

MtlMulti latigo_vraymeshmtl {
  mtls_list=List(
    VRayMtl1@material,
    VRayMtl2@material,
    VRayMtl3@material,
    VRayMtl4@material,
    VRayMtl4@material,
    VRayMtl4@material,
    VRayMtl4@material,
    VRayMtl4@material,
    VRayMtl4@material,
    VRayMtl4@material,
    VRayMtl4@material,
    VRayMtl4@material,
    VRayMtl4@material,
    VRayMtl4@material,
    VRayMtl3@material
  );
  ids_list=ListInt(
    0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15);
}

Just to confirm, there isn’t a way to read a .abc file once, and then have @mesh# access to it’s individual sub-meshes for render stats and geometry settings? For me, that would be the best of both worlds for characters.

So you could have:


pointerToOpenedCache="P:/..../Finaling/Geocache/Alembic/Latigo_Rig.Latigo_cacheFlowSet/Latigo_Rig.Body_cacheFlowSet.abc";

GeomMeshFile geoShape@mesh3{
    file=pointerToOpenedCache;
    index=0;

GeomMeshFile geoShape@mesh4{
    file=pointerToOpenedCache;
    index=1;

I did discover something quite by accident! Your VRayMesh node reads .abc files just fine! And it publishes the shape names in the VRayMeshMaterial node! That’s pretty great! I’m still thinking about how that impacts our current workflow.

By the way, having the scene export in .1 seconds vs 1 second per frame… awesome! (Using GPU cache for testing)

19

Render stats in V-Ray are done through materials, so it should be perhaps not too hard to implement them, if you need it…

Just to confirm, there isn’t a way to read a .abc file once, and then have @mesh# access to it’s individual sub-meshes for render stats and geometry settings? For me, that would be the best of both worlds for characters.For the moment, this is not possible (but it may be in the future; we’ll see).

I did discover something quite by accident! Your VRayMesh node reads .abc files just fine! And it publishes the shape names in the VRayMeshMaterial node! That’s pretty great!Of course it does :slight_smile: that’s how we implemented the direct rendering of GPU Alembic caches - internally we just create a proxy for it.

Best regards,
Vlado

20

Well, it’s working great for regular alembic files too!

I’ll probably develop reading the .abc files as proxies and re-write our shader hookup to work with the MtlMulti. Then I’ll probably build something where the artist can use the alembic node if they need something more specific in the scene.

As far as gaining more control over the sub-meshes in the alembic file, I think the main efficiency would be in tweaking displacement settings. For organization/simplicity, say you wanted to have a single .abc for all the clothing, but only the coat needed high subdivs for displacement. Obviously you could write out a separate cache, (and that might be the right thing to do), but having that granular control seems nice. So the other way to think about this would be handling displacement like a MtlMulti - something that splits control for the mesh without splitting the mesh itself.

I guess we are really talking about complex scene assembly… katana is too complex for what I’m working with at the moment, but at some point, a tool like that would probably address most of these issues.