Automatic "psyop" ID mattes

[Fredrik Averpil]

Now this (Cryptomatte) would be completely fantastic to have. Any chance a new render element could be developed for this?
Perhaps we can figure out a way to build a gizmo to read this new render element in Nuke?

SIGGRAPH 2015 poster and description: http://www.psyop.tv/wp-content/uploa...tes_poster.pdf
Quick video demo: https://vimeo.com/136954966

Without having given this too much thought, I am assuming it is not possible to output the required data today using a combination of existing render elements, right?

[jasonhuang1115]

Seems like some gizmo based tool is needed in Nuke to use this efficiently??

[jenya.andersson]

This is pretty bad a*ss, is it using metadata in the exr to label ID names?

[Fredrik Averpil]

Seems like some gizmo based tool is needed in Nuke to use this efficiently??

That is correct. Here's a quote from Jonah, one of the authors of Cryptomatte:


About the Nuke side, a basic implementation can definitely be done with regular Nuke gizmos/nodes. In fact, when we've had to export Nuke setups to other studios, we use a tool called "decryptomatte" to turn all the cryptomatte keyers into expression nodes that will travel.

At the end of the day, the Nuke side is just a combination of id keys multiplied by corresponding coverage channels and summed to give you the resulting mattes. Making the implementation elegant requires some scripting, as we wanted to do things like reverse name lookups from selected colors (you pick a color and it tells you the name by finding the selected ID in a table stored in exr metadata), support for arbitrary "crypto-depth" (i.e., how many id/coverage pairs are used), and ID generation from names (you tell it a name, and it keys the corresponding color).

I think deeper integration into a compositing package could have really interesting performance advantages over using traditional mattes (fewer disk operations, less data in and out of RAM, etc), but from an end user's perspective the gizmo-based implementation works just fine.

[bigbossfr]

What is the difference between this a renderid from deep exr ?
Here we have a tool in nuke to pick something on the deep image and it will create a matte automatically.

[vlado]

Yep, I fail to understand how that's any different from deep images (or RPF images for that matter). One can certainly code a render element plugin for V-Ray to produce these mattes, but deep images are so much more versatile.

Best regards,
Vlado

[jasonhuang1115]

Just FYI.
Cryptomatte just went public and open standard.
https://github.com/Psyop/Cryptomatte

Blog post: http://www.psyop.com/news/view/new-c...h-beyond-psyop

[vlado]

Yes, I know. And I still don't understand why people don't just use regular deep files for the same purpose - without the need of *any* additional plugins or code. I asked the Psyop guys; they couldn't really give any answer why they think their method is better.

Best regards,
Vlado

[jasonhuang1115]

Hey Vlado, Andy (from Psyop I suppose) explained a bit in his last reply. Believe you've got and read it. You definitely know the best and maybe it's still not good enough to justify for supporting it.

[vlado]

I did read it, but it still doesn't make a lot of sense to me unfortunately. I still think deep images (or Evotis for that manner) are a better way to go. If we have to, we will implement it, but it would be such a sad way to spend developer time. Deep images are already there; they work perfectly without any extra plugins or anything. Why make things more complicated?

Best regards,
Vlado

[FSGFX]

isnt the trick of crypotmatte just that you have all named in nuke...?
just pick the object by name...

[vlado]

isnt the trick of crypotmatte just that you have all named in nuke...? just pick the object by name...

The naming convention and the fixed way to convert between object names and hashes is useful indeed, and I can totally see a use for standardizing that. But the weird way to encode subpixel mask information into a flat OpenEXR file - not so much.

Best regards,
Vlado

[chuckie7413]

Hey Vlado,

Thanks for you take on it.

We do not use deep compositing at our studio so my knowledge is limited but if we were, you say you could get the same effect from deep but would we not have create a mask for every object in the scene? Especially if you want to single out an individual object element? If you rendered all the objects together I am sure you could pick and area in 3d point space in deep to isolate it but it would be an area rather than a specific object? Especially if objects are grouped close together in space.

It seems like crypotmatte, openexr/id by gorilla or Evotis could be easier in production and less of a storage strain than deep.

Your comment about it being a sad day for your developers having to code to support this makes me not want to push you more about it but just wondering how easy it actually is to get the same usability with deep.

Thanks,

Richard

[monsteradurm]

Yes, I know. And I still don't understand why people don't just use regular deep files for the same purpose - without the need of *any* additional plugins or code. I asked the Psyop guys; they couldn't really give any answer why they think their method is better.

Best regards,
Vlado

If this is correct ( and I certainly believe you that it is ) then is all we need to do to get multimatteIds working correctly (with aliasing ) in nuke a simple deepexpression and if so would anyone know what this is?

For example, exporting out of vray 3.4 for maya with multimatteId and coverage pass I can use something like..

multimatteid == 5, to isolate my materialid.

However, im unsure at this point what to do with the coverage pass.

(objectid == 1) / floor(coverage), seems to give a decent AA over the whole object, but it doesnt work with the materialIds.

(multimatteId == 5) * coverage, doesnt seem to alias the object at all.

Any pointers would be great..

[instinct]

There seem to be some misconceptions in regards to deep images and how VRay can store information in deep.

@chuckie:
I have asked questions like vlado for both guerillarender and cryptomatte and have not yet had a real answer either. By convention if you are using deep for volumetrics you have z-range based sample merging. This is up to the renderer though. VRay by default merges samples by renderID rather than z-depth. Which gives you object seperation without a waste of additional samples. It still needs to cheat a little and there is also a fair share of issues left in Deep, but i still find it superior. I have hops that evotis will solve some of the issues of deep. Will see. I don't buy that it is smaller than deep yet (when writing equivalent info, that is no volumetrics), and i am not so keen on something non-standard and closed-source. But that's way to early to judge.

See also below.

@monsteradmurm:
Yes, that's kind of cumbersome but does work. You do not render and MMREs anymore. Only MtlID or ObjectID. To pull a Matte you could do the following in a DeepExpression:

other.VRayMtlID == 35 ? 1 : 0

This will pull a pixel-perfect Matte similar to a Multimatte based on MtlID 35. You can also combine multiple IDs like so:

(other.VRayMtlID == 35)||(other.VRayMtlID == 8) ? 1 : 0

That beeing said, we have some internal nodes that can create MMREs from arbitrary selections of IDs as well as nodes that extend the available toolset in nuke to actually mask based on deep IDs.
The latter brings a lot of additional possibilities because you can actually grade SAMPLE-perfect rather than pixel perfect.

You can also mask objects sample perfect (and then tweak and re-combine with a simple plus) by adding the following to red/green/blue respectively:

(other.VRayMtlID == 35)||(other.VRayMtlID == 8) ? rgba.red : 0
(other.VRayMtlID == 35)||(other.VRayMtlID == 8) ? rgba.green : 0
(other.VRayMtlID == 35)||(other.VRayMtlID == 8) ? rgba.blue : 0

This will isolate all MaterialID 35 and 8.

Cheers,
Thorsten