Is it possible to animate render masks (from the image sampler rollout)

[ThomasMiller]

Is there a way to animate render masks? I'm talking about the option in the image sampler rollout

I work with a lot of product configurators where we need to render multiple options for parts, colors, etc, that all need to be rendered separately--but still need to be rendered in the context of the rest of the image.
My current workflow is to animate all the options (color options, visibility of extra parts that the users will be able to turn on and off) but the time consuming part is having to select the render mask individually for each object and rendering them one at a time.

I'm wondering if there's a way to animate the render masks as well so the workflow can be more seamless and not as manual every time changes need to be made in the renders.

Thanks

[Joelaff]

We make animated image sequences of render masks in Nuke or similar.

Be sure to load them in as Screen space, and for some reason Bitmap node seems more reliable in this one case than VrayBitmap (otherwise always use VrayBitmap).

The Progressive sampler is known to produce rough edges with render masks (the edges of the mask are slightly rough). You will get a warning to this effect in the logs. Bucket has no problem. This is really only an issue if you try to use exact sized render masks. Just make them a little bigger and comp over with some feathered edge if required.

[ThomasMiller]

Thanks for the tip, Joelaff. That's exaclty what I was looking for.

Just out of curiosity, I asked ChatGPT the same question and it produced a surprisingly good result:

Yes, you can animate the render mask option in V-Ray for 3ds Max, although it requires a bit of manual setup since the render mask itself doesn’t have built-in animation controls. Here’s a general approach to achieve this:

### Steps to Animate Render Mask in V-Ray for 3ds Max

1. **Create the Render Mask**:
- Use the **Render Mask** option in the V-Ray settings. This allows you to define which parts of your scene will be rendered based on a mask.

2. **Set Up a Bitmap Mask**:
- Create a bitmap image (like a black and white image) that represents the areas you want to include/exclude in your render. You can create this mask in an image editing software like Photoshop.

3. **Use the Material Editor**:
- In the Material Editor, create a new material that will use the bitmap mask. Assign this material to a dummy object or plane that will be placed in your scene.

4. **Animate the Mask**:
- To animate the mask, you can modify the bitmap over time. You can do this by:
- **Animating the bitmap in an external software**: Change the mask image frame by frame in an image editor.
- **Using a Multi/Sub-Object Material**: Create multiple materials with different masks and animate the opacity or the material IDs to switch between them.

5. **Use Controllers**:
- You can use a controller to drive the changes in your mask. For example, if you are animating the visibility or the material ID, set keyframes for the parameters that influence which mask is displayed.

6. **Rendering**:
- When you render, make sure that the mask settings are correctly applied. You may need to adjust the render settings to ensure the mask behaves as expected.

### Tips:
- **Preview Your Masks**: Use the V-Ray Frame Buffer to preview how your masks are affecting your render in real time.
- **Test Renders**: Perform test renders to ensure that your animated masks are working correctly.

This approach gives you a way to dynamically control which parts of your scene are rendered over time, effectively allowing for an animated render mask effect. If you have specific requirements or a certain scenario in mind, feel free to ask!

[Joelaff]

Like most ChatGPT it's *almost* good enough... Confused on some things, missing others, etc. The new preview model is a little better at least.

[ThomasMiller]

Hey Joelaff, I'm getting pretty bad rough/jagged edges even using bucket, regular bitmap, etc. What file format do you usually use for the masks?

Or is this normal behavior and I'm just expecting too much?

[Joelaff]

Oh, I expand my masks using something like dilate/erode so that you get some excess rendered data... Then I comp them back through a clean matte (cryptomatte, multimatte, etc.)

Also, might need to use dissolve rather than comp, or make sure you are handling premultiplication correctly, depending on the comp app.

[ThomasMiller]

Got it, thanks Joelaff. Looking at the documentation, it seems texture map render masks are completely binary, meaning pure black pixels are not rendered, and anything else that's not pure black renders--no gradients or anything in between.
Unfortunately, that defeats the purpose... I was looking for something that would bypass the compositing/post-production step altogether. Right now we're averaging 300 options per view, with 3 views for 4 products. That adds up to 3600 different images. If you assume that it would take us 1 minute to adjust the mask for each image, that would amount to 60 hours of work -- work that is prone to human error. And even if we automated the mask creation process, it still wouldn't be an ideal process as it would add another step and make iterations and implementation of the renders into the builder/configurator even slower.

The solution I found this morning was to use the "selected" render mask option (which creates perfect masks IMO) and create a script with the help of ChatGPT that selects the objects I want for each frame. As for color adjusments/post-production, I'll handle that in the VFB. It's working perfectly so far

I uploaded a screenshot of the script if anyone's curious (forum is giving me a hard time posting the script in writing for some reason)

Hope this helps someone who might come across a similar challenge.

[Joelaff]

Yeah, my method requires manual work. Not much work, but at that kind of volume it sure would add up. I suppose it could be automated with the method I described as well, but would be many more steps. I think Draft with deadline could do a lot of it.

Looks like you find a good solution. Thanks for sharing.

Note that the progressive sampler can make jagged edges with the selected option. So use the bucket sampler.

[M.Max]

What I usually do when I need to fix part of the animation is to make a plane facing the camera flying over the scene with an empty material ,then I animate it to cover the parts that's I need to fix .then I use it in the Render Mask Include List to render only what's visible behind it .
You can of course use multiple planes and do all kind of animations like scaling rotating or link it to moving objects ..etc
I've been using this technique for many years and it works pretty well for most cases.

[ThomasMiller]

That's a very creative way of doing an animated "render region," M.Max. It wouldn't work in my case as I need need perfect masks for isolated objects. Cool trick to keep in mind though

[Joelaff]

Yeah, I like that idea with the plane. Been doing it with render masks for a while now, usually built from a dilated alpha of some previous render.

For multiple versions of different elements like you describe I would render various “layers” using state sets so that I could comp the different versions over a clean background in post with perfectly clean edges. Or look at animating the materials or possibly VrayUserColor or Scalar or VraySwitchMtl to change properties based on environment variables at render time, or based on the current frame, etc. any way to keep things as a sequence makes all the later processing easier.

Lots of ways to do it, and sounds like you have already put a lot of research into it.