Bake to vertex color set

[gisi]

What is the workflow for the "bake to vertex color set" option? I can`t manage to get a result. I want to make an occlusion baking to the vertices of a high poly mesh without an workable uv mapping. Can`t find useful hints in the documentation or any tutorial for Vray/Maya. At the moment I am using Xnormal for this, but it would be more feasible directly in Maya.

Thanks, Gisi

[yolov]

Basically, the idea like you've guessed, is to bake any color (and lighting) information to a vertex color set for a given object, instead of relying on UVs. We'll probably have to make a tutorial so this is better explained, but until we do that, here's the gist of it in short:

1. You can apply and shaders and textures to your object, add any lights too and so on.
2. With your object selected, go to the render settings > common tab > bake engine and choose "bake to vertex color set"
3. add a name for "color set name" - a new color set will be created for your object using this new name and the baked information will be written there.
4. disable "bake to vertex color set"
5. add a new shader to your object
6. connect a VRayUserColor node to (for example) the diffuse slot
7. in the VRayUserColor you will need to enter the name of the color set that holds the baked information. You can easily get the name by inspecting the "color set editor" with your object selected. Keep in mind that if you have also baked "alpha" information (from the bake to color set options), you will have two resulting sets - <setName>_RGB color (holding the color information) and <setName>_Alpha (hodling the alpha information). Enter the name of the "RGB color" color set in the VRayUserColor node and render.

Does this help?

[Miro]

EDIT: Deleted post

[gisi]

Hi Alex,

thanks for your instruction. My problem is point number 3: how do I initiate the baking process? When I only add a name for the color set (with a selected object), nothing happens.
When I choose Lighting/Shading -> Bake with V-Ray, a texture baking process is started, although "Bake to vertex color set" is choosen. No color set appears in the color set editor.
Render -> Batch render doesn`t work. I`m missing a "Bake with V-Ray" under the MeshDisplay menue ...

Gisi

[yolov]

Ah yes, sorry about that. The lighting/shading baking options have their own "bake with vray" button that starts the baking (but they don't support baking to vertex colors).
To bake to vertex colors with the settings in the render settings > common tab, you just need to select your object and hit "render" as you would do for a normal render and that's it.

[gisi]

ok, so easy if you know it, thank you. And I can use Render Elements etc., great! But I made some testrenderings with a simple scene, light mtl and dirtmap texture for ambient occlusion and I am not sure of the render settings. I can improve the sampling in the dirtmap, but the general quality is still very bad. In my screenshot is a comparison with the result in xNormal.

[yolov]

I would say that this is one of the downsides of using vertex colors - you will need a very large amount of vertices to hold high-precision information, i.e. your objects need to have a lot of segments.

[vlado]

I'm not sure it is a matter of segments, Alex - this looks like it might be a bug. Can you reproduce the problem here?

Best regards,
Vlado

[gisi]

But it is the same geometrie and amount of vertices! The left sphere and plane I made a vertex baking with V-Ray inside of Maya. The right sphere and plane I exported as FBX to xNormal. Vertice Baking in xNormal and reimport. It`s not a comparison of vertex and texture baking, both are baked vertices. In my real project I use CAD Data with a hugh amount of vertices and no suitable UV mapping. Thats why I would prefer vertice baking in spite of texture baking.

[yolov]

I ran some tests and I had no luck in reproducing this. The only way I could replicate it is if after baking to vertex colors I apply the baked color set for rendering, but accidentally bake a second time and then render.
gisi, could you please try and do another bake in a new scene and after baking, turn the baking engine off, assign your baked color set to a new shader's diffuse with a userColor node, render and see if renders fine?

[gisi]

For me, it`s easy to reproduce in a new scene. Noticable: the plane is smooth and clean, the sphere not. Tried the same with a torus, look at the screenshot. If you assign a simple Maya lambert after baking and toggle the display color attributes you are able to see it directly in the viewport. Rendering with V-Ray and userColor node is the same result.

[yolov]

Could you share one such scene before the baking and one after the baking? It will make it easier for us to figure this one out. Thanks in advance.

[gisi]

while preparing the scenes for you I could locate the problem: it`s a matter of size. I am working in real world dimensions, my sphere is 100 cm. Scaling the objects down to 10% gives a clean result. Is this a bug, or can I change any settings?

[vlado]

I think there should be a bias parameter in the texture baking settings that may need to be increased.

Best regards,
Vlado

[thescopeteam]

Is there any way to control the quality of the vertex paint? A colleague has a script for Houdini and this result looks much better then the vray baking on the same mesh.