I am trying to work out how to do shadow/matte passes in the same way that I do in the scanline render. (max)
What I am trying to achieve is to have a alpha on the image that is just all black apart form the shadow, witch should be white. I am trying to do this using the Vray Properties tab on the object..
I set MATTE (tick) | Affect Alpha (tick) | shadows (tick) | Alpha contribution = -1.0
The problem is that the matte is all weird.. I get this strange checker pattern and I am not sure why.
the way you are getting a shadow is correct. The problem is in the mesh however. You have a box object with the thikness of 0, which means that faces from the backside to the front side intersect eachother. If you make the object single sided this should work. What I did was i went back to your modifier stack history (box) and added thikness in height to be more then 0.
After that, if you apply your standart vray shadow matte approach, it will be fine.