Camera Projection/Mapping distortion in single render pass from audience's perspective

[colorbleed]

Hi,

We are working on a new project that requires us to do some Camera Projections from multiple beamers (oriented on a multitude of angles) projecting on relatively flat surfaces. Now we're looking to render the "fake perspective" from the audience's perspective and are trying to figure out the best solutions with Maya + V-Ray.

Some of what we are trying to achieve is similar to what is described here under Virtual Camera's perspective is different to real world projector position specifically the Projecting on two sides of a house with one perspective example.

So we basically have a bunch of (off-axis) beamers projecting content as if correct from the audience's view, which can be described as "projecting from the audience's view" and rendering what the beamer would see... then beam that. Which is a technique that would work in 3D as well, however would require a two-step render process. 1. First render from the audience, then 2. render the projection from each beamer.

Currently we've tested a technique where we rendered from the audience's view in the "beamer camera" by using the camera's "Film Translate" properties and alike, but it didn't end up as perfect as we'd expected as differently oriented surfaces/beamers would have odd differences around the edges (where we had expected the "audience's view projection" would work, and it should, so I guess something must be wrong with our assumptions of the 'camera offset' technique)

• Is there a way to perform such a ''projection-render" in one go in V-Ray by applying some special tricks?
• Any other pointers on Camera Projection setups like this are welcome.

[Mihail.Djurev]

Hi,

I think the two-step process is the least complicated.

However, there's another approach that uses the V-Ray physical camera's lens distortion with a texture.

Let's assume you have the audience camera, a projector and a building that you project onto. The projector will actually be a camera in Maya.
See the attached sample scenes to make this clearer.

Step 1: you'll need to generate the distortion texture. To do this, we'll need a way to map a pixel position from the projector's view to a pixel position from the audience's view.
You can create a simple UV texture by combining two ramps (left red, right black + top green, bottom black):




Then, you'll need to feed this into a Maya projection tex. The projection should be perspective, from the audience camera. Note: make sure that fit type is "Match camera resolution", otherwise it may come out wrong.
Apply the projection tex on a VRayLightMtl and put it on the building.
Rendering from the camera with the projector's point of view will produce the distortion texture (distortion_map_building.exr in the file samples).


Note that you only need to do this once, as long as you don't move the audience camera, projector camera or change the building.

Step 2:
Set your render camera to be the audience camera. You should place the thing you want to render behind the building, as it will be the screen.
Hide the building. Set the audience camera to be a V-Ray Physical camera. From "Distortion type" choose texture and apply the distortion texture rendered in step 1.
Render and you should see a distorted image (rendered_scene_distorted.exr in the samples). Projecting this image from your projector should make it look undistorted from the audience's point of view.



Step 3 (optional):
This is only to make sure you've set things correctly without doing actual field work.
Hide the scene and show the building only. Apply a VRayLightMtl with a projection texture. Set the projection texture to be perspective, from the projector camera. Note: make sure that fit type is "Match camera resolution", otherwise it may come out wrong.
Connect the image rendered in step 2 to the input of the projection texture.
Render from the audience camera. Note: Make sure you have disabled the distortion on the physical camera.



You should be able to see the scene, as if it was projected from the projector, but distorted in such a way that it looks correct from the audience's point of view.

That's it! The sample scenes roughly correspond to steps 1,2 and 3. You'd probably need to fix the paths though.

Cheers,
Mihail

[Mihail.Djurev]

Here's how the scene looks in Maya

[dgruwier]

Cinema 4D has a feature that makes this a one step project, basically a shader that is itself a live view from a different camera, something I've requested previously. that's the only way I can think of to make this a single step process.

Are you having any technical problems with it being a two step process, other than convenience? I figure the second render should be pretty fast

edit: Didn't see Mihail.Djurev 's coment before I submitted my own, that is an interesting technique