Tutorial: Stereoscopic panorama with Corona + Cardboard

TO AVOID ANY CONFUSION: This thread was started on 17.12.2015. Since then, native support for VR (stereo spherical camera) was added to Corona, and now probably different approach is required. However, maybe there is still some helpful info in this thread.

UPDATE: see the new “better” guide here: Tutorial: Stereoscopic panorama with Corona + Cardboard - Corona for 3ds Max :: Tutorials & Guides - Chaos Forum

NOTE: Corona alread… soon will have built-in sterescopic pano controls. I am posting this guide mainly as a curiosity. Maybe it will serve as a base for further experimenting for users with better knowledge of the topic.
NOTE #2: example files are below.

So, here we go:

What we will need:
-3ds Max - it was chosen as the host application, we will be using Max Script so it won’t work with any other host, unless you can create a similar script/tool
-Corona - any version with spherical camera support should work
-krpano - make sure you download the 1.19 version!
-VR-ready devices (I used my good old Moto G + basic version of Google Cardboard which cost about 2€), you can also view your pano on the monitor, but it’s not so much fun :wink:
-software to view the vr pano - I am using Firefox for Android
-some file browser for your Android (I am using Amaze, but anything which can go to a location and save it’s path to clipboard should be ok)

1. Configure cameras - basically it is a head+eyes setup:
My setup was based on Rambambulli’s one. Basically I have 3 cameras. Camera-center is used for viewport/IR preview only.
cameraL is moved 3,5cm to the left from Camera-center. - it is ESSENTIAL to use the name cameraL for the object unless you edit the script
cameraR is moved 3,5cm to the right from Camera-center. -it is ESSENTIAL to use the name cameraR for the object unless you edit the script
All cameras’ targets are placed in one point, which is centered (so that it aligns with Camera-center), and moved away from the cameras (about 1-2m should be fine, I did not experiment with this, basically this is the exact point you are looking at).
The cameras and their targets are linked to a circle (rotation-helper). The circle is animated, it does a full 360 degree rotation through the whole animation range.
The cameras are aligned to the very center of the circle helper, and then are moved a little (7 cm) towards front. This is because your eyes are away from the spine - in this case the pivot of the circle helper.
All of the cameras have Corona Camera modifier applied with spherical projection type enabled (you can disable it for the center camera for previews if you wish).
The whole rig should be at some realistic height, so I guess 160cm+ - remember your eyes are a little lower than your total height.

2. Animate the helper:
It needs to do a full 360-degree spin which takes the same amount of frames as the whole animation. Remember that 0-100 frames is actually 101 frames in total.

3. Model the environment:
You can place whatever you want around the cameras. there will be probably some problems if something is extremely close to the camera. I’m not sure.

4. Render setup and animation length:
This is very important! The way the script is constructed requires the following:
-The total number of frames should be the same as width of the render output. So if you are rendering 2400x1200px image - use 2400 frames of animation. If you are rendering a 600x300px image - 600 frames
-The circle helper must do the 360-degree spin in that number of frames

Render setup:
-set pass limit
-disable “lock sampling pattern” in performance tab
-you can set VFB to none
-remember about the correct 2:1 ratio in output size (2400x1200, 600x300 etc)

5. Run the script:
Note: the script was made by Rambambulli. All the credit goes to him.

Make sure you have the correct viewport selected.

Go to MAXScript > MAXScript Editor…
Paste the script:


--set render mode to crop
setRenderType #crop

-- change so the output width of a image divided by the stripwidth makes a whole number 
stripwidth = 1 

-- the left eye image 
PanLeft = bitmap renderwidth renderheight
-- the right eye image 
PanRight = bitmap renderwidth renderheight 

--set a region to render a crop 
EditRenderRegion.EditRegion()
EditRenderRegion.UpdateRegion()
viewport.setRegionRect viewport.activeViewport (Box2 (renderwidth/2) 0 1 renderheight) 
EditRenderRegion.UpdateRegion()
EditRenderRegion.EditRegion()

parts = renderwidth/stripwidth 
-- for testing parts = 100  or whatever you like 

-- for the left eye

for t = 0 to parts do ( 
display panLeft 
--make sure the left eye camera is cold cameraL. Or change the name here 
viewport.setCamera $cameraL 
strip = render rendertype:#crop vfb:false frame:t  
    pastebitmap strip panLeft [0,0] [(t*stripwidth),0] 
) 

-- and for the right eye

for t = 0 to parts do ( 
display panRight 
--make sure the right eye camera is cold cameraR. Or change the name here 
viewport.setCamera $cameraR 
strip = render rendertype:#crop vfb:false frame:t  
	pastebitmap strip panRight [0,0] [(t*stripwidth),0] 

)  

Then left-click in the pink text field in the lower-left side of the 3ds Max ui. Once the cursor appears in it, press Esc key - you should then see interrupted text in the lower, white text field. If something goes wrong, you should do this again before running the script again.

Get back to the script editor window, click Tools > Evaluate All.

If everything goes fine, you should see standard 3ds Max VFB pop up and a vertical line going from left to right with the rendered content appearing. You will need to wait for the two panoramas to render out this way.

6. Save the outputs:
After the rendering finishes, save the two images as pano_l.jpg and pano_r.jpg (IT IS REQUIRED TO SAVE THE FILES UNDER THE NAMES SHOWN HERE! the first one which renders out is the left one, the second one is right).

7. Create the stereoscopic VR tour:
Open your krpano installation folder. Select the two images (pano_l.jpg and pano_r.jpg) in explorer, grab them, and drag-and-drop them onto “MAKE VTOUR (MULTIRES) droplet.bat”. You should see a command window that will do its job and then tell you to press any key.
Press any key.

This should convert your images into tiles, and save them in a folder structure in the place where they were originally stored. Go to that location.

In that location, go to “vtour” folder and open the file tour.xml in notepad (or any other text editor). We will need to do the following:

-remove the whole second section: http://s29.postimg.org/fbjo3u1qv/remove.jpg
-add stereo=“true” stereolabels=“l|r” to the tag: http://s17.postimg.org/5jk61wm4v/image_add.jpg
-in the change ‘pano_l.tiles’ to ‘pano_%t.tiles’
-save the file (just Ctrl+S it - resave under the same name)

After that is done, you can run the tour.html in your web browser on your PC to check how it works. You should see the panorama running, with a bottom menu where you can pick the Google Cardboard icon.

8. Send it to your phone:
I just connected my phone via USB and uploaded the whole folder with the original images and the pano directory to internal storage. For some reason it did not want to work when I uploaded it to the SD card. Maybe it’s a useful hint, maybe I made some mistake.

Then, on my phone I used Amaze file browser (but I guess you can use anything), tracked the folder with the tour.html file, copied the path to clipboard (in Amaze you long-press on the path).

Switch to Firefox. Click on address bar. Paste the path, then add “tour.html” at the very end of it. Go!

Then just put your phone inside the Carboard/other viewer and enjoy the random teapots around you.

If you have done everything right, it should just work. Let me know if something goes wrong, there is high chance I made a mistake while typing all of this. :smiley:


Known issues:
-Once you run the script, it cannot be paused/canceled (at least I do not know a way) so you will probably need to kill 3dsmax process. Incremental saves are advised. :wink:
-The demo version of krpano leaves ugly watermarks all over the panorama - this can be probably fixed by using other software or buying the full version

Thanks for everyone who helped me do this, especially the guys over krpano forum who had to read my posts where I bitched how hard it is to find basic info, while it was my mistakes. :wink:

ready-panorama.zip (5.84 MB)
pano-max-file.zip (47.7 KB)

I’m surprised there is no interest in this. :frowning:
I have just found a faster and less painful way with no watermarks. And with daily builds’ VR support you do not have to use any scripts. I will post everything soon.

Don’t worry, I am interested. But at the moment trying to catch a deadline, …rendering an animation.

I’d really love to see panos in 3D, but i haven’t device in which i could do so.

of course im interested in a clean easy 360 stereo pano solution for corona !!
so far i could only do it in vray or octane with their rendertime solution…

i will test your way asap ! …and share my outcome…

thanks a lot man!

It depends whether you want to get some really nice experience, or just see how it works. If you just want to see it, then I would recommend getting a cheap Cardboard viewer (AFAIR I bought mine for about 2€ just for fun) and a budget Android phone. I am using a moto G which I bought at the beginning of 2015 for about 150€ and I am still hapy about using it - no freezing or lags, checked with some 3d games and they run pretty smooth too. It could be a good excuse for getting yourself a new phone. Scientific purposes. :wink:

well,here is a good place to start : https://labs.chaosgroup.com/index.php/portfolio/guide-to-virtual-reality-2/

its from chaos group but the pdf guide is very informative. and the images they rendered in vray cube vr are amazing in 360 stereo vr.(18k wide !!!)

a client of mine gave me a samsung gear vr and a samsung galaxy s6 edge to go with it, so im telling you the experience is superior over any cardboard or some other
cheap plastics. the FOV is very large and you are immersed in the 360 world.

this is what pushed me to learn more about vr and making and rendering content for it.
i will try maru’s script and workflow, hoping to get something from the dev team as an addon to corona.

cheers mates !

To see how it works would be enough. Thing is, i already have android device, but it hasn’t gyro, so it’s not very useful for VR experience. Or am i wrong?

I;m waiting for your tutorial Maru!

New guide!
Android + any VR viewer such as Cardboard again. This time it’s much more simple and has no watermarks.

You will need:

Either: the script mentioned in the 1st post - this will give you 2 images which you can just use
Or: new version of Corona with the built-in VR support - this will give you one image, which you will need to split into two
This app on your phone: PSViewer

Requirements:
-Each of the images needs to be wider than 2000px
-Width needs to be 2x height (2200x1100 etc)
-left eye image needs to end with _l.jpg
-right - surprise… _r.jpg

Howto:

  1. Render the image(s)
  2. Save as xxx_l.jpg and xxx_r.jpg - each one’s width has to be above 2000px!
  3. Copy the files to your phone, into one folder
  4. Start PSVIewer, pick “load photospheres”, find the directory where you copied the files, pick one of the images (for some reason it displays only the _l.jpg image for me - it is fine), pick “add photospheres to gallery”
  5. Pick “view gallery”
  6. It should just work. To switch to next panorama, you need to tilt your head (either left or right, depends on the sensors in your phone)

The End

Thanks a lot Maru! I’ve just tried it and it works like a charm!


Great! Did you use the script or Corona’s DB?

I used Corona Cam modifier with VR on, resolution 4000x4000, then I separated files in external program. Using PSViewer with correct phone (The program doesn’t care about screen size, so when it’s too big you have to have big exotropia to see it correctly!) it looks fantastic. Now it’s time to test some interiors.

soap bubbles and godrays, somehow the 3d effects are not that apparent on my device


meado-vr.zip (4.49 MB)

I’ve done this one just for another quick test, 3d effects are clear and strong. I use
Eye separation =6,3cm
Eye front offset =8cm
Converge eyes = unticked

Another problem in PSViewer is that on some phones it won’t keep the horizon in horizontal position, so you can’t tilt your head too much.


is the Corona Cam modifier with VR support a part of a new daily build ?
how can i get it?

Thanks !

Yep, you can get it with newest daily build.

to Maru:
After checking your Meadow I also noticed that 3d effect is visible only a little - was the scale of scene corresponding with VR camera and IPD (Eye Separation)?

Another important thing to feel comfortable in VR is front eye offset.

Yup, you need to get a daily build and then apply Corona Camera Mod. It has built in VR controls. Sorry, I forgot to write about it. :smiley:

to 88qba88, yes, the scale is correct, I used 7cm for eye offset and 7cm for front offset. Converge was set to 2m as far as I remember.

dear maru, also on gear vr both the meadow and 88qba88 image is very flat,the depth is almost non present…maybe because of no foreground and background stuff…

Some of the bubbles were placed really close to the camera. And they were in 3 groups - each one was in different distance from the camera.