A common complaint about “live action” camera shot 360 degree imagery is that it’s not really VR. This seems a little misguided, certainly there are different levels of virtual reality, and I think there is definitely space in the immersive media field for non-computer generated photography. There are a number of characteristics of computer generated VR experiences that are lacking in much of the live action shot today. But these will eventually be addressed allowing us to capture a live scene and completely synthesize it in VR, in full stereo, from multiple angles.

Stereo is the most basic aspect that is required for a faithful reproduction in VR. Flat non-stereo images don’t take advantage of VR headset, and quite honestly just look like a flat wall, not very immersive at all! But stereo capture in the field can be complex. In fact many feature films sporting 3D were in fact shot in mono and “fake” depth added. There are whole companies dedicated to this 3D conversion. But stereo capture is very possible today, generally with a spherical camera rig with 6 to 12 separate lenses.

Next is “6DOF” or 6 degrees of freedom, another aspect of “real” virtual reality that is often missing in camera shot footage. So called lightfield cameras that can capture a scene in 360 degrees full stereo from multiple vantage points are very few and far between. Lytro had been pushing their cameras for this application, but have been absent as of late. It is possible to fake a small amount of camera movement with the creation of a depth map, and an appropriate viewer. This is where Fraunhofer’s Realception comes in. Part of their technology is a viewer plugin for Unreal Engine that can play in realtime images with depth information:

Realception^® for Unreal Engine enables real-time playback of live-action video light-fields in VR worlds on different head-mounted displays. Its main application is to provide a windowed 6DoF experience with a mix of live-action footage and VR footage.

This represents the next level of immersive “VR photography” and it is exciting to see where this technology will lead.