China's 4DV AI Releases WebXR Demo Of 6DoF Volumetric Video

China's 4DV AI released a WebXR demo of volumetric 6DoF videos created from 20 camera views, and you can try it on your VR headset right now.

4DV's technology uses gaussian splatting, of course, leveraging advances in machine learning to "train" a volumetric output from the video inputs. In recent years, gaussian splatting has done for realistic volumetric rendering what large language models (LLMs) did for chatbots, propelling the technology from an expensive niche to shipping products like Varjo Teleport and Niantic's Scaniverse.

While Gracia shipped a moving splats demo last year, that system took six minutes to train a single frame. In contrast, 4DV's paper claims its technology is 30 times faster, training a whole second of video in that time.

Further, Gracia's scenes require expensive and non-portable studios with dozens of cameras in a sphere around the subjects, which are captured without most of the background, while 4DV is focusing on capturing the whole frontal arc of the scene from around 20 regular cameras on one side of it. It's still far from a point-and-shoot solution, but a professional videographer could easily set up a capture solution like this.

4DV AI's data efficiency is also impressive, with each second of footage taking around 12.5 megabytes, meaning in theory for a 100 megabit per second internet connection it would take a second of loading time per second of footage.

The result, as you can see for yourself in the WebXR demo, is like a 180° 3D video but with 6DoF, meaning you can lean around without the entire scene moving as if fixed to your head.

The further you move from the central area of overlap of the cameras, including into entirely occluded areas, the more blur and artifacts you'll see. The reconstruction here is impressive, though still not perfect. And yet, the ability to lean is a major improvement over traditional immersive video.

You can try 4DV AI's WebXR demo in the web browser of your standalone headset, such as Quest, Pico, or Apple Vision Pro, or on PC VR via a browser with WebXR support such as Chrome.

The demo features a series of eight 10-second clips, as well as a few static scenes too. All the scenes are also viewable on flatscreen, if your headset isn't handy right now.

On Apple Vision Pro and PC VR I found the demo performed relatively well, with minor judder, while on Quest 3 I saw flickering and major performance issues, so your mileage may vary between devices.