For the layperson, volumetric video capture is the process of filming an environment in three dimensions. The top performances in the world can now be seen through augmented reality, virtual reality, and holographic platforms. All throughout the world, people are coming to the conclusion that volumetric video is the next frontier for the development of the media production sector. With the explosion of the virtual reality (VR) and augmented reality (AR) industries, volumetric video is emerging as a crucial technological development. This research presents a novel approach to the collection and analysis of volumetric footage, dubbed 3D Human Body Reconstruction (3DHBR).
The core of the system is sixteen stereo camera pairs capable of capturing 360-degree video of a moving object. Modern stereo techniques gather depth information from every conceivable direction to create a 360-degree point cloud. Following a meshing and mesh reduction procedure, the resulting meshes will be compatible with the majority of 3D modelling software. This paves the way for the use of convincing, lifelike, and dynamic 3D reconstructions of humans in motion in VR and AR applications.
Since depth-sensing cameras can capture the performer’s every nuance, this method of creating an immersive experience is particularly well-suited to live-action content. Volumetric video provides a more realistic representation of the three-dimensional world than regular video. It’s like ordinary video in that it captures a likeness of the subject in motion, but each frame also records the subject’s geometric shape.
A completely realised 3D “hologram” that appears and moves just like a living subject may now be viewed by the public without the assistance of digital artists or developers.
Exploring volumetric video in real-time 3D is the finest method to experience its benefits. Converting the game into a 2D video would be a waste of the 3D effects used in it. Instead, we’ll employ gaming engine technologies to present the imagery in a lively fashion. Incorporating our expertise in augmented and virtual reality, our team enables volumetric video to be viewed in a purely AR/VR setting. This is because we have a large number of case studies showcasing their previous successes across many different types of projects.
A new industry study predicts that the volumetric capture market will expand from $578.3 million in 2018 to $2.78 billion in 2023.
Media coverage of volumetric cameras like the Manifold has mostly focused on the potential for such devices to enhance the quality of virtual reality (VR) experiences. Users are given unrestricted access to previously inaccessible worlds by combining volumetric video with a VR headset that supports positional tracking, such as Facebook’s Oculus Rift or Quest.
The viewer, in a sense, assumes the role of director, as it is his or her choice as to what happens in each individual frame. However, the same potential will also provide new resources for more conventional filmmakers.
What Does that Mean?
Red CTO Uday Mathur stated that the VFX capture’s potential, virtual green screen and the plate capture was way more enthralling than its applications for 360 video and VR. In fact, they believe this will be the camera’s primary use until the VR ecosystem matures.
“At the moment, we envision this working in tandem with a conventional camera to capture a whole distinct perspective of the situation that can be changed in post-production,” Mathur explained. It removes the burden of capture and lightens the pressure of what must occur on the set. It affords the post-production team creative latitude.”
According to Mathur, the possibility of employing the camera to capture immersive content is secondary to its promise for conventional film productions. “For the traditional cinematographer, this is an entirely new type of instrument that enables a different kind of experience.”
How It Works
It is essential to know how the volumetric video actually works so first, it captures various images from the screen creating various perspectives and the data from those images is then used to generate a depth map of the area. For example, each Manifold can record data and that too in several directions using a minimum of four sensors and, in most cases, seven sensors.
The Manifold is outfitted with 16 RED Helium 8K sensors, each paired with a custom Schneider 180-degree, 8mm fisheye lens with an F/4.0 aperture.
The Manifold, like the vast majority of other 360-degree cameras, can stitch these images together to create a 3D 360-degree video. Using these numerous camera angles, however, Facebook can generate a depth map (for which it has partnered with Otoy) as well as an infinite number of perspectives on the scene.
Once you capture a volumetric video, there is a plethora of ways in which you can intend to use it even far beyond its original intent. It could, for example, be used to create additional assets for the project for which it was designed, but it could also be used as a virtual set for an infinite number of other projects.
The classic ideal according to Kate Wurzbacher, Head of Camera at Here Be Dragons, is to “plop down two of these cameras somewhere in the forest, take a few frames, re-light the whole thing later, and throw some actors in.” “That, I believe, is the pinnacle of all dreams. Right now, we’re just getting started.”
The Future of Volumetric
The next stage after recording events on camera is to produce volumetric video.
Positional tracking, spatial computing, and VR/AR are all converging. Volumetric video provides the spectator with an unrivalled look and feel of the captured performers, which will “really indulge” them. This enhances comprehension and retention.
The Bottom Line
Volumetric video is the new technology of the future. Volumetrics will be utilised extensively in the micro world, meta-verse, spatial internet, and virtually any other setting imaginable. All of this boils down to documenting every aspect of our life digitally. A volumetric video can even help in redefining different forms of entertainment. A portion of the future concerns the amount of resources available. To learn more, please visit our website and read about it.
