Intruduction

1. Volumetric capture technology
   1. Technology examples
      1. Full volumetric capture studios
         1. https://metastage.com/
         2. https://4drstudios.com/en
         3. https://www.dimensionstudio.co/
      2. Depth sensor technology
         1. https://ef-eve.com/
         2. https://holocap.com
         3. Microsoft Kinect
            1. Usually, specifically the Azure Kinect
         4. LIDAR sensors, such as the ones on the more recent iPads
      3. Mobile Capture Device and Apps
         1. Volu
         2. Record3D
         3. Hoxel

---

Points of Interest

1. Introduction
   1. Volumetric video is a videography technique allowing for the capture of a three-dimensional space, as opposed to the usual two-dimensional cameras we use today. The methods we utilize currently to capture these three-dimentional spaces vary wildly in scale and price, ranging from photogrammetry apps and taking multiple pictures of a subject, to LIDAR scanners taking densely-packed laser-sampled points of a subject of space in order to turn them into processible data cloud, to mass-merket technologies such as the Microsoft Kinect and Azure Kinect, photogrammetry, to full-on volumetric capture studios specifically designed to be cutting-edge in this space, utilizing hundreds or even thousands of cameras to capture a three-dimensional rendering of a space and/or subject.
   2. 
   3. 
   4. In particular, one form of this tech I wanted to point to was Microsoft’s Mixed Reality Capture Studios, one of many in-house/partnered studios devoted to volumetric capture and hologram generation. These studios use over 106+ RGB/infrared cameras and green screen backgrounds, along with various forms of processing to create these three-dimensional captures of people, objects, performances, etc, to later be able to be output to various format like custom .mp4, OBJ PNG, etc.
2. Why is it a good use for VR/AR?
   1. One major benefit of this kind of technology is the fact that a capture can be reused, re-purposed, and even changed post-capture in a three-dimensional context. One can, for example, repose a volumetrically-captured model to fit better within a 3D CGI scene, or much more accurately capture a space directly from a volumetric capture device, and use that as a model for, say, tourism or museum work.
   2. As for this specific technology center, it represents a form of volumetric technology where money is not much of (or is not, given Microsoft’s size) an issue, giving content creatorsa much more high-grade way to acheive volumetric capture, compared to, say, a LIDAR sensor, or photogrammetry from a phone.
   3. studio_example
3. Who are its intended users?
   1. This technology could extend to nearly anyone with a need to capture a model or space in some way, shape, or form. The average person in the future might, instead of simply taking a photo of a place or event, might instead use the LIDAR scanner on their mobile device to take a 3D scan of the space, and throw it into some VR/AR compatible device application and be able to walk around in the space they’ve captured. Large studios, especially those who make heavy use of CGI or similar technology, architects or museum/history runners may, instead of painstakingly modelling out some space they want to show off, might elect to simply go in and capture the actual space itself, and throw the resulting models into something like Autodesk Maya or Blender, and be able to integrate it directly into their production pipelines and achieve a much tuer sense of immersion than what we currently have to work with.
4. What are the issues (positive and negative) with it?
   1. In and of itself, it’s a fairly benign technology, simply a video capturing technology. One of the main issues is not just slow(ish) adoption in the industry, but also the currently limited capabilities and scope of technology, and an overall lack of standardization in the space (similar to the current lack of standardization in the VR/AR space). Large studio pipelines and even more so the general public are unable to easily accommodate the much more intensive technology requirements of current-day volumetric technology, especially with the current level of technology we have now.
   2. Cost is also a huge roadblock in adoption, both physically, monetarily, and even in terms of the data itself: enormous sets of data need to be streamed or processed for volumetric capture, and we simply don’t have many good ways of dealing with this (most in this space have some sort of proprietary application to handle this, which further exacerbates the standardization issue mentioned earlier.)

---

Various Examples of these Capture Studios in Use

1. Afterworld: Balenciaga, Streamline Media Group, 2021
   1. Links
      1. https://www.unrealengine.com/en-US/spotlights/what-balenciaga-s-afterworld-the-age-of-tomorrow-tells-us-about-the-future-of-fashion
      2. https://videogame.balenciaga.com/en/
   2. Description
      1. One example is a collaboration between fashion design house Balenciaga and a software development company named Streamline Media Group to create a video game named Afterworld, for one of Balenciaga Fall 2021 collections. Within this development, one of Microsoft’s partnered studios, Dimension Studios, provided volumetric video capture for 50 digital models to be used as game characters in the Afterworld game.
2. Natural History Museum Virtual Tours with Sir David Attenborough, 2018
   1. Links
      1. https://news.microsoft.com/en-gb/2018/03/23/sir-david-attenborough-is-giving-behind-the-scenes-tours-of-the-natural-history-museum-as-a-hologram/
   2. Description
      1. Here, Sir David Attenborough (you might recognize his voice from the Planet Earth docuseries) was filmed at Microsoft’s Mixed Reality Capture Studio, and along with some photogrammetry to capture the museum, was digitized as a hologram for a virtual tour application of the Natural History Museum, with users of this app being able to select tours of different parts of the museum and have him sit beside and narrate them throughout the tour experience.
      2. 
      3. 
3. Beat The Blitz, Gatorade, Peyton Manning, Kyle Gracey
   1. https://www.kylegracey.com/beat-the-blitz
   2. https://www.youtube.com/watch?v=Vit4VWyw9NI
   3. Here, the Microsoft capture studio was used to capture Peyton Manning as a game model able to fully moved and shown from any angle within the game, and also capture various movements and motions such as throws, speaking, etc. This model was then integrated into a VR dehydration and football simulator/game, Beat the Blitz.

---

Conclusion

Admittedly, these examples are mainly of huge corporations and studios, and it’s not something the average person can just walk into (though it seems like one can simply contact/make appointments for these various studios), but these are a few examples of the various cameras and software/hardware being used for high-grade volumetric capture, and even being able to be used to large production pipelines.