Volumetric video capture (VVC) technologies are finding applications in multiple sectors that desire realistic and immersive mixed reality content. However, VVC is also emerging as a tool for traditional filmmaking: in theory, the capture technology allows directors to view a scene from every angle and choose how to frame the story all in post-production. To investigate this, we conducted two pilot studies. The first was a set of preliminary interviews with film industry practitioners, who demonstrated a cautious curiosity for the technology’s applications and expanded on its implications for direction and performance. The second study is a comparison exercise where two versions of the same devised scene were captured and developed for viewing: one was filmed via conventional means, the other was filmed with a VVC solution. The process of production and post-production was recorded for both versions, supplemented by observational notes from the actors and an auto-ethnographic account from the author. The final stage of this study was a feedback session with returning interviewees, who reacted positively to the VVC version and reflected on this in light of their previous speculations. Lastly, this article investigates the issue of access and democratisation in relation to VVC and illustrates further pathways for future human-led research for volumetric capture technologies.
AngehrnFWangOAksoyY, et al. (2014) MasterCam FVV: robust registration of multiview sports video to a static high-resolution master camera for free viewpoint video. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 3474–3478. IEEE.
3.
BiXLvTChengZ, et al. (2023) Enhancing health training education experience with volumetric video and wearable mixed reality. In: 2023 IEEE 6th Eurasian Conference on Educational Innovation (ECEI), pp. 131–133. IEEE.
ColletAChuangMSweeneyP, et al. (2015) High-quality streamable free-viewpoint video. ACM Transactions on Graphics34(4): 1–13. https://doi.org/10.1145/2766945
6.
DillVMattos FlachLHocevarR, et al.2012. Evaluation of the uncanny valley in CG characters. In HutchisonDKanadeT, et al. (Eds.). I Virtual Agents. Springer Berlin Heidelberg, (Vol. 7502, pp. 511–513). Series Title: Lecture Notes in Computer Science. https://doi.org/10.1007/978-3-642-33197-8_62
7.
DooleyKEmeryS. 2023. Creating screen stories with game engines: challenges and opportunities for students and researchers working collaboratively across disciplines. Media Practice and Education24, 121–34. https://doi.org/10.1080/25741136.2022.2153002
8.
ElayatYZananiriEWongM-L, et al. (2017) Zero days VR. In: ACM SIGGRAPH 2017 Computer Animation Festival (SIGGRAPH ’17). Association for Computing Machinery, (Vol. 37, pp. 37). https://doi.org/10.1145/3078280.3105235
GuoKLincolnPDavidsonP, et al.2019. The relightables: volumetric performance capture of humans with realistic relighting. ACM Transactions on Graphics38: 217:1–217:19. https://doi.org/10.1145/3355089.3356571
14.
HabermannMHabermannMHabermannM, et al. (2019) LiveCap: real-time human performance capture from monocular video. ACM Transactions on Graphics38(2): 14–17. https://doi.org/10.1145/3311970MAGID:2894878561
15.
HackettMMakledBMizrochE, et al. (2021) Volumetric Video and Mixed Reality for Healthcare Training. Interservice/Industry Training, Simulation, and Education Conference.
HeagertyJLiSLeeE, et al. (2024) HoloCamera: advanced volumetric capture for cinematic-quality VR applications. IEEE Transactions on Visualization and Computer Graphics30(5): 2767–2775. https://doi.org/10.1109/TVCG.2024.3372123
18.
HogueAPorembaCSzkudlarekV, et al. (2022) A visual programming interface for experimenting with volumetric video. In: 2022 IEEE Games, Entertainment, Media Conference (GEM), (pp. 1–6). IEEE.
KachkovskiGVVasilyevDKukM, et al. (2019) Exploring the effects of violating the 180-Degree rule on film viewing preferences. Communication Research46(7): 948–964. https://doi.org/10.1177/0093650219838959
KavakliMCremonaC (2022) The virtual production studio concept – an emerging game changer in filmmaking. In: 2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), (pp. 29–37). IEEE.
23.
KerblBKopanasGLeimkuehlerT, et al. (2023) 3D gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics42(4): 1–14. https://doi.org/10.1145/3592433
24.
KowalskiMNaruniecJDanilukM (2015) Livescan3D: a fast and inexpensive 3D data acquisition system for multiple kinect v2 sensors. In: 2015 International Conference on 3D Vision, (pp. 318–325). IEEE Computer Society.
LeeKYiJLeeY, et al. (2020) GROOT: a real-time streaming system of high-fidelity volumetric videos. In: Proceedings of the 26th Annual International Conference on Mobile Computing and Networking (Mobicom ’20). Association for Computing Machinery, pp. 1–14.
27.
LiuYJinQZhangZ, et al. (2024) HoloClass: enhancing VR classroom with live volumetric video streaming. In: Adjunct Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology (UIST Adjunct ’24). Association for Computing Machinery, pp. 1–3.
28.
LuitenJKopanasGLeibeB, et al. (2023) Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis. https://arxiv.org/abs/2308.09713
29.
McGheeJBourkeCLawtherR, et al. (2024) 3DCrewCap: applying 3D volumetric video capture for XR helicopter rescue crew training and simulation. In: ACM SIGGRAPH 2024 Posters (SIGGRAPH ’24). Association for Computing Machinery, pp. 1–2.
30.
MoriMMacDormanKFKagekiN (2012) The Uncanny Valley [From the Field]. IEEE Robotics and Automation Magazine19(2): 98–100. https://doi.org/10.1109/MRA.2012.2192811
31.
O’DwyerNJohnsonNBatesE, et al. (2017) Virtual play in free-viewpoint video: reinterpreting samuel Beckett for virtual reality. In: 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct), (pp. 262–267). IEEE.
32.
O’DwyerNYoungGWJohnsonN, et al. (2020) Mixed reality and volumetric video in cultural Heritage: expert opinions on augmented and virtual reality. In: Culture and Computing, 8th International Conference, C&C 2020, Held as Part of the 22nd HCI International Conference, HCII 2020, Copenhagen, Denmark, July 19–24, 2020, (pp. 195–214).
33.
PagésRAmplianitisKMonaghanD, et al. (2018) Affordable content creation for free-viewpoint video and VR/AR applications. Journal of Visual Communication and Image Representation53: 192–201. https://doi.org/10.1016/j.jvcir.2018.03.012
PeterW (1980) Cinema and technology: a historical overview. In: HeathS (ed) The Cinematic Apparatus, Teresa de Lauretis. Palgrave Macmillan UK, pp. 14–25.
PietroszekK (2020) Vera – crossing the fourth wall. In: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (CHI EA ’20). Association for Computing Machinery, pp. 1–4.
38.
PietroszekKRebolMLakeB (2022) The meeting: Volumetric participatory theatre play in mixed realityProceedings of the 10th International Conference on Human-Agent Interaction (HAI ’22). Association for Computing Machinery, pp. 330–332.
StarSLGriesemerJR (1989) Institutional ecology, ‘Translations’ and boundary objects: amateurs and professionals in Berkeley’s museum of vertebrate zoology, 1907-39. Social Studies of Science19(3): 387–420. https://doi.org/10.1177/030631289019003001
42.
SterzentsenkoVKarakottasAPapachristouA, et al. (2018) A low-cost, flexible and portable volumetric capturing system. In: 2018 14th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS). Institute of Electrical and Electronics Engineers (IEEE), pp. 200–207.
WangPZhangZWangL, et al. (2024) Vˆ3: Viewing Volumetric Videos on Mobiles via Streamable 2D Dynamic Gaussians. ACM Transactions on Graphics (TOG)43: 1–13. Available at: https://doi.org/10.48550/arXiv.2409.13648
WenttolaV (2017) The Use of Modelling Technology and its Effects on the Music Industry. Tampereen ammattikorkeakoulu. Available at: https://urn.fi/URN:NBN:fi:amk-201705158217.
48.
WuGYiTFangJ, et al. (2024) Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 4D Gaussian Splatting for Real-Time Dynamic Scene Rendering.
YoungGWO’DwyerNMoynihanM, et al. (2022b) Audience experiences of a volumetric virtual reality music video. In: 2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), (pp. 775–781). IEEE.
51.
ZhongJZhangHJiaQ, et al. (2024) Low-bitrate volumetric video streaming with depth image. In:Proceedings of the 2024 SIGCOMM Workshop on Emerging Multimedia Systems (EMS ’24). Association for Computing Machinery, pp. 39–44.
