With the increasing adoption of mixed reality (MR) headsets with video passthrough functionality, concerns over perceptual and social effects have surfaced. Building on prior qualitative findings,1 this study quantitatively investigates the impact of video passthrough on users. Forty participants completed a body transfer task twice, once while wearing a headset in video passthrough and once without a headset. Using video passthrough induced simulator sickness, created social absence (another person in the physical room feels less present), altered self-reported body schema, and distorted distance perception. On the other hand, compared with past research that showed perceptual aftereffects from video passthrough, the current study found none. We discuss the broader implications for the widespread adoption of MR headsets and their impact on theories surrounding presence and body transfer.
Get full access to this article
View all access options for this article.
References
1.
BailensonJN, BeamsB, BrownJ, et al.Seeing the world through digital prisms: Psychological implications of passthrough video usage in mixed reality. Technology, Mind, and Behavior, 2024; 5(2):1–16; doi: 10.1037/tmb0000129
DoughtyM, GhugreNR, WrightGA. Augmenting performance: A systematic review of optical see-through head-mounted displays in surgery. J Imaging, 2022; 8(7):203; doi: 10.3390/jimaging8070203
5.
RollandJP, HollowayRL, FuchsH. Comparison of optical and video see-through, head-mounted displays. In: Telemanipulator and Telepresence Technologies. SPIE; 1995;Vol. 2351, pp. 293–307. 10.1117/12.197322
6.
ZengW, IloC, BowmanD, et al.Virtual loupes: A pilot study on the use of video passthrough augmented reality in plastic surgery. Surg Innov, 2024; 31(5):493–501.
7.
PetrovskiD, TiroshO, McCarthyC, et al.Video See-Through Pipelines for Virtual Reality Headsets and their Impact on Gait. In: 2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). 2023; pp. 1–4.
8.
SchierF, ZeidlerD, ChandranK, et al.ViewR: Architectural-Scale Multi-User mixed reality with mobile head-mounted displays. IEEE Trans Vis Comput Graph, 2023; 30(8):5609–5622.
9.
GuoZ, WangH, DengH, et al.Breaking the Isolation: Exploring the Impact of Passthrough in Shared Spaces on Player Performance and Experience in VR Exergames. IEEE Trans Vis Comput Graph, 2024; 30(5):2580–2590.
10.
KaufeldM, MundtM, ForstS, et al.Optical see-through augmented reality can induce severe motion sickness. Displays, 2022; 74:102283.
11.
KimSY, LeeJH, ParkJH. The effects of visual displacement on simulator sickness in video see-through head-mounted displays. In: Proceedings of the 2014 ACM International Symposium on Wearable Computers. 2014; pp. 79–82.
WienrichC, ObremskiD, IsraelJH. Repeated experience or a virtual nose to reduce simulator sickness?–Investigating prediction of the sensorial conflict theory and the rest-frame hypothesis in two virtual games. Entertainment Computing, 2022; 43:100514.
14.
VovkA, WildF, GuestW, et al.Simulator sickness in augmented reality training using the Microsoft HoloLens. In: Proceedings of the 2018 CHI conference on human factors in computing systems. 2018; pp. 1–9.
15.
ŠkolaF, LiarokapisF. Examining the effect of body ownership in immersive virtual and augmented reality environments. Vis Comput, 2016; 32(6–8):761–770.
16.
NimcharoenC, ZollmannS, CollinsJ, et al.Is that me?—Embodiment and body perception with an augmented reality mirror. In: 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). IEEE; 2018; pp. 158–163.
17.
WaltemateT, GallD, RothD, et al.The impact of avatar personalization and immersion on virtual body ownership, presence, and emotional response. IEEE Trans Vis Comput Graph, 2018; 24(4):1643–1652.
DöllingerN, WolfE, MalD, et al.Virtual reality for mind and body: Does the sense of embodiment towards a virtual body affect physical body awareness?. In: CHI Conference on Human Factors in Computing Systems Extended Abstracts. 2022; pp. 1–8.
20.
RollandJP, BioccaFA, BarlowT, et al.Quantification of adaptation to virtual-eye location in see-thru head-mounted displays. In: Proceedings Virtual Reality Annual International Symposium ‘95. IEEE; 1995; pp. 56–66.
21.
BioccaFA, RollandJP. Virtual eyes can rearrange your body: Adaptation to visual displacement in see-through, head-mounted displays. Presence, 1998; 7(3):262–277.
22.
PfeilK, MasnadiS, BelgaJ, et al.Distance Perception with a Video See-Through Head-Mounted Display. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 2021; pp. 1–9. 10.1145/3411764.3445223
23.
ParkM, SerefoglouS, SchmidtL, et al.Hand-eye coordination using a video see-through augmented reality system. TOERGJ, 2008; 1(1):46–53; doi: 10.2174/1875934300801010046
KamijoK, NishihiraY, HigashiuraT, et al.The interactive effect of exercise intensity and task difficulty on human cognitive processing. Int J Psychophysiol, 2007; 65(2):114–121.
26.
MillerMR, JunH, HerreraF, et al.Social interaction in augmented reality. PLoS One, 2019; 14(5):e0216290.
27.
RiesB, InterranteV, KaedingM, et al.The effect of self-embodiment on distance perception in immersive virtual environments. In: Proceedings of the 2008 ACM symposium on Virtual reality software and technology. 2008; pp. 167–170.
28.
ChenY, FanchiangHD, HowardA. Effectiveness of virtual reality in children with cerebral palsy: A systematic review and meta-analysis of randomized controlled trials. Phys Ther, 2018; 98(1):63–77.
29.
OzdemırM, SahınC, ArcagokS, et al.The effect of augmented reality applications in the learning process: A meta-analysis study. EJER, 2018; 18(74):1–22.
30.
CaoW, YuZ. The impact of augmented reality on student attitudes, motivation, and learning achievements—a meta-analysis (2016–2023). Humanit Soc Sci Commun, 2023; 10(1):1–2.
31.
KennedyRS, LaneNE, BerbaumKS, et al.Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness. The International Journal of Aviation Psychology, 1993; 3(3):203–220.
32.
BouchardS, BerthiaumeM, RobillardG, et al.Arguing in favor of revising the simulator sickness questionnaire factor structure when assessing side effects induced by immersions in virtual reality. Front Psychiatry, 2021; 12:739742.
33.
StanneyKM, KennedyRS, DrexlerJM. Cybersickness is not simulator sickness. In: Proceedings of the Human Factors and Ergonomics Society annual meeting. 1997 vol 41, pp 1138–1142. 10.1177/107118139704100292
34.
PeckTC, Gonzalez-FrancoM. Avatar embodiment. a standardized questionnaire. Front Virtual Real, 2021; 1:575943.
35.
RothD, LatoschikME. Construction of the Virtual Embodiment Questionnaire (VEQ). IEEE Trans Vis Comput Graph, 2020; 26(12):3546–3556; doi: 10.1109/TVCG.2020.3023603
MillerMR, BailensonJN. Social presence outside the augmented reality field of view. Front Virtual Real, 2021; 2:656473.
38.
OhCS, BailensonJN, WelchGF. A systematic review of social presence: Definition, antecedents, and implications. Front Robot AI, 2018; 5:409295.
39.
KhammarA, YarahmadiM, MadadizadehF. What is analysis of covariance (ANCOVA) and how to correctly report its results in medical research? Iran J Public Health, 2018; 49(5):1016–1017.
40.
GuoJ, WengD, ZhangZ, et al.Mixed reality office system based on maslow’s hierarchy of needs: Towards the long-term immersion in virtual environments. In: 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE; 2019; pp. 224–235.
41.
WolfE, MalD, FrohnapfelV, et al.Plausibility and perception of personalized virtual humans between virtual and augmented reality. In: 2022 IEEE international symposium on mixed and augmented reality (ISMAR). IEEE; 2022; pp. 489–498.
42.
WolfE, DöllingerN, MalD, et al.Body weight perception of females using photorealistic avatars in virtual and augmented reality. In: 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE; 2020; pp. 462–473.
43.
SchroederPA, GehrerNA, ReentsM, et al.Body dissatisfaction directs avatar perception: Embodiment and selective visual attention in body mass-modified self-avatars. Cyberpsychol Behav Soc Netw, 2023; 26(11):850–860.
44.
WangJ, ShiR, ZhengW, et al.Effect of frame rate on user experience, performance, and simulator sickness in virtual reality. IEEE Trans Vis Comput Graph, 2023;PP(5):2478–2488.
45.
Ihemedu-SteinkeQC, ErbachR, HaladyP, et al.Virtual reality driving simulator based on head-mounted displays. Automotive User Interfaces: Creating Interactive Experiences in the Car, 2017:401–428.
46.
IbánezML, PeinadoF, PalmieriO. Walking in VR: Measuring presence and simulator sickness in first-person virtual reality games. In: Proceedings of the Third Congress of the Spanish Society for Video Games Sciences. 2016.
47.
BrownP, PowellW. Pre-exposure cybersickness assessment within a chronic pain population in virtual reality. Front Virtual Real, 2021; 2:672245.
48.
GarrettB, TavernerT, GromalaD, et al.Virtual reality clinical research: Promises and challenges. JMIR Serious Games, 2018; 6(4):e10839.
49.
BosJE, de VriesSC, van EmmerikML, et al.The effect of internal and external fields of view on visually induced motion sickness. Appl Ergon, 2010; 41(4):516–521.
Nguyen-VoT, RieckeBE, StuerzlingerW. Simulated reference frame: A cost-effective solution to improve spatial orientation in VR. In: 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE; 2018; pp. 415–422.
52.
AkizukiH, UnoA, AraiK, et al.Effects of immersion in virtual reality on postural control. Neurosci Lett, 2005; 379(1):23–26.
53.
DraperMH, ViireES, FurnessTA, et al.Effects of image scale and system time delay on simulator sickness within head-coupled virtual environments. Hum Factors, 2001; 43(1):129–146.
54.
KeshavarzB. Exploring behavioral methods to reduce visually induced motion sickness in virtual environments. In: Virtual, Augmented and Mixed Reality: 8th International Conference, VAMR 2016, Held as Part of HCI International 2016, Toronto, Canada, July 17-22, 2016. Proceedings 8. Springer International Publishing; 2016; pp. 147–155.
55.
AdamsH, StefanucciJ, Creem-RegehrS, et al.Depth Perception in Augmented Reality: The Effects of Display, Shadow, and Position. In: 2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 2022; pp. 792–801. 10.1109/VR51125.2022.00101
56.
RennerRS, VelichkovskyBM, HelmertJR. The perception of egocentric distances in virtual environments-a review. ACM Comput Surv, 2013; 46(2):1–40.
57.
WitmerBG, KlinePB. Judging perceived and traversed distance in virtual environments. Presence, 1998; 7(2):144–167.
58.
BuckLE, YoungMK, BodenheimerB. A comparison of distance estimation in HMD-based virtual environments with different HMD-based conditions. ACM Trans Appl Percept, 2018; 15(3):1–15.
59.
VienneC, MasfrandS, BourdinC, et al.Depth perception in virtual reality systems: Effect of screen distance, environment richness and display factors. IEEE Access, 2020; 8:29099–29110.
60.
LeyrerM, LinkenaugerSA, BülthoffHH, et al.Eye height manipulations: A possible solution to reduce underestimation of egocentric distances in head-mounted displays. ACM Trans Appl Percept, 2015; 12(1):1–23.
61.
WilkinsonM, BrantleyS, FengJ. A mini review of presence and immersion in virtual reality. In: Proceedings of the human factors and ergonomics society annual meeting. SAGE Publications: Los Angeles, CA. 2021 Vol. 65, pp. 1099–1103.
62.
BerkmanMI, AkanE. Presence and immersion in virtual reality. In: Encyclopedia of Computer Graphics and Games. Springer International Publishing: Cham; 2024; pp. 1461–1470.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
