Sage Journals: Discover world-class research

Abstract

This case study focuses on quantifying the impact of saccadic eye movements in VR/AR environments, which differ from natural conditions and contribute to visual discomfort. We develop a predictive model based on VR/AR usage characteristics and suggest ergonomic improvements for these systems. Our methodology involves constructing a mathematical model to analyze saccadic phenomena and their relationship with visual strain, aiming to mimic natural eye movements to enhance comfort and promote wider adoption of VR/AR technologies. Key research questions include: (a) How do saccadic movements in VR/AR differ from natural conditions and affect visual discomfort? (b) Can we predict visual discomfort from these movements using VR/AR characteristics? And (c) What design changes can reduce discomfort in VR/AR headsets? We also stress the need for standardized metrics in testing VR and AR devices to ensure safety and comfort, advocating for industry-wide benchmarks to prevent health issues and enhance user experience.

Keywords

virtual reality augmented reality eye tracking headsets ergonomics

Objectives

The advancement of virtual and augmented Reality (VR/AR) technologies offers immersive experiences that enhance user interaction. Eye tracking is a crucial element for user interaction in VR/AR, yet it presents ergonomic and functional challenges affecting comfort and satisfaction. As VR/AR devices are increasingly used for longer durations, considering the needs of users with visual impairments such as astigmatism, myopia, or hyperopia becomes critical (Ivaniuk & Kalinina, 2021; Kellnhofer et al., 2020). Among these challenges, saccadic drift strain emerges as a significant ergonomic issue for VR/AR devices, impairing the overall immersive experience by inducing excessive visual fatigue and discomfort.

Saccadic movements, fundamental to human visual perception, involve periodic jumps of the eyes from one point of interest to another, occurring so swiftly in natural conditions that they appear smooth (Bahill & Stark, 1979; Braun et al., 2017; Hutton 2008). In VR/AR environments, however, these movements can significantly deviate from normal conditions, thereby increasing visual fatigue. The discrepancy between physical and virtual distances, coupled with the need to process an abundance of information, further exacerbates fatigue (Selvan et al., 2023). While previous research has explored the vergence-accommodation conflict (Hoffman et al., 2008; Zhou et al., 2021), the effect of saccades in VR/AR on user comfort remains insufficiently explored. Although some solutions have dynamically adjusted elements for enhanced comfort (Osman & Roberts, 2020), the exploration of biological parameters has not been as thorough.

This research aims to quantify the impact of saccadic movements, predict key parameters of VR/AR system use, and suggest ergonomic design improvements for VR/AR systems. Guiding research questions include:

How do saccadic eye movements during VR/AR use differ from those in natural conditions, and what are the implications for visual discomfort?

Can a model predict visual discomfort from saccadic movements based on VR/AR usage characteristics?

What design modifications can alleviate discomfort in VR/AR headsets?

The methodological framework involves constructing a mathematical model to examine the saccadic phenomenon and its influence on visual strain. This process entails defining the research problem, identifying parameters affecting visual strain (Liversedge et al., 2011), formulating hypotheses based on literature and theoretical aspects of saccades and eye strain, and correlating biological parameter measurements to mathematical functions rooted in physiological approaches to visually related problems (Fomins et al., 2023; Yarbus, 1967).

A mathematical model incorporating expressions for physiological variables will delineate the relationship between saccadic eye movements and visual strain. Expected outcomes include a predictive model capable of forecasting the impact of saccades on visual strain, identifying factors contributing to strain (Fan et al., 2023; Reding & Berek, 1983), and developing ergonomic design recommendations to mimic natural eye movement, reduce ergonomic impact, enhance user comfort, and ultimately facilitate broader adoption of VR/AR technologies (Kruchinina & Yakushev, 2018).

The necessity for standardized metrics in testing VR and AR headsets lies in establishing industry-wide benchmarks for internal testing, ensuring products meet essential safety, and comfort standards before market launch. Without a universal standard, companies lack a reference point, leading to avoidable issues such as headaches, eye strain, or seizures in users (Won & Kim, 2021). Standardization not only aims to prevent such outcomes but also allows for optimization beyond the baseline, enhancing user experience. The absence of such standards risks public health, turning the sector into a “Wild West” without safety thresholds (Ferguson, 2020; X Reality Safety Intelligence [XRSI], n.d.). Therefore, a unified testing framework is crucial to predict and mitigate risks, and provide warnings for atypical cases, ensuring consumer safety, and industry accountability (Hayes et al., 2023).

Approach

Formulation of a mathematical model to understand and mitigate saccadic strain in users of VR/AR headsets starts with defining the purposes and success criteria of the model, such as minimizing visual strain in users and improving the ergonomic comfort of users over an extended period. This ensures that the model is geared towards specific users and use-cases in VR/AR environments; the next step is to define all variables that are relevant to these purposes. These variables can be independent (e.g., how frequently and how large are users’ saccades?) and dependent (e.g., how much visual strain do users experience?). Hypotheses regarding relationships between these variables are then developed.

Each of these hypothesized relationships are translated into mathematical form: we select appropriate mathematical functions (such as linear, quadratic, logarithmic, etc.) that most accurately reflect the real-world behavior of the variables. All these functions are included within a unified mathematical model for the interactions between the variables that then explains how all the variables together affect visual strain (see the Appendix).

This review covers the modeling process, from the statement of an objective to detailing the model. It does not cover essential phases such as model verification and refinement, which involves conducting empirical tests and subsequent iterations to make the model more accurate and applicable to a variety of contexts.

Findings

Parameters for predicting VR/AR headset strain from user saccadic movements include saccade frequency, saccade amplitude, interpupillary distance (IPD), viewing angle, distance to content, and headset use duration. These elements build an achievable predictive model for headset strain expected from future use. Such a model could be used by a designer to fine-tune adjustable VR/AR settings. These are our key predictive model parameters:

Saccade Frequency: Measures how often the gaze shifts.

Saccade Amplitude: Reflects the gaze shift distance.

IPD: Affects 3D content perception.

Viewing Angle: Influences ergonomic comfort (Zhang et al., 2023).

Content Distance: Impacts focus and potential strain (Hirzle et al., 2019).

Usage Duration: Key in visual fatigue development.

The following predictive model aspires to integrate saccadic dynamics with anatomical factors and VR/AR environmental aspects to create a consistent scheme for analytically quantifying user strain for informed iterative design. Research by Kruchinina and Yakushev (2018) on assessing saccadic eye movements with VR/AR head-mounted display technology provides essential insights into the biomechanical behaviors that are relevant to our understanding of user strain and comfort.

S = β_{1} f + β_{2} a_{2} + β_{3} \log (IPD) + β_{4} c o s (θ) + β_{5} d + β_{6} t^{2} + ϵ

(1)

Here, S stands for predicted strain, with β₁ to β₆ being the effects of the different parameters on strain, and ϵ representing error or variability. The model combines linear and non-linear terms to describe how each parameter independently influences strain. It is common to use quadratic terms for amplitude and duration. Increasing one of these strain variables will exacerbate the strain. Following the principles of psychophysical scaling, our model incorporates logarithmic terms for IPD to accurately reflect the diminishing perceptual impact of changes at higher IPD values (Meilgaard et al., 2016). The effect of viewing angle also follows a non-linear relation because it oscillates (cosine). It is possible to use this relation to determine what changes can be made to lessen strain.

Takeaways

We introduce a mathematical framework to the domain of human factors and ergonomics, focusing on the ergonomic challenges induced by saccadic movements within VR/AR settings. By placing ergonomic recommendations on a quantitative basis, this study addresses a major void in the empirical knowledge related to the interplay between rapid eye movements and visual strain when using VR/AR. The application of this quantifiable framework makes it possible to create user interfaces and headset features in line with users’ physiological limits and capacities (Hossfeld et al., 2023). These mathematical foundations open up the possibility for defining predictive models that influence the way VR/AR technologies are designed.

They also provide mathematical inclusion thresholds that can predict where and how to influence good design in order to effectively and directly address the needs and constraints of a diverse user population (Stanney et al., 2020). This could enhance VR adoption by fostering deeper immersion through thoughtful and intentional design by human factors professionals.

Furthermore, this study translates insights from psychology and perceptual literature into a mathematical framework, instigating interdisciplinary efforts that draw from human factors engineering, mathematics, and computer science to address the complex science of VR/AR technology design ergonomics. With an interdisciplinary approach, we accelerate the innovation and technology development in this area. We underscore that using mathematics-driven frameworks for technological innovation significantly enriches the use of ergonomics principles.

Footnotes

Appendix: Mathematical and Optimization Models for VR/AR Ergonomics

This additional content introduces several key mathematical relationships and optimization equations that address critical aspects of user comfort, visual strain, and ergonomic design in VR/AR systems.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Bahill

A. T.

Stark

(1979). The trajectories of saccadic eye movements. Scientific American, 240(1), 108–117.

Braun

D. I.

Schütz

A. C.

Gegenfurtner

K. R.

(2017). Visual sensitivity for luminance and chromatic stimuli during the execution of smooth pursuit and saccadic eye movements. Vision Research, 136, 57–69. https://doi.org/10.1016/j.visres.2017.05.008

Fan

Wang

Song

Dong

Bao

Wang

(2023). Eye movement characteristics and visual fatigue assessment of virtual reality games with different interaction modes. Frontiers in Neuroscience, 17, 1173127. https://doi.org/10.3389/fnins.2023.1173127

Ferguson

(2020). Virtual reality and safety training. Safety+Health Magazine. safetyandhealth-magazine.com

Fomins

Ikaunieks

Krumina

(2023). The effect of stimulus contrast and spatial position on saccadic eye movement parameters. Vision, 7(4), 68. https://doi.org/10.3390/vision7040068

Hayes

A. T.

Bradshaw-Ward

Howard

McAdams

Cockerham

(2023). Engaging novice VR users to VR for education: Enhancing user comfort, health and safety in VR and AR. 2023 IEEE conference on virtual reality and 3D user interfaces abstracts and workshops (VRW) (pp. 530–531). https://doi.org/10.1109/VRW58643.2023.00116

Hoffman

D. M.

Girshick

A. R.

Akeley

Banks

M. S.

(2008). Vergence–accommodation conflicts hinder visual performance and cause visual fatigue. Journal of Vision, 8(3), 33. https://doi.org/10.1167/8.3.33

Hossfeld

Seufert

Loh

Wunderer

Davies

(2023). Industrial user experience index vs. quality of experience models. IEEE Communications Magazine, 61, 98–104. https://doi.org/10.1109/MCOM.001.2200044

Hirzle

Gugenheimer

Geiselhart

Bulling

Rukzio

(2019). A design space for Gaze interaction on head-mounted displays. Proceedings of the 2019 CHI conference on human factors in computing systems. http://dl.acm.org/citation.cfm?id=3300855

10.

Hutton

S. B

. (2008). Cognitive control of saccadic eye movements. Brain and Cognition, 68(3), 327–340.

11.

Ivaniuk

Kalinina

(2021). Prescription customized augmented reality display. AR, VR, MR. SPIE. https://doi.org/10.1117/12.2588768

12.

Kellnhofer

Butler

D. J.

Wetzstein

(2020). Characterizing visual acuity in the use of head mounted displays. Virtual Reality, 24(4), 451–463. https://doi.org/10.1007/s10055-020-00451-0

13.

Kruchinina

A. P.

Yakushev

A. G.

(2018). A mathematical model of optimal saccadic eye movement by a pair of muscles. Biophysics, 63, 241–247. https://doi.org/10.1134/S0006350918020161

14.

Liversedge

Gilchrist

Everling

(Eds.). (2011). The Oxford handbook of eye movements. Oxford University Press.

15.

Meilgaard

M. C.

Carr

B. T.

Civille

G. V.

(2016). Sensory evaluation techniques (5th ed.). CRC Press.

16.

Osman

Roberts

(2020). U.S. patent no. 10,712,900. U.S. Patent and Trademark Office.

17.

Reding

G. R.

Berek

(1983). Effect of saccadic and pursuit eye movements on spatial and temporal resolution. Neurology, 33(7), 858. https://doi.org/10.1212/WNL.33.7.858

18.

Selvan

Mina

Abdelmeguid

Gulsha

Vincent

Sarhan

(2023). Virtual reality headsets for perimetry testing: A systematic review. Eye, 38, 1041–1064. https://doi.org/10.1038/s41433-023-02843-y

19.

Stanney

K. M.

Fidopiastis

C. M.

Foster

(2020). Virtual reality is sexist: But it does not have to be. Frontiers in Robotics and AI, 7, 4. https://www.ncbi.nlm.nih.gov/pubmed/33501173

20.

Won

J. H.

Kim

Y. S.

(2021). A study on visually induced VR reduction method for virtual reality sickness. Applied Sciences, 11(14), 6339. https://doi.org/10.3390/app11146339

21.

X Reality Safety Intelligence (XRSI). (n.d.). An imperative: Developing standards for safety and security in XR environments. XRSI. XRSI.org

22.

Yarbus

A. L.

(1967). Eye movements and vision. Plenum Press.

23.

Zhang

Chen

Sun

(2023). Toward optimized VR/AR ergonomics: Modeling and predicting user neck muscle contraction. ACM SIGGRAPH 2023 conference proceedings. http://dl.acm.org/citation.cfm?id=3591495

24.

Zhou