Optical flow visual stimuli induce body sway and visually-induced motion sickness in virtual reality system: A preliminary study

Abstract

Objectives

Visually-induced motion sickness (VIMS) remains an unsolved issue when using head-mounted displays (HMD) in immersive virtual reality (VR) systems. We investigate the influence of restricted visual field on sensorimotor incongruence of postural control in an immersive VR environment, as a surrogate marker of VIMS while standing steadily and viewing ambiguous optical-flow visual stimuli.

Methods

Twenty-seven healthy participants wore HMD and viewed optical flow visual stimuli (called the plaid motion pattern) using a VR system. Visual stimuli were presented on a full-screen display, either only in the center or periphery of the visual field, or nowhere (Natural). To evaluate standing posture stability, we measured spatial (length and area of the center of the pressure (COP)) and temporal dynamics of body sway using a stabilometer. Subjective feelings of VIMS were assessed for each visual stimulus condition.

Results

The full-screen condition significantly worsened the COP measurements and feelings. The COP areas of standing balance under the center of the visual field condition were significantly smaller than those under the full-screen condition; however, the periphery of the visual field condition was comparable to the full-screen condition. The condition effects for the conditions (natural, center of the visual field, periphery of the visual field and full-screen) were observed in the COP measurements and subjective feelings.

Conclusion

Optical-flow visual stimuli can induce body sway, suggesting sensorimotor incongruence in postural control. Ancillary findings demonstrated that subsequent to full-screen presentation, the periphery of the visual field potentially contributes to spatial and temporal dynamics of body sway and the sensorimotor incongruence rather than the central area.

Introduction

In recent years, virtual reality (VR) systems have been developed to enable people to enjoy realistic and immersive experiences with metaverses, such as videos, games, concerts and plays. In addition to the clinical applications of VR-based neurorehabilitation in post-stroke paralysis and phantom limb pain,^1,2 in the medical education fields, VR is emerging as a new method of delivering medical simulation sessions that require relatively low costs and provide haptic feedback while promoting greater accessibility and flexibility. Furthermore, VR can be used to provide an immersive simulation of both technical and non-technical skills in surgical education. ³ Since VR can have additive effects in combination with physical stimulators, it has been integrated into the medical curriculum in some educational hospitals. However, viewing in VR environments can result in discomfort and undesirable effects such as dizziness, nausea, vomiting and headache. This phenomenon is called visually-induced motion sickness (VIMS), which is similar to motion sickness when traveling by car, ship or airplane.

Notably, VIMS remains an unresolved complication of VR use and requires further study to determine its cause and means of mitigation. Moreover, it is particularly noticeable in immersive VR environments that use head-mounted displays. ⁴ Postural control is achieved by unconsciously-integrating visual, vestibular and proprioceptive sensory information and producing motor outputs. Following viewing of visual images in an immersive VR environment, changes in one's ability to control posture and balance become prominent, preceding the onset of VIMS symptoms.^5,6 A mismatch between the information delivered by these sensory systems can occur in an immersive VR environment; this mismatch is not in accordance with the internal model that is prepared and predicted for movements, eventually causing VIMS. ⁷

To reduce the undesirable effects of VIMS during visually-interactive tasks using head-mounted displays, the critical role of the entire visual image motion against the background, namely optical flow, has been discussed. ⁸ Several lines of restriction techniques of the user's visual field have been investigated to reduce optical flow and the amount of visual information available to users, expecting to mitigate VIMS. However, consistent results have not yet been obtained. Some techniques could successfully mitigate VIMS but others could not. ⁹ Inconsistent results, where the location of the visual field should be restricted (the center or periphery of the visual field), were also observed. Further, adaptation to optical flow is another promising strategy to mitigate VIMS, and some successes were practically reported. ¹⁰ However, adaptation to optical flow seems to be largely dependent on the user and the VR environments.

We aimed to disentangle the theoretical considerations of VIMS and investigate the influence of restricted visual field on postural instability and sensorimotor incongruence of postural control in an immersive VR environment.

Method

This study was a controlled study, conducted at outpatient clinic of Department of Pain and Palliative Medicine, The University of Tokyo Hospital, and Kadinche Corporation during the periods of March 2023 to November 2023.

Participants

The study protocol was approved by the institutional review board (Ethics committee of Graduate school and Faculty of Medicine, The University of Tokyo; Approval ID 10263).

Participants were recruited by opportunity sampling, subject to key inclusion criteria of being neurologically healthy without any neurological distress and who are aged 20 through 80 years. Exclusion criteria were; participants have any neurological signs and symptoms (e.g., cognitive decline, hearing loss, eye nystagmus, dizziness or postural instability); participants can not understand Japanese without any assistance; if consent to this research is not obtained; and other cases that researchers determine inappropriate such as the participant being unable to express intentions accurately. Twenty-seven participants (11 females; age: 20s, 5; 30s, 13; 40s, 7; and 50s, 2) gave their written informed consent to participate in this study. The study was conducted in a small, commonly used council chamber.

Experimental conditions of visual stimuli

The participants wore a head-mounted display (HMD, resolution: 1832 × 1920 per eye; field-of-view: horizontal 106°/vertical 96°; 90-Hz refresh) through which they first viewed a real-world image of the experimental chamber, which was live footage from a digital camera placed in front of the HMD (natural condition, Figure 1(a)). The experimental presentations of the optical flow visual stimuli were of the following three types. The first involved viewing stimuli occupying the full screen (FS condition, Figure 1(b)). Participants perceived the visual stimuli as completely filling their visual field, approximately 3.5 m in front of them. The second involved viewing stimuli on a circular screen located at the center of the visual field (CVF condition, Figure 1(c)). This circular screen had a radius of approximately 45 visual degrees, corresponding to the inner limit of the extreme peripheral visual field. ¹¹ This circular screen always located at the CVF by anchoring the head movements, indicating the retinal eccentricity. Outside the boundary of this circular screen, participants viewed live footage of the chamber. The third involved viewing stimuli presented on the extreme periphery and far visual screen, corresponding to the area outside the circular screen (PVF condition, Figure 1(d)). This condition completely inverted the CVF condition: visual stimuli were always presented outside the circular boundary, with live footage at the center of the visual field. All participants were exposed to all the study conditions (Natural, FS, CVF and PVF).

Figure 1.

Examples of optical flow visual stimuli (plaid motion pattern) by the virtual reality system. (a) Real-world footage of the experiment chamber (Natural condition); (b) Full-screen presentation (FS condition); (c) Center of the visual field (CVF condition) and (d) Peripheral of the visual field (PVF condition).

We used a plaid motion pattern, ¹² which is a form of perceptual rivalry, as the experimental visual stimulus. A sine-wave plane was created by superimposing two drifting sinusoidal gratings. The two gratings had the same spatial frequency (about 0.3 cycle/degree), duty cycle and speed, and the gratings were therefore completely symmetric. The plaid pattern contained one component moving downward and the other moving leftward. We used two sets of drifting gratings to produce quasi-regular but non-uniform perceptual alternations between a coherent moving plaid of diamond-shaped intersections and two sets of component ‘sliding’ grating. Because the two gratings appeared to slide across one another in a quasi-regular but non-uniform frequency, the directions of the optical flow changed without visual adaptation to the ambiguous plaid motion stimuli.

The respective visual conditions were successively presented for 40s with 5-s intervals and black presentation, and the order of the experimental conditions was randomized.

Measurements

We measured the balance and postural control of the subjects under their respective visual conditions using a stabilometer. The participants were asked to maintain a standing posture on the stabilometer comfortably but as steadily as possible. At that time, the participants did not have any constraints on moving their eyes, head and neck, trunk, or limbs. Using the stabilometer, we measured the center of pressure (COP) of the body projected vertically onto the floor below as a component of standing balance. COP is an important metric for balance stability assessment given that it closely approximates body sway. The validity and reliability of the stabilometer for assessing standing balance were confirmed at a sampling rate of (∼30–50 Hz). ¹³ When standing on the stabilometer, the stabilometer measured body sway in the ‘side to side (mediolateral)’ and ‘front to back (anteroposterior)’ directions based on downward force sensor data generated at each corner of the stabilometer. This body sway information, along with the total downward force (i.e., vertical ground reaction force), was transmitted with minimal time lag via wireless (i.e., Bluetooth) technology. The stabilometer wireless signals were obtained using laptops through freely available stabilometer software applications (https://www.adinstruments.com/support/downloads/windows/wii-balance-board-0). The total lengths of the COP traces and the areas of COP movement in the anteroposterior and mediolateral directions were compared among the experimental conditions. Further, we measured the COP fluctuations, to demonstrate the regularity in temporal postural control dynamics, by calculating sample entropy in the anteroposterior and mediolateral directions. ¹⁴ Sample entropy indexes the regularity of a given time-series, and is used to analyze complex stochastic systems that include both deterministic and random processes. ¹⁵ Furthermore, we used the power spectral density, which was obtained by calculating the COP displacement with a fast Fourier transform using the periodogram method. The power spectrum was analyzed in three frequency bands (0–0.3 Hz, 0.3–1.0 HZ and 1–3 Hz) in the anteroposterior and mediolateral directions at these low, medium and high frequencies. ¹⁶ These were also compared among the experimental conditions.

The participants were asked to recall and provide their subjective feelings regarding VIMS for the respective experimental conditions after completing the COP measurement. We obtained an 11-point numerical rating scale of VIMS, with 0 = ‘the participant does not feel any discomfort’ and 10 = ‘the participant feels strong discomfort and is going to throw up’, and the total sickness score by using the simulator sickness questionnaire (SSQ). ¹⁷ According to the original study, the oculomotor (i.e., eyestrain, difficulty focusing, blurred vision, headache), disorientation (i.e., dizziness, vertigo) and nausea (nausea, stomach awareness, increased salivation, burping) subscales were weighted. The total score of SSQ was obtained by summing all the weighted subscale scores and applying the conversion formula.

Statistical analysis

We used a within-participants design, and compared data among the respective conditions using the Kruskal-Wallis test, followed by the Bonferroni test as a post-hoc analysis, if necessary. Based on the classical notion that the sensitivity of motion detection of visual stimuli is higher in the peripheral visual field than in the center visual field, ¹⁸ we investigated the effect of the conditions in order (Natural < CVF < PVF < FS) using the Spearman rank correlation test. Additionally, we analyzed the respective data change over time across the sequence of trials to evaluate the trial effects. In general, female subjects are more vulnerable to VIMS. We performed the Pearson correlation test to analyze the relationships between subjective motion sickness (the total score of SSQ) and respective objective measurements. We analyzed sex differences of all measurements and symptom items by the Mann-Whitney test. Statistical significance was set at p < 0.05.

Results

Measurements of postural control

The total lengths of the COP traces were significantly different among the four conditions (K–W test, p = 0.028; Figure 2). Specifically, under the FS condition, the traces were significantly longer than those under the natural condition (Bonferroni test, p = 0.045); however, there were no other differences. The condition effect of the total length of COP traces was significant (p = 0.014). The areas of COP movement were also significantly different among the four conditions (K–W test, p = 0.004; Figure 2). Importantly, the FS condition demonstrated larger areas than the natural (Bonferroni test, p = 0.010) and CVF (Bonferroni test, p = 0.015) conditions, but not the PVF (Bonferroni test, p = 0.057). The areas in the CVF and PVF conditions were comparable; however, the condition effect in order of the COP movement area was also significant (p = 0.002). Both length and area data of the COP did not demonstrate any changes over time across the sequence of trials (length, p = 0.94; area, p = 0.57). The sex difference was not observed in length of the COP under each conditions. The sex difference of the area of the COP was observed in the FS condition. Female subjects demonstrated significantly smaller areas of the COP (3.71 ± 3.75) than those of male subjects (7.40 ± 4.72). Under other conditions, differences of the area of the COP did not reach significance between male and female subjects.

Figure 2.

Length and areas of the center of pressure movements during visual stimuli conditions. Bars and error bars indicate mean ± standard deviation. *p < 0.05 (Bonferroni test).

The sample entropies in the anteroposterior direction were different among the four conditions (K–W test, p = 0.049, Figure 3). This sample entropy of the natural condition was significantly smaller than those of the CVF (Bonferroni test, p = 0.022) and FS conditions (Bonferroni test, p = 0.020), respectively. There was no difference of this sample entropy between natural and PVF conditions (p = 0.404), and the condition effect was not observed in the anteroposterior sample entropy (p = 0.080). On the other hand, there were no differences of the sample entropy in the mediolateral direction, among the four conditions (K–W test, p = 0.93, Figure 3). The condition effect of the mediolateral sample entropy was not observed (p = 0.87). Any changes over time across the sequence of trials were not observed in both sample entropies of the anteroposterior direction (p = 0.22) and the mediolateral direction (p = 0.051). There were no sex differences in both anteroposterior and mediolateral sample entropies among each condition.

Figure 3.

Anteroposterior and mediolateral sample entropies of the center of pressure movements during visual stimuli conditions. Bars and error bars indicate mean ± standard deviation. *p < 0.05 (Bonferroni test).

The power spectrum density analyses revealed that there were significant differences among experimental conditions, except for low frequency of the anteroposterior direction (K–W test, p = 0.47). The FS condition of each frequency demonstrated higher density in these significant frequency densities (Figure 4), but the high frequency density of the anteroposterior direction (p = 0.758) and the low frequency density of the mediolateral direction (p = 0.063) did not reach significance between the FS and the PVF conditions. The condition effects in order of the medium frequency of anteroposterior direction and the three frequencies of mediolateral direction were significant (p < 0.05, respectively). Any changes over time across the sequence of trials were not observed in all frequencies of both directions. There were no sex differences in both anteroposterior and mediolateral sample entropies among each condition.

Figure 4.

The power spectrum density of the center of pressure movements during visual stimuli conditions. The power spectrum density is set as low (0–0.3 Hz), medium (0.3–1.0 Hz) and high (1–3 Hz) frequency bands in the anteroposterior (AP) and mediolateral (ML) directions. Bars and error bars indicate mean ± standard deviation. *p < 0.05, **p < 0.001, ***p < 0.0001 (Bonferroni test).

Subjective feelings of VIMS

Regarding subjective feelings of VIMS, none of the participants reported any VIMS under the natural conditions. An 11-point numerical rating scale of VIMS in the FS condition was significantly higher than that in the CVF (Bonferroni test, p < 0.001) and PVF (Bonferroni test, p = 0.001) conditions (K–W test, p < 0.001). Additional analyses of VIMS symptoms revealed no significant differences in VIMS ratings between the CVF and PVF conditions (Bonferroni test, p = 0.258); however, the condition effect was significant among the three experimental conditions in order (p < 0.001). There were no significant data changes of respective conditions over time across the sequence of trials to evaluate the trial effects. The sex difference of the numerical rating scale of the VIMS were not observed (Mann-Whitney test, p = 0.34).

Subscales and the total score of SSQ followed the numerical rating scale of VIMS. There were significant differences among the three experimental conditions (Figure 5), and the FS condition showed significantly higher scores than the CVF and PVF conditions for the three subscales and the total score. However, all the SSQ scores of the CVF were not different from those of the PVF (Figure 5). The total score of SSQ was linearly correlated with most objective measurements in this study; however, both sample entropies in the anteroposterior direction (p = 0.056) and the mediolateral direction (p = 0.062) and the low frequency density of the anteroposterior direction (p = 0.212) did not correlated with the total SSQ score (Table 1). The trial sequence effect of respective SSQ scores was not observed. Further, any sex differences were not observed in three subscales and the total score of SSQ.

Figure 5.

Subjective feelings of visually-induced motion sickness during visual stimuli conditions. Of the simulator sickness questionnaires, the total score, nausea score, oculomotor score and disorientation score are presented. Bars and error bars indicate mean ± standard deviation. *p < 0.0001, **p < 0.001 (Bonferroni test).

Table 1.

Correlation of measured variables and the total score of simulator sickness questionnaire.

Conditions			Correlation coefficient	p-value a
The center of pressure	Length of traces		0.283	0.010
	Area of movements		0.396	0.0002
Sample entropy	Anteroposterior		0.213	0.056
	Mediolateral		−0.208	0.062
Power spectrum density	Anteroposterior	Low	0.144	0.212
		Medium	0.462	<0.0001
		High	0.382	0.0005
	Mediolateral	Low	0.245	0.0312
		Medium	0.388	0.004
		High	0.444	<0.0001

^aPearson correlation test.

Discussion

To elucidate the underlying mechanism of VIMS, we measured spatial dynamics (length and area of the COP trajectories) and temporal dynamics (anteroposterior and mediolateral sample entropies and the power spectrum densities of low, medium and high frequency in anteroposterior and mediolateral directions) of body sway as a surrogate marker of VIMS while standing steadily and viewing ambiguous optical-flow visual stimuli. The most robust finding of the present study was that full-screen presentation of the visual stimuli basically caused significant body sway and a subjective feeling of discomfort. Differences in both body sway dynamics and subjective feelings were not significantly different between the visual presentations at the center and periphery of the visual fields. However, ancillary findings demonstrated that visual stimuli in the center of the visual field induce significantly smaller areas of spatial dynamics of body sway than that of the full-screen presentation and that visual stimuli in the periphery did not induce the body sway, compared to those in the full-screen presentation. We obtained another ancillary finding that presentations of the full-screen, only the periphery, and only the center worsened both spatial dynamics of body sway in sequence. As well, such condition-effect was observed with discomfort.

Different from spatial dynamics of body sway, the temporal dynamics did not demonstrate consistent results according to experimental conditions. Healthy physiological systems are often characterized by an irregular and complex type of fluctuation. Sample entropy quantitatively indexes regularity of time-series of the system. For example, cerebral stroke patients demonstrated the trajectories of the COP more regularly (as indexed by reduced sample entropy) than healthy elderly, indicating that a decrease in sample entropy (i.e., more regular body sway fluctuations) is interpreted as a decrease in the effectiveness of postural control. ¹⁴ More irregular COP fluctuations (as indexed by an increase in sample entropy) is interpreted as an increase in the efficacy of postural control. In the present study, the healthy participants demonstrated increase or no change of sample entropy during viewing the plaid motion pattern, of which the directions of the optical flow of the two gratings quasi-regularly but non-uniformly change without visual adaptation. We aimed to use the plaid motion pattern as experiment visual stimuli because of this nature. Observed increase or no change of sample entropy were expected results, indicating least visual adaptation. Visual information provides advanced knowledge of potentially destabilizing situations and assists in orienting toward the environment to determine the visual vertical and positional awareness between the body and the surrounding environment. ¹⁹ Optical-flow visual stimuli can lead to postural instability and sensorimotor incongruence during postural control. Although of maintenance or increase in the efficacy of postural control (as indexed by no change or an increase in sample entropy) without visual adaptation, more irregular COP fluctuations were observed during viewing the optical-flow visual stimuli. In line with the sample entropies, another temporal dynamics (namely, the power spectrum density) followed maintenance or increase in the efficacy of postural control. Lower and higher components of COP fluctuations indicate explanatory and reactive posture controls respectively. ²⁰ During uncertain sensory inputs which potentially destabilize body posture, the body stiffness control strategy can be adopted more to reduce body sway, which produce more power in higher frequencies. ²¹ In this study using healthy participants, lower components of COP fluctuations in the anteroposterior direction did not differ among visual conditions; however, optical-flow visual stimuli more elicited higher components, indicating postural instability and sensorimotor incongruence during postural control. By one definition, motion sickness is postulated to be caused by alterations of postural control. ²² Postural instability is suggested to occur in novel environments when failing to perceive the new dynamics or being unable to assemble and execute the postural control actions that are appropriate for the new dynamics. Though of individual variance in this study, higher body sway in response to optical-flow visual stimuli as indexed by objective measurements was in general correlated with higher subjective motion sickness severity. Our present findings might be consistent with the hypothetical mechanism of VIMS. ²³

Following the full-screen presentation of the optical-flow visual stimuli, which induced full-field visual motion at the retina without reference to the visual vertical of the surrounding space, visual stimuli presented exclusively in the peripheral visual field tended to perturb postural control. Notably, vision in the center of the visual field remained normal and enabled reference to the visual vertical. Because the periphery of the visual field, particularly the far peripheral field, has a higher sensitivity of motion detection than does the center of the visual field, ²⁴ information from the periphery of the visual field is classically known to be more important for postural equilibrium than that from the center of the visual field. ²⁵ Our present findings could replicate previous findings that more prominent postural instability responses to optical-flow presented to the periphery of the visual field than to the optical-flow in the center of the visual field. ²⁶ Such consistent results support the importance of information from the periphery of the visual field for postural equilibrium more than that from the center of the visual field, and suggest that visual information in the peripheral visual field is the key component in the emergence of VIMS. Supporting evidence comes from a recent report in which optical flow in the periphery of the visual field through the head-mounted displays increased subjective VIMS symptoms. ²⁷

Electrical technologies of the VR system have continuously advanced over the last few years; however, VIMS remains an issue for today's VR systems mainly because of the nature of content and/or user-related factors.^28,29 The reported incidence rates and severity of VIMS are highly variable; however, the premature termination rates of VR sessions can be as high as 60%. ³⁰ A recent meta-analysis revealed three main factors that affect the likelihood of VIMS in VR users. ³⁰ First, VR games cause significantly more VIMS than static 360° videos or minimalistic scenes do, indicating that the type of VR content is important. Second, VR scenes with more visual information (i.e., content with rapid visual changes) provoke more VIMS than scenes with less visual information do. Finally, the type of locomotion used to navigate the virtual environment is more crucial for VIMS than controller-based locomotion or static scenes (i.e., no visual or physical motion of the user or scene). Adaptation to optical-flow visual information changes (namely, habituation) remains the most effective method for reducing VIMS. Our study revealed that, under the circumstance where such visual adaptation did not occur, optical flow in the periphery of the visual field increased postural instability and subjective VIMS symptoms. The modulation or restriction of visual information in the peripheral visual field might promise to represent a viable solution for improving and/or preventing VIMS.

This study has some limitations. Due to ethical considerations, the present study with short-term visual stimuli intentionally never elicited clinically relevant symptoms of VIMS or moderate-to-severe discomfort. Therefore, we did not identify the sickness-effect sensitive participants. And thereby, we could not confirm that our present findings are equally applicable in sickness-effect sensitive individuals and non-sensitive individuals. Further investigation is required to confirm this solution for clinically relevant VIMS symptoms and discomfort. We should raise another limitation of this study. Due to the exploratory nature of this research, we did not perform sample size calculations. Our sample size might be inadequate to make a decisive inference about the influence of optical-flow visual stimuli in the periphery of the visual field on postural instability. Based on sample size calculations, in future studies we should recruit sufficient numbers of participants to realize the condition effects.

Conclusion

Measuring spatial and temporal dynamics of body sway using a stabilometer when viewing optical-flow visual stimuli through HMD, the stimuli could induce body sway, suggesting sensorimotor incongruence in postural control. Full-screen presentation demonstrated the significant influence on body sway and visually-induced motion sickness. An ancillary finding demonstrated that the visual stimuli presented in the periphery of the visual field potentially contribute to body sway and the sensorimotor incongruence rather than those in the central area. Future studies should be conducted to confirm the condition-effects on body sway and evident visually-induced motion sickness, with sufficient numbers of participants.