Detection method for unrecognized spatial disorientation based on optical flow stimuli

Abstract

BACKGROUND:

Flight accidents caused by spatial disorientation (SD) greatly affect flight safety.

OBJECTIVE:

Few studies have been devoted to the evaluation of SD.

METHODS:

10 pilots and 10 non-pilots were recruited for the experimental induction of SD. Videos for giving optical flow stimuli were played at two different flow speeds to induce SD. Subjective judgment and center of foot pressure (CoP) data were collected from the tests. The data were combined to determine the occurrence of SD and analyze the SD types.

RESULTS:

The number of self-reported SD events was slightly smaller in the pilots than in the non-pilots. The average upper bound of the confidence interval for the standard deviation of CoP was 0.32 $\pm$ 0.09 cm and 0.38 $\pm$ 0.12 cm in the pilots and non-pilots, respectively. This indicator was significantly lower in the pilots than in the non-pilots ( $P=$ 0.03). The success rate of the experimental induction of unrecognized SD was 26.7% and 45.0% in the pilots and non-pilots, respectively.

CONCLUSION:

The method offered a new to analyze unrecognized SD. We could determine the occurrence unrecognized SD. This is an essential means of reducing flight accidents caused by unrecognized SD.

1. Introduction

Spatial disorientation (SD) is a pilot’s erroneous perception of position, attitude, or motion in relation to the gravitational vertical angle and the Earth’s surface. Examples include illusions of self-motion (or vection) [1] and autokinetic illusion [2]. SD has caused many flight accidents so far. From 1993 to 2013, 11% of the flight accidents in the US troops were related to SD [3], with a mortality of 69% [4]. Hence, SD poses a serious threat to flight safety [5].

SD-induced flight accidents are generally divided into three categories [6]: unrecognized SD in this category, a pilot gets disoriented but does not recognize it [7]; recognized SD is aware of SD [8]; and uncontrollable SD is aware of SD, but the pilots gets incapacitated and loses control of the aircraft [7]. According to the latest research, the three types of SD correspond to the three stages of human brain cognition. The three of them are interconnected, each building on the previous one [9]. The judgment method varies for each type of SD. Recognized SD and incontrollable SD can be more intuitively determined, for example, by subjective judgment [10] and rating scales [11]. However, the judgment method for unrecognized SD is not yet mature. In this study, we focused on the detection method for unrecognized SD to solve these mentioned problems.

Although unrecognized SD goes undetected by the human, it influences human behaviors subconsciously. From 2002 to the present, the changes in the center of foot pressure (CoP) were detected using a force plate in several studies on vision-induced vection illusion and postural deviation [12]. CoP deviation was found to be larger during self-reported vection illusion than in its absence.

In 2002, Thurrell et al. [13] examined whether the conscious awareness of self-motion affected the amplitude and directional accuracy of vision-induced postural response. Among studies on vision-induced vection illusion and postural deviation, the changes in CoP were detected using a force plate. CoP deviation was larger during self-reported vection illusion than in its absence. The results confirmed that the direction of vision-induced postural response changed with the variation in the visual stimuli varied. In 2005, Fushiki et al. [14] conducted a similar study and used video-based motion analysis equipment to discuss the subjects’ head and trunk deviations during the experiment. They concluded that postural balance was affected by vection illusion.

Da-Silva et al. [15] conducted an experimental study in 2015 using dynamic virtual vision simulation to induce vection. The CoP data in different directions and under different speeds of the visual scene were analyzed. Their study showed that the displacement of CoP upon vection had the same direction as the visual scene. Their study also showed that the postural balance was disrupted by vection, resulting in significant changes in CoP.

The aforementioned studies were concerned with postural balance upon vection. However, very few focused on recognized SD. Mounting evidence shows that the differences in postural instability can be used to predict who will experience strong illusory self-motions (vection) and become sick when exposed to global patterns of optical flow [16, 17]. Observing a moving visual scene is very likely to generate a feeling of disorientation [18], which inevitably results in postural adjustments [19]. When SD is induced by only visual information, very small discrepancies are noted in posture and individuals’ susceptibility to SD. The subjects may have little awareness of the occurrence of SD. This situation is known as unrecognized SD, which poses the greatest threat to flight safety. Even if the pilots themselves are unaware of SD, we can still detect it from minor postural swings. On this basis, we could determine the occurrence of SD and differentiate between different types of SD events, especially unrecognized SD. Then, enhancing SD-targeted training based on this understanding is an essential means of reducing flight accidents.

In the present study, we applied a judgment methodology for unrecognized SD published in 2020 [20] and 2022 [9] to practice. CoP was used as an indicator to determine the occurrence of unrecognized SD. CoP was combined with subjective judgment to determine the occurrence of unrecognized SD. The present study explored the feasibility of the judgment methodology in two different populations: pilots and non-pilots. We investigated the feasibility and reliability of this methodology and assessed the similarities and differences in the experimental induction of unrecognized SD in pilots and non-pilots. Therefore, we hope that this study can provide reference for reducing flight accidents and casualties.

2. Method

2.1 Participants

The study included 20 participants, all of whom were male. Ten participants were pilots, whereas the other 10 were non-pilots. The pilots were aged 24–28 years, with an average of 25 $\pm$ 1.79 years. The non-pilots were aged 23–28 years, with an average of 22.6 $\pm$ 1.68 years. All participants were right-handed and had no history of systemic diseases or nervous system disorders. They also had no history of mental illnesses affecting their cognitive ability. This research complied with the tenets of the Declaration of Helsinki and was approved by the Institutional Review Board at the ethics committee of Hebei Medical University (approval number: 2022001). Informed consent was obtained from each participant.

2.2 Experimental process

Each participant received two groups of tests using optical flow stimuli. One group consisted of six repetitive control tests, with each lasting for 30 s. The other group consisted of six repetitive tests with optical flow stimuli, each lasting for 30 s. The participants were asked whether they experienced any vection at the end of each test. twelve tests were given for each participant, with each lasting for 30 s. The two tests were spaced 1–5 min apart. It was ensured that each participant had enough time to rest.

The lights were turned off during the tests so that the tests were conducted in complete darkness. The participants kept two feet together during the tests, standing on the force plate with two hands down. The participants were told to maintain an upright position during the tests. A projection screen was placed at 1 m in front of the participants for playing the video stimuli. Both the vertical and horizontal angles of view of the projection screen were 90^∘.

2.3 Visual stimuli

The optical flow stimuli were given to induce SD. The video was shown on a 2 $\times$ 2 m² projection screen. Only two colors were displayed, that is, black (R: 255, G: 255, and B: 255) and white (R:1, G:1, and B:1). A visual stimulus was projected on the screen [21] to simulate the scene in which the light observable on the ground from a high-speed aircraft at night was moving toward the pilot due to relative motion.

Figure 1.

Optical flow stimuli.

2.4 Control tests

In the control tests, the video was paused. That is, the picture was static (Fig. 1), while all other test conditions were the same as in the optical flow stimuli tests. Only one variable was different between the control and optical flow stimuli tests: whether the picture was flowing. The test results were then analyzed on this basis.

2.5 Experimental group

The optical flow stimulus was given to simulate the scene in which the light observable on the ground from a high-speed aircraft at night was moving toward the pilot due to relative motion [21]. After the test began, the lines in the videos flowed in the direction shown by the arrowhead (Fig. 1). Two flow speeds of the visual scene were used: high and low. The width of the distant mountain and the horizon in the video was 2.5 cm. The width of the arrow line was 1 cm. Every two adjacent arrow lines among seven were spaced 45^∘ apart. In the high-speed flow stimulus tests, the middle arrow line moved downward for 45 cm every second. In the low-speed flow stimulus tests, the middle arrow line moved downward for 15 cm every second. The high flow speed was about three times the low flow speed. Each test lasted for 30 s.

2.6 Data collection

2.6.1 Subjective judgment

Since we focused on unrecognized SD, the participants were asked whether they had experienced vection during tests after each test. Here, vection was defined as a subjective feeling of oneself moving forward. Two different subjective judgments concerning the aforementioned situation were recorded:

a)

Y, feeling oneself moving forward, that is, a visual illusion of self-motion;

b)

N, not feeling oneself moving forward, that is, no visual illusion of self-motion.

2.6.2 Postural deviation

Bertec force plate (Bertec FP4060-08, Bertec Corp, OH, USA) was immobilized on the laboratory floor, covering an area of 400 $\times$ 600 mm². The duration of each test was set to 30 s, with 100-Hz low-pass filtering. CoP data in both $x$ -axis (CoPx) and $y$ -axis (CoPy) were collected. It was assumed that when a physically stationary individual in a standing position experienced vection caused by dynamic video stimuli, CoP deviation and swing were usually more significant than in the absence of SD [15].

2.7 Processing of indicator data

We calculated the standard deviation ( $S$ ) [22] of CoP as a primary indicator to measure the deviation of the CoP. Standard deviation measures the dispersion of the observed values relative to the true values. S represented the degree of dispersion of CoP distribution relative to the means. The overall standard deviation ( $S$ ) of CoP along the $x$ - and $y$ -axes was used. $S$ was calculated as follows:

$S=\sqrt{\sum [{\left(x_m-x_i\right)}^2+{\left(y_m-y_i\right)}^2]/n}$(1)

where $x_m=\sum x_i/n$ , $y_m=\sum y_i/n$ , $x_i$ and $y_i$ are the coordinates of the $i$ -th sampling point along the $x$ - and $y$ -axes, respectively, $n$ is the sampling frequency, $n=$ 30,000.

2.8 Statistical analysis

$S$ was used to determine whether the participant underwent some change in them postural control when receiving visual stimuli. Each participant received twelve tests: six control tests and six optical flow stimuli tests (three tests belong to high and three tests belong to low). A 95% confidence interval of CoP was calculated from the six control tests [9] as the normal range of CoP in the control test in the absence of SD.

When $S$ in the experimental group fell on the right side of the confidence interval, a significant difference in CoP was noted between the optical flow stimuli test and the control test (Y); when $S$ in the experimental group fell within or on the left side of the confidence interval, no significant difference in CoP was observed between the optical flow stimuli test and the control test (N).

Table 1

Judgment method for unrecognized SD

No.	Reported	$S$ fell on the right side of the confidence interval	Result
1	Y	Y	Recognized SD
2	Y	N	Recognized SD
3	N	Y	Unrecognized SD
4	N	N	Non-SD

2.9 Judgment method for unrecognized SD

We conducted an in-depth investigation into the judgment method for unrecognized SD based on the studies conducted in 2020 [20] and 2022 [9]. Following the judgment method, the subjective judgment data and CoP data from all tests were classified into the following four categories, as shown in Table 1:

a)

The participant reported a vection (Y), and $S$ fell on the right side of the confidence interval (Y). In this situation, the participant recognized the occurrence of vection but could not maintain postural balance. A recognized SD event occurred.

Table 2

Upper bounds of the 95% confidence interval of $S$ in two different groups receiving the control tests

No.	Upper bounds of the 95% confidence interval of $S$ (cm)		No.
	Pilot group	Non-pilot group
1	0.18	0.18	7
7	0.25	0.20	4
8	0.26	0.22	1
4	0.27	0.38	3
3	0.28	0.41	5
2	0.30	0.42	8
5	0.36	0.45	2
9	0.41	0.47	6
10	0.42	0.50	9
6	0.50	0.55	10
Mean value	0.32	0.378	Mean value
Standard deviation	0.09	0.119	Standard deviation

b)

The participant reported a vection (Y), and $S$ fell within or on the left side of the confidence interval (N). A recognized SD event occurred.

c)

The participant did not report a vection (N), and $S$ fell on the right side of the confidence interval (Y). The participant experienced a vection but did not recognize it. An unrecognized SD event occurred.

d)

The participant did not report a vection (N), and $S$ fell within or on the left side of the confidence interval (N). The participant did not experience vection, and no SD event occurred.

3. Results

3.1 Subjective judgment

Twenty participants were tested (pilot group and non- pilot group), each given three repetitive control tests and three optical flow stimuli tests. Each group receiving the low-speed optical flow stimuli + high-speed optical flow stimuli received 30 $+$ 30 tests in total.

The test was marked positive for recognized SD if the participant reported SD after the test. The numbers of tests marked positive for recognized SD were as follows:

a)

Pilot group: Two for the low-speed optical flow stimuli and three for the high-speed optical stimuli.

b)

Non-pilot group: Two for the low-speed optical flow stimuli and four for the high-speed optical stimuli.

Unrecognized SD occurred only in tests for which no participants reported SD. The number of tests without self-reported SD is shown in Fig. 2:

1. a)

Pilot group: 28 for the low-speed optical flow stimuli and 27 for the high-speed optical stimuli.

b)

Non-pilot group: 28 for the low-speed optical flow stimuli and 26 for the high-speed optical stimuli.

The two flow speeds chosen for the optical flow stimuli were small. Therefore, the pilots reported recognized SD less frequently than the non-pilots. Further, we analyzed CoP in participants reporting no SD

3.2 Postural swing analysis

We first calculated the $S$ of CoP for each participant in the control tests. The upper bound of the 95% confidence interval was calculated using the standard deviation of the six control tests for each participant. These upper bounds within each group were arranged in ascending order, as shown in Table 2 and Fig. 2. Then, the $t$ test ( $\alpha =$ 0.05) was used to compare the confidence intervals of the mean. The $t$ test showed significant differences in the means between pilots and non-pilots ( $P=$ 0.03). This result indicated a significantly lower upper bound of the confidence interval in pilots than in non-pilots. The physically stationary pilots better controlled standing balance despite potential vection, indicating a better spatial orientation ability than in non-pilots.

Table 3

$S$ values of each test in participants from the two groups. The values marked by ^∗ fell on the right side of the confidence interval

No.	Pilot group						Non-pilot group
	Low-speed tests			High-speed tests			Low-speed tests			High-speed tests
	1	2	3	1	2	3	1	2	3	1	2	3
1	0.17	0.14	0.11	0.14	0.15	0.10	0.36	0.40*	0.37	0.43*	0.39*	0.47*
2	0.20	0.24	0.27	0.30*	0.33*	0.19	0.74*	0.40	0.55*	0.38	0.50*	0.56*
3	0.26	0.20	0.24	0.44*	0.27	0.25	0.19	0.17	0.78*	0.20*	0.26*	0.24*
4	0.29*	0.24	0.19	0.26	0.32*	0.36*	0.15	0.24*	0.26*	0.24*	0.20	0.17
5	0.35	0.51*	0.40*	0.29	0.33	0.31	0.57*	0.59*	0.51*	0.43	0.41	0.39
6	0.49	0.33	0.48	0.45	0.47	0.82*	0.82*	0.54	0.77*	0.58*	0.63*	0.71*
7	0.25*	0.26*	0.28*	0.21	0.24	0.19	0.38	0.30	0.40	0.43*	0.51*	0.39
8	0.38*	0.28*	0.23	0.31*	0.25	0.40*	0.13	0.17	0.15	0.12	0.10	0.16
9	0.39	0.49*	0.39	0.55*	0.40	0.33	0.52*	0.43	0.40	0.61*	0.41	0.57*
10	0.40	0.36	0.30	0.48*	0.40	0.37	0.80*	0.37	0.42	0.86*	0.49	0.76*

Figure 2.

Upper bounds of the 95% confidence interval in the two groups receiving the control tests.

After obtaining the upper bounds of the 95% confidence interval, we further calculated the standard deviation of CoP for each participant in each test (Table 3). If $S$ for a specific participant in a test was above the upper bound of the confidence interval, the $S$ of CoP of this participant in this test fell on the right side of the confidence interval. In that case, the swing amplitude would be much more significant than that in the control group. Such postural imbalance was caused by SD. Table 3 shows the values of $S$ of CoP for each participant in each test in the pilot and non-pilot groups, respectively. The values marked by ^∗ fell on the right side of the confidence interval.

As shown in Table 3, in low-speed optical flow stimuli tests from pilots, the $S$ values fell on the right side of the confidence interval in nine tests. Postural imbalance events occurred in these participants, accounting for one third of all tests. In high-speed optical flow stimuli tests from pilots, $S$ values fell on the right side of the confidence interval in 10 tests. Postural imbalance events occurred in these participants, accounting for one third of all tests. In low-speed optical flow stimuli tests from non-pilots, $S$ values fell on the right side of the confidence interval in 13 tests. Postural imbalance events occurred in these participants, accounting for two thirds of all tests. In high-speed optical flow stimuli tests from non-pilots, $S$ values fell on the right side of the confidence interval in 18 tests. Postural imbalance events occurred in these participants, accounting for one-half of all tests.

3.3 Judgment and analysis of unrecognized SD

The subjective judgment data were combined with the confidence intervals of CoP to identify unrecognized SD events (Table 1). When the participant did not report vection but the $S$ value of the participant for this test fell on the right side of the confidence interval, the participant was considered to experience unrecognized SD. The judgment results for unrecognized SD are shown in Table 4. Here, “ $+$ ” represents the occurrence of unrecognized SD in the participants; “ $++$ ” represents the occurrence of recognized SD in the participants; “–” represents no occurrence of SD whatever.

Table 4

Judgment results for unrecognized SD in the two groups Here, “ $+$ ” represents the occurrence of unrecognized SD in the participants; “ $++$ ” represents the occurrence of recognized SD in the participants; “–” represents no occurrence of SD

No.	Pilot group						Non-pilot group
	Low-speed tests			High-speed tests			Low-speed tests			High-speed tests
	1	2	3	1	2	3	1	2	3	1	2	3
1								$+$	–	$++$	$+$	$+$
2	$-$	$-$	$-$	$+$	$+$	$-$	$+$	$-$	$+$	$-$	$+$	$+$
3	$-$	$-$	$-$	$+$	$-$	$-$		$++$	$+$	$+$	$+$	$+$
4	$++$	$-$	$-$		$+$	$+$		$+$	$+$	$++$	$-$	$-$
5	$-$	$+$	$+$				$++$	$+$	$+$	$-$	$-$	$-$
6	$++$	$-$	$-$	$++$	$-$	$+$	$+$	$-$	$+$	$+$	$+$	$+$
7	$+$	$+$	$+$							$+$	$+$	$-$
8	$+$	$+$	$-$	$+$	$-$	$+$
9		$++$	$-$	$+$	$-$	$-$	$+$	$-$	$-$	$+$	$-$	$+$
10	$-$	$-$	$-$	$++$	$-$	$-$	$++$	$++$	$-$	$+$	$-$	$+$
Total	“ $+$ ” (Unrecognized SD): 16; “ $++$ ” (Recognized SD): 5; “–” (Non-SD): 39.						“ $+$ ” (Unrecognized SD): 27; “ $++$ ” (Recognized SD): 6; “–” (Non-SD): 27.

Table 4 shows that the total number of unrecognized SD events in pilots was lower than non-pilots. An insignificant difference in the number of unrecognized SD events was observed between the two groups. The success rate of the experimental induction of unrecognized SD was 26.7% and 45.0% in the pilots and non-pilots, respectively. Non-pilots were more likely to experience unrecognized SD than pilots (more by 9 events).

The two groups of participants show differences according to the test results. Then, we performed a multivariate analysis of variance to verify whether this difference was statistically significant. The three different judgment results were taken as dependent variables, and two-factor analysis of variance was conducted considering the interaction effect. The analysis results for factor A (two levels: optical flow stimuli with low and high flow speeds) and factor B (two levels: pilot and non-pilot) were indicated no interaction effect between the flow speed and the population. These two factors had little impact on the type of SD. No significant difference was found in the occurrence of SD evoked by low- and high-speed optical flow stimuli in each group. To be specific, the difference in this respect was only statistically insignificant.

4. Discussion

We administered video-based optical flow stimuli to 10 pilots and 10 non-pilots for the experimental induction of SD. A set of criteria were set up for identifying unrecognized SD. The purpose was to discuss the feasibility of the judgment method for unrecognized SD and the difference between pilots and non-pilots in terms of susceptibility to SD.

The analysis of subjective judgment data from the participants showed that, under the visual stimuli at the same flow speed, vection was reported less frequently in pilots than in non-pilots. One possible reason was that the pilots received flight simulator training addressing the vection problem as part of their routine training. In contrast, non-pilots were rarely exposed to such scenes and were more likely to experience recognized vection compared with pilots. The aforementioned finding proved the effectiveness of flight simulator training in reducing SD [23, 24]. Some researchers believed that the three types of SD events were three successive steps in human perception. Increasing the level of cognition could help with the timely recognition of SD [20].

The present study demonstrated the feasibility of the judgment method for unrecognized SD. The subjective judgment data were combined with the statistical analysis of CoP to identify participants and tests positive for unrecognized SD. Twenty participants recruited in the present study received 120 control tests in total. The statistical analysis of the confidence intervals calculated from the standard deviation ( $S$ ) of CoP in the control tests in the two groups showed that the pilots better controlled their postural balance when being physically stationary in a standing position compared with non-pilots. In the tests, $S$ values of CoP falling on the right side of the confidence interval indicated that the postural balance was disrupted. In other words, the participants could not maintain postural balance in a specific test. This finding agreed with those of PDa-Silva et al. [15] in 2015: if the postural balance was disrupted by vection, CoP would change considerably.

We used the proposed judgment method for unrecognized SD. The unrecognized SD events totaled 43 in 20 participants, accounting for 35.8% of all tests. This result agreed with the general occurrence rule of unrecognized SD [25]. The total number of unrecognized SD events occurring in pilots was reduced by 18.3% compared with that in non-pilots. Although the difference was not statistically significant, we have reasons to believe that pilots are less likely to experience SD under optical visual flow stimuli, probably due to frequent training addressing SD. It was thus proved feasible to combine subjective and objective judgments to determine the occurrence of recognized SD. Our study shed new light on the detection of unrecognized SD [26] and laid the basis for subsequent experimental induction and statistical analysis of SD. Hence, enhancing SD-targeted training may be an effective pathway to reduce SD-related accidents.

Two-way ANOVA, considering the interaction effect between variables based on all test data, indicated no significant difference in SD occurrence under different population and flow speed factors. This result was inconsistent with the findings of Da Silva et al. [15] . The latter believed that the flow direction of the visual scene affected people’s control of postural balance. The discrepancy might be explained by the slight difference between the two flow speeds adopted for our tests. Another probable reason was the limited number of participants recruited for the present study, leading to an insignificant difference. We will follow up on this issue in the future.

To conclude, we identified participants and tests positive for unrecognized SD by combining subjective judgment with the standard deviations ( $S$ ) of CoP. Under optical flow stimuli, the number of recognized and unrecognized SD events in pilots was smaller than that in non-pilots. Besides, the pilots had better control of postural balance in the control tests and the optical flow stimuli tests than the non-pilots. These results demonstrated that weak stimuli or visual stimuli alone resulted in a very small amplitude of postural imbalance. The participants themselves were hardly aware of the occurrence of SD. This situation is known as unrecognized SD [27], which poses the greatest threat to flight safety. Even if the pilots themselves are unaware of SD, we can still detect them from minor postural swings. Combining subjective judgment with postural swing analysis proved effective for identifying unrecognized SD [26]. Besides, our results agreed with the findings of Nam-Gyoon Kim and Beom-Su Kim [28], confirming the assumption that severe SD was associated with worse control of postural balance.

The aforementioned findings suggested that the amplitude of postural deviation was closely related to the occurrence of SD. Many studies have shown that the involuntary deviation of the head or trunk occurs first upon SD, resulting in CoP deviation. In future research, we will use this method to analyze more subjects. We plan to combine this judgment method with the EEG signals. Used to analyze whether there are observable characteristic changes in EEG signals when unrecognized SD occurs, and further improve the judgment method for unrecognized SD.

5. Conclusion

Our research methodology opened a new direction for the study of unrecognized SD. The test results showed that combining subjective judgment with CoP data was feasible for identifying unrecognized SD. Based on this method, we will look for the characteristic changes in objective physiological signals accompanying the occurrence of unrecognized SD. Our method laid a basis for identifying unrecognized SD, screening for pilots below the cognitive threshold, and enhancing the training addressing unrecognized SD. Our study might contribute to the reduction of SD-related flight accidents.

Footnotes

Acknowledgments

This work was supported in part by the Health Commission of Hebei Province (20220079, 20230214), Science and Technology Project of Hebei Education Department (BJK2023074, QN2022020, QN2023029), and the Innovation Experiment Program of College Students in Hebei Medical University (USIP2022075, USIP2022214, USIP2023161, USIP2023219).

Conflict of interest

None to report.