Sage Journals: Discover world-class research

Abstract

Objective

The aim was to evaluate the effectiveness of a Transport Canada Level 2 Instrument Proficiency Check Flight Training Device to elicit the disorientation caused by the black hole illusion. To evaluate the role of gender, we measured the relative susceptibility of men and women.

Background

Spatial disorientation is a well-known causative factor in aviation mishaps. However, there is no simulation-based training protocol for visual illusions that cause spatial disorientation.

Method

We simulated an approach-and-land scenario using an ALSIM simulator. Trainee pilots were instructed to maintain a 3° approach and land the aircraft under conditions with (nighttime) and without (daytime) the black hole illusion. We computed altitude errors by differencing the daytime and nighttime flight paths. Glideslope errors were calculated as deviations from the 3° approach. To assess the contribution of spatial abilities, participants completed a mental rotation test.

Results

Most pilots showed a shallow final approach during night flight relative to day flights. The pilots who experienced the illusion had lower mental rotation scores than those who did not. Men had higher mental rotation scores, on average, than women, and showed less negative altitude and glideslope errors in the night relative to day conditions. These errors were not mitigated by flight experience.

Conclusion

We reproduced the effects of the black hole illusion in a relatively low-cost aviation simulator. Gender and mental rotation skills were factors in black hole disorientation.

Application

It is feasible to implement simulated visual illusion scenarios in aviation training. It is important to consider gender in designing and assessing flight scenarios.

Keywords

simulation and training situation awareness experimental design vision pilot decision making

Introduction

Spatial disorientation is a known contributing and causative factor in military, commercial, and general aviation mishaps (Benson & Burchard, 1973; Gillingham & Previc, 1993). In this context, spatial disorientation refers to the pilot’s erroneous judgments regarding flight path, altitude, airspeed, vertical velocity, attitude, or general motion and position of the aircraft relative to Earth (Benson & Burchard, 1973; Gillingham, 1992; Gillingham & Previc, 1993). The severity and prevalence of spatial disorientation-related mishaps in aviation vary across countries, aircraft type (e.g., fixed or rotary-wing), and aircrew population (i.e., general aviation, air force, naval, or army). For example, in the United States, Class A mishaps (i.e., involving a fatality, permanent total disability, aircraft destruction, or damage exceeding a specified cost threshold) had prevalence rates related to spatial disorientation ranging from 5 to 15.3% (Bellenkes et al., 1992; Gibb & Olson, 2008; Kirkham et al., 1978; Lyons et al., 2006; Poisson & Miller, 2014). All fatal mishaps within this category reported rates that ranged from 2.5 to 26% (Collins & Dollar, 1996; Kirkham et al., 1978; Moser, 1969). In Canada, spatial disorientation was a factor in 22.5% of Category A accidents (i.e., accidents in which an aircraft is designated missing or destroyed) within the Canadian Forces from 1982 to 1992 (Cheung et al., 1995). These prevalence rates over multiple decades indicate that spatial disorientation is a significant, ongoing factor in aviation mishaps.

It is estimated that half to nearly all pilots experience at least one episode of spatial disorientation in their careers (Chimonas et al., 2002; Sipes & Lessard, 2000; Takada et al., 2009; Tu et al., 2021). The percentage of international aircrews that report an episode of severe spatial disorientation posing a risk to flight safety ranges from 4 to 44% (Davidson et al., 1991; Durnford, 1992; Holmes et al., 2003; Matthews et al., 2002; Pennings et al., 2020; Tu et al., 2021). When disoriented, pilots may rely on their vision to control aircraft trajectory rather than instrumentation, either by choice or by necessity (e.g., instruments are unavailable). Further, during visually guided flight, pilots are more susceptible to spatial distortions from visual illusions (Gillingham, 1992; Previc, 2004). Thus, to maintain a safe flight, pilots must recognize and avoid such disorientation. Unfortunately, opportunities to experience and learn to counter these illusions during training are limited. The aim of this project was to evaluate the effectiveness of a Transport Canada Level 2- Instrument Proficiency Check Flight Training Device to reproduce common aviation-relevant visual illusions. To do so, we simulated one of the most frequently reported visual illusions: the black hole illusion. Surveys suggest that 60–80% of aircrew have experienced this illusion in their careers (Holmes et al., 2003; Matthews et al., 2002; Pennings et al., 2020; Sipes & Lessard, 2000).

The black hole illusion occurs at night during the approach-and-land phase of flight. This illusion typically occurs when an approach is made over a featureless terrain and an unlit sky lacking a discernible horizon; in the worst-case scenario, only the runway lights are visible (Figure 1—left). The black hole illusion has been identified as a contributing factor in several well-documented accidents, including Pan Am Flight 806 (NTSB, 1989), Air Sunshine Cessna (NTSB, 1998), Korean Air Flight 801 (NTSB, 1997), FedEx Flight 1478 (NTSB, 2002), and Dassault Falcon 20 (FSF, 2004). Visual illusions (including black hole) play a role in a notable portion of controlled flight into terrain (CFIT) events (Kelly & Efthymiou, 2019), and the International Air Transport Association lists the black hole illusion as a continuing “environmental threat” in its annual Safety Report (IATA, 2021). Human vision is not well adapted for the spatial judgments required in night flights, relying heavily on contextual cues and horizon references (Roscoe, 1979). In the absence of ambient out the window visual cues, the only information available to land is the runway size and shape, which can lead pilots to misjudge their glide path and altitude. In this context, altitude refers to the aircraft’s height above the ground, and glideslope refers to the angle of descent toward the runway. Pilots tend to overestimate the aircraft’s altitude and initiate an aggressive early descent (Figure 1—right), which results in a shallow final approach, descending below the proper glide path at a lower-than-normal altitude (Kim et al., 2010; Previc, 2004). Typically, the low approach occurs less than 5 NM from the runway (Gibb, 2007). Black hole illusion assessments often use full-flight simulators with realistic motion (Teifer et al., 2023), experienced pilots (Kraft, 1978; Lewis & Mertens, 1979; Mertens & Lewis, 1982; Robinson et al., 2020) or nonpilots (Gibb et al., 2008) to study spatial disorientation. Our study was designed to evaluate if a simple, low-cost fixed-base simulator could reliably reproduce the disorientation caused by this illusion in trainee pilots.

Figure 1.

Image on the left shows an illustration of a black hole environment where only runway edge lights are visible. Image on the right shows an illustration of a shallow approach due to the black hole illusion; when initially on a normal glide path, the pilot misperceives the aircraft’s altitude as too high. To compensate, the pilot begins an aggressive descent, flying a low and potentially dangerous final approach (adapted from Figure 1 from Gibb (2007)).

Most research on visual illusions in aviation involves anecdotal, incident, or accident reports. Further, while extant experimental studies are more rigorous, many only include men or fail to specify gender (Bulkley et al., 2009; Gibb et al., 2008; Kim et al., 2010; Kraft, 1978; Lewis & Mertens, 1979; Mertens & Lewis, 1982; Robinson et al., 2020). This omission is notable given evidence that key aviation-related cognitive skills—such as spatial ability (Barron & Rose, 2013; Dror et al., 1993; Egan, 1978; Verde et al., 2018), particularly mental rotation ability (Verde et al., 2013)—often differ by gender (Boone & Hegarty, 2017; Kheloui et al., 2021; Voyer et al., 1995). Meta-analyses consistently demonstrate a male advantage in three-dimensional mental rotation tasks, moderated by task characteristics such as rotation angle and time limits (Voyer et al., 1995). These differences stem partly from variations in strategy use, confidence, and problem-solving approach (Boone & Hegarty, 2017). However recently, it has been reported that spatial ability differences may reflect an interplay of biological, sociocultural, and psychological factors rather than biological sex alone (Kheloui et al., 2021). Despite these well-documented gender differences in spatial perception, little research has examined whether mental rotation ability influences susceptibility to the black hole illusion directly. Research has focused primarily on misperceptions of slant and visual spatial orientation during landing approaches (Jakicic et al., 2022; Perrone, 1984), without explicitly examining the role of mental rotation ability. With growing initiatives to promote gender balance among the aviator population, the lack of data from underrepresented genders may have important implications for training and risk assessment.

Accordingly, the aim of this study was to (1) establish whether an aviation training simulator effectively generates the disorientation expected from the black hole illusion in trainee pilots, (2) evaluate the susceptibility of this illusion in men and women, and (3) assess whether individual differences in mental rotation ability are associated with susceptibility to the illusion. To test our first hypothesis (H1) that trainee pilots can effectively demonstrate the effect of the black hole illusion in a simulator, we evaluated their approach and landing performance during a nighttime simulated black hole scenario. Given that the shallow approach elicited by the black hole phenomenon occurs during the final stages of the approach, we evaluated pilots at two starting distances from the runway (3.5 and 5 NM). To quantify the illusion, we assessed the altitude and glideslope of the simulated aircraft relative to daytime approaches without the illusion.

Based on prior research linking spatial ability and flight performance, we hypothesized (H2) that limitations in mental rotation ability may contribute to the perceptual misjudgments underlying the black hole illusion. Specifically, we propose that the illusion may arise, in part, from challenges in mentally transforming spatial information during approach and landing. Furthermore, given established gender differences in mental rotation ability, we hypothesized (H3) that the female pilots may demonstrate a higher prevalence of the black hole illusion which could be explained by differences in mental rotation ability. Together, these hypotheses aim to clarify the mechanisms contributing to black hole illusion susceptibility and inform training approaches.

Methods

Participants

A total of 30 trainee pilots (men = 15 and women = 15) between the ages of 19–31 (M = 20.7, SD = 2.5) completed the flight scenarios. They were recruited from the student population in the University of Waterloo Science and Aviation program, the Waterloo Wellington Flight Centre, the University of Waterloo Aviation Society, and the University of Waterloo Aviation Alumni Group. All pilots held a Private Pilot License and a Transport Canada Medical Category 1 certificate. Their flight hours ranged from 52 to 335 (M = 168.7, SD = 92). Visual acuity was assessed before the experiment with a requirement of 20/20 with optical correction. If necessary, participants wore their optical correction during testing. Both biological sex and self-reported gender were collected from all participants, and in all cases, their biological sex was consistent their declared gender; all analyses are reported by gender only.

Apparatus

All testing was conducted using the ALSIM AL250 FSTD fixed-base Flight Training Device at the University of Waterloo’s Institute for Sustainable Aeronautics (WISA) facility (Figure 2). The device has a variety of weather, auditory and tactile settings, which create flexible environments for advanced aviation training and research. The simulator has a panoramic 250° by 49° high-definition display and a minimum frame rate of 60 frames per second. The display screen was approximately 85 cm from the eyepoint in the cockpit.

Figure 2.

The cockpit view of the ALSIM, showing controls and panoramic visual displays.

Procedure

Prior to testing, participants completed a consent form, demographic questionnaire, visual acuity test, and the Vandenberg and Kuse Mental Rotation Test-A (MRT-A) (Peters et al., 1995). We selected the MRT-A as our measure of spatial ability because the black hole illusion likely relies on object-centered 3D rotational transformations. The MRT-A specifically measures the speed and accuracy of such rotations and has been widely used in individual-differences research (Peters et al., 1995; Vandenberg & Kuse, 1978). The demographic questionnaire included questions regarding total flight hours, hours flying at night, and simulator hours. Following this, participants completed a 5-min practice landing under daylight conditions with full instruments available to become familiar with the simulated aircraft. For practice trials, we used a simulated approach to Pembroke airport (ICAO: CYTA, N 45° 51.87′ W 77° 15.09′, magnetic heading 352°).

To simulate the black hole illusion, we selected a location and weather conditions to generate a Night condition that included an unlit sky, featureless terrain, and an invisible horizon and moon, with only runway lights visible. To prevent pilots from learning the characteristics of the runway and landscape, all flight scenarios were completed in the Night condition before the Day condition. This also reduced the possibility of practice effects between the Night and Day conditions, since the visual information available in the Night condition was sparse. For all test conditions, we used a rural northern Canadian airport in Fort Severn, Ontario (ICAO: CYER, N 56° 01.14′ W 87° 40.57′, magnetic heading 150°). We required pilots to fly using vision alone, simulating an instrument failure that increases susceptibility to the illusion. Thus, all instruments, except the airspeed indicator and tachometer, were disabled and covered by a strip of black card. Before each trial, the trim was neutralized, and flaps were up. Participants were free to adjust the flaps while flying. The runway had no markings, and both the precision approach path indicator (PAPI) and approach lighting system (ALS) were turned off. After each flight, a MATLAB^TM script recorded data from the simulator. Pilots were instructed to maintain a 3° approach and land the aircraft. We started pilots on the ideal 3° glide path. Thus, the altitude for each Starting Distance (5 and 3.5 NM) was consistent with a 3° glideslope (1630 ft and 1150 ft above ground, respectively). Starting distances were counterbalanced, and each participant completed 4 trials in total (2 Starting Distances × 2 Time-of-Day conditions). Between each trial, we asked trainee pilots about their confidence and how difficult they found each approach. We included a debrief questionnaire after the flight scenarios, where we stated the purpose of the study and asked pilots for their feedback on task difficulty and flight strategies. The study took 1 hour to complete.

Analysis

The independent variables were Time-of-Day (Night or Day), Starting Distance (3.5 and 5 NM), Gender (men or women), MRT-A score, or Observer Group (“black hole” or “no black hole” group). The dependent variables were the altitude and glideslope deviation time series. Glideslope errors were calculated as the difference in approach angle relative to the 3° approach for the Night and Day conditions. The instantaneous approach angle was calculated as the arctangent of the ratio of recorded altitude to distance from the runway threshold. Given the lack of flight instruments and the trainee status of the participants, we expected considerable variability in their flight trajectories, particularly in the black hole environment. Altitude (i.e., height above ground) errors were defined as the signed difference in altitude between the Day and Night conditions (altitude_D-N). Positive altitude errors indicate that nighttime altitudes exceeded daytime altitudes, whereas negative errors indicate that nighttime altitudes were lower than daytime altitudes. For each analysis, we used individual linear mixed-effects models to determine the relationship between select predictor variable(s) (i.e., Time of Day, Starting Distance, MRT-A score, Gender, and Observer Group) on a single dependent variable (i.e., altitude at Night relative to Day (altitude_D-N) or glideslope error). Each model included a random intercept for each participant to account for individual differences and repeated within-participant measurements, and, where appropriate, random slopes were included to allow the effect of predictors to vary across observers, if doing so improved model fit. Repeated measures of altitude_D-N and glideslope error, collect across multiple Time-of-Day and Starting Distance trials, were nested within observers. Each linear mixed-effects model had a Kenward-Roger correction on degrees of freedom. Statistical significance was evaluated at p < .05 and the Holm’s correction for family-wise error was applied throughout the analyses (Holm, 1979). The models were fit by a restricted maximum likelihood (REML) procedure. We used the “lmer” function in the “lmerTest” package in R (Kuznetsova et al., 2017) to compute linear mixed-effects models. Partial eta-squared effect sizes for the linear mixed-effect models were computed using the “t_to_eta2” function from the “effectsize” (Ben-Shachar et al., 2020) package. Bayesian and other independent samples tests and corresponding effect sizes were calculated using JASP statistical software (JASP, 2024).

Results

As discussed above, the black hole disorientation results in a shallow final landing approach. We found that almost two-thirds of approaches in the Night condition followed a shallow path that was below the altitude of the ideal 3° glide path (Figure 3). Thus, our night approach-and-land scenario reliably recreated the black hole disorientation. A low approach was particularly evident less than 1 NM from the runway threshold (right plots in Figure 3). Although the LOESS (i.e., locally estimated scatterplot smoothing) fit in Figure 3 appears slightly above the ideal 3° glide path, this represents the average across all pilots. Given that some pilots flew below the 3° path while others exceeded it, the smoothed fit is on average slightly above the ideal path in the final approach. For purposes of analysis, pilots who demonstrated a negative mean altitude_D-N (i.e., a lower night altitude than day altitude) in the last 1 NM of their final approach were categorized as having experienced black hole disorientation.

Figure 3.

Individual flight trajectories for the 5 (top) and 3.5 NM (bottom) Starting Distances during the Day (control) and Night (black hole) conditions. The left plots show the flight trajectories for the entire flight path. The right plots show the flight trajectories for the last 1 NM of the flight. The start of the runway is equivalent to zero NM. The solid black line represents an LOESS fit. The single-dashed black line represents a 3° glideslope.

To measure black hole disorientation, we calculated mean altitude_D-N and glideslope error for the last 1 NM of the final approach (Figure 4). A linear mixed-effects random intercept model of altitude_D-N regressed onto Starting Distance revealed that there was no significant difference between 3.5 and 5 NM, b = −11.49, t (29.00) = −0.74, p_adj = .46, η_p² = .02, CI_.95 = [−42.59, 19.61]. Similarly, to determine if the glideslope error differed between the Day and Night flights, we first compared the glideslope error between the two Starting Distances in each Time-of-Day condition. In another linear mixed-effects model with random slopes allowed for Time-of-Day, with glideslope error regressed onto Starting Distance and Time-of-Day, we found that glideslope error was similar between 3.5 and 5 NM from the runway in the Day, b = 0.04, t (58.00) = 0.21, p_adj = 1.00, η_p² = .00, CI_.95 = [−0.34, 0.42], and the Night condition, b = −0.14, t (58.00) = −0.71, p_adj = 1.00, η_p² = .00, CI_.95 = [−0.51, 0.24].

Figure 4.

The left plot shows the average altitude_D-N (feet) for the 3.5 (yellow circles) and 5 NM (blue triangles) Starting Distances. In this plot, the y-axis represents the difference in altitude in the Night relative to the Day condition. The right plot shows the glideslope error for the Day and Night conditions at both Starting Distances. The glideslope error was calculated as the difference between the observed glideslope and the predicted 3° glide path. A positive glideslope is steeper than the predicted 3° glide path, and a negative glideslope is shallower than the ideal glide path. Altitude_D-N and glideslope error are averaged over the last 1 NM of flight. The boxplot represents the interquartile range, and the solid horizontal line represents the median. The horizontal dashed lines represent zero error.

Given that Starting Distance did not impact altitude_D-N or glideslope error, it was not included as a predictor in subsequent analyses. When excluding Starting Distance as a predictor, glideslope error was significantly more negative in the Night compared to Day performance, b = −0.98, t (29.00) = −2.81, p_adj = .009, η_p² = .21, CI_.95 = [−1.68, −0.29], as can be seen in Figure 4 (right). This finding is consistent with the effects of the black hole illusion. We did the same analysis with altitude to confirm a shallow approach in the night conditions. As expected, we found that the differences between observed altitude and altitude corresponding to a 3-degree approach were more negative in the Night condition compared to the Day, b = −46.51, t (29.00) = −2.74, p_adj = .01, η_p² = .21, CI_.95 = [−80.30, −12.72]. Although glideslope was significantly lower in the night condition compared to the day condition, the glideslope in the night condition was closer to the ideal 3-degree glideslope, with greater variability across observers than in the day condition.

To evaluate the factors related to experiencing the black hole illusion, we divided pilots based on whether they experienced the illusion. To do so, we placed pilots into two groups based on their altitude_D-N, averaged across the two start distances (Figure 5). Participants with negative altitude_D-N were placed in the “black hole” (BH) group (19 observers or 63.3%) and the remaining observers were placed in the “no black hole” (NBH) group as indicated by vertical brackets in Figure 5. To determine if the mental rotation ability influenced pilots’ susceptibility to the black hole illusion, we compared MRT-A scores between the BH and NBH groups (Figure 6—left). An independent samples t-test (with a Welch correction for unequal sample size) confirmed that pilots in the BH group had lower mental rotation scores than pilots in the NBH group on average, t (18.74) = −3.10, p = .006, CI_.95 = [−9.26, −1.79]; Hedges’ g = −1.16, CI_.95 = [−1.98, −0.32]. Further, a Bayesian independent samples t-test confirmed strong evidence supporting higher mental rotation scores for pilots that did not show the effect of the illusion, BF₁₀ = 11.90; median: −1.01, CI_.95 = [−1.85, −0.23].

Figure 5.

The altitude_D-N averaged across the two Starting Distances. The vertical brackets represent BH (below zero ft) and the NBH (above zero ft) groups. The horizontal solid line indicates the median. The horizontal dashed lines show zero error.

Figure 6.

The left plot shows the MRT-A scores for individuals who showed the black hole effect (BH) and those who did not (NBH). The MRT-A score represents the number of correctly identified 3D rotations. The right plot shows the MRT-A scores as a function of gender. The horizontal lines represent the median.

To determine if the difference in mental rotation scores between the BH and NBH was partly due to gender differences, we compared MRT-A scores between men and women (Figure 6—right). An independent samples t-test showed that, on average, men had significantly higher MRT-A scores than women, t (28.00) = 2.81, p = .009, CI_.95 = [1.30, 8.30]; Cohen’s d = 1.02, [0.25, 1.78]. A Bayesian independent samples t-test confirmed strong evidence that MRT-A scores were higher in men than women, BF₁₀ = 5.49, median = 0.84, CI_.95 = [0.12, 1.62]. This is consistent with the literature that shows men tend to have higher mental rotation skills than women (Voyer et al., 1995).

Gender

Although men tended to have higher mental rotation scores, our analyses revealed that neither altitude_D-N nor glideslope error was influenced by the interaction between MRT-A score and gender (see Appendix A for full analysis). Focusing on gender alone, Figure 7 (left) showed that women had more negative altitude_D-N than men and this was confirmed with a linear mixed-effects random intercept model, b = −119.84, t (28.00) = −3.40, p_adj = .004, η_p² = .29, CI_.95 = [−188.85, −50.83]. Second, we evaluated the relationship between gender, Time-of-Day (i.e., Day and Night flights), and glideslope error with another linear mixed-effects model (Figure 7- right). The interaction between gender and Time-of-Day was significant, F (1, 28) = 8.94, p_adj = .01. However, glideslope error did not significantly differ between men and women in both the Day, b = 0.93, t (28.00) = 2.12, p_adj = .22, η_p² = .14, CI_.95 = [0.07, 1.78], and Night conditions, b = −0.93, t (28.00) = −1.41, p_adj = .51, η_p² = .07, CI_.95 = [-2.21, 0.36]. While men made similar errors under Day and Night conditions, b = −0.05, t (28.00) = −0.13, p_adj = 1.00, η_p² = .00, CI_.95 = [−0.91, 0.80], women showed significantly more negative glideslope error (−1.91°) in the Night compared to the Day condition, b = −1.91, t (28.00) = −4.35, p_adj = .001, η_p² = .40, CI_.95 = [−2.76, −1.05]. Thus, women’s performance was more affected by the black hole scenario than men’s.

Figure 7.

Mean altitude_D-N (left) and glideslope error (right) for men (dark green) and women (orange) at Starting Distances of 3.5 (circle) and 5 NM (triangle). The horizontal solid lines indicate the median. The horizontal dashed lines show zero error.

Given that women demonstrated more negative altitude_D-N than men, we compared the proportion of men and women in the BH and NBH groups. Overall, there were 7 men and 12 women in the BH group (36.8 and 63.2%, respectively), and 8 men and 3 women in the NBH group (72.7 and 27.3 %, respectively). Thus, a larger proportion of women were susceptible to the effects of the black hole illusion. Lastly, we also confirmed that these differences were not due to differences in flight experience (Appendix B).

Mental Rotation

Given that NBH pilots tended to have higher mental rotation scores, we evaluated whether altitude_D-N and glideslope error depended on mental rotation skills and Observer Group (BH or NBH, Figures 8 and 9). A linear mixed-effects revealed the slope of altitude_D-N as a function of MRT-A score did not significantly differ between the BH and NBH pilots, b = −8.52, t (26.00) = −1.60, p_adj = .26, η_p² = .09, CI_.95 = [−18.56, 1.53]. The slope between altitude_D-N and MRT-A score did not significantly differ from zero for BH, b = 1.26, t (26.00) = 0.36, p_adj = .72, η_p² = .00, CI_.95 = [−5.27, 7.78], or NBH pilots, b = −7.26, t (26.00) = −1.79, p_adj = .26, η_p² = .11, CI_.95 = [-14.90, 0.38]. Overall, altitude_D-N was not significantly related to MRT-A score, b = 7.58, t (28.00) = 1.98, p_adj = .06, η_p² = .12, CI_.95 = [0.09, 15.07].

Figure 8.

Mean altitude_D-N as a function of MRT-A score in the BH (green) and NBH (purple) groups at Starting Distances of 3.5 (circle) and 5 NM (triangle). The solid lines are linear regression lines with confidence intervals indicated by shaded areas. The horizontal dashed lines show zero error.

Figure 9.

Mean glideslope error as a function of MRT-A score for the BH (green) and NBH (purple) groups for the 3.5 (circle) and 5 NM (triangle) Starting Distances. The solid lines are linear regression lines with confidence intervals indicated by shaded areas. The horizontal dashed lines show zero error.

To determine if mental rotation ability and Observer Group affected glideslope error, we modeled glideslope error as a function of MRT-A score, Observer Group, and Time-of-Day using a linear mixed-effects random intercept model (Figure 9). There were no significant three-way or two-way interactions. Further, MRT-A score was not significantly associated with glideslope error in either the Day or Night condition for either group (Table 1).

Table 1.

Analysis of Glideslope Error as a Function of MRT-A Score and Observer Group.

	Coefficient [CI_.95]	df	t-value	p _adj	η _p ²
Day
BH	−0.03 [−0.16, 0.10]	33.42	−0.45	1.00	.00
NBH	0.05 [−0.10, 0.20]	33.42	0.59	1.00	.01
Slope change from Night to Day
BH	−0.02 [−0.11, 0.07]	86.00	−0.36	1.00	.00
NBH	0.15 [0.04, 0.25]	86.00	2.65	.15	.08
Night
BH	−0.01 [−0.14, 0.11]	33.42	−0.20	1.00	.00
NBH	−0.10 [−0.25, 0.05]	33.42	−1.25	1.00	.04
Slope change from BH to NBH
Day	0.08 [−0.12, 0.27]	33.42	0.75	1.00	.02
Night	−0.09 [−0.28, 0.11]	33.42	−0.82	1.00	.02

Given the absence of MRT-A effects on glideslope error for the BH and NBH groups, we investigated the relationship between MRT-A score and error independent of Observer Group. A mixed-effects model regressing glideslope error onto MRT-A score and Time-of-Day (with random slopes for Time-of-Day) showed that relationship between MRT-A score and glideslope error was similar between the Night and Day conditions, b = −0.12, t (28.00) = −1.80, p_adj = .33, η_p² = .10, CI_.95 = [−0.25, 0.01]. Further, neither the Day, b = −0.02, t (28.00) = −0.39, p_adj = .71, η_p² = .00, CI_.95 = [−0.11, 0.07], nor the Night, b = 0.10, t (28.00) = 1.58, p_adj = .37, η_p² = .08, CI_.95 = [−0.02, 0.22], condition had a slope significantly different from zero. There was no relationship between MRT-A score and altitude_D-N when excluding Observer Group as a predictor, b = 7.58, t (28.00) = 1.98, p_adj = .06, η_p² = .12, CI_.95 = [0.09, 15.07].

Pilots who experienced black hole disorientation scored lower on the MRT-A (mental rotation) test than those who did not, suggesting a link between spatial ability and susceptibility to the illusion. Since MRT-A scores differed by gender, with women scoring lower than men, and women also showed more negative altitude_D-N, we investigated whether mental rotation ability mediated the relationship between gender and altitude_D-N. To test this, we conducted a statistical causal mediation analysis (Appendix C). In brief, we found no support for the proposal that mental rotation mediates the relationship.

Discussion

We found that most trainee pilots produced a shallow glide path during the final approach, consistent with the spatial disorientation expected from the black hole illusion (Figure 3). Thus, our study confirmed that it is possible to reproduce the black hole disorientation in a fixed aviation training simulator (H1). This is an effective, low-cost solution for implementing simulated visual illusion scenarios in aviation training and research. Interestingly, in postflight interviews, some pilots stated they adopted strategies in their approaches (e.g., flying at higher altitudes) to counteract the illusion. Despite this, half of these pilots still demonstrated a shallow approach. Thus, just being aware of the illusion is insufficient to mitigate its impact. Additionally, neither total flight experience nor night flying experience was associated with performance in the black hole scenario (Appendix B). This further suggests that general flight experience (including night flights) does not help aviators avoid the effects of this illusion. From these results, we conclude that training should not only convey the effects of the illusion but also focus on the specific flight strategies necessary to counteract the spatial disorientation and land safely. For example, black hole illusion countermeasures and staged visual approach training (i.e., starting with pure nighttime runway conditions and gradually adding distractions) have been proposed to improve pilots’ spatial-judgment skills (Curtis et al., 2009; Patterson et al., 2021). Incorporating black hole illusion exercises in simulators could provide low-risk practice and help trainees build resistance to related visual errors in simulated real-world flight conditions.

Baseline performance did not differ between men and women; they had equivalent flight skills without the influence of the illusion. However, women’s performance was most affected by the illusion (H3), with more negative glideslope error in the Night compared to Day, and more negative altitude_D-N than men (Figure 7). Overall, men tended to have higher mental rotation scores than women (Figure 6—right). Therefore, we evaluated whether differences in mental rotation ability contributed to these gender differences in flight performance using a causal mediation analysis. However, we found mental rotation scores did not mediate the relationship between gender and negative altitude_D-N.

The fact that those who experienced black hole disorientation had lower mental rotation scores suggests that mental rotation skills play a role in performance in the black hole scenario to some degree (H2). This is understandable given that the shape and scale of the runway lights were the only information available to complete the task. To successfully achieve a 3° glideslope, pilots had to maintain the position and shape of the runway during the entire approach, which is akin to maintaining the orientation of a slanted plane. Short-term visuospatial training could improve scores on such standardized spatial tests (Harris et al., 2013; Rehfeld, 2006). If exposure to the black hole illusion trains the same transformations measured by the MRT-A, it could potentially improve MRT scores. Since we measured MRT-A only at baseline, we cannot assess whether experience with the illusion causally affects mental rotation ability; however, this is an avenue for future research.

In postflight interviews, most pilots reported that they found the black hole scenario difficult and felt uncertain about their altitude, which may have elicited more random visual scanning behavior (Allsop & Gray, 2014), impairments in decision making (Causse et al., 2011) or led to increased cognitive load during landing (Li & Lajoie, 2021). In the face of this additional uncertainty, participants may have relied more on mental rotation-based strategies. However, this remains speculative and future studies are needed to evaluate the causal nature of this relationship.

Overall, we have shown that low-cost aviation simulators can be used to elicit the spatial disorientation associated with the black hole illusion in trainee pilots. Given the often-catastrophic consequences of spatial disorientation, exposure to these potentially dangerous scenarios in a safe, controlled environment is invaluable, especially when opportunities to learn to counter visual illusions during training are limited. Further, the influence of mental rotation skills and gender on flight performance should be considered when designing and assessing simulated flight scenarios during training. Although the black hole illusion is specific to aviation, the underlying mechanisms (e.g., spatial perception, mental rotation, and reliance on visual cues) reflect general principles of human perception. Similar perceptual challenges may occur in other operational settings, such as driving, maritime navigation, and remote vehicle operation, suggesting that these findings may inform broader human factors approaches to training and spatial performance under degraded visual conditions.

Key Points

A relatively low-cost aviation simulator used in pilot training is effective in eliciting the common black hole visual illusion

Training should include strategies to counteract the illusion

Women trainee pilots were more susceptible to the illusion than men

Mental rotation skills play a role in mitigating black hole disorientation

Footnotes

Acknowledgments

Special thanks to Kamal Ben and Rafael Pastorin Repato for their feedback on our flight scenarios and management of the flight simulator, Carolyn Machan and Allison Lynch for coordinating participant recruitment, and the Waterloo Institute for Sustainable Aeronautics (WISA) team for their collaboration and access to their training and research facility.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was funded by the Canada First Research Excellence Fund (CFREF): Vision Sciences to Applications (VISTA).

Brittney Hartle

Robert S. Allison

Elizabeth L. Irving

Laurie M. Wilcox

Appendix

Author Biographies

Arleen Aksay is a PhD student in psychology at York University. She obtained her MA (2020) from York University.

Brittney Hartle is a postdoctoral researcher at York University, where she studies 3D vision in immersive technology. She obtained her PhD (2022) from York University.

Robert S. Allison is professor of Electrical Engineering and Computer Science at York University, where he directs the Centre for Vision Research. He obtained his PhD (1998) from York University and his MASc (1994) from the University of Toronto.

Elizabeth L. Irving is a professor emeritus at the School of Optometry and Vision Science, University of Waterloo. She obtained her PhD in vision science in 1994 from the University of Waterloo.

Sion Jennings is a senior research officer at the National Research Council of Canada. He earned his MASc (1992) in systems design engineering from the University of Waterloo.

Laurie M. Wilcox is a professor of psychology where she currently holds a York Research Chair (in 3D Vision) and is Scientific Director for the CFREF program Vision: Science to Applications. She obtained her MA (1987) and PhD (1992) in psychology from Western University, Canada.

References

Allsop

Gray

(2014). Flying under pressure: Effects of anxiety on attention and gaze behavior in aviation. Journal of Applied Research in Memory and Cognition, 3(2), 63–71. https://doi.org/10.1016/j.jarmac.2014.04.010

Barron

L. G.

Rose

M. R.

(2013). Relative validity of distinct spatial abilities: An example with implications for diversity. International Journal of Selection and Assessment, 21(4), 400–406. https://doi.org/10.1111/ijsa.12049

Bellenkes

Bason

Yacavone

D. W.

(1992). Spatial disorientation in naval aviation mishaps: A review of class A incidents from 1980 through 1989. Aviation Space & Environmental Medicine, 63(2), 128–131.1546941

Ben-Shachar

Lüdecke

Makowski

(2020). Effectsize: Estimation of effect size indices and standardized parameters. Journal of Open Source Software, 5(56), 2815. https://doi.org/10.21105/joss.02815

Benson

A. J.

Burchard

(1973). Spatial disorientation in flight: A handbook for aircrew. Advisory Group for Aerospace Research and Development.

Boone

A. P.

Hegarty

(2017). Sex differences in mental rotation tasks: Not just in the mental rotation process. Journal of Experimental Psychology: Learning, Memory, and Cognition, 43(7), 1005–1019. https://doi.org/10.1037/xlm0000370

Bulkley

Dyre

B. P.

Lew

Caufield

(2009). A peripherally-located virtual instrument landing display affords more precise control of approach path during simulated landings than traditional instrument landing displays. Proceedings of the Human Factors and Ergonomics Society - Annual Meeting, 53(1), 31–35. https://doi.org/10.1177/154193120905300108

Causse

Dehais

Pastor

(2011). Executive functions and pilot characteristics predict flight simulator performance in general aviation pilots. The International Journal of Aviation Psychology, 21(3), 217–234. https://doi.org/10.1080/10508414.2011.582441

Cheung

Money

Wright

Bateman

(1995). Spatial disorientation-implicated accidents in Canadian forces, 1982-92. Aviation Space & Environmental Medicine, 66(6), 579–585. 24279238.

10.

Chimonas

Diamantopolous

Markou

Stathogiannis

(2002). A spatial disorientation survey of Hellenic air force pilots. [Paper presented]. Research and Technology Organization and Human Factors and Medicine Symposium on Spatial Disorientation in Military Vehicles: Causes, Consequences and Cures, La Coruna, Spain. (HFM-085, RTO-MP-086).

11.

Collins

W. E.

Dollar

C. S.

(1996). Fatal general aviation accidents involving spatial disorientation: 1976-1992. Federal Aviation Administration. (No. DOT/FAA/AM-96/21).

12.

Curtis

M. T.

Jentsch

F. G.

Maraj

(2009). How differences in spatial ability influence inexperienced users in a visual perceptual aviation task. In 2009 international symposium on aviation psychology (pp. 545–550). https://corescholar.libraries.wright.edu/isap_2009/25

13.

Davidson

Beevis

Buick

Donati

A. L. M.

Kantor

Bannister

S. H. R.

Brooke

E. A.

Rochefort

J. A. P.

Turner

J. R.

(1991). Human factors in the CF-18 pilot environment. Defense and Civil Institute of Environmental Medicine. Report number 91-11.

14.

Dror

I. E.

Kosslyn

S. M.

Waag

W. L.

(1993). Visual-spatial abilities of pilots. Journal of Applied Psychology, 78(5), 763–773. https://doi.org/10.1037/0021-9010.78.5.763

15.

Durnford

S. J.

(1992). Disorientation and flight safety: A survey of UK Army aircrew. In Proceedings of the advisory group for aerospace research and development. Aircraft accidents: Trends in aerospace medical investigation techniques. Neuilly-Sur-Seine, France. AGARD-CP-532, 32-1-14.

16.

Egan

D. E.

(1978). Characterizing spatial ability: Different mental processes reflected in accuracy and in latency scores (Research report no. 1250). Naval Aerospace Medical Research Laboratory.

17.

Flight Safety Foundation . (2004). Nonadherence to approach procedure cited in Falcon 20 CFIT in Greenland. Accident Prevention, 61(1), 1–5.

18.

Gibb

Schvaneveldt

Gray

(2008). Visual misperception in aviation: Glide path performance in a black hole environment. Human Factors, 50(4), 699–711. https://doi.org/10.1518/001872008X288619

19.

Gibb

R. W.

(2007). Visual spatial disorientation: Revisiting the black hole illusion. Aviation Space & Environmental Medicine, 78(8), 801–808.17760289

20.

Gibb

R. W.

Olson

(2008). Classification of air force aviation accidents: Mishap trends and prevention. The International Journal of Aviation Psychology, 18(4), 305–325. https://doi.org/10.1080/10508410802346913

21.

Gillingham

Previc

(1993). Spatial orientation in flight (AL-TR-1993-0022) (p. 138). Armstrong Lab, Brooks Air Force Bace, Air Force Materiel Command.

22.

Gillingham

K. K.

(1992). The spatial disorientation problem in the United States Air Force. Journal of Vestibular Research: Equilibrium & Orientation, 2(4), 297–306.

23.

Harris

Hirsh-Pasek

Newcombe

N. S.

(2013). Understanding spatial transformations: Similarities and differences between mental rotation and mental folding. Cognitive Processing, 14(2), 105–115. https://doi.org/10.1007/s10339-013-0544-6

24.

Holm

(1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6(2), 65–70.

25.

Holmes

S. R.

Bunting

Brown

D. L.

Hiatt

K. L.

Braithwaite

M. G.

Harrigan

M. J.

(2003). Survey of spatial disorientation in military pilots and navigators. Aviation Space & Environmental Medicine, 74(9), 957–965. 14503674.

26.

IATA . (2021). Safety report: 2020 edition. International Air Transport Association. Available at: https://libraryonline.erau.edu/online-full-text/iata-safety-reports/IATA-Safety-Report-2020.pdf

27.

Jakicic

Boyer

Francis

(2022). Investigating a computational explanation of the black hole illusion. The International Journal of Aerospace Psychology, 32(4), 203–226. https://doi.org/10.1080/24721840.2022.2084096

28.

JASP Team . (2024). JASP (version 0.19.0). [Computer software]. Retrieved from. https://jasp-765stats.org/

29.

Kelly

Efthymiou

(2019). An analysis of human factors in fifty controlled flight into terrain aviation accidents from 2007 to 2017. Journal of Safety Research, 69(1), 155–165. https://doi.org/10.1016/j.jsr.2019.03.009

30.

Kheloui

Brouillard

Rossi

Marin

M. F.

Mendrek

Paquette

Juster

R. P.

(2021). Exploring the sex and gender correlates of cognitive sex differences. Acta Psychologica, 221(1), 103452. https://doi.org/10.1016/j.actpsy.2021.103452

31.

Kim

Palmisano

S. A.

Ash

Allison

R. S.

(2010). Pilot gaze and glideslope control. ACM Transactions on Applied Perception, 7(3), 1–18. https://doi.org/10.1145/1773965.1773969

32.

Kirkham

Collins

Grape

Simpson

Wallace

(1978). Spatial disorientation in general aviation accidents. Aviation Space & Environmental Medicine, 49(9), 1080–1086.

33.

Kraft

C. L.

(1978). A psychophysical contribution to air safety: Simulator studies of visual illusions in night visual approaches. In Pick

H. L.

Leibowitz

H. W.

Singer

J. E.

Steinschneider

Stevenson

H. W.

(Eds.), Psychology: From research to practice (pp. 363–385). Plenum Press.

34.

Kuznetsova

Brockhoff

P. B.

Christensen

R. H. B.

(2017). lmerTest package: Tests in linear mixed-effects models. Journal of Statistical Software, 82(13), 1–26. https://doi.org/10.18637/jss.v082.i13

35.

Lewis

M. F.

Mertens

H. W.

(1979). Pilot performance during simulated approaches and landings made with various computer-generated visual Glidepath indicators. Aviation Space & Environmental Medicine, 50(10), 991–1002.

36.

Lajoie

(2021). Predicting aviation training performance with multimodal affective inferences. International Journal of Training and Development, 25(3), 301–315. https://doi.org/10.1111/ijtd.12232

37.

Lyons

T. J.

Ercoline

O’Toole

Grayson

(2006). Aircraft and related factors in crashes involving spatial disorientation: 15 years of U.S. air force data. Aviation Space & Environmental Medicine, 77(7), 720–723.16856357

38.

Matthews

R. S. J.

Previc

Bunting

(2002). USAF spatial disorientation survey. [Paper presented]. Research and Technology Organization and Human Factors and Medicine Symposium on Spatial Disorientation in Military Vehicles: Causes, Consequences and Cures, La Coruna, Spain. (HFM-085, RTO-MP-086).

39.

Mertens

H. W.

Lewis

M. F.

(1982). Effect of different runway sizes on pilot performance during simulated night landing approaches. Aviation Space & Environmental Medicine, 53(5), 463–471.

40.

Moser

(1969). Spatial disorientation as a factor in accidents in an operational command. Aerospace Medicine, 40(2), 174–176.

41.

National Transportation Safety Board . (1997). Controlled flight into terrain, Korean air flight 801, Boeing 747-300, HL7468. National Transportation Safety Board. (Aircraft Accident Report NTSB/AAR-99/02).

42.

National Transportation Safety Board . (1989). Accident investigation report: Pan American world airways, Inc., Boeing 707–321B, N454PA. U.S. Government Printing Office. (NTSB-AAR-74-01).

43.

National Transportation Safety Board . (1998). Safety recommendation (Report no. A-98-87). National Transportation Safety Board.

44.

National Transportation Safety Board . (2002). Collision with trees on final approach: Federal express flight 1478, Boeing 727-232, N497FE. National Transportation Safety Board. (Aircraft accident report NTSB/AAR-02/03).

45.

Patterson

F. R.

Williams

H. P.

Folga

R. V.

Arnold

R. D.

(2021). Aviator black hole illusion: Validated training countermeasures for newly identified causal factors (Report no. A1159041). Wright-Patterson Air Force Base: Naval Medical Research Unit.

46.

Pennings

H. J. M.

Oprins

E. A. P. B.

Wittenberg

Houben

M. M. J.

Groen

E. L.

(2020). Spatial disorientation survey among military pilots. Aerospace Medicine and Human Performance, 91(1), 4–10. https://doi.org/10.3357/AMHP.5446.2020

47.

Perrone

J. A.

(1984). Visual slant misperception and the ‘black-hole’ landing situation. Aviation Space & Environmental Medicine, 55(11), 1020–1025.

48.

Peters

Laeng

Latham

Jackson

Zaiyouna

Richardson

(1995). A Redrawn Vandenberg & Kuse mental rotations test: Different versions and factors that affect performance. Brain and Cognition, 28(1), 39–58. https://doi.org/10.1006/brcg.1995.1032

49.

Poisson

R. J.

Miller

M. E.

(2014). Spatial disorientation mishap trends in the U.S. Air Force 1993-2013. Aviation Space & Environmental Medicine, 85(9), 919–924. https://doi.org/10.3357/ASEM.3971.2014

50.

Previc

F. H.

(2004). Chapter 7: Visual illusions in flight. In Previc

F. H.

Ercoline

W. R.

(Eds.), Spatial disorientation in aviation, progress in astronautics and aeronautics series (p. 203). American Institute of Aeronautics and Astronautics, Inc.

51.

Rehfeld

(2006). The impact of mental transformation training across levels of automation on spatial awareness in human-robot interaction (p. 830). Electronic Theses and Dissertations.

52.

Robinson

F. E.

Williams

Horning

Biggs

A. T.

(2020). A comparative evaluation of hypotheses to explain the black hole illusion. The International Journal of Aerospace Psychology, 30(1-2), 54–68. https://doi.org/10.1080/24721840.2020.1719010

53.

Roscoe

S. N.

(1979). When day is done and shadows fall, we miss the airport most of all. Human Factors, 21(6), 721–731. https://doi.org/10.1177/001872087912210609

54.

Rosseel

(2012). Lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36. https://doi.org/10.18637/jss.v048.i02

55.

Sipes

W. E.

Lessard

C. S.

(2000). A spatial disorientation survey of experienced instructor pilots. IEEE Engineering in Medicine and Biology Magazine: The Quarterly Magazine of the Engineering in Medicine & Biology Society, 19(2), 35–42. https://doi.org/10.1109/51.827403

56.

Takada

Hisada

Kuwada

Sakai

Akamatsu

(2009). Survey of severe spatial disorientation episodes in Japan air self-defense force fighter pilots showing increased severity in night flight. Military Medicine, 174(6), 626–630. https://doi.org/10.7205/milmed-d-01-6308

57.

Teifer

Sikora

Paraskevas

(2023). Black hole illusion in aviation – A simulator experiment to examine predominant criteria in a real–life environment. In Praetorius

Sellberg

Patriarca

(Eds.), Human factors in transportation. AHFE (2023) international conference. AHFE open access (p. 95). AHFE International. https://doi.org/10.54941/ahfe1003849

58.

Tingley

Yamamoto

Hirose

Keele

Imai

(2014). Mediation: R package for causal mediation analysis. Journal of Statistical Software, 59(5), 1–38. https://doi.org/10.18637/jss.v059.i05. https://www.jstatsoft.org/v59/i05/

59.

Cheng

Hsin

Huang

Chiang

Lai

(2021). Analysis of in-flight spatial disorientation among military pilots in Taiwan. Journal of Medical Sciences, 41(1), 22–28. https://doi.org/10.4103/jmedsci.jmedsci_94_20

60.

Vandenberg

S. G.

Kuse

A. R.

(1978). Mental rotations, a group test of three-dimensional spatial visualization. Perceptual and Motor Skills, 47(2), 599–601. https://doi.org/10.2466/pms.1978.47.2.599

61.

Verde

Angelino

Piccolo

Carrozzo

Bottiglieri

Lugli

Piccardi

Nori

(2018). Spatial orientation and directional judgments in pilots vs. nonpilots. Aerospace Medicine and Human Performance, 89(10), 857–862. https://doi.org/10.3357/AMHP.5023.2018

62.

Verde

Piccardi

Bianchini

Trivelloni

Guariglia

Tomao

(2013). Gender effects on mental rotation in pilots vs. nonpilots. Aviation Space & Environmental Medicine, 84(7), 726–729. https://doi.org/10.3357/ASEM.3466.2013

63.

Voyer

Bryden

M. P.

(1995). Magnitude of sex differences in spatial abilities: A meta-analysis and consideration of critical variables. Psychological Bulletin, 117(2), 250–270. https://doi.org/10.1037/0033-2909.117.2.250

Visual Illusions in Aviation: Simulating the Black Hole Phenomenon

Abstract

Objective

Background

Method

Results

Conclusion

Application

Keywords

Introduction

Methods

Participants

Apparatus

Procedure

Analysis

Results

Gender

Mental Rotation

Discussion

Key Points

Footnotes

Acknowledgments

Declaration of Conflicting Interests

Funding

Appendix

Author Biographies

References