Unveiling Virtual Reality Overheads and Their Potential Impact on Flightcrew Training

Abstract

Virtual reality (VR) technology offers immersive training environments for pilots, enhancing safety, cost-effectiveness, and access to varied scenarios. However, significant overheads, including technical challenges, technology learning curve, costs, and simulation sickness, can hinder VR adoption for flightcrew training. This paper explores these potential roadblocks through a combined approach. First, we conduct a review of recent existing literature. Second, we employ thematic analysis of qualitative data gathered from commercial aviation pilots. The analysis resulted in a thematic map that visually captured the key challenges and opportunities associated with VR overheads. Finally, we leverage these insights to formulate research propositions that can guide airlines, collegiate aviation institutions, and aviation training organizations on effectively integrating VR into the flightcrew training pipeline.

Introduction

Flightcrew training is a critical aspect of aviation safety, and there is a rich history of human factors research in pilot training. Flightcrew training has traditionally relied on physical simulators—including, flight training devices, flight simulation training devices, and full flight simulators—which can be expensive to procure, maintain, and operate. These simulators may have limited scalability and flexibility in accommodating various training scenarios. Virtual reality (VR) technologies have been considered as a potential supplement to these devices, promising immersive and interactive simulations that may enhance flightcrew training experiences (Duruaku et al., 2023; Nguyen et al., 2023). However, concerns persist (Jerald, 2015) regarding not only the fidelity of VR devices for supplemental training, but also issues such as technical (Ali & Nasser, 2017; Gandhi & Patel, 2018), cost (Farra et al., 2019), technology learning curve (Kavanagh et al., 2017), and simulation sickness (Chang et al., 2020; Rebenitsch & Owen, 2016). Using VR, whether in the capacity of a developer, provider or end user, demands a substantial amount of specialized expertise. Even seemingly straightforward tasks—such as donning and calibrating a headset, initiating a simulation, and interacting with the hardware and software—may necessitate that users have a minimum proficiency in prerequisite knowledge, skill, and practical experience distinct from those employed in the use of traditional flight simulation devices. These issues can impact the methods employed and hinder the adoption of VR in aviation training.

In line with suggestions proposed by Duruaku et al. (2023), this study carefully considered a set of challenges for the use of VR which—though not directly moderating training outcomes, may be necessary for supporting successful implementation, adoption, and user experience with VR. These considerations include limitations (e.g., accessibility), user experience challenges (e.g., technology learning curve), cost-related barriers (e.g., assets and personnel), and the potential health implications (physical, physiological, and psychological) associated with acute prolonged VR exposure. We discuss how these factors, collectively referred to as “VR Overhead,” may influence the implementation of VR in flightcrew training.

Methodology

This study aimed to identify the “VR Overhead” associated with implementing VR in flightcrew training contexts. While prior research has explored the challenges of VR training in high-risk industries, including aviation, this work used a distinct approach combining three qualitative methods. First, we conducted a review of the literature using keywords such as “virtual reality,” “training,” “flight simulation,” “pilot training,” “simulation sickness,” and “technology learning curve,” and included articles related to research specific to airlines and collegiate pilots, VR simulation, the use of head-mounted displays (HMDs), and the potential impact on pilot training.

We complemented this with a qualitative analysis of firsthand insights from 30 active airline pilots obtained during an empirical study conducted in collaboration with a major U.S. air carrier (Nguyen et al., 2023). This study involved airline pilots using either (a) a VR HMD device, or (b) a PC desktop monitor or tablet, to interact with demonstrations of an external preflight inspection task (Sonnenfeld et al., 2023) or an initial preflight setup trainer (Reaction Simulation, 2021). As part of that study, we elicited participants’ feedback during semi-structured interviews on topics including usability and instrumentality (Nguyen et al., 2023). For our current work, we analyzed participants’ perspectives on the demonstrations they’d interacted with, applying a qualitative content analysis (QCA) framework (Elo & Kyngäs, 2008) to identify VR Overheads grounded in participants’ feedback. Then, we applied a descriptive thematic analysis approach (Braun & Clarke, 2006) to categorize recurrent themes pertaining to the concept of VR Overheads and used these themes to inform the creation of a thematic map (Braun & Clarke, 2006) to capture the key VR Overheads for flightcrew training.

Through this work, we sought to answer the following research questions: (i) What technical challenges may constrain the implementation of VR systems in flightcrew training? (ii) How might the “technology learning curve” for VR affect training? (iii) How do the upfront and ongoing costs associated with VR technology influence the adoption of VR in flightcrew training programs? Through this work, we seek to offer practical propositions for airlines, collegiate aviation institutions, and aviation training organizations considering the integration of VR for flightcrew relevant training.

Findings

Our qualitative analysis, supported by our narrative review of existing literature, indicated that several factors contributed to the concept of “VR Overhead” in flightcrew training contexts. The themes centered around realism, training implementation, and the importance of tutorials in bridging the familiarity gap. Other themes included the role of individual differences in technology familiarity and adaptation and the roles and responsibilities of instructors in VR training, such as ensuring the accuracy and effectiveness of the training environment, troubleshoot technical issues, and assessing pilot performance to ensure that each pilot acquires the necessary skills and confidence to operate aircraft safely and efficiently

Pilots tended to perceive a steeper technology learning curve when using VR than the PC/mobile devices. Discussion of the plausibility of VR systems being introduced in flightcrew training contexts often included notes on the challenges posed by the technology learning curve associated with VR adoption. Participants expressed the potential need for extensive support and tutorials to effectively navigate and use VR training simulations. Furthermore, analyses uncovered various technical challenges encountered during the use of VR systems for flightcrew training, including hardware malfunctions, software glitches, and compatibility issues with existing training infrastructure. We describe VR and its potential capabilities for flightcrew training, then we discuss the specific VR Overheads that may be anticipated in implementing these technologies for flightcrew training, as well as a series of propositions to guide practitioners in implementing these technologies in training programs. Overall, the findings from the content analyses provided valuable insights into the potential barriers and considerations associated with the integration of VR technology into flightcrew training contexts.

Virtual Reality for Pilot Training

VR has been accepted as a transformative tool for training in high-risk industries including aviation because it provides a safe, interactive, accessible, and immersive environment for flightcrews to practice essential skills relevant to flying. VR simulations can replicate different flight scenarios, from maneuvers to emergency procedures, adverse weather conditions to system failures and complex airspace environments allowing pilots to gain experience without the risks associated with real-world training on an aircraft (Sonnenfeld, Nguyen, Boesser, Duruaku, et al., 2021; Sonnenfeld, Nguyen, Boesser, & Jentsch, 2021; Thomas et al., 2023). VR can be a valuable pre-flight training tool, allowing pilots to familiarize themselves with the aircraft and procedures before entering the simulator or a real aircraft (Fussell & Truong, 2020). VR can also be used for post-flight debriefing and after-action reviews, allowing flight instructors to revisit and review training and provide specific feedback (Kleygrewe et al., 2024). This feedback loop may help pilots identify and correct errors more efficiently than via traditional methods.

Many air carriers and collegiate aviation programs have become increasingly interested in using VR for flight training. Regulators worldwide have expressed interest in the potential of VR for pilot training but has yet to establish specific regulations or guidelines for its use. However, they have collaborated with industry stakeholders to develop best practices for VR-based training programs. Despite the benefits, it was crucial to identify the important considerations and challenges that must be addressed to ensure the efficient use of VR within the flightcrew training pipeline.

VR Overheads

In this section, we discuss VR overheads including technical challenges, the technological learning curve, costs, and simulation sickness and their potential impact on flightcrew training.

Technical Challenges

Technical challenges are significant overheads in the integration of virtual reality technology for flightcrew training, potentially impacting training. These challenges include hardware limitations, software development challenges, and integration issues (Xie et al., 2021), all of which are required and can influence the overall success of VR-based training programs. To start with, hardware may be a challenge and limiting factor to successful VR implementation because high-quality VR headsets and other peripherals like motion tracking sensors and sometimes controllers and haptic feedback gloves may be essential for delivering immersive and realistic training experiences (Alalwan et al., 2020). According to the airline pilots, the accessibility and distribution of the VR assets may be difficult for airlines with many pilots (see Figure 1), particularly when additional technical personnel may be needed to support their use and maintenance.

Figure 1.

VR overhead: technical limitations.

Also, limited or clunky VR controls can hinder a pilot’s ability to interact with the virtual environment. Imprecise hand-tracking might make it difficult to practice manipulating aircraft instruments effectively, while wand controllers may reduce the functional fidelity of such interaction. Additionally, software development may present another hurdle (Alalwan et al., 2020). In cases where physical flight panels or controls are not used to support mixed reality configurations, creating immersive VR environments may demand elaborate software development, involving detailed 3D modeling, data, programming, realistic simulations, and intuitive user interfaces (Guthridge, 2022). This process can be complex and resource-intensive, requiring skilled teams and posing significant challenges in software stability (i.e., the ability of the software to perform consistently and reliably without crashing or producing errors) and cross-platform compatibility. With multiple consumer VR hardware options, ensuring cross-platform compatibility and accessibility is an obstacle for VR software development.

Developing VR software that accurately simulates aircraft systems, flight dynamics, and environmental conditions also demands expertise in both aviation principles and VR technology (Guthridge, 2022), particularly considering that the effectiveness of VR systems depends on their level of fidelity (Champney at al., 2017). Software issues could result in defective training scenarios because when the system keeps having an error or crashing, the aim of the technology may be defeated, and this may impact usability and learning. Furthermore, Integrating VR systems with existing technology infrastructures poses challenges due to compatibility issues. Scalability is also critical as VR applications must handle increasing numbers of users and complex scenarios without degradation in performance or quality. From the above, we propose that technical limitations can negatively impact flightcrew training.

Proposition I

The successful implementation of VR systems in flightcrew training may be hindered by technical challenges such as hardware limitations, software compatibility issues, and the complexity of integrating VR technology into existing training frameworks.

Technology Learning Curve

This VR overhead refers to the time, resources and effort required for instructors and flightcrews to become proficient with new VR technology, affecting various aspects of the training process. Since VR introduces an environment with new visual, auditory, and haptic information, the users may require additional time to learn to use the technology effectively. For flightcrew training, the technology learning curve compounds that are required to achieve proficiency in flight skills by requiring trainees to first learn how to operate and interact with the VR system itself. Donning the headset and peripherals, using controllers, and navigating the user interface can be a hurdle for instructors and flightcrews who are unfamiliar with VR technology, thus making initial VR experiences cognitively demanding (Reddy et al., 2022). Air carrier pilots also found the technology learning curve to be steeper with immersive VR compared to the non-immersive approach (see Figure 2).

Figure 2.

VR overhead: design constraints.

Instructors, and supporting designers/developers, also face a technology learning curve as they must adapt their instructional strategies to the capabilities and limitations of VR technologies (Alalwan et al., 2020). They need to be trained not just in the use of the technology but also in how to effectively integrate VR into their existing training curricula, and vice versa (Alalwan et al., 2020). Developing this expertise can be time-consuming and may necessitate collaboration with subject-matter experts in other domains (e.g., human factors, instructional design, learning engineering). In line with current literature and perspectives from the airline pilots, the familiarity with, adaptability to and acceptance of VR technology can vary widely among trainees (Thomas et al., 2023).

Usability, fidelity, and instructional design should also be considered in VR training environments as they can also introduce challenges that may affect VR implementation. Poor usability can steepen the learning curve, leading to longer adaptation, cognitive overload, and frustration. Fidelity on the other hand influences how intuitive a VR system is. VR systems with low fidelity may be more complicated to learn because flightcrews would have to adapt to the variation between the VR simulation and their real-world experiences (Lowell & Tagare, 2023). Finally, the instructional design may influence the learning process because it determines how effective the content in VR facilitates or inhibits learning (Lowell & Tagare, 2023; Mikropoulos & Natsis, 2011). Designing content in VR requires an understanding of learning principles and VR technology and this may be a learning curve for instructional designers who are new to VR

From the above, we propose that VR-based training has a steeper initial technology learning curve compared to conventional training methods for flightcrews with limited prior relevant experience with VR or associated gaming technologies. These issues may be exacerbated by familiarity, adaptation, and task complexity.

Proposition II

The learning curves associated with VR-based training differ from conventional methods due to factors such as familiarity with technology, cognitive adaptation to immersive environments, and the effectiveness of the design in VR simulations.

Costs

The cost of VR technology can be a significant overhead for pilot training programs. VR systems, including headsets, computing hardware, and other peripherals can be expensive to acquire and maintain (Farra et al., 2019). Additionally, the development of high-quality VR training scenarios and content can require significant investment in software development (Gandhi & Patel, 2018). These costs could be the initial investment, ongoing maintenance, and the training and implementation costs for personnel. The cost of purchasing VR equipment, which includes head-mounted devices, motion tracking devices, and the necessary computing hardware to run the simulations, can be significant (Adebowale & Agumba, 2024). The equipment needs to not only provide high resolution but also be durable enough to withstand frequent use in a training environment. Developing and maintaining VR training software can be costly. Licenses for existing VR simulation software can be expensive, and creating custom VR experiences requires specialized programming expertise. VR equipment requires ongoing maintenance and potential upgrades to keep pace with technological advancements.

According to the airline pilots, these maintenance and upgrade costs contribute to the total ownership cost over the VR equipment’s lifespan. This also adds to the total cost over time. Instructors and trainees may need to be trained in how to effectively use VR, hence increasing personnel cost (see Figure 3). Additionally, the space required to safely operate VR equipment can lead to higher facility costs.

Figure 3.

VR overhead: costs.

The impact of VR cost on pilot training can be huge ranging from $10,000 to $100,000 (Mohammadi & Garrido Martins, 2023). Smaller regional airlines or those with limited budgets may struggle to afford the upfront and ongoing costs of VR, which can limit their ability to provide trainees with the benefits of immersive training. Furthermore, the cost of VR technology can also impact the scalability of pilot training programs. As the number of trainees increases, the overall cost of VR hardware and software can quickly escalate, making it challenging to provide VR-based training to many pilots. This can limit the accessibility of VR-based training and potentially slow the adoption of this technology in the aviation industry. Based on the above, we propose costs will influence the adoption and effectiveness of VR in flight crew training programs.

Proposition III

The adoption and effectiveness of VR in flightcrew training programs are influenced by the upfront costs of acquiring VR technology, ongoing maintenance expenses, and the perceived return on investment in terms of training outcomes.

Simulation Sickness

Simulation sickness is a significant issue in VR-based pilot training that can impact the effectiveness and efficiency of the training (Marron et al., 2024; Oh & Son, 2022). It refers to the undesirable symptoms that can arise from exposure to VR environments, including dizziness, nausea, fatigue, and disorientation. These symptoms can be severe enough to cause trainees to discontinue using the VR system (Reddy et al., 2022). VR simulation sickness is primarily caused by latency issues where there is a delay between the user’s actions and the visual feedback, low refresh rates of displays, and discrepancies between visual motion cues and the lack of corresponding vestibular cues (sensation of movement in the inner ear). These factors can lead to symptoms such as nausea, dizziness, headaches, and disorientation (Kourtesis et al., 2024). The symptoms of simulation sickness can interfere with the learning process, leading to decreased training efficiency. Pilots experiencing these symptoms may require more frequent breaks, shortening actual training time. In severe cases, some trainees may not be able to continue using VR at all, especially for certain high-intensity training scenarios, which are essential for preparing pilots for complex real-world situations. Simulation sickness can lead to changes in training schedules and adaptations in the training program to accommodate individual susceptibility. Instructors might need to modify VR sessions, either by reducing their duration or by altering the simulation settings to minimize discomfort. We propose that there will be a negative correlation between the severity of simulation sickness experienced by flightcrew trainees during VR training and their successful implementation.

Proposition IV

Simulation sickness among flightcrew members will impact successful implementation of VR-based flightcrew training

Conclusions

This study explored the practical hurdles of incorporating virtual reality (VR) into flightcrew training, offering valuable insights for developers, educators, and aviation organizations. By analyzing the “overheads” linked to VR adoption—encompassing technical constraints, financial aspects, user learning curves, and simulation sickness—we can guide for optimizing VR applications and ensuring seamless integration into existing training programs.

Beyond practical implications, this study contributes to the broader comprehension of VR technology adoption and the reasons for high VR overhead. By emphasizing the interconnectedness of these overhead elements, we gain insights into the mechanisms influencing user adaptation and training outcomes in VR environments. This enriches existing theoretical frameworks in human-computer interaction, educational technology, and training methodologies.

Ultimately, this research offers a roadmap for maximizing VR’s potential in flightcrew training. By addressing the identified overheads, developers can develop more accessible and efficient VR applications, while educators and organizations can utilize this technology to revolutionize training methods and bolster flight safety. The future of VR in flight training holds promise, and this study empowers stakeholders to navigate its complexities and unlock its transformative capabilities.