Augmented Reality in Extra-Vehicular Activities: Optimizing Alert Detection and Cognitive Workload

Abstract

This study investigates the impact of augmented reality (AR) system complexity (active vs. passive) and lighting conditions (normal, dark, glare) on task performance, cognitive workload, and alert detection during simulated extravehicular activities (EVAs). Fifty-eight undergraduate students participated, using AR systems to complete tasks. Key metrics included task completion rate, completion time, alert detection, and cognitive workload assessed by the NASA Task Load Index (NASA-TLX). Results showed no significant differences in task performance or cognitive workload across lighting conditions and AR types. Notably, less than 15% of participants noticed any alerts, suggesting AR alerts may lack sufficient salience. The study’s lack of significant findings is attributed to the small sample size and potential insensitivity of measures. Future research should involve larger samples and refined methodologies to better understand AR’s impact on astronaut performance and safety during EVAs.

Introduction

In the context of extravehicular activities (EVAs) on Mars or the moon, the ability to receive real-time, task-relevant information is crucial due to limited communication with the ground team. Leveraging augmented reality (AR) technology within astronauts’ helmets presents a promising solution for enhancing task performance and reducing cognitive workload. AR systems can provide task instructions, wayfinding, and communication support, which are vital for the success and safety of EVAs. However, challenging lighting conditions, such as dark environments and glare, can impede task completion and increase cognitive demands on astronauts. Further, presentation of information may interfere with alert visibility.

Inattentional blindness, a well-documented phenomenon where individuals fail to notice stimuli due to high cognitive engagement, is a critical consideration for AR systems. For example, when drivers engage in more complex conversations, they tend to fail to notice pedestrians or that the leading vehicle has slowed down (McKnight, 1993). Similarly, in a flight deck, inattentional blindness tends to occur as a result of higher workload and greater cognitive demands of the current task (White, 2022). Astronauts, during EVAs, face significant cognitive workloads, making it essential to design AR systems that optimize alert detection and task performance without overwhelming the user.

Previous research has explored the impact of AR heads-up display complexity on the detection of stimuli. One study found that more complex displays led to higher detection errors (Ruskin, 2021). However, the comparison between active AR assistants, providing just-in-time instructions, and passive assistants, presenting a full checklist of steps, remains underexplored. Active assistants may enhance task performance but could increase the risk of missing alerts due to the dynamic nature of information presentation. In contrast, passive assistants, despite their higher complexity, may offer more consistent alert visibility.

Previous research has also showed that lighting can influence the usability of an AR system. In one study, the ability of participants to use the AR system was greatly impaired by a very bright environment (Kim & Lee, 2020). When the environment is too bright, the AR overlay is perceived only faintly, making it hard to interact with the system. On the other hand, an AR system can reduce the cognitive workload of persons completing a task in dim lighting (Kumar et al., 2023). Thus, the usefulness of an AR system may be dependent on the lighting environment it is used in. For astronauts conducting EVAs, lighting conditions will vary greatly. It is important to understand how lighting may influence cognitive workload and alert detections to optimize an AR system.

This research aims to address the gaps in understanding the interaction between AR systems, lighting conditions, and astronaut performance during EVAs. By examining the effects of dark and glare conditions on task performance and cognitive workload and evaluating the comparative effectiveness of passive and active AR systems, we seek to optimize AR design for space exploration missions. The insights gained will inform the development of AR technologies that enhance task performance, reduce cognitive workload, and ultimately improve the safety and success of future space missions.

Method

Participants

Fifty-eight undergraduate students participated in this study. Participants were recruited through the undergraduate research pool at Rice University and were compensated with course credit.

Materials

The study utilized Unity 2022 and the Mixed Reality Toolkit plugin to design augmented reality (AR) scenes, which were projected onto a Microsoft HoloLens II device via Microsoft Visual Studio. Two distinct AR scenes were created: an active scene and a passive scene. Both scenes featured a heads-up display (HUD) showing static information relevant to astronauts, such as oxygen levels, coordinates, and the current task.

In the active scene, task instructions were presented sequentially, requiring participants to advance to the next step upon completion. In contrast, the passive scene displayed all task steps simultaneously in sequential order. Alerts were randomly integrated into both scenes, appearing in one of the four quadrants of the HUD. These alerts prompted users to perform specific actions, such as writing a number on a sheet of paper, to indicate acknowledgment. Each alert remained visible for 5 s, with a total of four alerts presented throughout the task.

Procedures

Participants were randomly assigned to one of the three lighting conditions: normal, dark, or glare (Figure 1) and one of two tasks: active or passive. They were instructed to follow the HUD instructions to construct a simple origami boat. Key metrics recorded included completion rate, completion time, and the number of correctly identified alerts.

Figure 1.

Experimental set-up for varying lighting conditions: (a) glare, (b) normal, and (c) dark.

Cognitive workload was assessed using the NASA Task Load Index (NASA-TLX). This setup allowed for the evaluation of performance across different lighting conditions and AR scene types.

Results

The study aimed to investigate the impact of different lighting conditions (normal, dark, glare) and scenes (active, passive) on task performance and cognitive workload during the construction of a simple origami boat. Key metrics recorded included completion rate, completion time, the number of correctly identified alerts, and cognitive workload as assessed by the NASA Task Load Index (NASA-TLX).

From the 58 participants who completed the task, 12% (N = 7) identified and responded to the presented alerts. 0% of participants noticed at least 2 alerts (see Figure 2). 4 participants responded to the alert presented in the bottom right corner of the visual display, 1to the top left corner, and 2 responded to the bottom left and top right corners (see Figure 3). Alerts were noticed across lighting conditions and scenes.

Figure 2.

Number of alerts noticed by participants.

Figure 3.

Number of alerts noticed per location.

An ANOVA was used to assess the number of correctly identified alerts across different lighting conditions and scene types. The analysis revealed no significant differences in alert identification accuracy based on lighting conditions or scene types. There were also no significant interaction effects between lighting conditions and the active and passive conditions.

The NASA-TLX scores were analyzed to determine the cognitive workload experienced by participants under different lighting conditions and scene types. The ANOVA results indicated that there were no significant differences in cognitive workload scores between the various lighting conditions or between the active and passive scenes. No significant interaction effects were found between lighting conditions and scene types.

Discussion

The present study aimed to evaluate the effects of lighting conditions and AR scene types on task performance and cognitive workload during simulated EVAs. No significant differences were found across the conditions tested. Importantly, less than 15% of participants noticed any alerts, regardless of our manipulations. This may suggest that visual alerts presented in AR are not salient enough to attract attention. Due to the very small sample of participants who noticed alerts, it was not feasible to analyze this group versus the group who did not notice any alerts.

The lack of significant findings may be attributed to the small sample size, which limits the statistical power of the study. Additionally, the measures employed may not have been sensitive enough to detect subtle differences in performance and workload. The variability inherent in the tasks performed could have also contributed to the non-significant results.

Implications for Future Research

Future research should consider larger sample sizes to enhance statistical power and explore alternative methodologies that may be more sensitive to detecting differences in performance and cognitive workload. Investigating other factors, such as the duration of exposure to AR systems or different types of tasks, may also yield more significant results.

Although no significant differences were observed, the study provides valuable insights into the usability of AR systems under varying lighting conditions. These findings contribute to the broader understanding of AR technology’s potential and limitations in space exploration contexts. This information is crucial for the ongoing development and optimization of AR systems to support astronauts during EVAs, ultimately enhancing the safety and success of future space missions.