Multimodal Cueing in Attitude Tracking: Predicting Pilot Mental Workload Through Physiological Measurements

Abstract

Objective

To investigate the relationship between variations in physiological signals and mental workload (MWL) during the execution of a helicopter roll-attitude compensatory tracking task.

Background

Current perceptual models in human–machine piloting have been focused on visual and vestibular cues, overlooking somatosensory and auditory inputs and their interactions. This creates a knowledge gap in understanding shared perception strategies for piloting in environments with impaired sensory channels or enhanced secondary cues.

Methods

Fifteen healthy participants performed an attitude-tracking task under eleven cueing modalities combining visual, degraded visual, haptic, and auditory cues. Physiological signals—cardiac activity, respiration, brain activity, skin temperature, and electrodermal activity—were analyzed in relation to self-reported MWL using statistical tests and GLMM (Generalized Linear Mixed Models).

Results

Participants reported low perceived MWL under good visual conditions, with supplementary auditory and haptic cues helping to reduce MWL with degraded or absent visual input. Physiological signals discriminated between MWL levels and multivariate analysis showed that while combined signals revealed an evident explanation of their variance, individual differences underscored the importance of personalized modeling.

Conclusion

MWL assessment through physiological signals validated during a helicopter tracking task demonstrated that multimodal cueing in complex scenarios can reduce cognitive load, leading to potential safety risk mitigation.

Application

This research has the objective of providing a novel approach for safety enhancement and mitigating risks in rotorcraft operations by integrating visual, auditory, and somatosensory cues with physiological-based MWL assessment.