Dynamic characteristics and classification of mental workload in excavator operators based on multimodal physiological feature fusion in a VR environment

Abstract

To address the insufficient systematic analysis of excavator operators’ mental workload and the underexplored EEG phase information, this research introduces a method that utilizes a VR-based experiment and multimodal physiological feature fusion to investigate the dynamic characteristics and classification of mental workload. By designing a high-fidelity VR-based excavator operation experiment, EEG signals, eye-tracking data and the subjective mental workload data are collected. In addition, EEG phase synchronization features are extracted using the Weighted Phase Lag Index to comprehensively reveal the dynamic patterns of mental workload. Subsequently, multimodal physiological features were integrated with machine learning algorithms to propose a three-category assessment model for excavator operators’ mental workload. The research comprehensively reveals the dynamic changes of mental workload. Subsequently, multimodal physiological features are integrated with machine learning algorithms to propose a three-classification evaluation model for excavator operators’ mental workload. Results showed that, as the experimental tasks progressed, the Theta power in the F3 and F8 channels, the (α + θ)/β ratio in the T7 channel, and the Frontal-Central Beta WPLI value all gradually increased, while the average fixation duration gradually decreased and the blink count gradually increased. The SVM-RBF classification model using multimodal feature fusion performed optimally in the three-classification task for mental workload, achieving a classification accuracy rate of 86.9% and a macro-average AUC of 0.954, outperforming single-modal classification models. These findings provide a theoretical basis for applying multimodal physiological feature fusion to mental workload classification in excavator operators.