Investigating Automation Level and Automation Reliability in Hybrid Automation Systems

Abstract

As autonomous systems become more prevalent, understanding how humans interact with varying levels and reliability of automation is critical. This study investigates the effects of automation level and automation reliability on performance, mental workload, and situation awareness in hybrid automation systems. Using OpenMATB, 45 participants completed a multitasking simulation under different automation levels (manual to fully automated) and reliability conditions (50%, 70%, 99%). Results indicated significant effects of both automation level and reliability on task performance, situational awareness, and heart rate variability, with higher automation and reliability generally improving performance but lowering physiological markers of workload. Participants also showed a tendency to prefer lower automation levels when system reliability was low. Findings highlight the importance of designing adaptive systems that account for user preferences and reliability expectations. Future work may focus on modeling user transition behavior to inform adaptive automation strategies that support performance while maintaining engagement and situational awareness.

Keywords

human-automation interaction cognitive workload adaptive systems

Get full access to this article

View all access options for this article.

References

Calhoun

(2022). Adaptable (not adaptive) automation: Forefront of human–automation teaming. Human Factors: The Journal of the Human Factors and Ergonomics Society, 64(2), 269–277. https://doi.org/10.1177/00187208211037457

Delliaux

Delaforge

Deharo

J. C.

Chaumet

. (2019). Mental workload alters heart rate variability, lowering non-linear dynamics. Frontiers in Physiology, 10, 565. https://doi.org/10.3389/fphys.2019.00565

Endsley

M. R.

(2018). Automation and situation awareness. In Automation and human performance (pp. 163–181). CRC Press. https://www.taylorfrancis.com/chapters/edit/10.1201/9781315137957-8/automation-situation-awareness-mica-endsley

Hyde

Sibi

Johns

Fischer

Sirkin

(2020). Assessing the effects of failure alerts on transitions of control from autonomous driving systems. 2020 IEEE Intelligent Vehicles Symposium (IV), 1956–1963. https://ieeexplore.ieee.org/abstract/document/9304627/

Huang

Pugh

Z. H.

Kim

Nam

C. S.

(2024). Brain dynamics of mental workload in a multitasking context: Evidence from dynamic causal modeling. Computers in Human Behavior, 152, 108043. https://doi.org/10.1016/j.chb.2023.108043

Kaber

D. B.

Endsley

M. R.

(2004). The effects of level of automation and adaptive automation on human performance, situation awareness and workload in a dynamic control task. Theoretical Issues in Ergonomics Science, 5(2), 113–153. https://doi.org/10.1080/1463922021000054335

Novak

V. D.

Hass

Hossain

M. S.

Sowers

A. F.

Clapp

J. D.

(2024). Effects of adaptation accuracy and magnitude in affect-aware difficulty adaptation for the multi-attribute task battery. International Journal of Human-Computer Studies, 183, 103180. https://doi.org/10.1016/j.ijhcs.2023.103180

Pichot

Roche

Celle

Barthelemy

J.-C.

Chouchou

(2016). HRVanalysis: A free software for analyzing cardiac autonomic activity. Frontiers in Physiology, 7, 557.

Spence

Feng

(2010). Video Games and Spatial Cognition. Review of General Psychology, 14(2), 92–104. https://doi.org/10.1037/a0019491