The Dynamics of Attention Across Automated and Manual Driving Modes: A Driving Simulation Study

Abstract

Objective

This study aims to explore the dynamics of driver attention to various zones, including road, central mirror, center stack, and instrument cluster, across different driving modes in AVs.

Background

The integration of automated vehicles (AVs) into transportation systems has introduced critical safety concerns, particularly regarding driver re-engagement during mode transitions. Past crashes underscore the risks when drivers overly rely on automation and highlight the need to understand dynamic attention allocation to support safety during automated driving.

Method

A high-fidelity driving simulation was conducted to examine drivers’ visual attention. Eye-tracking technology was employed to measure fixation duration, fixation count, and time to first fixation (TFF) across distinct driving modes (automated, manual, and transition). These indicators were analyzed to capture both sustained attention (fixation duration/count) and attentional reallocation/latency (time to first fixation), which are the critical aspects of visual attention to different areas of interest (AOIs).

Results

Findings show that drivers’ attention varies significantly across driving modes. In manual mode, attention consistently focuses on the road, while in automated mode, prolonged fixation on center stack was observed. During the handover and takeover phases, attention shifts dynamically between environmental and technological elements.

Conclusion

The study reveals that driver attention allocation is mode dependent. These findings inform the design of center stacks in AVs that align with drivers’ attention patterns. By presenting relevant information according to the driving context, such systems can enhance driver-vehicle interaction, support effective transitions, and improve overall safety.

Application

Systematic analysis of visual attention dynamics across driving modes is gaining prominence, as it informs center stack designs and driver readiness interventions. The generalized linear mixed model (GLMM) findings can be directly applied to the design of center stack or driver training programs to enhance attention and improve safety.

Keywords

attention automated driving eye movement human-automation interaction

Get full access to this article

View all access options for this article.

References

Abbasi

J. A.

Mullins

Ringelstein

Reilhac

Jones

Glavin

(2022). An analysis of driver gaze behaviour at roundabouts. IEEE Transactions on Intelligent Transportation Systems, 23(7), 8715–8724. https://doi.org/10.1109/TITS.2021.3085492

Ahlström

Kircher

Nyström

Wolfe

(2021). Eye tracking in driver attention research—how gaze data interpretations influence what we learn. Frontiers in Neuroergonomics, 2, 778043. https://doi.org/10.3389/fnrgo.2021.778043

Banks

V. A.

Plant

K. L.

Stanton

N. A.

(2018). Driver error or designer error: Using the perceptual cycle model to explore the circumstances surrounding the fatal Tesla crash on 7th May 2016. Safety Science, 108, 278–285. https://doi.org/10.1016/j.ssci.2017.12.023

Bates

Mächler

Bolker

Walker

(2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67, 1–48. https://doi.org/10.18637/jss.v067.i01

Beggiato

Hartwich

Schleinitz

Krems

Othersen

Petermann-Stock

(2015). What would drivers like to know during automated driving? Information needs at different levels of automation. Paper presented at the 7th International Conference. Munich, Germany: Tagung Fahrerassistenz. https://doi.org/10.13140/RG.2.1.2462.6007

Brainard

(2018). News at a glance. Science, 359(6382), 1310–1312. https://doi.org/10.1126/science.359.6382.1310

Chen

Liu

(2019). HMI design for autonomous cars: Investigating on driver’s attention distribution. In Krömker

(Ed.), HCI in mobility, transport, and automotive systems (pp. 92–102). Springer International Publishing. https://doi.org/10.1007/978-3-030-22666-4_7

Chen

Sawaragi

Hiraoka

(2023). Comparing driver reaction and mental workload of visual and auditory take-over request from perspective of driver characteristics and eye-tracking metrics. Transportation Research Part F: Traffic Psychology and Behaviour, 97, 396–410. https://doi.org/10.1016/j.trf.2023.07.012

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed). L. Erlbaum Associates.

10.

Deng

Yan

Qin

Ngo

Manjunath

B. S.

(2020). How do drivers allocate their potential attention? Driving fixation prediction via convolutional neural networks. IEEE Transactions on Intelligent Transportation Systems, 21(5), 2146–2154. https://doi.org/10.1109/TITS.2019.2915540

11.

Ding

Murzello

Cao

Samuel

(2024). Exploring the relationship between drivers’ stationary Gaze entropy and situation awareness in a Level-3 automation driving simulation. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 68(1), 879–884. https://doi.org/10.1177/10711813241275910

12.

Engström

Markkula

Victor

Merat

(2017). Effects of cognitive load on driving performance: The cognitive control hypothesis. Human Factors, 59(5), 734–764. https://doi.org/10.1177/0018720817690639

13.

Eriksson

Petermeijer

S. M.

Zimmermann

de Winter

J. C. F.

Bengler

K. J.

Stanton

N. A.

(2019). Rolling out the red (and green) carpet: Supporting driver decision making in automation-to-manual transitions. IEEE Transactions on Human-Machine Systems, 49(1), 20–31. https://doi.org/10.1109/THMS.2018.2883862

14.

Espié

Auberlet

J. M.

(2007). ARCHISIM: A behavioral multi-actors traffic simulation model for the study of a traffic system including ITS aspects. International Journal of ITS Research, 5(1), 7–16. https://hal.science/hal-00542272v1.

15.

Faul

Erdfelder

Lang

A.-G.

Buchner

(2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175–191. https://doi.org/10.3758/BF03193146

16.

Gaspar

Carney

(2019). The effect of partial automation on driver attention: A naturalistic driving study. Human Factors, 61(8), 1261–1276. https://doi.org/10.1177/0018720819836310

17.

Gold

Körber

Lechner

Bengler

(2016). Taking over control from highly automated vehicles in complex traffic situations: The role of traffic density. Human Factors, 58(4), 642–652. https://doi.org/10.1177/0018720816634226

18.

Guo

Wang

Guan

(2024). Modelling attention allocation and takeover performance in two-stage takeover system via a cognitive computational model: Considering the role of multiple monitoring requests. Ergonomics, 67(11), 1–20. https://doi.org/10.1080/00140139.2024.2340671

19.

Hecht

Kratzert

Bengler

(2020). The effects of a predictive HMI and different transition frequencies on acceptance, workload, usability, and gaze behavior during urban automated driving. Information, 11(2), 73. Article 2. https://doi.org/10.3390/info11020073

20.

Hergeth

Lorenz

Vilimek

Krems

J. F.

(2016). Keep your scanners peeled: Gaze behavior as a measure of automation trust during highly automated driving. Human Factors, 58(3), 509–519. https://doi.org/10.1177/0018720815625744

21.

Lin

Chu

Cheng

L.-L.

(2020). Laplace approximation, penalized quasi-likelihood, and adaptive Gauss–Hermite quadrature for generalized linear mixed models: Towards meta-analysis of binary outcome with sparse data. BMC Medical Research Methodology, 20(1), 152. https://doi.org/10.1186/s12874-020-01035-6

22.

Kraft

A.-K.

Naujoks

Wörle

Neukum

(2018). The impact of an in-vehicle display on glance distribution in partially automated driving in an on-road experiment. Transportation Research Part F: Traffic Psychology and Behaviour, 52, 40–50. https://doi.org/10.1016/j.trf.2017.11.012

23.

Lee

J. D.

(2014). Dynamics of driver distraction: The process of engaging and disengaging. Annals of Advances in Automotive Medicine, 58, 24–32. https://pmc.ncbi.nlm.nih.gov/articles/PMC4001670/.24776224

24.

Schroeter

Rakotonirainy

Kuo

Lenné

M. G.

(2023). Get ready for take-overs: Using head-up display for drivers to engage in non–driving-related tasks in automated vehicles. Human Factors, 65(8), 1759–1775. https://doi.org/10.1177/00187208211056200

25.

Liang

Yang

Prakah-Asante

K. O.

Curry

Blommer

Swaminathan

Pitts

B. J.

(2021). Using eye-tracking to investigate the effects of pre-takeover visual engagement on situation awareness during automated driving. Accident Analysis & Prevention, 157, 106143. https://doi.org/10.1016/j.aap.2021.106143

26.

Louw

Kountouriotis

Carsten

Merat

(2015, November). Driver inattention during vehicle automation: How does driver engagement affect resumption of control? [Proceedings Paper]. ARRB group. 4th International driver distraction and inattention conference. 4th international conference on driver distraction and inattention (DDI2015), Sydney: Proceedings. https://trid.trb.org/view/1402294

27.

Zhang

Lian

Chan

C.-Y.

(2020). Drivers’ visual attention characteristics under different cognitive workloads: An On-Road driving behavior study. International Journal of Environmental Research and Public Health, 17(15), 5366. Article 15. https://doi.org/10.3390/ijerph17155366

28.

Martinez-Conde

Macknik

Hubel

(2004). The role of fixational eye movements in visual perception. Nat Rev Neurosci, 5, 229–240. https://doi.org/10.1038/nrn1348

29.

McKerral

Pammer

Gauld

(2023). Supervising the self-driving car: Situation awareness and fatigue during highly automated driving. Accident; Analysis and Prevention, 187, 107068. https://doi.org/10.1016/j.aap.2023.107068

30.

McMains

S. A.

Kastner

(2009). Visual attention. In Binder

M. D.

Hirokawa

Windhorst

(Eds.), Encyclopedia of neuroscience (pp. 4296–4302). Springer. https://doi.org/10.1007/978-3-540-29678-2_6344

31.

Morgan

P. L.

Alford

Williams

Parkhurst

Pipe

(2018). Manual takeover and handover of a simulated fully autonomous vehicle within urban and extra-urban settings. In Stanton

N. A.

(Ed.), Advances in human aspects of transportation (pp. 760–771). Springer International Publishing. https://doi.org/10.1007/978-3-319-60441-1_73

32.

Nelder

J. A.

Wedderburn

R. W. M.

(1972). Generalized linear models. Journal of the Royal Statistical Society. Series A (General), 135(3), 370–384. https://doi.org/10.2307/2344614

33.

Peng

Iwaki

(2020). Visual attention of young and older drivers in takeover tasks of highly automated driving. In Gao

Zhou

(Eds.), Human aspects of IT for the aged population. Technologies, design and user experience (pp. 210–221). Springer International Publishing. https://doi.org/10.1007/978-3-030-50252-2_16

34.

Pipkorn

Dozza

Tivesten

(2024). Driver visual attention before and after take-over requests during automated driving on public roads. Human Factors, 66(2), 336–347. https://doi.org/10.1177/00187208221093863

35.

Powell

M. J. D.

(2010). The BOBYQA algorithm for bound constrained optimization without derivatives (no. 11137). Optimization Online. https://optimization-online.org/?p=11137

36.

Schartmüller

Weigl

Löcken

Wintersberger

Steinhauser

Riener

(2021). Displays for productive non-driving related tasks: Visual behavior and its impact in conditionally automated driving. Multimodal Technologies and Interaction, 5(4), 21. Article 4. https://doi.org/10.3390/mti5040021

37.

Victor

T. W.

Harbluk

J. L.

Engström

J. A.

(2005). Sensitivity of eye-movement measures to in-vehicle task difficulty. Transportation Research Part F: Traffic Psychology and Behaviour, The Relationship Between Distraction and Driving Performance: Towards a Test Regime for In-Vehicle Information Systems, 8(2), 167–190. https://doi.org/10.1016/j.trf.2005.04.014

38.

Walch

Lange

Baumann

Weber

(2015). Autonomous driving: Investigating the feasibility of car-driver handover assistance. In Proceedings of the 7th international conference on automotive user interfaces and interactive vehicular applications, AutomotiveUI ’15 (pp. 11–18). Association for Computing Machinery. https://doi.org/10.1145/2799250.2799268

39.

Hasegawa

Kihara

(2025). How to request drivers to prepare for takeovers during automated driving. Transportation Research Part F: Traffic Psychology and Behaviour, 109, 938–950. https://doi.org/10.1016/j.trf.2025.01.017

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.70 MB

0.00 MB