With a Little Help From My Car: Sharing Automated Vehicle’s Situation Awareness Reduces Driver-Initiated Automation Disengagement Without Delaying Takeover Response Time

Abstract

Objective

This study investigated the effect of sharing vehicle situation awareness (VSA) on driver takeover behavior in complex urban environments.

Background

As automated vehicles (AV) expand their operational design domain, little is known about driver interactions with driving automation in complex urban settings. Drivers often become either overly reliant on automation or fail to rely on it even when capable, leading to misuse or disuse. Sharing VSA information could enhance drivers’ awareness of the AV system and response when AVs request manual control in complex situations.

Methods

A driving simulator tested sharing VSA information via augmented reality head-up displays (AR HUDs) during takeover scenarios. Participants were assigned to control or experimental groups that received different combinations of VSA elements: perception (object highlighting), comprehension (confidence assessment), and projection (trajectory information). Two urban driving scenarios (parking lane and intersection) were tested.

Results

Sharing VSA information reduced driver-initiated automation disengagement before takeover requests without delaying response times. Perception information alone showed no significant difference from baseline, but adding egocentric projection information significantly reduced driver-initiated overrides, while allocentric projection did not. Adding confidence assessment further enhanced effectiveness. The parking lane scenario was associated with quicker responses, fewer full takeovers, and softer braking.

Conclusion

Specific combinations of VSA information reduced driver-initiated disengagement from automation without compromising response times. The type and presentation of shared information significantly affect human-automation interaction.

Application

These findings can guide the design of AV systems that better support driver-vehicle interaction in complex urban environments.

Keywords

advanced driver assistance systems automated driving takeover situation awareness

Get full access to this article

View all access options for this article.

References

Ackerman

Xargay

Talleur

D. A.

Carbonari

R. S.

Kirlik

Hovakimyan

Gregory

I. M.

Belcastro

C. M.

Trujillo

Seefeldt

B. D.

(2015). Flight envelope information-augmented display for enhanced pilot situational awareness. AIAA Infotech @ Aerospace. AIAA Infotech @ Aerospace, Kissimmee, FL, 5 January 2015. https://doi.org/10.2514/6.2015-1112

Bates

Mächler

Bolker

Walker

(2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01

Beller

Heesen

Vollrath

(2013). Improving the driver–automation interaction: An approach using automation uncertainty. Human Factors, 55(6), 1130–1141. https://doi.org/10.1177/0018720813482327

Bengler

Götze

Pfannmüller

Zaindl

(2015). To see or not to see: Innovative display technologies as enablers for ergonomic cockpit concepts. In Proceedings of the Electronic Displays Conference (Nuremberg, Germany).

Bürkner

P.-C.

(2017). brms: An R package for Bayesian multilevel models using Stan. Journal of Statistical Software, 80(1), 1–28. https://doi.org/10.18637/jss.v080.i01

Campbell

J. L.

Brown

J. L.

Graving

J. S.

Richard

C. M.

Lichty

M. G.

Sanquist

Bacon

Woods

Williams

D. N.

Morgan

J. F.

(2016). Human factors design guidance for driver-vehicle interfaces (Report No. DOT HS 812 360). National Highway Traffic Safety Administration.

Carsten

Martens

M. H.

(2019). How can humans understand their automated cars? HMI principles, problems and solutions. Cognition, Technology & Work, 21(1), 3–20. https://doi.org/10.1007/s10111-018-0484-0

Casner

S. M.

Hutchins

E. L.

Norman

(2016). The challenges of partially automated driving. Communications of the ACM, 59(5), 70–77. https://doi.org/10.1145/2830565

Colley

Gruler

Woide

Rukzio

(2021). Investigating the design of information presentation in take-over requests in automated vehicles. Proceedings of MobileHCI 2021 - ACM international conference on mobile human-computer interaction: Mobile apart, mobile together, Toulouse & Virtual, France, 27 September 2021. https://doi.org/10.1145/3447526.3472025

10.

Damböck

Weißgerber

Kienle

Bengler

(2012). Evaluation of a contact analog head-up display for highly automated driving. 4th international conference on applied human factors and ergonomics, San Francisco, CA, 2012.

11.

Dönmez Özkan

Mirnig

A. G.

Meschtscherjakov

Demir

Tscheligi

(2021). Mode awareness interfaces in automated vehicles, robotics, and aviation: A literature review. 13th international conference on automotive user interfaces and interactive vehicular applications, pp. 147–158. https://doi.org/10.1145/3409118.3475125

12.

Drüke

Semmler

Bendewald

(2018). The “HMI tool kit” as a strategy for the systematic derivation of user-oriented HMI concepts of driver assistance systems in urban areas. Springer.

13.

Zhou

Tilbury

D. M.

Robert

L. P.

Yang

X. J.

(2024). Behavioral and physiological responses to takeovers in different scenarios during conditionally automated driving. Transportation Research Part F: Traffic Psychology and Behaviour, 101, 320–331. https://doi.org/10.1016/j.trf.2024.01.008

14.

Endsley

M. R.

(1995). Toward a theory of situation awareness in dynamic systems. Human Factors, 37(1), 32–64. https://doi.org/10.1518/001872095779049543

15.

Endsley

M. R.

(2017). From here to autonomy: Lessons learned from human–automation research. Human Factors, 59(1), 5–27. https://doi.org/10.1177/0018720816681350

16.

Feierle

Beller

Bengler

(2019). Head-Up displays in urban partially automated driving: Effects of using augmented reality. 2019 IEEE intelligent transportation systems conference, ITSC 2019, Auckland, New Zealand, 27–30 October 2019, pp. 1877–1882. https://doi.org/10.1109/ITSC.2019.8917472

17.

Frison

A. K.

Liu

Wintersberger

Riener

(2019). Why do you like to drive automated? A context-dependent analysis of highly automated driving to elaborate requirements for intelligent user interfaces. International conference on intelligent user interfaces, proceedings IUI, part F1476, Marina del Ray, CA, 17 March 2019, pp. 528–537. https://doi.org/10.1145/3301275.3302331

18.

Gaspar

Carney

(2019). The effect of partial automation on driver attention: A naturalistic driving study. Human Factors: The Journal of the Human Factors and Ergonomics Society, 61(8), 1261–1276. https://doi.org/10.1177/0018720819836310

19.

Ghori

Mackowiak

Bautista

Beuter

Drumond

Diego

Ommer

(2018). Learning to forecast pedestrian intention from pose dynamics. In IEEE Intelligent Vehicles Symposium (IV) (pp. 1277–1284). Changshu, China: IEEE. https://doi.org/10.1109/IVS.2018.8500657 [Accessed 2018].

20.

Helldin

Falkman

Riveiro

Davidsson

(2013). Presenting system uncertainty in automotive UIs for supporting trust calibration in autonomous driving. In Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (pp. 210–217). https://doi.org/10.1145/2516540.2516554

21.

Israel

Seitz

Bubb

Senner

Khalid

Hedge

Ahram

T. Z.

(2010). Contact analog information in the head up display: How much information supports the driver? In Advances in ergonomics modeling and usability evaluation. CRC Press. https://doi.org/10.1201/EBK1439835036-20

22.

Klein

Woods

D. D.

Bradshaw

J. M.

Hoffman

R. R.

Feltovich

P. J.

(2004). Ten challenges for making automation a “team player” in joint human-agent activity. IEEE Intelligent Systems, 19(06), 91–95. https://doi.org/10.1109/MIS.2004.74

23.

Koo

Kwac

Steinert

Leifer

Nass

(2015). Why did my car just do that? Explaining semi-autonomous driving actions to improve driver understanding, trust, and performance. International Journal on Interactive Design and Manufacturing, 9(4), 269–275. https://doi.org/10.1007/s12008-014-0227-2

24.

Kunze

Summerskill

S. J.

Marshall

Filtness

A. J.

(2019). Automation transparency: Implications of uncertainty communication for human-automation interaction and interfaces. Ergonomics, 62(3), 345–360. https://doi.org/10.1080/00140139.2018.1547842

25.

Lee

J. R.

Koenig

M. L.

Douglas

J. R.

Domeyer

J. E.

Lee

J. D.

Toyoda

(2022). Incorporating driver expectations into a taxonomy of transfers of control for automated vehicles. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 66, No. 1, pp. 340–344). SAGE Publications. https://doi.org/10.1177/1071181322661241

26.

Lee

J. D.

Liu

S. Y.

Domeyer

DinparastDjadid

(2021). Assessing drivers’ trust of automated vehicle driving styles with a two-part mixed model of intervention tendency and magnitude. Human factors, 63(2), 197–209.

27.

Lee

Rheem

Lee

J. D.

Szczerba

J. F.

Rajavenkatanarayanan

Mathieu

(2023). Sharing vehicle situation awareness reduces driver-initiated overrides in urban environments. Proceedings of the Human Factors and Ergonomics Society - Annual Meeting, 67(1), 21695067231195827. https://doi.org/10.1177/21695067231195827

28.

Lee

Rheem

Lee

J. D.

Szczerba

J. F.

Tsimhoni

(2022). Teaming with your car: Redefining the driver–automation relationship in highly automated vehicles. Journal of Cognitive Engineering and Decision Making, 17(1), 155534342211326–155534342211374. https://doi.org/10.1177/15553434221132636

29.

Lee

J. D.

See

K. A.

(2004). Trust in automation: Designing for appropriate reliance. Human Factors, 46(1), 50–80. https://doi.org/10.1518/hfes.46.1.50_30392

30.

Lindemann

Lee

T.-Y.

Rigoll

(2018). Catch my drift: Elevating situation awareness for highly automated driving with an explanatory windshield display user interface. Multimodal Technologies and Interaction, 2(4), 71. https://doi.org/10.3390/mti2040071

31.

Happee

Cabrall

C. D. D.

Kyriakidis

de Winter

J. C. F.

(2016). Human factors of transitions in automated driving: A general framework and literature survey. Transportation Research Part F: Traffic Psychology and Behaviour, 43, 183–198. https://doi.org/10.1016/j.trf.2016.10.007

32.

Maggi

Romano

Carsten

Winter

J. C. F. D.

(2022). When terminology hinders research: The colloquialisms of transitions of control in automated driving. Cognition, Technology & Work, 24(3), 1–12. https://doi.org/10.1007/s10111-022-00705-3

33.

Mahajan

Large

D. R.

Burnett

Velaga

N. R.

(2021). Exploring the benefits of conversing with a digital voice assistant during automated driving: A parametric duration model of takeover time. Transportation Research Part F: Traffic Psychology and Behaviour, 80, 104–126. https://doi.org/10.1016/j.trf.2021.03.012

34.

Morando

Gershon

Mehler

Reimer

(2020). Driver-initiated tesla autopilot disengagements in naturalistic driving. 12th international conference on automotive user interfaces and interactive vehicular applications, Virtual Event, DC, 20 September 2020, pp. 57–65. https://doi.org/10.1145/3409120.3410644

35.

Parasuraman

Riley

(1997). Humans and automation: Use, misuse, disuse, abuse. Human Factors, 39(2), 230–253. https://doi.org/10.1518/001872097778543886

36.

Pfannmüller

Kramer

Senner

Bengler

(2015). A comparison of display concepts for a navigation system in an automotive contact analog head-up display. Procedia Manufacturing, 3, 2722–2729. https://doi.org/10.1016/j.promfg.2015.07.678

37.

Rittger

Götze

(2018). HMI strategy: Recommended action. In Bengler

Drüke

Hoffmann

Manstetten

Neukum

(Eds.), UR:BAN human factors in traffic. Springer Vieweg. https://doi.org/10.1007/978-3-658-15418-9_7

38.

Rusch

M. L.

Schall

M. C.

Gavin

Lee

J. D.

Dawson

J. D.

Vecera

Rizzo

(2013). Directing driver attention with augmented reality cues. Transportation Research Part F: Traffic Psychology and Behaviour, 16, 127–137. https://doi.org/10.1016/j.trf.2012.08.007

39.

Samuel

Borowsky

Zilberstein

Fisher

D. L.

(2016). Minimum time to situation awareness in scenarios involving transfer of control from an automated driving suite. Transportation Research Record, 2602(1), 115–120. https://doi.org/10.3141/2602-14

40.

Schuet

Lombaerts

Acosta

Kaneshige

Wheeler

Shish

(2017). Autonomous flight envelope estimation for loss-of-control prevention. Journal of Guidance, Control, and Dynamics, 40(4), 847–862. https://doi.org/10.2514/1.G001729

41.

Stapel

El Hassnaoui

Happee

(2022). Measuring driver perception: Combining eye-tracking and automated road scene perception. Human Factors: The Journal of the Human Factors and Ergonomics Society, 64(4), 714–731. https://doi.org/10.1177/0018720820959958

42.

Swain

Kaye

S.-A.

Rakotonirainy

(2024). Shared intention and shared awareness for conditional automated driving: An online, randomized video experiment. Traffic Injury Prevention, 1–9(4), 398–406. https://doi.org/10.1080/15389588.2024.2421215

43.

Van Paassen

M. M.

Borst

Ellerbroek

Mulder

Flach

J. M.

(2018). Ecological interface design for vehicle locomotion control. IEEE Transactions on Human-Machine Systems, 48(5), 541–555. https://doi.org/10.1109/THMS.2018.2860601

44.

Vehtari

Gelman

Gabry

(2017). Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Statistics and Computing, 27(5), 1413–1432. https://doi.org/10.1007/s11222-016-9696-4

45.

Wickham

Averick

Bryan

Chang

McGowan

François

Grolemund

Hayes

Henry

Hester

Kuhn

Pedersen

Miller

Bache

Müller

Ooms

Robinson

Seidel

Spinu

Yutani

(2019). Welcome to the Tidyverse. Journal of Open Source Software, 4(43), 1686. https://doi.org/10.21105/joss.01686

46.

Wintersberger

von Sawitzky

Frison

A. K.

Riener

(2017). Traffic augmentation as a means to increase trust in automated driving systems. In Proceedings of the 12th Biannual Conference of the Italian SIGCHI Chapter. https://doi.org/10.1145/3125571.3125600