Investigating take-over, driving, non-driving related task (NDRT) performance, and trust of conditionally automated vehicles (AVs) in critical transitions on a test track.
Background
Most experimental results addressing driver take-over were obtained in simulators. The presented experiment aimed at validating relevant findings while uncovering potential effects of motion cues and real risk.
Method
Twenty-two participants responded to four critical transitions on a test track. Non-driving related task modality (reading on a handheld device vs. auditory) and take-over timing (cognitive load) were varied on two levels. We evaluated take-over and NDRT performance as well as gaze behavior. Further, trust and workload were assessed with scales and interviews.
Results
Reaction times were significantly faster than in simulator studies. Further, reaction times were only barely affected by varying visual, physical, or cognitive load. Post-take-over control was significantly degraded with the handheld device. Experiencing the system reduced participants’ distrust, and distrusting participants monitored the system longer and more frequently. NDRTs on a handheld device resulted in more safety-critical situations.
Conclusion
The results confirm that take-over performance is mainly influenced by visual-cognitive load, while physical load did not significantly affect responses. Future take-over request (TOR) studies may investigate situation awareness and post-take-over control rather than reaction times only. Trust and distrust can be considered as different dimensions in AV research.
Application
Conditionally AVs should offer dedicated interfaces for NDRTs to provide an alternative to using nomadic devices. These interfaces should be designed in a way to maintain drivers’ situation awareness.
Précis
This paper presents a test track experiment addressing conditionally automated driving systems. Twenty-two participants responded to critical TORs, where we varied NDRT modality and take-over timing. In addition, we assessed trust and workload with standardized scales and interviews.
AutomobiltechnikD.-N. (2015). Road vehicles – measurement of driver visual behaviour with respect to transport information and control systems – Part 1: Definitions and parameters (ISO15007-1:2014). (Tech. Rep.). ISO.
2.
BaileyB. P.KonstanJ. A. (2006). On the need for attention-aware systems: Measuring effects of interruption on task performance, error rate, and affective state. Computers in Human Behavior, 22(4), 685–708. (Attention aware systems). https://doi.org/10.1016/j.chb.2005.12.009.
3.
BrouwerW. H.WaterinkW.Van WolffelaarP. C.RothengatterT. (1991). Divided attention in experienced young and older drivers: Lane tracking and visual analysis in a dynamic driving simulator. Human Factors, 33(5), 573–582.
de WinterJ.StantonN.EismaY. B. (2021). Is the take-over paradigm a mere convenience?
6.
EndsleyM. R.KirisE. O. (1995). The out-of-the-loop performance problem and level of control in automation. Human Factors, 37(2), 381–394. https://doi.org/10.1518/001872095779064555.
7.
ErikssonA.BanksV. A.StantonN. A. (2017). Transition to manual: Comparing simulator with on-road control transitions. Accident Analysis & Prevention, 102, 227-234.
8.
ErikssonA.StantonN. A. (2017). Takeover time in highly automated vehicles: Noncritical transitions to and from manual control. Human Factors, 59(4), 689–705. https://doi.org/10.1177/0018720816685832.
9.
FrisonA.-K.WintersbergerP.SchartmüllerC.RienerA. (2019). The real t (h) or: Evaluation of emergency take-over on a test track. In Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications: Adjunct Proceedings, Utrecht Netherlands, 21–25 September 2019 (pp. 478–482).
10.
FrisonA.-K.WintersbergerP.SchartmüllerC.RienerA.BoyleL. N.MillnerE.WeiglK. (2019). In ux we trust - the impact of hmi design and system performance on the perception/experience of automated driving. In Proceedings of the Sigchi Conference on Human Factors in Computing Systems, Glasgow Scotland Uk, 4–9 May 2019 (pp. 159–166).
11.
GerberM. A.SchroeterR.XiaomengL.ElhenawyM. (2020). Self-interruptions of non-driving related tasks in automated vehicles: Mobile self-interruptions of non-driving related tasks in automated vehicles: Mobile vs head-up display. In Proceedings of the Conference on Human Factors in Computing Systems CHI’20, Honolulu, HI, USA, 25–30 April 2020. ACM. (Issue: April). https://doi.org/10.1145/3313831.3376751.
GoldC.KörberM.HohenbergerC.LechnerD.BenglerK. (2015). Trust in automation–before and after the experience of take-over scenarios in a highly automated vehicle. Procedia Manufacturing, 3, 3025-3032. https://doi.org/10.1016/j.promfg.2015.07.847.
14.
GoldC.KörberM.LechnerD.BenglerK. (2016). Taking over control from highly automated vehicles in complex traffic situations: The role of traffic density. Human Factors, 58(4), 642–652. https://doi.org/10.1177/0018720816634226.
HergethS.LorenzL.KremsJ. F. (2017). Prior familiarization with takeover requests affects drivers’ takeover performance and automation trust. Human Factors, 59(3), 457–470. https://doi.org/10.1177/0018720816678714.
17.
HergethS.LorenzL.KremsJ. F.ToenertL. (2015). Effects of take-over requests and cultural background on automation trust in highly automated driving.
18.
HergethS.LorenzL.VilimekR.KremsJ. F. (2016). Keep your scanners peeled: Gaze behavior as a measure of automation trust during highly automated driving. Human Factors, 58(3), 509–519. https://doi.org/10.1177/0018720815625744.
19.
JaroschO.GoldC.NaujoksF.WandtnerB.MarbergerC.WeidlG.SchraufM. (2019). The impact of non-driving related tasks on take-over performance in conditionally automated driving–a review of the empirical evidence. 9. tagung automatisiertes fahren.
20.
JianJ.-Y.BisantzA. M.DruryC. G. (2000). Foundations for an empirically determined scale of trust in automated systems. International Journal of Cognitive Ergonomics, 4(1), 53–71. https://doi.org/10.1207/S15327566IJCE0401_04.
21.
KnappeG.KeinathA.BenglerK.MeineckeC. (2007). Driving simulator as an evaluation tool–assessment of the influence of field of view and secondary tasks on lane keeping and steering performance. In 20th International Technical Conference on the Enhanced Safety of Vehicles (esv) National Highway Traffic Safety Administration, Lyon , France, 18–21 June 2007.
22.
KubitzkiJ.FastenmeierW. (2016). Ablenkung durch moderne informations- und kommunikationstechniken und soziale interaktion bei autofahrern. Allianz AG.
23.
LargeD.BurnettG.SalanitriD.LawsonA.BoxE. (2019). A longitudinal simulator study to explore drivers’ behaviour in level 3 automated vehicles.
LiM.HolthausenB. E.StuckR. E.WalkerB. N. (2019). No risk no trust: Investigating perceived risk in highly automated driving. In Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Utrecht Netherlands, 21–25 September 2019 (pp. 177–185).
26.
LinzC. (2017, September 21). Handy am steuer wird teurer. Stuttgarter Nachrichten. https://goo.gl/FopUCo)
27.
LiX.SchroeterR.RakotonirainyA.KuoJ.LennéM. G. (2020, April). Effects of different non-driving-related-task display modes on drivers’ eye-movement patterns during take-over in an automated vehicle. Transportation Research Part F: Traffic Psychology and Behaviour, 70(4), 135–148. (Publisher: Elsevier Ltd)https://doi.org/10.1016/j.trf.2020.03.001
28.
LuZ.CosterX.De WinterJ. (2017). How much time do drivers need to obtain situation awareness? a laboratory-based study of automated driving. Applied Ergonomics, 60, 293-304. https://doi.org/10.1016/j.apergo.2016.12.003.
29.
MaciejJ.VollrathM. (2009). Comparison of manual vs. speech-based interaction with in-vehicle information systems. Accident Analysis & Prevention, 41(5), 924–930. https://doi.org/10.1016/j.aap.2009.05.007.
30.
MartelaroN.TeevanJ.IqbalS. T. (2019). An exploration of speech-based productivity support in the car. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI ’19, Glasgow, Scotland, UK, 4–9 May 2019 (pp. 1–12).
31.
McDonaldA. D.AlambeigiH.EngströmJ.MarkkulaG.VogelpohlT.DunneJ.YumaN. (2019). Toward computational simulations of behavior during automated driving takeovers: A review of the empirical and modeling literatures. Human Factors, 61(4), 642–688. https://doi.org/10.1177/0018720819829572.
32.
MeratN.JamsonA. H.LaiF. C. H.DalyM.CarstenO. M. J. (2014). Transition to manual: Driver behaviour when resuming control from a highly automated vehicle. Transportation Research Part F: Traffic Psychology and Behaviour, 27(Part B), 274–282. https://doi.org/10.1016/j.trf.2014.09.005.
33.
MokB.JohnsM.YangS.JuW. (2017). Reinventing the wheel: Transforming steering wheel systems for autonomous vehicles. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology - UIST, 17, 229-241, New York, NY: Association for computing Machinery. https://doi.org/10.1145/3126594.3126655.
34.
Morales-AlvarezW.SmirnovN.MatthesE.Olaverri-MonrealC. (2020). Vehicle automation field test: Impact on driver behavior and trust. In 2020 ieee 23rd International Conference on Intelligent Transportation Systems (itsc), Rhodes, Greece, 20–23 September 2020 (pp. 1–6).
35.
NaujoksF.PuruckerC.WiedemannK.MarbergerC. (2019). Noncritical state transitions during conditionally automated driving on german freeways: Effects of non–driving related tasks on takeover time and takeover quality. Human Factors, 61(4), 596–613. https://doi.org/10.1177/0018720818824002.
36.
NaujoksF.WiedemannK.SchömigN. (2017). The importance of interruption management for usefulness and acceptance of automated driving. In Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Oldenburg Germany, 24–27 September 2017 (pp. 254–263). ACM. https://doi.org/10.1145/3122986.3123000.
37.
NjusE. (2017, September 20). Oregon’s new distracted driving law takes effect oct. 1: What you need to know. The Oregonian. https://goo.gl/W9bCKK)
38.
Oviedo-TrespalaciosO.NandavarS.HaworthN. (2019). How do perceptions of risk and other psychological factors influence the use of in-vehicle information systems (ivis)?Transportation Research Part F: Traffic Psychology and Behaviour, 67, 113-122. https://doi.org/10.1016/j.trf.2019.10.011.
39.
ParasuramanR.SheridanT. B.WickensC. D. (2000). A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 30(3), 286–297. https://doi.org/10.1109/3468.844354.
40.
ParduziA.VenrooijJ.MarkerS. (2019). A method to assess the effect of vertical dynamics on driving performance in driving simulators: A behavioural validation study. In International Conference on Human-Computer Interaction (pp. 176–189).
41.
Petermann-StockI.HackenbergL.MuhrT.MerglC. (2013). Wie lange braucht der fahrer-eine analyse zu übernahmezeiten aus verschiedenen nebentätigkeiten während einer hochautomatisierten staufahrt. 6. Tagung Fahrerassistenzsysteme. Der Weg zum automatischen Fahren.
42.
PflegingB.RangM.BroyN. (2016). Investigating user needs for non-driving-related activities during automated driving. In Proceedings of the 15th International Conference on Mobile and Ubiquitous Multimedia, Rovaniemi Finland, 12–15 December 2016 (pp. 91–99).
43.
PolitisI.BrewsterS.PollickF. (2015). Language-based multimodal displays for the handover of control in autonomous cars. In Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Nottingham United Kingdom, 1–3 September 2015 (pp. 3–10). ACM. https://doi.org/10.1145/2799250.2799262.
44.
RadlmayrJ.BenglerK. (2015). Literature survey and description of methods for the development of highly automated driving. FAT Schriftenreihe.
45.
RadlmayrJ.GoldC.LorenzL.FaridM.BenglerK. (2014). How traffic situations and non-driving related tasks affect the take-over quality in highly automated driving. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 58(1), 2063–2067. https://doi.org/10.1177/1541931214581434.
46.
Sadeghian BorojeniS.BollS. C. J.HeutenW.BülthoffH. H.ChuangL. (2018). Feel the movement: Real motion influences responses to take-over requests in highly automated vehicles. In Proceedings of the 2018 chi Conference on Human Factors in Computing Systems, Montreal QC Canada, 21–26 April 2018 (pp. 246:1–246:13). ACM. https://doi.org/10.1145/3173574.3173820.
47.
Sadeghian BorojeniS.ChuangL.HeutenW.BollS. (2016). Assisting drivers with ambient take-over requests in highly automated driving. In Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Ann Arbor MI USA, 24–26 October 2016 (pp. 237–244).
48.
SadeghianS.WallbaumT.HeutenW.BollS. (2017). Comparing shape-changing and vibro-tactile steering wheels for take-over requests in highly automated driving. In Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Oldenburg Germany, 24–27 September 2017 (pp. 221–225).
49.
SadeghianS.WeberL.HeutenW.BollS. (2018). From reading to driving: Priming mobile users for take-over situations in highly automated driving. Association for Computing Machinery. https://doi.org/10.1145/3229434.3229464.
50.
SAE. (2018). Sae j3016 taxonomy and definitions for terms related to on-road motor vehicle automated driving systems. SAE International.
51.
SchartmüllerC.RienerA.WintersbergerP.FrisonA.-K. (2018). Workaholistic: On balancing typing-and handover-performance in automated driving. In Proceedings of the 20th International Conference on Human-Computer Interaction With Mobile Devices and Services, Barcelona Spain, 3–6 September 2018 (p. 16).
52.
SchartmüllerC.WeiglK.LöckenA.WintersbergerP.RienerA. (2021) Displays for productive non-driving related tasks: Visual behavior and its impact in conditionally automated driving. 1–20.
53.
SchartmüllerC.WeiglK.WintersbergerP.RienerA.SteinhauserM. (2019). Text comprehension: Heads-up vs. auditory displays: Implications for a productive work environment in sae level 3 automated vehicles. In Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Utrecht Netherlands, 21–25 September 2019 (pp. 342–354).
54.
SchartmüllerC.WintersbergerP.RienerA. (2017). Interactive demo: Rapid, live data supported prototyping with U.S.E. In Adjunct Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Oldenburg Germany, 24–27 September 2017 (pp. 239–241). ACM. https://doi.org/10.1145/3131726.3132046.
StrauchC.MühlK.PatroK.GrabmaierC.ReithingerS.BaumannM.HuckaufA. (2019). Real autonomous driving from a passenger’s perspective: Two experimental investigations using gaze behaviour and trust ratings in field and simulator. Transportation Research Part F: Traffic Psychology and Behaviour, 66, 15-28. https://doi.org/10.1016/j.trf.2019.08.013.
57.
TelpazA.RhindressB.ZelmanI.TsimhoniO. (2015). Haptic seat for automated driving: Preparing the driver to take control effectively. In Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Nottingham United Kingdom, 1–3 September 2015 (pp. 23–30).
TsimhoniO.SmithD.GreenP. (2004). Address entry while driving: Speech recognition versus a touch-screen keyboard. Human Factors, 46(4), 600–610. https://doi.org/10.1518/hfes.46.4.600.56813.
60.
van der HeidenR. M.IqbalS. T.JanssenC. P. (2017). Priming drivers before handover in semi-autonomous cars. In Proceedings of the 2017 chi Conference on Human Factors in Computing Systems, Denver Colorado USA, 6–11 May 2017 (pp. 392–404).
61.
VictorT. W.TivestenE.GustavssonP.JohanssonJ.SangbergF.Ljung AustM. (2018). Automation expectation mismatch: Incorrect prediction despite eyes on threat and hands on wheel. Human Factors, 60(8), 1095–1116. https://doi.org/10.1177/0018720818788164.
62.
WiedemannK.NaujoksF.WörleJ.Kenntner-MabialaR.KaussnerY.NeukumA. (2018). Effect of different alcohol levels on take-over performance in conditionally automated driving. Accident Analysis & Prevention, 115, 89-97. https://doi.org/10.1016/j.aap.2018.03.001.
63.
WintersbergerP.FrisonA.-K.RienerA.von SawitzkyT. (2018). Fostering user acceptance and trust in fully automated vehicles: Evaluating the potential of augmented reality. Presence: Teleoperators and Virtual Environments, 27(1), 46–62. https://doi.org/10.1162/pres_a_00320.
64.
WintersbergerP.GreenP.RienerA. (2017). Am i driving or are you or are we both? A taxonomy for handover and handback in automated driving. In 9th International Driving Symposium on Human Factors in Driver Assessment, Training, and Vehicle Design, Equinox Manchester Village, Vermont, 26–29 June 2017. University of Iowa.
65.
WintersbergerP.RienerA.SchartmüllerC.FrisonA.-K.WeiglK. (2018). Let me finish before i take over: Towards attention-aware device integration in highly automated vehicles. In Proceedings of the 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Toronto ON Canada, 23–25 September 2018.
66.
WintersbergerP.SchartmüllerC.RienerA. (2019). Attentive user interfaces to improve multitasking and take-over performance in automated driving: The auto-net of things. International Journal of Mobile Human Computer Interaction (IJMHCI), 11(3), 40-58. https://doi.org/10.4018/IJMHCI.2019070103.
67.
ZeebK.BuchnerA.SchraufM. (2015). What determines the take-over time? An integrated model approach of driver take-over after automated driving. Accident Analysis & Prevention, 78, 212-221. https://doi.org/10.1016/j.aap.2015.02.023.
68.
ZeebK.HärtelM.BuchnerA.SchraufM. (2017). Why is steering not the same as braking? The impact of non-driving related tasks on lateral and longitudinal driver interventions during conditionally automated driving. Transportation Research Part F: Traffic Psychology and Behaviour, 50, 65-79. https://doi.org/10.1016/j.trf.2017.07.008.
69.
ZhangB.de WinterJ.VarottoS.HappeeR.MartensM. (2019). Determinants of take-over time from automated driving: A meta-analysis of 129 studies. Transportation Research Part F: Traffic Psychology and Behaviour, 64, 285-307. https://doi.org/10.1016/j.trf.2019.04.020.
