The study aims to investigate the potential of using HUD (head-up display) as an approach for drivers to engage in non–driving-related tasks (NDRTs) during automated driving, and examine the impacts on driver state and take-over performance in comparison to the traditional mobile phone.
Background
Advances in automated vehicle technology have the potential to relieve drivers from driving tasks so that they can engage in NDRTs freely. However, drivers will still need to take-over control under certain circumstances.
Method
A driving simulation experiment was conducted using an Advanced Driving Simulator and real-world driving videos. Forty-six participants completed three drives in three display conditions, respectively (HUD, mobile phone and baseline without NDRT). The HUD was integrated with the vehicle in displaying NDRTs while the mobile phone was not. Drivers’ visual (e.g. gaze, blink) and physiological (e.g. ECG, EDA) data were collected to measure driver state. Two take-over reaction times (hand and foot) were used to measure take-over performance.
Results
The HUD significantly shortened the take-over reaction times compared to the mobile phone condition. Compared to the baseline condition, drivers in the HUD condition also experienced lower cognitive workload and physiological arousal. Drivers’ take-over reaction times were significantly correlated with their visual and electrodermal activities during automated driving prior to the take-over request.
Conclusion
HUDs can improve driver performance and lower workload when used as an NDRT interface.
Application
The study sheds light on a promising approach for drivers to engage in NDRTs in future AVs.
BenedekM.KaernbachC. (2010). A continuous measure of phasic electrodermal activity. Journal of Neuroscience Methods, 190, 80-91.
2.
CharissisV.PapanastasiouS. (2010). Human–machine collaboration through vehicle head up display interface. Cognition, Technology & Work, 12(1), 41-50.
3.
ChenK. T.ChenH. Y. W. (2021). Manipulating music to communicate automation reliability in conditionally automated driving: A driving simulator study. International Journal of Human-Computer Studies, 145, 102518.
4.
De WinterJ. C.HappeeR.MartensM. H.StantonN. A. (2014). Effects of adaptive cruise control and highly automated driving on workload and situation awareness: A review of the empirical evidence. Transportation Research Part F: Traffic Psychology and Behaviour, 27, 196-217.
5.
DuN.YangX. J.ZhouF. (2020a). Psychophysiological responses to takeover requests in conditionally automated driving. Accident Analysis & Prevention, 148, 105804.
DuN.ZhouF.PulverE. M.TilburyD. M.RobertL. P.PradhanA. K.YangX. J. (2020c). Examining the effects of emotional valence and arousal on takeover performance in conditionally automated driving. Transportation Research Part C: Emerging Technologies, 112, 78-87.
8.
FaureV.LobjoisR.BenguiguiN., (2016). The effects of driving environment complexity and dual tasking on drivers' mental workload and eye blink behavior. Transportation Research Part F: Traffic Psychology and Behaviour, 40, 78-90.
9.
GerberM. A.SchroeterR.VehnsJ. (2019). A video-based automated driving simulator for automotive UI prototyping, UX and behaviour research. Proceddings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (pp. 21-25), Utrecht, Netherlands, September 2019.
10.
GerberM. A.SchroeterR.XiaomengL.ElhenawyM. (2020). Self-interruptions of non-driving related tasks in automated vehicles: Mobile vs head-up display. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1-9), Honolulu, USA, April 2020.
11.
GoldC.BerishaI.BenglerK. (2015). Utilisation of drivetime–performing non-driving related tasks while driving highly automated. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 59(1), 1666-1670): Los Angeles, CA: SAGE Publications.
12.
HenschA. C.RauhN.SchmidtC.HergethS.NaujoksF.KremsJ. F.KeinathA. (2020). Effects of secondary tasks and display position on glance behavior during partially automated driving. Transportation Research Part F: Traffic Psychology and Behaviour, 68, 23-32.
13.
JanssenC. P.IqbalS. T.KunA. L.DonkerS. F. (2019). Interrupted by my car? Implications of interruption and interleaving research for automated vehicles. International Journal of Human-Computer Studies, 130, 221-233.
14.
KhattakZ. H.FontaineM. D.BoatengR. A. (2018). Evaluating the impact of adaptive signal control technology on driver stress and behavior using real-world experimental data. Transportation Research Part F: Traffic Psychology and Behaviour, 58, 133-144.
15.
KimJ.KimH. S.KimW.YoonD. (2018). Take-over performance analysis depending on the drivers' non-driving secondary tasks in automated vehicles. 2018 International Conference on Information and Communication Technology Convergence (ICTC) (pp. 1364-1366), Jeju, Korea (South), October 2018.
16.
LargeD. R.BurnettG.SalanitriD.LawsonA.BoxE. (2019). A Longitudinal simulator study to explore drivers' behaviour in level 3 automated vehicles. Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (pp. 222-232), Utrecht, Netherlands, September 2019.
17.
LarueG. S.RakotonirainyA.PettittA. N. (2011). Driving performance impairments due to hypovigilance on monotonous roads. Accident Analysis & Prevention, 43(6), 2037-2046.
18.
LiangY.LeeJ. D.YekhshatyanL. (2012). How dangerous is looking away from the road? Algorithms predict crash risk from glance patterns in naturalistic driving. Human Factors, 54(6), 1104-1116.
19.
LiangN.YangJ.YuD.Prakah-AsanteK. O.CurryR.BlommerM.PittsB. J. (2021). Using eye-tracking to investigate the effects of pre-takeover visual engagement on situation awareness during automated driving. Accident Analysis & Prevention, 157, 106143.
20.
LiX.RakotonirainyA.YanX.ZhangY. (2018). Driver's visual performance in rear-end collision avoidance process under the influence of cell phone use. Transportation Research Record, 2672(37), 55-63.
21.
LiX.SchroeterR.RakotonirainyA.KuoJ.LennéM. G. (2020b). Effects of different non-driving-related-task display modes on drivers' eye-movement patterns during take-over in an automated vehicle. Transportation Research Part F: Traffic Psychology and Behaviour, 70, 135-148.
22.
LiuY.DuS. (2018). Psychological stress level detection based on electrodermal activity. Behavioural Brain Research, 341, 50-53.
23.
LiuY. C.WenM. H. (2004). Comparison of head-up display (HUD) vs. head-down display (HDD): driving performance of commercial vehicle operators in Taiwan. International Journal of Human-Computer Studies, 61(5), 679-697.
24.
LiX.VaezipourA.RakotonirainyA.DemmelS. (2019). Effects of an in-vehicle eco-safe driving system on drivers' glance behaviour. Accident Analysis & Prevention, 122, 143-152.
25.
LiX.VaezipourA.RakotonirainyA.DemmelS.Oviedo-TrespalaciosO. (2020a). Exploring drivers' mental workload and visual demand while using an in-vehicle HMI for eco-safe driving. Accident Analysis & Prevention, 146, 105756.
26.
LohaniM.PayneB. R.StrayerD. L. (2019). A review of psychophysiological measures to assess cognitive states in real-world driving. Frontiers in Human Neuroscience, 13, 57.
27.
LouwT.KuoJ.RomanoR.RadhakrishnanV.LennéM. G.MeratN. (2019). Engaging in NDRTs affects drivers' responses and glance patterns after silent automation failures. Transportation Research Part F: Traffic Psychology and Behaviour, 62, 870-882.
28.
MilakisD.Van AremB.Van WeeB. (2017). Policy and society related implications of automated driving: A review of literature and directions for future research. Journal of Intelligent Transportation Systems, 21(4), 324-348.
29.
NaujoksF.HöflingS.PuruckerC.ZeebK. (2018). From partial and high automation to manual driving: relationship between non-driving related tasks, drowsiness and take-over performance. Accident Analysis & Prevention, 121, 28-42.
30.
NeubauerC.MatthewsG.SaxbyD. (2012). The effects of cell phone use and automation on driver performance and subjective state in simulated driving. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 56(1), 1987-1991.
31.
RecarteM. Á.PérezE.ConchilloÁ.NunesL. M., (2008). Mental workload and visual impairment: Differences between pupil, blink, and subjective rating. The Spanish Journal of Psychology, 11 (2), 374-385.
32.
RuscioD.BosA. J.CiceriM. R. (2017). Distraction or cognitive overload? Using modulations of the autonomic nervous system to discriminate the possible negative effects of advanced assistance system. Accident Analysis & Prevention, 103, 105-111.
33.
SAE International (2018). Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles. Standard J3016_201806. https://saemobilus.sae.org/content/j3016_201806
34.
SanoA.PicardR. W. (2011). Toward a taxonomy of autonomic sleep patterns with electrodermal activity. 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (pp. 777-780), Boston, USA, September 2011.
35.
SchroeterR.OxtobyJ.JohnsonD. (2014). AR and gamification concepts to reduce driver boredom and risk taking behaviours. Proceedings of the 6th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (pp. 1-8), Seattle, USA, September 2014.
36.
TianR.RuanK.LiL.LeJ.GreenbergJ.BarbatS. (2019). Standardized evaluation of camera-based driver state monitoring systems. IEEE/CAA Journal of Automatica Sinica, 6(3), 716-732.
37.
VicenteF.HuangZ.XiongX.De la TorreF.ZhangW.LeviD. (2015). Driver gaze tracking and eyes off the road detection system. IEEE Transactions on Intelligent Transportation Systems, 16(4), 2014-2027.
38.
WittmannM.KissM.GuggP.SteffenA.FinkM.PöppelE.KamiyaH. (2006). Effects of display position of a visual in-vehicle task on simulated driving. Applied Ergonomics, 37(2), 187-199.
39.
YoonS. H.JiY. G. (2019). Non-driving-related tasks, workload, and takeover performance in highly automated driving contexts. Transportation Research Part F: Traffic Psychology and Behaviour, 60, 620-631.
40.
YoonS. H.LeeS. C.JiY. G. (2021). Modeling takeover time based on non-driving-related task attributes in highly automated driving. Applied Ergonomics, 92, 103343.
41.
ZeebK.HärtelM.BuchnerA.SchraufM. (2017). Why is steering not the same as braking? The impact of non-driving related tasks on lateral and longitudinal driver interventions during conditionally automated driving. Transportation Research Part F: Traffic Psychology and Behaviour, 50, 65-79.
42.
ZhangY.YanX.LiX.WuJ. (2019). Changes of drivers' visual performances when approaching a signalized intersection under different collision avoidance warning conditions. Transportation Research Part F: Traffic Psychology and Behaviour, 65, 584-597.
