When and how the driver should be intervened to relieve the fatigue status in Society of Automotive Engineers Level 3 automated driving has raised numerous debates. In this paper, we identify the driver’s fatigue level according to the driver’s facial features before the system issues the take-over request (TOR), and then perform the fatigue warning (FW) intervention to investigate whether the driver would be better prepared for taking over the vehicle safely and efficiently. In a simulator-based study, we compared the driver’s driving performance under the condition of whether a FW intervention was provided before a possible TOR, with the aim to gain insights into the ability of fatigued drivers to regain manual control and situation awareness after automated driving. A FW + TOR condition was compared with a TOR-only condition using a within-subject design with 30 participants. Under the effect of FW, participants exhibited better take-over performance; for example, they gazed at the road and placed their hands on the steering wheel earlier before taking over the vehicle. Moreover, in the FW + TOR state, they also exhibited better take-over performance, with a shorter average braking reaction time and a higher average speed, which reflects that the driver’s speed is more stable during the take-over of the vehicle and that the take-over task can be completed with a smaller deceleration. Furthermore, it is concluded that the FW (5 s before TOR) + TOR mode has greater potential to increase the safety and acceptance of automated driving, and could reduce the driver’s safety risk in a fatigue state when driving automatically compared with the FW (10 s before TOR) + TOR mode. It is concluded that FW (5 s before TOR) + TOR mode has the potential to increase safety and acceptance of automated driving as compared with systems that provide only TORs. In addition, we need to design take-over scenarios and non-driving-related tasks with different levels of complexity to satisfy subsequent research. The fatigue detection method based on image-processing techniques also needs further improvement.
LuZ.ZhangB.FeldhuetterA.HappeeR.MartensM.De WinterJ. C. F.Beyond Mere Take-Over Requests: The Effects of Monitoring Requests on Driver Attention, Take-Over Performance, and Acceptance. Transportation Research Part F: Traffic Psychology and Behaviour, Vol. 63, 2019, pp. 22–37.
2.
LuZ.CosterX.De WinterJ. C. F.How Much Time do Drivers Need to Obtain Situation Awareness? A Laboratory-Based Study of Automated Driving. Applied Ergonomics, Vol. 60, 2017, pp. 293–304.
3.
SamuelS.BorowskyA.ZilbersteinS.FisherD. L.Minimum Time to Situation Awareness in Scenarios Involving Transfer of Control From an Automated Driving Suite. Transportation Research Record: Journal of the Transportation Research Board, 2016. 2602: 115–120.
4.
LargeD. R.BurnettG. E.MorrisA.MuthumaniA.MatthiasR.Design Implications of Drivers’ Engagement with Secondary Activities During Highly Automated Driving - A Longitudinal Simulator Study. Proc., Road Safety and Simulation International Conference (RSS2017), The Hague, The Netherlands, 2017.
5.
ZeebK.HärtelM.BuchnerA.SchraufM.Why is Steering Not the Same as Braking? The Impact of Non-Driving Related Tasks on Lateral and Longitudinal Driver Interventions During Conditionally Automated Driving. Transportation Research Part F: Traffic Psychology and Behaviour, Vol. 50, 2017, pp. 65–79.
6.
XiaoZ.HuZ.GengL.ZhangF.WuJ.LiY.Fatigue Driving Recognition Network: Fatigue Driving Recognition via Convolutional Neural Network and Long Short-Term Memory Units. IET Intelligent Transport Systems, Vol. 13, No. 9, 2019, pp. 1410–1416.
7.
EndsleyM. R.KirisE. O.The Out-of-the-Loop Performance Problem and Level of Control in Automation. Human Factors, Vol. 37, No. 2, 1995, pp. 381–394.
8.
ParasuramanR.DaviesD. R.A Taxonomic Analysis of Vigilance Performance. In Vigilance (MackieR. R., ed.), Springer, 1977, pp. 559–574.
9.
KaberD. B.PerryC. M.SegallN.McClernonC. K.PrinzelL. J.Situation Awareness Implications of Adaptive Automation for Information Processing in an Air Traffic Control-Related Task. International Journal of Industrial Ergonomics, Vol. 36, No. 5, 2006, pp. 447–462.
10.
SchömigN.HarguttV.NeukumA.Petermann-StockI.OthersenI.The Interaction Between Highly Automated Driving and the Development of Drowsiness. Procedia Manufacturing, Vol. 3, 2015, pp. 6652–6659.
11.
DeganiA.ShaftoM.KirlikA.Modes in Human-Machine Systems: Constructs, Representation, and Classification. The International Journal of Aviation Psychology, Vol. 9, No. 2, 1999, 125–138.
12.
EndsleyM. R.KaberD. B.Level of Automation Effects on Performance, Situation Awareness and Workload in a Dynamic Control Task. Ergonomics, Vol. 42, No. 3, 1999, 462–492.
13.
VogelpohlT.KühnM.HummelT.GehlertT.VollrathM.Transitioning to Manual Driving Requires Additional Time After Automation Deactivation. Transportation Research Part F: Traffic Psychology and Behaviour, Vol. 55, 2018, pp. 464–482.
14.
GonçalvesJ.HappeeR.BenglerK.Drowsiness in Conditional Automation: Proneness, Diagnosis and Driving Performance Effects. Proc., 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC), Rio de Janeiro, Brazil, IEEE, 2016, pp. 873–878.
15.
FeldhütterA.GoldC.SchneiderS.BenglerK.How the Duration of Automated Driving Influences Take-Over Performance and Gaze Behavior. 62. Kongress der Gesellschaft für Arbeitswissenschaft, Aachen, Germany, 2016.
16.
ToffettiA.WilschutE. S.MartensM. H.SchiebenA.RambaldiniA.MeratN.FlemischF.CityMobil: Human Factor Issues Regarding Highly Automated Vehicles on eLane. Transportation Research Record: Journal of the Transportation Research Board, 2009. 2110: 1–8.
17.
KörberM.GoldC.LechnerD.BenglerK.The Influence of Age on the Take-Over of Vehicle Control in Highly Automated Driving. Transportation Research Part F: Traffic Psychology and Behaviour, Vol. 39, 2016, pp. 19–32.
18.
MokB.JohnsM.MillerD.JuW.Tunneled in: Drivers With Active Secondary Tasks Need More Time to Transition From Automation. Proc., 2017 CHI Conference on Human Factors in Computing Systems, Denver, 2017, pp. 2840–2844.
19.
CarstenO.MartensM. H.How Can Humans Understand Their Automated Cars? HMI Principles, Problems and Solutions. Cognition, Technology & Work, Vol. 21, No. 1, 2018, pp. 3–20.
20.
KuehnM.VogelpohlT.VollrathM.KuehnM.VogelpohlT.VollrathM.Takeover Times in Highly Automated Driving, Level 3. Paper 17-0027. National Highway Traffic Safety Administration, Washington, D.C., 2017.
21.
ErikssonA.StantonN. A.Takeover Time in Highly Automated Vehicles: Noncritical Transitions to and From Manual Control. Human Factors, Vol. 59, No. 4, 2017, pp. 689–705.
22.
GoldC.DamböckD.BenglerK.LorenzL.Partially Automated Driving as a Fallback Level of High Automation. 6. Tagung Fahrerassistenzsysteme, 2013.
23.
IdogawaK.On the Brain Wave Activity of Professional Drivers During Monotonous Work. Behaviormetrika, Vol. 18, No. 30, 1991, pp. 23–34.
24.
VogelpohlT.KühnM.HummelT.VollrathM.Asleep at the Automated Wheel-Sleepiness and Fatigue During Highly Automated Driving. Accident Analysis & Prevention, Vol. 126, 2019, pp. 70–84.
25.
MattssonK.In-Vehicle Prediction of Truck Driver Sleepiness: Lane Positions Variables. M.S. thesis. Division of Media Technology, Department of Computer Science and Electrical Engineering, Lulea University of Technology, Sodertalje, Sweden, 2007.
26.
MalikH.NaeemF.ZuberiZ.ul HaqR.Vision Based Driving Simulation. Proc., International Conference on Cyberworlds, Tokyo, Japan, IEEE, 2004, pp. 255–259.
27.
MardiZ.AshtianiS. N.MikailiM.EEG-Based Drowsiness Detection for Safe Driving Using Chaotic Features and Statistical Tests. Journal of Medical Signals and Sensors, Vol. 1, 2011, pp. 130–137.
28.
JingD.ZhangS.GuoZ.Fatigue Driving Detection Method for Low-Voltage and Hypoxia Plateau Area: A Physiological Characteristic Analysis Approach. International Journal of Transportation Science and Technology, Vol. 9, No. 2, 2020, pp. 148–158.
29.
ReddyB.KimY. H.YunS.SeoC.JangJ.Real-Time Driver Drowsiness Detection for Embedded System Using Model Compression of Deep Neural Networks. Proc., IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, 2017, pp. 121–128.
30.
GhoddoosianR.GalibM.AthitsosV.A Realistic Dataset and Baseline Temporal Model for Early Drowsiness Detection. Proc., IEEE Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, 2019.
YoungM. S.StantonN. A.Back to the Future: Brake Reaction Times for Manual and Automated Vehicles. Ergonomics, Vol. 50, No. 1, 2007, pp. 46–58.
33.
ZeebK.BuchnerA.SchraufM.What Determines the Take-Over Time? An Integrated Model Approach of Driver Take-Over After Automated Driving. Accident Analysis & Prevention, Vol. 78, 2015, pp. 212–221.
34.
CranorL. F.A Framework for Reasoning About the Human in the Loop. Proc., 1st Conference on Usability, Psychology, and Security, San Francisco, CA, 2008.
35.
ErikssonA.StantonN. A.>The Chatty Co-Driver: A Linguistics Approach to Human-Automation-Interaction. Paper presented at the Contemporary Ergonomics and Human Factors, Daventry, 2016.
36.
GoldC.DamböckD.LorenzL.BenglerK.“Take Over!” How Long Does it Take to Get the Driver Back Into the Loop?Paper presented at the Proceedings of the Human Factors and Ergonomics Society Annual Meeting, SAGE Publications, 2013.
37.
MokB.JohnsM.LeeK. J.MillerD.SirkinD.IveP.JuW.Emergency, Automation Off: Unstructured Transition Timing for Distracted Drivers of Automated Vehicles. Proc., 2015 IEEE 18th International Conference on Intelligent Transportation Systems (ITSC), Gran Canaria, Spain, IEEE, 2015, pp. 2458–2464.
38.
BelderbosC.Authority Transition Interface: A Human Machine Interface for Taking over Control from a Highly Automated Truck. TU Delft, Delft University of Technology, 2015.
39.
MelcherV.RauhS.DiederichsF.WidlroitherH.BauerW.Take-Over Requests for Automated Driving. Procedia Manufacturing, Vol. 3, 2015, pp. 2867–2873.
40.
SoukupovaT.CechJ.Eye Blink Detection Using Facial Landmarks. Proc., 21st Computer Vision Winter Workshop, Rimske Toplice, Slovenia, 2016.
41.
CampbellJ. L.RichardC. M.BrownJ. L.McCallumM.Crash Warning System Interfaces: Human Factors Insights and Lessons Learned. Department of Transportation, National Highway Traffic Safety Administration, Washington, D.C., 2007.
42.
GrayR.Looming Auditory Collision Warnings for Driving. Human Factors, Vol. 53, No. 1, 2011, pp. 63–74.
43.
ScottJ. J.GrayR.A Comparison of Tactile, Visual, and Auditory Warnings Forrear-End Collision Prevention in Simulated Driving. Human Factors, Vol. 50, No. 2, 2008, pp. 264–275.
44.
HailinW.HanhuiL.ZhumeiS.Fatigue Driving Detection System Design Based on Driving Behavior. Proc., 2010 International Conference on Optoelectronics and Image Processing, Haikou, China, IEEE, Vol. 1, 2010, pp. 549–552.
45.
BianD.WadeJ. W.ZhangL.BekeleE.SwansonA.CrittendonJ. A.SarkarM.WarrenZ.SarkarN.A Novel Virtual Reality Driving Environment for Autism Intervention. Proc., International Conference on Universal Access in Human-Computer Interaction, Springer, Berlin, Heidelberg, 2013, pp. 474–483.
46.
JohansenS. A.San AgustinJ.SkovsgaardH.HansenJ. P.TallM.Low Cost vs. High-End Eye Tracking for Usability Testing. CHI’11 Extended Abstracts on Human Factors in Computing Systems. Vancouver, BC, Canada, ACM, 2011, pp. 1177–1182.
47.
ChalderT.BerelowitzG.PawlikowskaT.WattsL.WallaceE. P.Development of A Fatigue Scale. Journal of Psychosomatic Research, Vol. 37, No. 2, 1993, pp. 147–153.
48.
HartS. G.StavelandL. E.Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. Advances in Psychology, Vol. 52, 1988, pp. 139–183.
49.
JianJ. Y.BisantzA. M.DruryC. G.Foundations for an Empirically Determined Scale of Trust in Automated Systems. International Journal of Cognitive Ergonomics, Vol. 4, 2000, pp. 53–71.
