Restricted accessResearch articleFirst published online 2024-6
Predicting Fatigue-Associated Aberrant Driving Behaviors Using a Dynamic Weighted Moving Average Model With a Long Short-Term Memory Network Based on Heart Rate Variability
This study proposed a moving average (MA) approach to dynamically process heart rate variability (HRV) and developed aberrant driving behavior (ADB) prediction models by using long short-term memory (LSTM) networks.
Background
Fatigue-associated ADBs have traffic safety implications. Numerous models to predict such acts based on physiological responses have been developed but are still in embryonic stages.
Method
This study recorded the data of 20 commercial bus drivers during their routine tasks on four consecutive days and subsequently asked them to complete questionnaires, including subjective sleep quality, driver behavior questionnaire and the Karolinska Sleepiness Scale. Driving behaviors and corresponding HRV were determined using a navigational mobile application and a wristwatch. The dynamic-weighted MA (DWMA) and exponential-weighted MA were used to process HRV in 5-min intervals. The data were independently separated for training and testing. Models were trained with 10-fold cross-validation strategy, their accuracies were evaluated, and Shapley additive explanation (SHAP) values were used to determine feature importance.
Results
Significant increases in the standard deviation of NN intervals (SDNN), root mean square of successive heartbeat interval differences (RMSSD), and normalized spectrum of high frequency (nHF) were observed in the pre-event stage. The DWMA-based model exhibited the highest accuracy for both driver types (urban: 84.41%; highway: 80.56%). The SDNN, RMSSD, and nHF demonstrated relatively high SHAP values.
Conclusion
HRV metrics can serve as indicators of mental fatigue. DWMA-based LSTM could predict the occurrence of the level of fatigue associated with ADBs.
Application
The established models can be used in realistic driving scenarios.
AAA.Foundation (2009). Aggressive driving: Research update.
2.
AnundA.IhlstromJ.ForsC.KecklundG.FiltnessA. (2016). Factors associated with self-reported driver sleepiness and incidents in city bus drivers. Industrial health, 54(4), 337–346. https://doi.org/10.2486/indhealth.2015-0217
BaharavA.KotagalS.GibbonsV.RubinB. K.PrattG.KarinJ.AkselrodS. (1995). Fluctuations in autonomic nervous activity during sleep displayed by power spectrum analysis of heart rate variability. Neurology, 45(6), 1183–1187. https://doi.org/10.1212/wnl.45.6.1183
5.
BartelsR.NeumammL.PecanhaT.CarvalhoA. R. S. (2017). SinusCor: An advanced tool for heart rate variability analysis. Biomedical engineering online, 16(1), 110. https://doi.org/10.1186/s12938-017-0401-4
6.
BegumS. (2013). Intelligent driver monitoring systems based on physiological sensor signals: A review. 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).
7.
BenerA.YildirimE.ÖzkanT.LajunenT. (2017). Driver sleepiness, fatigue, careless behavior and risk of motor vehicle crash and injury: Population based case and control study. Journal of Traffic and Transportation engineering (English edition), 4(5), 496–502. https://doi.org/10.1016/j.jtte.2017.07.005
8.
BiggsH.DingsdagD.StensonN. (2009). Fatigue factors affecting metropolitan bus drivers: a qualitative investigation. Work, 32(1), 5–10. https://doi.org/10.3233/WOR-2009-0810
9.
BlincoeL.MillerT. R.ZaloshnjaE.LawrenceB. A. (2015). The economic and societal impact of motor vehicle crashes, 2010(Revised).
10.
BuendiaR.ForcolinF.KarlssonJ.Arne SjoqvistB.AnundA.CandefjordS. (2019). Deriving heart rate variability indices from cardiac monitoring-An indicator of driver sleepiness. Traffic injury prevention, 20(3), 249–254. https://doi.org/10.1080/15389588.2018.1548766
11.
CammA. J.MalikM.BiggerJ. T.BreithardtG.CeruttiS.CohenR. J.CoumelP.FallenE. L.KennedyH. L.KleigerR. E. (1996). Heart rate variability: Standards of measurement, physiological interpretation and clinical use. Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology. Circulation, 93(5), 1043–1065. https://doi.org/10.1161/01.cir.93.5.1043
12.
ChawlaN. V.BowyerK. W.HallL. O.KegelmeyerW. P. (2002). SMOTE: Synthetic minority over-sampling technique. Journal of artificial intelligence research, 16, 321–357. https://doi.org/10.1613/jair.953
13.
EdquistJ.Rudin-BrownC. M.LenneM. G. (2012). The effects of on-street parking and road environment visual complexity on travel speed and reaction time. Accident; analysis and prevention, 45, 759–765. https://doi.org/10.1016/j.aap.2011.10.001
14.
FortE.ChironM.DaveziesP.BergeretA.CharbotelB. (2013). Driving behaviors and on-duty road accidents: A French case-control study. Traffic injury prevention, 14(4), 353–359. https://doi.org/10.1080/15389588.2012.719091
15.
FurmanG. D.BaharavA.CahanC.AkselrodS. (2008). Early detection of falling asleep at the wheel: A heart rate variability approach. 2008 Computers in Cardiology.
16.
GalarzaE. E.EgasF. D.SilvaF. M.VelascoP. M.GalarzaE. D. (2018). Real time driver drowsiness detection based on driver’s face image behavior using a system of human computer interaction implemented in a smartphone. International Conference on Information Theoretic Security.
17.
GeislerF. C.KleinfeldtA.KubiakT. (2016). Restrained eating predicts effortful self-control as indicated by heart rate variability during food exposure. Appetite, 96, 502–508. https://doi.org/10.1016/j.appet.2015.10.020
18.
HealeyJ. A.PicardR. W. (2005). Detecting stress during real-world driving tasks using physiological sensors. IEEE Transactions on intelligent transportation systems, 6(2), 156–166. https://doi.org/10.1109/tits.2005.848368
19.
HsiungP.-A.LinY.-Y.HsuH.-C.UtomoD. (2018). Improving accuracy in fatigue detection/prediction by dynamic weighted moving average of heart-rate variabilities. 2018 IEEE 23rd International Conference on Digital Signal Processing (DSP).
20.
HsuC.-L.ChenC.-M.HongW.-Q.LiuY.-C. (2018). A system for prediction and warning of driving environment risk. T. I. P. Office. (Taiwan Patent No. M570495).
21.
HsuC.-L.ShihS.-J.ChenC.-L.HongW.-Q. (2019). Driving Function Evaluation System And Evaluation Device. T. I. P. Office. (Taiwan Patent No. I733293).
22.
KimH.JangT. W.KimH. R.LeeS. (2018). Evaluation for fatigue and accident risk of korean commercial bus drivers. The Tohoku journal of experimental medicine, 246(3), 191–197. https://doi.org/10.1620/tjem.246.191
23.
KokonoziA.ChouvardaI.MaglaverasN. (2008). EEG and HRV markers of sleepiness and loss of control during car driving. IEEE Engineering in Medicine and Biology Society.
24.
KoppelS.StephensA. N.CharltonJ. L.Di StefanoM.DarzinsP.OdellM.MarshallS. (2018). The driver behaviour questionnaire for older drivers: Do errors, violations and lapses change over time?Accident; analysis and prevention, 113, 171–178. https://doi.org/10.1016/j.aap.2018.01.036
25.
LawtonR.ParkerD.MansteadA. S.StradlingS. G. (1997). The role of affect in predicting social behaviors: The case of road traffic violations. Journal of applied social psychology, 27(14), 1258–1276. https://doi.org/10.1111/j.1559-1816.1997.tb01805.x
26.
LeiteA.RochaA. P.SilvaM. E. (2013). Beyond long memory in heart rate variability: an approach based on fractionally integrated autoregressive moving average time series models with conditional heteroscedasticity. Chaos, 23(2), 023103. https://doi.org/10.1063/1.4802035
27.
LiangW. C.YuanJ.SunD. C.LinM. H. (2009). Changes in physiological parameters induced by indoor simulated driving: effect of lower body exercise at mid-term break. Sensors (Basel), 9(9), 6913–6933. https://doi.org/10.3390/s90906913
28.
LinY.-Y.HsiungP.-A. (2017). An early warning system for predicting driver fatigue. 2017 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-TW).
29.
LuG.YangF.TaylorJ. A.SteinJ. F. (2009). A comparison of photoplethysmography and ECG recording to analyse heart rate variability in healthy subjects. Journal of medical engineering & technology, 33(8), 634–641. https://doi.org/10.3109/03091900903150998
30.
LuK.Sjörs DahlmanA.KarlssonJ.CandefjordS. (2022). Detecting driver fatigue using heart rate variability: A systematic review. Accident; analysis and prevention, 178, 106830. https://doi.org/10.1016/j.aap.2022.106830
31.
LuS.ZhaoH.JuK.ShinK.LeeM.ShelleyK.ChonK. H. (2008). Can photoplethysmography variability serve as an alternative approach to obtain heart rate variability information?Journal of clinical monitoring and computing, 22(1), 23–29. https://doi.org/10.1007/s10877-007-9103-y
32.
LundbergS. M.LeeS.-I. (2017). A unified approach to interpreting model predictions. Advances in neural information processing systems.
33.
MaZ.LiB. C.YanZ. (2016). Wearable driver drowsiness detection using electrooculography signal. 2016 IEEE Topical Conference on Wireless Sensors and Sensor Networks (WiSNet).
34.
MatuzA.van der LindenD.KisanderZ.HernadiI.KazmerK.CsathoA. (2021). Enhanced cardiac vagal tone in mental fatigue: Analysis of heart rate variability in Time-on-Task, recovery, and reactivity. PLoS One, 16(3), Article e0238670. https://doi.org/10.1371/journal.pone.0238670
35.
MaynardS.FiltnessA.MillerK.Pilkington-CheneyF. (2021). Bus driver fatigue: A qualitative study of drivers in London. Applied ergonomics, 92, 103309. https://doi.org/10.1016/j.apergo.2020.103309
36.
MokhtariK. E.HigdonB. P.BaşarA. (2019). Interpreting financial time series with SHAP values. Proceedings of the 29th Annual International Conference on Computer Science and Software Engineering.
37.
Muñoz-OrganeroM.Corcoba-MagañaV. (2017). Predicting upcoming values of stress while driving. IEEE Transactions on intelligent transportation systems, 18(7), 1802–1811. https://doi.org/10.1109/tits.2016.2618424
38.
MurataA.HiramatsuY. (2008). Evaluation of drowsiness by HRV measures-basic study for drowsy driver detection. Proceedings: Fourth International Workshop on Computational Intelligence & Applications.
39.
NaderiH.NassiriH.SahebiS. (2018). Assessing the relationship between heavy vehicle driver sleep problems and confirmed driver behavior measurement tools in Iran. Transportation research part F: traffic psychology and behaviour, 59, 57–66. https://doi.org/10.1016/j.trf.2018.08.007
NelsonD. M.PereiraA. C.de OliveiraR. A. (2017). Stock market's price movement prediction with LSTM neural networks. 2017 International joint conference on neural networks (IJCNN).
43.
QiW.WenH.WuY.QinL. (2017). Effect model of urban traffic congestion on driver’s lane-changing behavior. Advances in Mechanical Engineering, 9(9), 168781401772408. https://doi.org/10.1177/1687814017724087
44.
RajaratnamS. M.BargerL. K.LockleyS. W.SheaS. A.WangW.LandriganC. P.O'BrienC. S.QadriS.SullivanJ. P.CadeB. E.EpsteinL. J.WhiteD. P.CzeislerC. A.Harvard Work Hours Health and Safety Group (2011). Sleep disorders, health, and safety in police officers. JAMA, 306(23), 2567–2578. https://doi.org/10.1001/jama.2011.1851
45.
Rajendra AcharyaU.Paul JosephK.KannathalN.LimC. M.SuriJ. S. (2006). Heart rate variability: a review. Medical & biological engineering & computing, 44(12), 1031–1051. https://doi.org/10.1007/s11517-006-0119-0
46.
RibeiroM. T.SinghS.GuestrinC. (2016). " Why should I trust you?" Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining.
RocaJ.LupiáñezJ.López-RamónM.-F.CastroC. (2013). Are drivers’ attentional lapses associated with the functioning of the neurocognitive attentional networks and with cognitive failure in everyday life?Transportation research part F: traffic psychology and behaviour, 17, 98–113. https://doi.org/10.1016/j.trf.2012.10.005
49.
Rodriguez-IbañezN.García-GonzalezM. A.de la CruzM. A. F.Fernández-ChimenoM.Ramos-CastroJ. (2012). Changes in heart rate variability indexes due to drowsiness in professional drivers measured in a real environment. 2012 Computing in Cardiology.
50.
RundoF.ConociS.TrentaF.BattiatoS. (2019). Car-driver drowsiness monitoring by multi-layers deep learning framework and motion analysis. AISEM Annual Conference on Sensors and Microsystems.
51.
SakH.SeniorA.BeaufaysF. (2014). Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition. arXiv preprint arXiv:1402.1128https://doi.org/10.48550/arXiv.1402.1128
52.
SegerstromS. C.NesL. S. (2007). Heart rate variability reflects self-regulatory strength, effort, and fatigue. Psychological science, 18(3), 275–281. https://doi.org/10.1111/j.1467-9280.2007.01888.x
53.
ShafferF.GinsbergJ. P. (2017). An overview of heart rate variability metrics and norms. Frontiers in public health, 5, 258. https://doi.org/10.3389/fpubh.2017.00258
54.
ShahidA.WilkinsonK.MarcuS.ShapiroC. M. (2011). Karolinska sleepiness scale (KSS). In STOP, THAT and one hundred other sleep scales (pp. 209–210). Springer.
55.
ShinarZ.AkselrodS.DaganY.BaharavA. (2006). Autonomic changes during wake-sleep transition: a heart rate variability based approach. Autonomic neuroscience: basic & clinical, 130(1-2), 17–27. https://doi.org/10.1016/j.autneu.2006.04.006
56.
ŠtrumbeljE.KononenkoI. (2014). Explaining prediction models and individual predictions with feature contributions. Knowledge and information systems, 41(3), 647–665. https://doi.org/10.1007/s10115-013-0679-x
57.
TjollengA.JungK.HongW.LeeW.LeeB.YouH.SonJ.ParkS. (2017). Classification of a Driver's cognitive workload levels using artificial neural network on ECG signals. Applied ergonomics, 59(Pt A), 326–332. https://doi.org/10.1016/j.apergo.2016.09.013
58.
TonhajzerovaI.OndrejkaI.TurianikovaZ.JavorkaK.CalkovskaA.JavorkaM. (2012). Heart rate variability: An index of the brain-heart interaction. Tachycardia.
VicenteJ.LagunaP.BartraA.BailónR. (2016). Drowsiness detection using heart rate variability. Medical & biological engineering & computing, 54(6), 927–937. https://doi.org/10.1007/s11517-015-1448-7
61.
VigoD. E.DominguezJ.GuinjoanS. M.ScaramalM.RuffaE.SolernóJ.SiriL. N.CardinaliD. P. (2010). Nonlinear analysis of heart rate variability within independent frequency components during the sleep–wake cycle. Autonomic neuroscience: basic & clinical, 154(1-2), 84–88. https://doi.org/10.1016/j.autneu.2009.10.007
62.
WangX.XuC. (2016). Driver drowsiness detection based on non-intrusive metrics considering individual specifics. Accident; analysis and prevention, 95(Pt B), 350–357. https://doi.org/10.1016/j.aap.2015.09.002
63.
WangY.LiL.PratoC. G. (2019). The relation between working conditions, aberrant driving behaviour and crash propensity among taxi drivers in China. Accident; analysis and prevention, 126, 17–24. https://doi.org/10.1016/j.aap.2018.03.028
64.
WilliamsonA.LombardiD. A.FolkardS.StuttsJ.CourtneyT. K.ConnorJ. L. (2011). The link between fatigue and safety. Accident; analysis and prevention, 43(2), 498–515. https://doi.org/10.1016/j.aap.2009.11.011
65.
YouZ.GaoY.ZhangJ.ZhangH.ZhouM.WuC. (2017). A study on driver fatigue recognition based on SVM method. 2017 4th International Conference on Transportation Information and Safety (ICTIS).
66.
ZengC.WangW.ChenC.ZhangC.ChengB. (2020). Sex differences in time-domain and frequency-domain heart rate variability measures of fatigued drivers. International journal of environmental research and public health, 17(22), 8499. https://doi.org/10.3390/ijerph17228499
67.
ZhangC.VarvarigouV.ParksP. D.GautamS.BuenoA. V.MalhotraA.KalesS. N. (2012). Psychomotor vigilance testing of professional drivers in the occupational health clinic: A potential objective screen for daytime sleepiness. Journal of occupational and environmental medicine, 54(3), 296–302. https://doi.org/10.1097/JOM.0b013e318223d3d6
68.
ZhaoC.ZhaoM.LiuJ.ZhengC. (2012). Electroencephalogram and electrocardiograph assessment of mental fatigue in a driving simulator. Accident; analysis and prevention, 45, 83–90. https://doi.org/10.1016/j.aap.2011.11.019
69.
ZhaoZ.ChenH. P.HuangY.ZhangS. B.LiZ. H.FengT.LiuJ. K. (2017). LSTM network: a deep learning approach for short-term traffic forecast. Phytochemistry, 142(2), 68–75. https://doi.org/10.1016/j.phytochem.2017.06.020
70.
ZhaoZ.ZhouN.ZhangL.YanH.XuY.ZhangZ. (2020). Driver Fatigue Detection Based on Convolutional Neural Networks Using EM-CNN. Computational Intelligence and Neuroscience, 2020.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
