The Impacts of Temporal Variation and Individual Differences in Driver Cognitive Workload on ECG-Based Detection

Abstract

Objective

This paper aimed to investigate the robustness of driver cognitive workload detection based on electrocardiogram (ECG) when considering temporal variation and individual differences in cognitive workload.

Background

Cognitive workload is a critical component to be monitored for error prevention in human–machine systems. It may fluctuate instantaneously over time even in the same tasks and differ across individuals.

Method

A driving simulation study was conducted to classify driver cognitive workload underlying four experimental conditions (baseline, N-back, texting, and N-back + texting distraction) in two repeated 1-hr blocks. Heart rate (HR) and heart rate variability (HRV) were compared among the experimental conditions and between the blocks. Random forests were built on HR and HRV to classify cognitive workload in different blocks and for different individuals.

Results

HR and HRV were significantly different between repeated blocks in the study, demonstrating the time-induced variation in cognitive workload. The performance of cognitive workload classification across blocks and across individuals was significantly improved after normalizing HR and HRV in each block by the corresponding baseline.

Conclusion

The temporal variation and individual differences in cognitive workload affects ECG-based cognitive workload detection. But normalization approaches relying on the choice of appropriate baselines help compensate for the effects of temporal variation and individual differences.

Application

The findings provide insight into the value and limitations of ECG-based driver cognitive workload monitoring during prolonged driving for individual drivers.

Keywords

driver cognitive workload temporal variation individual differences heart rate variability multiclass classification baseline normalization

Get full access to this article

View all access options for this article.

References

Bagiella

Sloan

R. P.

Heitjan

D. F

. (2000). Mixed‐effects models in psychophysiology. Psychophysiology, 37, 13–20.doi:10.1111/1469-8986.3710013

10705763

Baldwin

C. L.

Roberts

D. M.

Barragan

Lee

J. D.

Lerner

Higgins

J. S

. (2017). Detecting and quantifying mind wandering during simulated driving. Frontiers in Human Neuroscience, 11, 406.doi:10.3389/fnhum.2017.00406

28848414

Ben-Shachar

Makowski

Lüdecke

. (2020). Compute and interpret indices of effect size. CRAN. R package. https://github.com/easystats/effectsize

Borghini

Astolfi

Vecchiato

Mattia

Babiloni

. (2014). Measuring neurophysiological signals in aircraft pilots and CAR drivers for the assessment of mental workload, fatigue and drowsiness. Neuroscience & Biobehavioral Reviews, 44, 58–75.doi:10.1016/j.neubiorev.2012.10.003

23116991

Breiman

. (2001). Random forests. Machine Learning, 45, 5–32.doi:10.1023/A:1010933404324

Brookhuis

K. A.

de Waard

. (2010). Monitoring drivers’ mental workload in driving simulators using physiological measures. Accident Analysis & Prevention, 42, 898–903.doi:10.1016/j.aap.2009.06.001

20380918

Cannon

Krokhmal

P. A.

Chen

Murphey

. (2012). Detection of temporal changes in psychophysiological data using statistical process control methods. Computer Methods and Programs in Biomedicine, 107, 367–381.doi:10.1016/j.cmpb.2011.01.003

21377752

Conway

Dick

Wang

Chen

. (2013). The effect of stress on cognitive load measurement. IFIP Conference on Human-Computer Interaction, 659–666.

de Waard

. (1996). The measurement of drivers’ mental workload. Groningen University, Traffic Research Center.

10.

Dehghani

Glatard

Shihab

. (2019). Subject cross validation in human activity recognition. arXiv preprint arXiv:1904, 02666.

11.

Dietterich

T. G

. (1998). Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation, 10, 1895–1923.doi:10.1162/089976698300017197

9744903

12.

Hancock

P. A.

Caird

J. K

. (1993). Experimental evaluation of a model of mental workload. Human Factors: The Journal of the Human Factors and Ergonomics Society, 35, 413–429.doi:10.1177/001872089303500303

8244409

13.

Healey

J. A.

Picard

R. W

. (2005). Detecting stress during real-world driving tasks using physiological sensors. IEEE Transactions on Intelligent Transportation Systems, 6, 156–166.doi:10.1109/TITS.2005.848368

14.

Hughes

A. M.

Hancock

G. M.

Marlow

S. L.

Stowers

Salas

. (2019). Cardiac measures of cognitive workload: A meta-analysis. Human Factors: The Journal of the Human Factors and Ergonomics Society, 61, 393–414.doi:10.1177/0018720819830553

30822151

15.

Jeppesen

Beniczky

Johansen

Sidenius

Fuglsang-Frederiksen

. (2015). Detection of epileptic seizures with a modified heart rate variability algorithm based on Lorenz plot. Seizure, 24, 1–7.doi:10.1016/j.seizure.2014.11.004

25564311

16.

Kantowitz

B. H.

Casper

P. A

. (1988). Human workload in aviation. Human Factors in Aviation, 157–187.

17.

Koenig

Rehg

Rasshofer

. (2015). Statistical sensor fusion of ECG data using automotive-grade sensors. Advances in Radio Science, 13, 197–202.doi:10.5194/ars-13-197-2015

18.

Kramer

A. F

. (1991). Physiological metrics of mental workload: A review of recent progress. Multiple-task Performance, 279–328.

19.

Kuznetsova

Brockhoff

P. B.

Christensen

R. H. B

. (2015). Package ‘lmertest’. R Package Version . 2(0).

20.

Lenth

Singmann

Love

Buerkner

Herve

. (2018). Emmeans: Estimated marginal means, AKA least-squares means. R Package Version, 1, 3.

21.

Liang

Reyes

M. L.

Lee

J. D

. (2007). Real-time detection of driver cognitive distraction using support vector machines. IEEE Transactions on Intelligent Transportation Systems, 8, 340–350.doi:10.1109/TITS.2007.895298

22.

Lohani

Payne

B. R.

Strayer

D. L

. (2019). A review of psychophysiological measures to assess cognitive states in real-world driving. Frontiers in Human Neuroscience, 13, 57.doi:10.3389/fnhum.2019.00057

30941023

23.

Mårtensson

Keelan

Ahlström

. (2018). Driver sleepiness classification based on physiological data and driving performance from real road driving. IEEE Transactions on Intelligent Transportation Systems, 20, 421–430.doi:10.1109/TITS.2018.2814207

24.

Matthews

Reinerman-Jones

L. E.

Barber

D. J.

Abich

. (2015). The psychometrics of mental workload: Multiple measures are sensitive but divergent. Human Factors, 57, 125–143.doi:10.1177/0018720814539505

25790574

25.

McDonald

A. D.

Ferris

T. K.

Wiener

T. A

. (2020). Classification of driver distraction: A comprehensive analysis of feature generation, machine learning, and input measures. Human Factors: The Journal of the Human Factors and Ergonomics Society, 62, 1019–1035.doi:10.1177/0018720819856454

31237788

26.

Mehler

Reimer

Dusek

J. A

. (2011). MIT AgeLab delayed digit recall task (n-back) (Vol. 17). Massachusetts Institute of Technology.

27.

Mehler

Reimer

Wang

. (2011). A comparison of heart rate and heart rate variability indices in distinguishing single-task driving and driving under secondary cognitive workload [Conference session]. Proceedings of the Sixth International Driving Symposium on Human Factors in Driver Assessment, Training and Vehicle Design. Iowa City, IA, United States.

28.

Miyaji

Kawanaka

Oguri

. (2009). Driver’s cognitive distraction detection using physiological features by the adaboost. 12th International IEEE Conference on Intelligent Transportation Systems, 1–6.

29.

Oinuma

Kubo

Otsuka

Yamanaka

Murakami

Matsuoka

Ohkawa

Cornélissen

Weydahl

Holmeslet

Hall

Halberg

. (2002). Graded response of heart rate variability, associated with an alteration of geomagnetic activity in a subarctic area. Biomedicine & Pharmacotherapy, 56, 284–288.doi:10.1016/S0753-3322(02)00303-7

30.

Pedregosa

Varoquaux

Gramfort

Michel

Thirion

Grisel

Vanderplas

. (2011). Scikit-learn: Machine learning in python. The Journal of Machine Learning Research, 12, 2825–2830.

31.

Peltola

M. A

. (2012). Role of editing of R-R intervals in the analysis of heart rate variability. Frontiers in Physiology, 3, 148.doi:10.3389/fphys.2012.00148

22654764

32.

Pinto

J. R.

Cardoso

J. S.

Lourenço

Carreiras

. (2017). Towards a continuous biometric system based on ECG signals acquired on the steering wheel. Sensors, 17, 2228.doi:10.3390/s17102228

28956856

33.

Sarkar

Ross

Ruberto

A. J.

Rodenburg

Hungler

Etemad

. (2019). Classification of cognitive load and expertise for adaptive simulation using deep multitask learning. arXiv Preprint arXiv:1908.00385.

34.

Schmidt

E. A.

Schrauf

Simon

Fritzsche

Buchner

Kincses

W. E

. (2009). Drivers’ misjudgement of vigilance state during prolonged monotonous daytime driving. Accident Analysis & Prevention, 41, 1087–1093.doi:10.1016/j.aap.2009.06.007

19664450

35.

Shaffer

Ginsberg

J. P

. (2017). An overview of heart rate variability metrics and norms. Frontiers in Public Health, 5, 258.doi:10.3389/fpubh.2017.00258

29034226

36.

Shahid

Wilkinson

Marcu

Shapiro

C. M

. (2011). Karolinska sleepiness scale (KSS). In STOP, THAT and one hundred other sleep scales(Springer Science & Business Media pp. pp. 209–210).

37.

Solovey

E. T.

Zec

Garcia Perez

E. A.

Reimer

Mehler

. (2014). Classifying driver workload using physiological anddriving performance data: Two field studies [Conference session].Proceedings of the SIGCHI Conference on Human Factors inComputing Systems 4057–4066.

38.

Thiffault

Bergeron

. (2003). Monotony of road environment and driver fatigue: A simulator study. Accident Analysis & Prevention, 35, 381–391.doi:10.1016/S0001-4575(02)00014-3

12643955

39.

Tjolleng

Jung

Hong

Lee

You

Son

Park

. (2017). Classification of a driver’s cognitive workload levels using artificial neural network on ECG signals. Applied Ergonomics, 59, 326–332.doi:10.1016/j.apergo.2016.09.013

27890144

40.

Toichi

Sugiura

Murai

Sengoku

. (1997). A new method of assessing cardiac autonomic function and its comparison with spectral analysis and coefficient of variation of R-R interval. Journal of the Autonomic Nervous System, 62, 79–84.doi:10.1016/S0165-1838(96)00112-9

9021653

41.

Tomasevic

Horberry

Young

Fildes

. (2019). Validation of a driving simulator for research into human factors issues of automated vehicles. Journal of the Australasian College of Road Safety, 30, 37.

42.

van Dellen

H. J.

Aasman

Mulder

L. J. M.

Mulder

. (1985). Time domain versus frequency domain measures of heart-rate variability. In Orlebeke

J. F.

Mulder

van Doornen

L. J. P.

(Eds.), The psychophysiology of cardiovascular control (pp. 353–374).

43.

Vicente

Laguna

Bartra

Bailón

. (2011). Detection of driver’s drowsiness by means of HRV analysis. 2011 Computing in Cardiology, 89–92.

44.

Wickens

C. D

. (2008). Multiple resources and mental workload. Human Factors: The Journal of the Human Factors and Ergonomics Society, 50, 449–455.doi:10.1518/001872008X288394

18689052

45.

Xie

Salvendy

. (2000). Prediction of mental workload in single and multiple tasks environments. International Journal of Cognitive Ergonomics, 4, 213–242.doi:10.1207/S15327566IJCE0403_3

46.

Yang

Ferris

T. K

. (2018). Cognitive efficiency in Human–Machine systems: Metrics of display effectiveness for supporting Multitask performance. Journal of Cognitive Engineering and Decision Making, 12, 153–169.doi:10.1177/1555343417712464

47.

Yang

Kuo

Lenné

M. G

. (2018). Analysis of gaze behavior to measure cognitive distraction in real-world driving [Conference session]. Proceedingsof the Human Factors and Ergonomics Society Annual Meeting, 62, 1944–1948. Los Angeles, CA, United States.doi:10.1177/1541931218621441

48.

Yanko

M. R.

Spalek

T. M

. (2013). Route familiarity breeds inattention: A driving simulator study. Accident Analysis & Prevention, 57, 80–86.doi:10.1016/j.aap.2013.04.003

23643937