We examine the spatiotemporal dynamics of neural activity and its correlates in heart rate and its variability (HR/HRV) during a fatiguing visuospatial working memory task.
Background
The neural and physiological drivers of fatigue are complex, coupled, and poorly understood. Investigations that combine the fidelity of neural indices and the field-readiness of physiological measures can facilitate measurements of fatigue states in operational settings.
Method
Sixteen healthy adults, balanced by sex, completed a 60-minute fatiguing visuospatial working memory task. Changes in task performance, subjective measures of effort and fatigue, cerebral hemodynamics, and HR/HRV were analyzed. Peak brain activation, functional and effective connections within relevant brain networks were contrasted against spectral and temporal features of HR/HRV.
Results
Task performance elicited increased neural activation in regions responsible for maintaining working memory capacity. With the onset of time-on-task effects, resource utilization was seen to increase beyond task-relevant networks. Over time, functional connections in the prefrontal cortex were seen to weaken, with changes in the causal relationships between key regions known to drive working memory. HR/HRV indices were seen to closely follow activity in the prefrontal cortex.
Conclusion
This investigation provided a window into the neurophysiological underpinnings of working memory under the time-on-task effect. HR/HRV was largely shown to mirror changes in cortical networks responsible for working memory, therefore supporting the possibility of unobtrusive state recognition under ecologically valid conditions.
Applications
Findings here can inform the development of a fieldable index for cognitive fatigue.
AckermanP. L.KanferR.ShapiroS. W.NewtonS.BeierM. E. (2010). Cognitive fatigue during testing: An examination of trait, time-on-task, and strategy influences. Human Performance, 23(5), 381–402. https://doi.org/10.1080/08959285.2010.517720
2.
AhnS.NguyenT.JangH.KimJ. G.JunS. C. (2016). Exploring neuro-physiological correlates of drivers’ mental fatigue caused by sleep deprivation using simultaneous EEG, ECG, and fNIRS data. Frontiers in Human Neuroscience, 10(848), 219. https://doi.org/10.3389/fnhum.2016.00219
3.
BakemanR. (2005). Recommended effect size statistics for repeated measures designs. Behavior Research Methods, 37(3), 379–384.
BarnettL.SethA. K. (2014). The MVGC multivariate Granger causality toolbox: A new approach to Granger-causal inference. Journal of Neuroscience Methods, 223, 50–68. https://doi.org/10.1016/j.jneumeth.2013.10.018
BurrR. L. (2007). Interpretation of normalized spectral heart rate variability indices in sleep research: A critical review. Sleep, 30(7), 913–919. https://doi.org/10.1093/sleep/30.7.913
8.
CausseM.ChuaZ.PeysakhovichV.Del CampoN.MattonN. (2017). Mental workload and neural efficiency quantified in the prefrontal cortex using fNIRS. Scientific Reports, 7(1), 5222. https://doi.org/10.1038/s41598-017-05378-x
9.
CausseM.DehaisF.PastorJ. (2011). Executive functions and pilot characteristics predict flight simulator performance in general aviation pilots. The International Journal of Aviation Psychology, 21(3), 217–234. https://doi.org/10.1080/10508414.2011.582441
10.
ChangC.MetzgerC. D.GloverG. H.DuynJ. H.HeinzeH.-J.WalterM. (2013). Association between heart rate variability and fluctuations in resting-state functional connectivity. Neuroimage, 68, 93–104. https://doi.org/10.1016/j.neuroimage.2012.11.038
11.
CondyE. E.FriedmanB. H.GandjbakhcheA. (2020). Probing neurovisceral integration via functional near-infrared spectroscopy and heart rate variability. Frontiers in Neuroscience, 14, 575589. https://doi.org/10.3389/fnins.2020.575589
12.
D’EspositoM.DetreJ. A.AlsopD. C.ShinR. K.AtlasS.GrossmanM. (1995). The neural basis of the central executive system of working memory. Nature, 378(6554), 279–281. https://doi.org/10.1038/378279a0
13.
DehaisF.DupresA.Di FlumeriG.VerdiereK.BorghiniG.BabiloniF.RoyR. (2018). Monitoring pilot’s cognitive fatigue with engagement features in simulated and actual flight conditions using an hybrid fNIRS-EEG passive BCI. In IEEE International Conference on Systems, Man, and Cybernetics (SMC), Miyazaki, Japan, 7–10 October 2018, pp. 544–549.
14.
DimitrakopoulosG. N.KakkosI.DaiZ.WangH.SgarbasK.ThakorN.BezerianosA.SunY. (2018). Functional connectivity analysis of mental fatigue reveals different network topological alterations between driving and vigilance tasks. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 26(4), 740–749. https://doi.org/10.1109/TNSRE.2018.2791936
FanJ.SmithA. P. (2017). The impact of workload and fatigue on performance. In International Symposium on Human Mental Workload: Models and Applications, Dublin, 28–30 June 2017, pp. 90–105.
17.
ForteG.FavieriF.CasagrandeM. (2019). Heart rate variability and cognitive function: A systematic review. Frontiers in Neuroscience, 13, 710. https://doi.org/10.3389/fnins.2019.00710
GarrisonK. E. (2020). The influence of performance incentives on subjective experiences of mental effort [PhD thesis, Texas A&M University].
20.
GergelyfiM.JacobB.OlivierE.ZénonA. (2015). Dissociation between mental fatigue and motivational state during prolonged mental activity. Frontiers in Behavioral Neuroscience, 9, 176. https://doi.org/10.3389/fnbeh.2015.00176
21.
GrechM. R.NealA.YeoG.HumphreysM.SmithS. (2009). An examination of the relationship between workload and fatigue within and across consecutive days of work: Is the relationship static or dynamic?Journal of Occupational Health Psychology, 14(3), 231–242. https://doi.org/10.1037/a0014952
22.
HopstakenJ. F.Van Der LindenD.BakkerA. B.KompierM. A. J. (2015a). A multifaceted investigation of the link between mental fatigue and task disengagement. Psychophysiology, 52(3), 305–315. https://doi.org/10.1111/psyp.12339
23.
HopstakenJ. F.van der LindenD.BakkerA. B.KompierM. A. J. (2015b). The window of my eyes: Task disengagement and mental fatigue covary with pupil dynamics. Biological Psychology, 110, 100–106. https://doi.org/10.1016/j.biopsycho.2015.06.013
24.
HuppertT. J. (2016). Commentary on the statistical properties of noise and its implication on general linear models in functional near-infrared spectroscopy. Neurophotonics, 3(1), 010401. https://doi.org/10.1117/1.NPh.3.1.010401
25.
HuppertT. J.DiamondS. G.FranceschiniM. A.BoasD. A. (2009). HomER: A review of time-series analysis methods for near-infrared spectroscopy of the brain. Applied Optics, 48(10), D280–D298. https://doi.org/10.1364/ao.48.00d280
26.
KaneM. J.EngleR. W. (2002). The role of prefrontal cortex in working-memory capacity, executive attention, and general fluid intelligence: An individual-differences perspective. Psychonomic Bulletin & Review, 9(4), 637–671. https://doi.org/10.3758/bf03196323
27.
KarthikeyanR.SmootM. R.MehtaR. K. (2021). Anodal tDCS augments and preserves working memory beyond time-on-task deficits. Scientific Reports, 11(1), 19134. https://doi.org/10.1038/s41598-021-98636-y
28.
KlaassenE. B.de GrootR. H. M.EversE. A. T.SnelJ.VeermanE. C. I.LigtenbergA. J. M.JollesJ.VeltmanD. J. (2013). The effect of caffeine on working memory load-related brain activation in middle-aged males. Neuropharmacology, 64, 160–167. https://doi.org/10.1016/j.neuropharm.2012.06.026
29.
KrimskyM.ForsterD. E.LlabreM. M.JhaA. P. (2017). The influence of time on task on mind wandering and visual working memory. Cognition, 169, 84–90. https://doi.org/10.1016/j.cognition.2017.08.006
30.
LimJ.WuW.WangJ.DetreJ. A.DingesD. F.RaoH. (2010). Imaging brain fatigue from sustained mental workload: An ASL perfusion study of the time-on-task effect. Neuroimage, 49(4), 3426–3435. https://doi.org/10.1016/j.neuroimage.2009.11.020
31.
LiC.ZhengC.TaiC. (1995). Detection of ECG characteristic points using wavelet transforms. IEEE Transactions on Biomedical Engineering, 42(1), 21–28. https://doi.org/10.1109/10.362922
32.
LohaniM.PayneB. R.StrayerD. L. (2019). A review of psychophysiological measures to assess cognitive states in real-world driving. Frontiers in Human Neuroscience, 13, 57. https://doi.org/10.3389/fnhum.2019.00057
33.
Luque-CasadoA.PeralesJ. C.CárdenasD.SanabriaD. (2016). Heart rate variability and cognitive processing: The autonomic response to task demands. Biological Psychology, 113, 83–90. https://doi.org/10.1016/j.biopsycho.2015.11.013
34.
MarkedV. (1995). Correction of the heart rate variability signal for ectopics and missing beats. In MalikM.CammA. J. (Eds.), Heart rate variability (pp. 75–85). Futura. https://ci.nii.ac.jp/naid/10018620750/
35.
MatthewsG.HancockP. A. (2017). The handbook of operator fatigue. CRC Press.
36.
MehtaR. K.AgnewM. J. (2015). Subjective evaluation of physical and mental workload interactions across different muscle groups. Journal of Occupational and Environmental Hygiene, 12(1), 62–68. https://doi.org/10.1080/15459624.2014.942455
37.
MehtaR. K.NuamahJ.PeresS. C.MurphyR. R. (2020). Field methods to quantify emergency responder fatigue: Lessons learned from sUAS deployment at the 2018 Kilauea volcano eruption. IISE Transactions on Occupational Ergonomics and Human Factors, 8(3), 166–174. https://doi.org/10.1080/24725838.2020.1855272
38.
MehtaR. K.PeresS. C.KannanP.RheeJ.ShortzA. E.MannanM. S. (2017). Comparison of objective and subjective operator fatigue assessment methods in offshore shiftwork. Journal of Loss Prevention in the Process Industries, 48, 376–381. https://doi.org/10.1016/j.jlp.2017.02.009
39.
MöckelT.BesteC.WascherE. (2015). The effects of time on task in response selection-an ERP study of mental fatigue. Scientific Reports, 5(1), 10113. https://doi.org/10.1038/srep10113
40.
NeeD. E.D’EspositoM. (2016). The representational basis of working memory. Behavioral Neuroscience of Learning and Memory, 37, 213–230. https://doi.org/10.1007/7854_2016_456
41.
NuamahJ. K.MantoothW.KarthikeyanR.MehtaR. K.RyuS. C. (2019). Neural efficiency of human–robotic feedback modalities under stress differs with gender. Frontiers in Human Neuroscience, 13, 287. https://doi.org/10.3389/fnhum.2019.00287
42.
O’DonnellR. D. (1986). Workload assessment methodology. In BoffK. R.KaufmanL.ThomasJ. P. (Eds.), Handbook of perception and human performance, Vol. 2. Cognitive processes and performance (pp. 1–49). John Wiley & Sons.
43.
OwenA. M.McMillanK. M.LairdA. R.BullmoreE. (2005). N-back working memory paradigm: A meta-analysis of normative functional neuroimaging studies. Human Brain Mapping, 25(1), 46–59. https://doi.org/10.1002/hbm.20131
44.
OwensM. M.DudaB.SweetL. H.& MacKillopJ. (2018). Distinct functional and structural neural underpinnings of working memory. NeuroImage, 174, 463–471. https://doi.org/10.1016/j.neuroimage.2018.03.022
45.
PerssonJ.WelshK. M.JonidesJ.Reuter-LorenzP. A. (2007). Cognitive fatigue of executive processes: Interaction between interference resolution tasks. Neuropsychologia, 45(7), 1571–1579. https://doi.org/10.1016/j.neuropsychologia.2006.12.007
46.
PetruoV. A.MückschelM.BesteC. (2018). On the role of the prefrontal cortex in fatigue effects on cognitive flexibility - a system neurophysiological approach. Scientific Reports, 8(1), 6395. https://doi.org/10.1038/s41598-018-24834-w
47.
QiP.RuH.GaoL.ZhangX.ZhouT.TianY.ThakorN.BezerianosA.LiJ.SunY. (2019). Neural mechanisms of mental fatigue revisited: New insights from the brain connectome. Engineering, 5(2), 276–286. https://doi.org/10.1016/j.eng.2018.11.025
48.
RheeJ.MehtaR. K. (2018). Functional connectivity during handgrip motor fatigue in older adults is obesity and sex-specific. Frontiers in Human Neuroscience, 12, 455. https://doi.org/10.3389/fnhum.2018.00455
Sala-LlonchR.Peña-GómezC.Arenaza-UrquijoE. M.Vidal-PiñeiroD.BargallóN.JunquéC.Bartrés-FazD. (2012). Brain connectivity during resting state and subsequent working memory task predicts behavioural performance. Cortex, 48(9), 1187–1196. https://doi.org/10.1016/j.cortex.2011.07.006
51.
ShigiharaY.TanakaM.IshiiA.TajimaS.KanaiE.FunakuraM.WatanabeY. (2013). Two different types of mental fatigue produce different styles of task performance. Neurology, Psychiatry and Brain Research, 19(1), 5–11. https://doi.org/10.1016/j.npbr.2012.07.002
52.
StrasserF.MumaM.ZoubirA. M. (2012). Motion artifact removal in ECG signals using multi-resolution thresholding. Proceedings of the 20th European signal processing conference (EUSIPCO), Bucharest, Romania, 27–31 August 2012, pp, 899–903.
53.
SunY.LimJ.KwokK.BezerianosA. (2014). Functional cortical connectivity analysis of mental fatigue unmasks hemispheric asymmetry and changes in small-world networks. Brain and Cognition, 85C(1), 220–230. https://doi.org/10.1016/j.bandc.2013.12.011
54.
TempleJ. G.WarmJ. S.DemberW. N.JonesK. S.LaGrangeC. M.MatthewsG. (2000). The effects of signal salience and caffeine on performance, workload, and stress in an abbreviated vigilance task. Human Factors, 42(2), 183–194. https://doi.org/10.1518/001872000779656480
55.
ThayerJ. F.HansenA. L.Saus-RoseE.JohnsenB. H. (2009). Heart rate variability, prefrontal neural function, and cognitive performance: The neurovisceral integration perspective on self-regulation, adaptation, and health. Annals of Behavioral Medicine, 37(2), 141–153. https://doi.org/10.1007/s12160-009-9101-z
56.
ThayerJ. F.HansenA. L.SollersJohnsenB. H. (2005 May). Heart rate variability as an index of prefrontal neural function in military settings. In Biomonitoring for physiological and cognitive performance during military. In operations (Vol. 5797, pp. 71–77). International Society for Optics and Photonics.
57.
ThayerJ. F.LaneR. D. (2000). A model of neurovisceral integration in emotion regulation and dysregulation. Journal of Affective Disorders, 61(3), 201–216. https://doi.org/10.1016/s0165-0327(00)00338-4
58.
WangH.LiuX.HuH.WanF.LiT.GaoL.BezerianosA.SunY.JungT.-P. (2020). Dynamic reorganization of functional connectivity unmasks fatigue related performance declines in simulated driving. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 28(8), 1790–1799. https://doi.org/10.1109/TNSRE.2020.2999599
59.
YuS.BezerianosA.ThakorN.LiJ. (2018). Functional brain network analysis reveals time-on-task related performance decline. In 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Honolulu, HI, 18–21 July 2018, pp. 271–274.
60.
ZhaoC.ZhaoM.YangY.GaoJ.RaoN.LinP. (2016). The reorganization of human brain networks modulated by driving mental fatigue. IEEE Journal of Biomedical and Health Informatics, 21(3), 743–755. https://doi.org/10.1109/JBHI.2016.2544061
61.
ZhuY.JankayR. R.PierattL. C.MehtaR. K. (2017). Wearable sensors and their metrics for measuring comprehensive occupational fatigue: A scoping review. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 61(1), 1041–1045. https://doi.org/10.1177/1541931213601744
62.
ZhuY.Rodriguez-ParasC.RheeJ.MehtaR. K. (2020). Methodological approaches and recommendations for functional near-infrared spectroscopy applications in HF/E research. Human Factors, 62(4), 613–642. https://doi.org/10.1177/0018720819845275
