We aimed to provide an assessment of the impact of workload manipulations on various cardiac measurements. We further sought to determine the most effective measurement approaches of cognitive workload as well as quantify the conditions under which these measures are most effective for interpretation.
Background:
Cognitive workload affects human performance, particularly when load is relatively high (overload) or low (underload). Despite ongoing interest in assessing cognitive workload through cardiac measures, it is currently unclear which cardiac-based assessments best indicate cognitive workload. Although several quantitative studies and qualitative reviews have sought to provide guidance, no meta-analytic integration of cardiac assessment(s) of cognitive workload exists to date.
Method:
We used Morris and DeShon’s meta-analytic procedures to quantify the changes in cardiac measures due to task load conditions.
Results:
Sample-weighted Cohen’s d values suggest that several metrics of cardiac activity demonstrate sensitivity in response to cognitive workload manipulations. Heart rate variability measures show sensitivity to task load, conditions of event rate, and task duration. Authors of future work should seek to quantify the utility of leveraging multiple metrics to understand workload.
Conclusion:
Results suggest that assessment of cognitive workload can be done using various cardiac activity indicators. Further, given the number of valid and reliable measures available, researchers and practitioners should base their selection of a psychophysiological measure on the experimental and practical concerns inherent to their task/protocol.
Applications:
Findings bear implications for future assessment of cognitive workload within basic and applied settings. Future research should seek to validate conditions under which measurements are best interpreted, including but not limited to individual differences.
AbernethyB.MaxwellJ. P.MastersR. S.Van Der KampJ.JacksonR. C. (2007). Attentional processes in skill learning and expert performance. In TenenbaumG.EklundR. C. (Eds.), Handbook of sport psychology (pp. 245–263). Hoboken, NJ: Wiley.
2.
AckermanP. L.CiancioloA. T. (2000). Cognitive, perceptual speed, and psychomotor determinants of individual differences during skill acquisition. Journal of Experimental Psychology: Applied, 6, 259–290.
3.
AguinisH.GottfredsonR. K.JooH. (2013). Best-practice recommendations for defining, identifying, and handling outliers. Organizational Research Methods, 16, 270–301.
4.
AndersonC. A.LindsayJ. J.BushmanB. J. (1999). Research in the psychological laboratory: Truth or triviality?Current Directions in Psychological Science, 8, 3–9.
5.
*AndersonN. B.LaneJ. D.MonouH.WilliamsR. B.Jr.HouseworthS. J. (1988). Racial differences in cardiovascular reactivity to mental arithmetic. International Journal of Psychophysiology, 6, 161–164.
6.
*AndersonN. B.LaneJ. D.TaguchiF.WilliamsR. B. (1989). Patterns of cardiovascular responses to stress as a function of race and parental hypertension in men. Health Psychology, 8, 525–540.
7.
*AndersonN. B.WilliamsR. B.LaneJ. D.HouseworthS.MuranakaM. (1987). Parental history of hypertension and cardiovascular responses to behavioral stress in young Black women. Journal of Psychosomatic Research, 31, 723–729.
8.
ArthurW.Jr.BennettW.HuffcuttA. I. (2001). Conducting meta-analysis using SAS. London, UK: Psychology Press.
9.
*BacksR. W. (1997). Psychophysiological aspects of selective and divided attention during continuous manual tracking. Acta Psychologica, 96, 167–191.
10.
*BacksR. W.LennemanJ. K.SicardJ. L. (1999). The use of autonomic components to improve cardiovascular assessment of mental workload in flight. International Journal of Aviation Psychology, 9, 33–47.
11.
*BacksR. W.SeljosK. A. (1994). Metabolic and cardiorespiratory measures of mental effort: The effects of level of difficulty in a working memory task. International Journal of Psychophysiology, 16, 57–68.
12.
*BashiriB.MannD. D. (2014). Heart rate variability in response to task automation in agricultural semi-autonomous vehicles. Ergonomics Open Journal, 7, 6–12.
13.
BeckerB. J. (2015). Model-based meta-analysis. In CooperH.HedgesL.ValentineJ. C. (Eds.), The handbook of research synthesis and meta-analysis (pp. 377–395). New York, NY: Russell Sage Foundation.
14.
BlockR. A.HancockP. A.ZakayD. (2010). How cognitive load affects duration judgments: A meta-analytic review. Acta Psychologica, 134, 330–343.
15.
BoglerC.MehnertJ.SteinbrinkJ.HaynesJ. (2014). Decoding vigilance with NIRS. PLoS ONE, 9(7), 1–12.
16.
BorensteinM.HedgesL. V.HigginsJ.RothsteinH. R. (2009). Introduction to meta-analysis. Chichester, UK: Wiley.
17.
BrookingsJ. B.WilsonG. F.SwainC. R. (1996). Psychophysiological responses to changes in workload during simulated air traffic control. Biological Psychology, 42, 361–377.
18.
*BrouwerA. M.HogervorstM. A.HolewijnM.van ErpJ. B. (2014). Evidence for effects of task difficulty but not learning on neurophysiological variables associated with effort. International Journal of Psychophysiology, 93, 242–252.
19.
*BunceD. J.BarrowcloughA.MorrisI. (1996). The moderating influence of physical fitness on age gradients in vigilance and serial choice responding tasks. Psychology and Aging, 11, 671–682.
20.
*CarrollD.HarrisM. G.CrossG. (1991). Haemodynamic adjustments to mental stress in normotensives and subjects with mildly elevated blood pressure. Psychophysiology, 28, 438–446.
21.
*CausseM.SénardJ. M.DémonetJ. F.PastorJ. (2010). Monitoring cognitive and emotional processes through pupil and cardiac response during dynamic versus logical task. Applied Psychophysiology and Biofeedback, 35, 115–123.
22.
CooperH.HedgesL. V.ValentineJ. C. (2009). The handbook of research synthesis and meta-analysis. New York, NY: Russell Sage Foundation.
23.
*CullenR. H.AgnewM. J. (2016). Comparing different measures of overall workload in a multimodal postural/auditory dual-task environment. IIE Transactions on Occupational Ergonomics and Human Factors, 4, 115–127.
24.
*DahlstromN.NahlinderS. (2009). Mental workload in aircraft and simulator during basic civil aviation training. International Journal of Aviation Psychology, 19, 309–325.
25.
*de WaardD.KruizingaA.BrookhuisK. A. (2008). The consequences of an increase in heavy goods vehicles for passenger car drivers’ mental workload and behaviour: A simulator study. Accident Analysis & Prevention, 40, 818–828.
26.
DriskellJ. E.MullenB.JohnsonC.HughesS.BatchelorC. L. (1992). Development of quantitative specifications for simulating the stress environment. Wright-Patterson AFB, OH: Armstrong Lab.
27.
*EasonR. G.BeardshallA.JaffeeS. (1965). Performance and physiological indicants of activation in a vigilance situation. Perceptual and Motor Skills, 20, 3–13.
28.
EggemeierF. T.O’DonnellR. D. (1982). A conceptual framework for development of a workload assessment methodology. Dayton, OH: Wright State University.
29.
EggemeierF. T.WilsonG. F.KramerA. F.DamosD. L. (1991). General considerations concerning workload assessment in multi-task environments. In DamosD. L. (Ed.), Multiple task performance (pp. 207–216). London, UK: Taylor & Francis.
30.
*FahrenbergJ.FoersterF.WilmersF. (1995). Is elevated blood pressure level associated with higher cardiovascular responsiveness in laboratory tasks and with response specificity?Psychophysiology, 32, 81–91.
31.
*FredericksT. K.ChoiS. D.HartJ.ButtS. E.MitalA. (2005). An investigation of myocardial aerobic capacity as a measure of both physical and cognitive workloads. International Journal of Industrial Ergonomics, 35, 1097–1107.
32.
*FredriksonM.TuomistoM.LundbergU.MelinB. (1990). Blood pressure in healthy men and women under laboratory and naturalistic conditions. Journal of Psychosomatic Research, 34, 675–686.
33.
*GendollaG. H.RichterM. (2005). Ego involvement and effort: Cardiovascular, electrodermal, and performance effects. Psychophysiology, 42, 595–603.
34.
GrahamF. K. (1978a). Constraints on measuring heart rate and period sequentially through real and cardiac time. Psychophysiology, 15, 492–495.
35.
GrahamF. K. (1978b). Normality of distributions and homogeneity of variance of heart rate and heart rate period samples. Psychophysiology, 15, 487–491.
36.
*GrassmannM.VlemincxE.von LeupoldtA.Van den BerghO. (2017). Individual differences in cardiorespiratory measures of mental workload: An investigation of negative affectivity and cognitive avoidant coping in pilot candidates. Applied Ergonomics, 59, 274–282.
37.
HancockP. A. (2013). In search of vigilance: The problem of iatrogenically created psychological phenomena. American Psychologist, 68, 97–109.
38.
HancockP. A.RossJ. M.SzalmaJ. L. (2007). A meta-analysis of performance response under thermal stressors. Human Factors, 49, 851–877.
39.
HancockP. A.SzalmaJ. L. (2003). The future of neuroergonomics. Theoretical Issues in Ergonomics Science, 4, 238–249.
40.
HancockP. A.WarmJ. S. (1989). A dynamic model of stress and sustained attention. Human Factors, 31, 519–537.
41.
HeltonW. S.HollanderT. D.WarmJ. S.TrippL. D.ParsonsK.MatthewsG.. . . HancockP. A. (2007). The abbreviated vigilance task and cerebral hemodynamics. Journal of Clinical and Experimental Neuropsychology, 29, 545–552.
42.
HeltonW. S.WarmJ. S.TrippL. D.MatthewsG.ParasuramanR.HancockP. A. (2010). Cerebral lateralization of vigilance: A function of task difficulty. Neuropsychologia, 48, 1683–1688.
43.
*HoogendoornR.HoogendoornS.BrookhuisK.DaamenW. (2011). Adaptation longitudinal driving behavior, mental workload, and psycho-spacing models in fog. Transportation Research Record: Journal of the Transportation Research Board, 2249, 20–28.
44.
*HsuB. W.WangM. J. J.ChenC. Y.ChenF. (2015). Effective indices for monitoring mental workload while performing multiple tasks. Perceptual and Motor Skills, 121, 94–117.
45.
HughesA. M.VolanteW. G.StowersK.LeyvaK.OglesbyJ. M.BisbeyT.SalasE. (2014). Cognition and physiological response: Towards a model of validated physiological measurement. In Proceedings of the Human Factors and Ergonomics Society 58th Annual Meeting (pp. 1009–1013). Santa Monica, CA: Human Factors and Ergonomics Society.
46.
HunterJ. E.SchmidtF. L. (2004). Methods of meta-analysis: Correcting error and bias in research. Thousand Oaks, CA: Sage.
47.
*KaidaK.ÅkerstedtT.KecklundG.NilssonJ. P.AxelssonJ. (2007). The effects of asking for verbal ratings of sleepiness on sleepiness and its masking effects on performance. Clinical Neurophysiology, 118, 1324–1331.
48.
KalsbeekJ. (1971). Sinus arrhythmia and the dual task method in measuring mental load. In FoxJ.WhitfieldD. (Eds.), Measurement of man at work. London, UK: Taylor & Francis.
49.
*Keltikangas-JärvinenL.HeponiemiT. (2004). Vital exhaustion, temperament, and cardiac reactivity in task-induced stress. Biological Psychology, 65, 121–135.
50.
KramerA. F. (1991). Physiological metrics of mental workload: A review of recent progress. In DamosD. L. (Eds.), Multiple-task performance (pp. 279–328). London, UK: Taylor & Francis.
51.
*LacknerH. K.GoswamiN.Hinghofer-SzalkayH.PapousekI.ScharfetterH.FurlanR.SchwabergerG. (2010). Effects of stimuli on cardiovascular reactivity occurring at regular intervals during mental stress. Journal of Psychophysiology, 24, 48–60.
52.
LandisR. S. (2013). Successfully combining meta-analysis and structural equation modeling: Recommendations and strategies. Journal of Business and Psychology, 28, 251–261.
53.
LevyM. N. (1971). Brief reviews: Sympathetic-parasympathetic interactions in the heart. Circulation Research, 29, 437–445.
54.
MarinescuA. C.SharplesS.RitchieA. C.Sanchez LopezT.McDowellM.MorvanH. P. (2018). Physiological parameter response to variation of mental workload. Human Factors, 60, 31–56.
55.
*MasalonisA. J.DuleyJ. A.ParasuramanR. (1999). Effects of manual and autopilot control on mental workload and vigilance during simulated general aviation flight. Transportation Human Factors, 1, 187–200.
56.
*MascordD. J.HeathR. A. (1992). Behavioral and physiological indices of fatigue in a visual tracking task. Journal of Safety Research, 23, 19–25.
57.
MatthewsG.CampbellS. (1998). Task-induced stress and individual styles of coping. In Proceedings of the Human Factors and Ergonomics 42nd Annual Meeting (pp. 821–825). Santa Monica, CA: Human Factors and Engineering Society.
58.
MatthewsG.CampbellS. E.FalconerS.JoynerL. A.HugginsJ.GillilandK.. . . WarmJ. S. (2002). Fundamental dimensions of subjective state in performance settings: Task engagement, distress, and worry. Emotion, 2, 315–340.
59.
*MatthewsG.Reinerman-JonesL. E.BarberD. J.AbichJ.IV. (2015). The psychometrics of mental workload: Multiple measures are sensitive but divergent. Human Factors, 57, 125–143.
60.
*McCubbinJ. A.PeachH.MooreD. D.PilcherJ. J. (2012). Decreased cognitive/CNS function in young adults at risk for hypertension: Effects of sleep deprivation. International Journal of Hypertension, 2012, 1–9.
61.
MehtaR. K.ParasuramanR. (2013). Neuroergonomics: A review of applications to physical and cognitive work. Frontiers in Human Neuroscience, 7, 889.
62.
*MelamedS.Kristal-BonehE.HarariG.FroomP.RibakJ. (1998). Variation in the ambulatory blood pressure response to daily work load: The moderating role of job control. Scandinavian Journal of Work, Environment & Health, 24, 190–196.
63.
*MetalisS. A. (1991). Heart period as a useful index of pilot workload in commercial transport aircraft. International Journal of Aviation Psychology, 1, 107–116.
64.
*MetzgerU.ParasuramanR. (2005). Automation in future air traffic management: Effects of decision aid reliability on controller performance and mental workload. Human Factors, 47, 35–49.
65.
MorrisS. B.DeShonR. P. (2002). Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychological Methods, 7, 105–125.
66.
*NeumannD. L. (2002). Effect of varying levels of mental workload on startle eyeblink modulation. Ergonomics, 45, 583–602.
67.
*NickelP.NachreinerF. (2003). Sensitivity and diagnosticity of the 0.1-Hz component of heart rate variability as an indicator of mental workload. Human Factors, 45, 575–590.
68.
NunnallyJ. C. (1978). Psychometric theory. New York, NY: McGraw Hill.
69.
ObristP. A.WebbR. A.SuttererJ. R.HowardJ. L. (1970). Cardiac deceleration and reaction time: An evaluation of two hypotheses. Psychophysiology, 6, 695–706.
70.
O’DonnellR. D.EggemeierF. T. (1986). Workload assessment methodology. In BoffK. R.ThomasJ. P. (Eds.), Handbook of perception and human performance (pp. 1–49). Oxford, UK: Wiley.
71.
*PendletonD. M.SakalikM. L.MooreM. L.TomporowskiP. D. (2016). Mental engagement during cognitive and psychomotor tasks: Effects of task type, processing demands, and practice. International Journal of Psychophysiology, 109, 124–131.
72.
*ReimerB.MehlerB. (2011). The impact of cognitive workload on physiological arousal in young adult drivers: A field study and simulation validation. Ergonomics, 54, 932–942.
RoscoeA. H. (1993). Heart rate as a psychophysiological measure for in-flight workload assessment. Ergonomics, 36(9), 1055–1062.
75.
RosenthalR. (1991). Meta-analysis: A review. Psychosomatic Medicine, 53, 247–271.
76.
SaulJ. P. (1990). Beat-to-beat variations of heart rate reflect modulation of cardiac autonomic outflow. Physiology, 5, 32–37.
77.
*SausE. R.JohnsenB. H.EidJ.ThayerJ. F. (2012). Who benefits from simulator training: Personality and heart rate variability in relation to situation awareness during navigation training. Computers in Human Behavior, 28, 1262–1268.
78.
SchulzeR. (2004). Meta-analysis: A comparison of approaches. Cambridge, MA: Hogrefe & Huber.
79.
*SchwarzeA.EhrenpfordtI.EggertF. (2014). Workload of younger and elderly drivers in different infrastructural situations. Transportation Research: Part F, 26, 102–115.
80.
SeeJ. E.HoweS. R.WarmJ. S.DemberW. N. (1995). Meta-analysis of the sensitivity decrement in vigilance. Psychological Bulletin, 117, 230–249.
81.
ShawT. H.WarmJ. S.FinomoreV.TrippL.MatthewsG.WeilerE.ParasuramanR. (2009). Effects of sensory modality on cerebral blood flow velocity during vigilance. Neuroscience Letters, 461, 207–211.
82.
*ShiP.HuS.YuH. (2015). Influence of computer work under time pressure on cardiac activity. Computers in Biology and Medicine, 58, 40–45.
83.
StrimbuK.TavelJ. A. (2010). What are biomarkers? Current Opinion in HIV and AIDS, 5, 463–466.
84.
*StuiverA.de WaardD.BrookhuisK.DijksterhuisC.Lewis-EvansB.MulderL. (2012). Short-term cardiovascular responses to changing task demands. International Journal of Psychophysiology, 85, 153–160.
85.
*StuiverA.MulderB. (2015). Cardiovascular state changes in simulated work environments. Frontiers in Neuroscience, 9, 145.
86.
*SvebakS.DalenK.StorfjellO. (1981). The psychological significance of task-induced tonic changes in somatic and autonomic activity. Psychophysiology, 18, 403–409.
87.
*SvenssonE.Angelborg-ThanderezM.SjöbergL.OlssonS. (1997). Information complexity-mental workload and performance in combat aircraft. Ergonomics, 40, 362–380.
88.
*SvenssonE. A.WilsonG. F. (2002). Psychological and psychophysiological models of pilot performance for systems development and mission evaluation. International Journal of Aviation Psychology, 12, 95–110.
89.
SzalmaJ. L. (2009). Individual differences in performance, workload, and stress in sustained attention: Optimism and pessimism. Personality and Individual Differences, 47, 444–451.
90.
SzalmaJ. L.HancockP. A. (2011). Noise effects on human performance: A meta-analytic synthesis. Psychological Bulletin, 137, 682–707.
91.
*TaelmanJ.VandeputS.VlemincxE.SpaepenA.Van HuffelS. (2011). Instantaneous changes in heart rate regulation due to mental load in simulated office work. European Journal of Applied Physiology, 111, 1497–1505.
92.
TeoG.Reinerman-JonesL.MatthewsG.BarberD.HarrisJ.HudsonI. (2016). Augmenting robot behaviors using physiological measures of workload state. In SchmorrowD.FidopiastisC. (Eds.), Foundations of augmented cognition: Neuroergonomics and operational neuroscience (pp. 404–415). New York, NY: Springer International.
93.
*TeoG.Reinerman-JonesL.MatthewsG.SzalmaJ. (2015). Comparison of measures used to assess the workload of monitoring an unmanned system in a simulation mission. Procedia Manufacturing, 3, 1006–1013.
94.
ThayerJ. F.HansenA. L.JohnsenB. H. (2008). Handbook of physiological research methods in health psychology. Thousand Oaks, CA: Sage.
95.
ThayerR. E. (1989). The biopsychological response of mood and arousal. New York, NY: Oxford University Press.
96.
*ThumM.BoucseinW.KuhmannW.RayW. J. (1995). Standardized task strain and system response times in human–computer interaction. Ergonomics, 38, 1342–1351.
97.
*TianZ.ZhuN.ZhengG.WeiH. (2011). Experimental study on physiological and psychological effects of acclimazation in extreme environments. Building and Environment, 46, 2033–2041.
98.
*TurnerJ. R.GirdlerS. S.SherwoodA.LightK. C. (1990). Cardiovascular responses to behavioral stressors: Laboratory-field generalization and inter-task consistency. Journal of Psychosomatic Research, 34, 581–589.
99.
*VeltmanJ. A. (2002). A comparative study of psychophysiological reactions during simulator and real flight. International Journal of Aviation Psychology, 12, 33–48.
100.
*VeltmanJ. A.GaillardA. W. K. (1993). Indices of mental workload in a complex task environment. Neuropsychobiology, 28, 72–75.
101.
*VerweyW. B.VeltmanH. A. (1996). Detecting short periods of elevated workload: A comparison of nine workload assessment techniques. Journal of Experimental Psychology: Applied, 2, 270–285.
102.
*VogtJ.HagemannT.KastnerM. (2006). The impact of workload on heart rate and blood pressure in en-route and tower air traffic control. Journal of Psychophysiology, 20, 297–314.
103.
WallenM. P.GomersallS. R.KeatingS. E.WisloffU.CoombesJ. S. (2016). Accuracy of heart rate watches: Implications for weight management. PLoS One, 11(5), e0154420.
104.
*WanyanX.ZhuangD.ZhangH. (2014). Improving pilot mental workload evaluation with combined measures. Bio-Medical Materials and Engineering, 24, 2283–2290.
105.
WeisdorfD.SpodickD. H. (1976). Duration of diastole versus cycle length as correlates of left ventricular ejection time. British Heart Journal, 89, 504–516.
106.
*WetherellM. A.CarterK. (2014). The multitasking framework: The effects of increasing workload on acute psychobiological stress reactivity. Stress and Health, 30, 103–109.
107.
WhitenerE. M. (1990). Confusion of confidence intervals and credibility intervals in meta-analysis. Journal of Applied Psychology, 75, 315–321.
108.
WickensC. D. (2008). Multiple resources and mental workload. Human Factors, 50, 449–455.
109.
WickensC. D.HollandsJ. G. (2000). Engineering psychology and human performance. Upper Saddle River, NJ: Prentice Hall.
110.
*WidyantiA.de WaardD.JohnsonA.MulderB. (2013). National culture moderates the influence of mental effort on subjective and cardiovascular measures. Ergonomics, 56, 182–194.
111.
WilsonG. F. (1992). Applied use of cardiac and respiration measures: Practical considerations and precautions. Biological Psychology, 34, 163–178.
112.
*WrightR. A.JuniousT. R.NealC.AvelloA.GrahamC.HerrmannL.JuniousS.WaltonN. (2007). Mental fatigue influence on effort-related cardiovascular response: Difficulty effects and extension across cognitive performance domains. Motivation and Emotion, 31, 219–231.
113.
XuX. (2014). Analysis on mental stress/workload using heart rate variability and galvanic skin response during design process (Doctoral dissertation). Retrieved from Spectrum, Concordia University. (ID code 978540)
114.
YoungM. S.StantonN. A. (2007). What’s skill got to do with it? Vehicle automation and driver mental workload. Ergonomics, 50, 1324–1339.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
