This simulation study investigated factors influencing sustained performance and fatigue during operation of multiple Unmanned Aerial Systems (UAS). The study tested effects of time-on-task and automation reliability on accuracy in surveillance tasks and dependence on automation. It also investigated the role of trait and state individual difference factors.
Background:
Warm’s resource model of vigilance has been highly influential in human factors, but further tests of its applicability to complex, real-world tasks requiring sustained attention are necessary. Multi-UAS operation differs from standard vigilance paradigms in that the operator must switch attention between multiple subtasks, with support from automation.
Method:
131 participants performed surveillance tasks requiring signal discrimination and symbol counting with a multi-UAS simulation configured to impose low cognitive demands, for 2 hr. Automation reliability was manipulated between-groups. Five Factor Model personality traits were measured prior to performance. Subjective states were assessed with the Dundee Stress State Questionnaire.
Results:
Performance accuracy on the more demanding surveillance task showed a vigilance decrement, especially when automation reliability was low. Dependence on automation on this task declined over time. State but not trait factors predicted performance. High distress was associated with poorer performance in more demanding task conditions.
Conclusions:
Vigilance decrement may be an operational issue for multi-UAS surveillance missions. Warm’s resource theory may require modification to incorporate changes in information processing and task strategy associated with multitasking in low-workload, fatiguing environments.
Application:
Interface design and operator evaluation for multi-UAS operations should address issues including motivation, stress, and sustaining attention to automation.
Barg-WalkowL. H.RogersW. A. (2016). The effect of incorrect reliability information on expectations, perceptions, and use of automation. Human Factors, 58(2), 242–260.
2.
BergumB. O.LehrD. J. (1963). End spurt in vigilance. Journal of Experimental Psychology, 66(4), 383-385.
3.
CalhounG. L.GoodrichM. A.DoughertyJ. R.AdamsJ. A. (2016). Human-autonomy collaboration and coordination toward multi-RPA missions. In CookeN. J.RoweL. J.BennettW.JoralmonD. Q. (Eds.), Remotely piloted aircraft systems: A human systems integration perspective (pp. 109–136). Chichester, UK: Wiley.
4.
CalhounG. L.RuffH. A.DraperM. H.WrightE. J. (2011). Automation-level transference effects in simulated multiple unmanned aerial vehicle control. Journal of Cognitive Engineering and Decision Making, 5(1), 55–82. https://doi.org/10.1177/1555343411399069
5.
CasnerS. M.SchoolerJ. W. (2015). Vigilance impossible: Diligence, distraction, and daydreaming all lead to failures in a practical monitoring task. Consciousness and Cognition, 35, 33–41. https://doi.org/10.1016/j.concog.2015.04.019
6.
ChappelleW. L.SalinasA.McDonaldK. (2011). Psychological health screening of remotely piloted aircraft (RPA) operators and supporting units (No. ADA582856). Wright-Patterson Air Force Base, OH: USAF School of Aerospace Medicine Department of Neuropsychiatry. Retrieved from http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA582856
7.
CohenJ. (1988). Statistical power analysis for the behavioral sciences (2nd ed). Hillsdale, NJ: Erlbaum.
8.
CummingsM. L.MastracchioC.ThornburgK. M.MkrtchyanA. (2013). Boredom and distraction in multiple unmanned vehicle supervisory control. Interacting with Computers, 25(1), 34–47.
9.
DesmondP. A.HancockP. A. (2001). Active and passive fatigue states. In HancockP. A.DesmondP. A. (Eds.), Stress, workload, and fatigue (pp. 455–465). Mahwah, NJ: Erlbaum.
EysenckM. W.DerakshanN. (2011). New perspectives in attentional control theory. Personality and Individual Differences, 50(7), 955–960. https://doi.org/10.1016/j.paid.2010.08.019
12.
FinomoreV.MatthewsG.ShawT.WarmJ. (2009). Predicting vigilance: A fresh look at an old problem. Ergonomics, 52(7), 791–808. https://doi.org/10.1080/00140130802641627
13.
FrauliniN. W.HancockG. M.NeigelA. R.ClaypooleV. L.SzalmaJ. L. (2017). A critical examination of the research and theoretical underpinnings discussed in Thomson, Besner, and Smilek (2016). Psychological Review, 124(4), 525–531. https://doi.org/10.1037/rev0000066
14.
GunnD. V.WarmJ. S.NelsonW. T.BoliaR. S.SchumskyD. A.CorcoranK. J. (2005). Target acquisition with UAVs: Vigilance displays and advanced cuing interfaces. Human Factors, 47(3), 488–497.
15.
GuznovS.MatthewsG.WarmJ. S.PfahlerM. (2017). Training techniques for visual search in complex task environments. Human Factors, 59(7), 1139–1152.
16.
HancockP. A. (2013). In search of vigilance: The problem of iatrogenically created psychological phenomena. American Psychologist, 68(2), 97–109. https://doi.org/10.1037/a0030214
17.
HartS. G.StavelandL. E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In HancockP. A.MeshkatiN. (Eds.), Advances in psychology (Vol. 52, pp. 139–183). Amsterdam: Elsevier.
18.
HendyK. C.HamiltonK. M.LandryL. N. (1993). Measuring subjective workload: When is one scale better than many?Human Factors, 35(4), 579–601. https://doi.org/10.1177/001872089303500401
19.
HockeyG. R. J. (1997). Compensatory control in the regulation of human performance under stress and high workload: A cognitive-energetical framework. Biological Psychology, 45, 73–93. https://doi.org/10.1016/S0301-0511(96)05223-4
20.
HunterJ. A.AbrahamE. H.HunterA. G.GoldbergL. C.EastwoodJ. D. (2016). Personality and boredom proneness in the prediction of creativity and curiosity. Thinking Skills and Creativity, 22, 48–57. https://doi.org/10.1016/j.tsc.2016.08.002
KamzanovaA. T.KustubayevaA. M.MatthewsG. (2014). Use of EEG workload indices for diagnostic monitoring of vigilance decrement. Human Factors, 56(6), 1136–1149.
23.
KidwellB.CalhounG. L.RuffH. A.ParasuramanR. (2012). Adaptable and adaptive automation for supervisory control of multiple autonomous vehicles. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 56(1), 428–432. https://doi.org/10.1177/1071181312561096
24.
LinJ.WohleberR. W.SzalmaJ. L.RuffH. A.CalhounG. L.FunkeG. J. (2018). Cognitive overload, stress and automation utilization: A simulation study of multiple Unmanned Aerial Systems (UAS) operation. Manuscript submitted for publication.
MackieR. R. (1987). Vigilance research—are we ready for countermeasures?Human Factors, 29(6), 707–723.
27.
MatthewsG.CampbellS. E. (2010). Dynamic relationships between stress states and working memory. Cognition & Emotion, 24(2), 357–373. https://doi.org/10.1080/02699930903378719
28.
MatthewsG.CampbellS. E.FalconerS.JoynerL. A.HugginsJ.GillilandK.. . . WarmJ. S. (2002). Fundamental dimensions of subjective state in performance settings: Task engagement, distress, and worry. Emotion, 2(4), 315–340. https://doi.org/10.1037//1528-3542.2.4.315
29.
MatthewsG.DesmondP. A. (2002). Task-induced fatigue states and simulated driving performance. The Quarterly Journal of Experimental Psychology Section A, 55(2), 659–686. https://doi.org/10.1080/02724980143000505
30.
MatthewsG.NeubauerC.SaxbyD. J.WohleberR. W.LinJ. (2018). Dangerous intersections? A review of studies of fatigue and distraction in the automated vehicle. Accident Analysis and Prevention. Online publication: https://doi.org/10.1016/j.aap.2018.04.004
31.
MatthewsG.Reinerman-JonesL.AbichJ.KustubayevaA. (2017). Metrics for individual differences in EEG response to cognitive workload: Optimizing performance prediction. Personality and Individual Differences, 118, 22–28. https://doi.org/10.1016/j.paid.2017.03.002
32.
MatthewsG.SzalmaJ. L.PanganibanA. R.NeubauerC.WarmJ. S. (2013). Profiling task stress with the Dundee Stress State Questionnaire. In CavalcantiL.AzevedoS. (Eds.), Psychology of stress: New research (pp. 49–91). Hauppauge, NY: Nova Science Publishers.
33.
MatthewsG.WarmJ. S.Reinerman-JonesL. E.LangheimL. K.WashburnD. A.TrippL. (2010). Task engagement, cerebral blood flow velocity, and diagnostic monitoring for sustained attention. Journal of Experimental Psychology: Applied, 16(2), 187–203. https://doi.org/10.1037/a0019572
34.
MatthewsG.WarmJ. S.ShawT. H.FinomoreV. S. (2014). Predicting battlefield vigilance: a multivariate approach to assessment of attentional resources. Ergonomics, 57(6), 856–875. https://doi.org/10.1080/00140139.2014.899630
35.
MatthewsG.WarmJ. S.SmithA. P. (2017). Task engagement and attentional resources: Multivariate models for individual differences and stress factors in vigilance. Human Factors, 59(1), 44–61.
36.
McCraeR. R. (2009). The Five-Factor-Model of personality traits. In CorrP. J.MatthewsG. (Eds.), The Cambridge handbook of personality psychology (pp. 148–161). Cambridge: Cambridge University Press.
37.
MorrisS. B.DeShonR. P. (2002). Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychological Methods, 7(1), 105–125. https://doi.org/10.1037//1082-989X.7.1.105
38.
MoulouaM.GilsonR.HancockP. A. (2003). Human-centered design of unmanned aerial vehicles. Ergonomics in Design, 11(1), 6–11.
39.
NeubauerC.MatthewsG.LangheimL.SaxbyD. (2012). Fatigue and voluntary utilization of automation in simulated driving. Human Factors, 54(5), 734–746. https://doi.org/10.1177/0018720811423261
40.
NormanD. A.BobrowD. G. (1975). On data-limited and resource-limited processes. Cognitive Psychology, 7(1), 44–64.
41.
OkenB. S.SalinskyM. C.ElsasS. M. (2006). Vigilance, alertness, or sustained attention: physiological basis and measurement. Clinical Neurophysiology, 117(9), 1885–1901. https://doi.org/10.1016/j.clinph.2006.01.017
ParasuramanR.ManzeyD. H. (2010). Complacency and bias in human use of automation: An attentional integration. Human Factors, 52(3), 381–410. https://doi.org/10.1177/0018720810376055
44.
ParasuramanR.RileyV. (1997). Humans and automation: Use, misuse, disuse, abuse. Human Factors, 39(2), 230–253.
45.
ParasuramanR.WickensC. D. (2008). Humans: Still vital after all these years of automation. Human Factors, 50(3), 511–520. https://doi.org/10.1518/001872008X312198
46.
ParsonsK. S.WarmJ. S.NelsonW. T.MatthewsG.RileyM. A. (2007). Detection-action linkage in vigilance: Effects on workload and stress. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 51, 1291–1295.
RiceS.GeelsK. (2010). Using system-wide trust theory to make predictions about dependence on four diagnostic aids. The Journal of General Psychology, 137(4), 362–375. https://doi.org/10.1080/00221309.2010.499397
49.
RussoM. B.SingH.SantiagoS.KendallA. P.JohnsonD.ThorneD.. . . RedmondD. (2004). Visual neglect: occurrence and patterns in pilots in a simulated overnight flight. Aviation, Space, and Environmental Medicine, 75(4), 323–332.
50.
SaucierG. (2002). Orthogonal markers for orthogonal factors: The case of the Big Five. Journal of Research in Personality, 36(1), 1–31. https://doi.org/10.1006/jrpe.2001.2335
51.
SauerJ.WastellD. G.RobertG.HockeyG. R. J.EarleF. (2003). Performance in a complex multiple-task environment during a laboratory-based simulation of occasional night work. Human Factors, 45(4), 657–669.
52.
SaxbyD. J.MatthewsG.WarmJ. S.HitchcockE. M.NeubauerC. (2013). Active and passive fatigue in simulated driving: Discriminating styles of workload regulation and their safety impacts. Journal of Experimental Psychology: Applied, 19(4), 287–300. https://doi.org/10.1037/a0034386
53.
SeeJ. E.HoweS. R.WarmJ. S.DemberW. N. (1995). Meta-analysis of the sensitivity decrement in vigilance. Psychological Bulletin, 117(2), 230–249.
54.
ShawT. H.MatthewsG.WarmJ. S.FinomoreV. S.SilvermanL.CostaP. T. (2010). Individual differences in vigilance: Personality, ability and states of stress. Journal of Research in Personality, 44(3), 297–308. https://doi.org/10.1016/j.jrp.2010.02.007
55.
SzalmaJ. L. (2009). Individual differences in human–technology interaction: incorporating variation in human characteristics into human factors and ergonomics research and design. Theoretical Issues in Ergonomics Science, 10(5), 381–397. https://doi.org/10.1080/14639220902893613
56.
SzalmaJ. L. (2014). On the application of motivation theory to human factors/ergonomics: Motivational design principles for human–technology interaction. Human Factors, 56(8), 1453–1471.
57.
SzalmaJ. L.TaylorG. S. (2011). Individual differences in response to automation: The five factor model of personality. Journal of Experimental Psychology: Applied, 17(2), 71–96. https://doi.org/10.1037/a0024170
58.
TempleJ. G.WarmJ. S.DemberW. N.JonesK. S.LaGrangeC. M.MatthewsG. (2000). The effects of signal salience and caffeine on performance, workload, and stress in an abbreviated vigilance task. Human Factors, 42(2), 183–194.
59.
ThomsonD. R.BesnerD.SmilekD. (2016). A critical examination of the evidence for sensitivity loss in modern vigilance tasks. Psychological Review, 123(1), 70–83. https://doi.org/10.1037/rev0000021
60.
TvaryanasA. P.LopezN.HickeyP.DaLuzC.ThompsonW. T.CaldwellJ. L. (2006). Effects of shift work and sustained operations: Operator performance in remotely piloted aircraft (OP-REPAIR) (Final No. HSW-PE-BR-TR-2006-0001). Brooks City-Base. TX: United States Air Force 331st Human Systems Wing. Retrieved from http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA443145
61.
TvaryanasA. P.MacphersonG. D. (2009). Fatigue in pilots of remotely piloted aircraft before and after shift work adjustment. Aviation, Space, and Environmental Medicine, 80(5), 454–461. https://doi.org/10.3357/ASEM.2455.2009
62.
WarmJ. S.DemberW. N. (1998). Tests of vigilance taxonomy. In HoffmanR. R.SherrickM. F.WarmJ. S. (Eds.), Viewing psychology as a whole: The integrative science of William N. Dember (pp. 87–112). Washington, DC: American Psychological Association. Retrieved from http://content.apa.org/books/10290-004
63.
WarmJ. S.DemberW. N.HancockP. A. (1996). Vigilance and workload in automated systems. In ParasuramanR.MoulouaM. (Eds.), Automation and human performance: theory and applications (pp. 183–200). Mahwah, NJ: Erlbaum.
64.
WarmJ. S.FinomoreV. S.VidulichM. A.FunkeM. E. (2015). Vigilance: A perceptual challenge. In HoffmanR. R. (Ed.), The Cambridge handbook of applied perception research (Vol. 1, pp. 241–283). New York, NY: Cambridge University Press.
65.
WarmJ. S.JerisonH. J. (1984). The psychophysics of vigilance. In WarmJ. S. (Ed.), Sustained attention in human performance (pp. 15–59). Chichester, UK: Wiley.
66.
WarmJ. S.MatthewsG.FinomoreV. S. (2008). Vigilance, workload, and stress. In HancockP. A.SzalmaJ. L. (Eds.), Performance under stress (pp. 115–141). Burlington, VT: Ashgate.
67.
WarmJ. S.ParasuramanR.MatthewsG. (2008). Vigilance requires hard mental work and is stressful. Human Factors, 50(3), 433–441. https://doi.org/10.1518/001872008X312152
68.
WarmJ. S.TrippL. D.MatthewsG.HeltonW. S. (2012). Cerebral hemodynamic indices of operator fatigue in vigilance. In MatthewsG.DesmondP. A.NeubauerC.HancockP. A. (Eds.), The handbook of operator fatigue (pp. 197–207). Burlington, VT: Ashgate.
69.
WickensC. D.DixonS. R. (2007). The benefits of imperfect diagnostic automation: A synthesis of the literature. Theoretical Issues in Ergonomics Science, 8(3), 201–212.
70.
WohleberR. W.CalhounG. L.FunkeG. J.RuffH.ChiuC.-Y. P.LinJ.MatthewsG. (2016). The impact of automation reliability and operator fatigue on performance and reliance. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 60(1), 211–215. https://doi.org/10.1177/1541931213601047
