Assess performance, trust, and visual attention during the monitoring of a near-perfect automated system.
Background
Research rarely attempts to assess performance, trust, and visual attention in near-perfect automated systems even though they will be relied on in high-stakes environments.
Methods
Seventy-three participants completed a 40-min supervisory control task where they monitored three search feeds. All search feeds were 100% reliable with the exception of two automation failures: one miss and one false alarm. Eye-tracking and subjective trust data were collected.
Results
Thirty-four percent of participants correctly identified the automation miss, and 67% correctly identified the automation false alarm. Subjective trust increased when participants did not detect the automation failures and decreased when they did. Participants who detected the false alarm had a more complex scan pattern in the 2 min centered around the automation failure compared with those who did not. Additionally, those who detected the failures had longer dwell times in and transitioned to the center sensor feed significantly more often.
Conclusion
Not only does this work highlight the limitations of the human when monitoring near-perfect automated systems, it begins to quantify the subjective experience and attentional cost of the human. It further emphasizes the need to (1) reevaluate the role of the operator in future high-stakes environments and (2) understand the human on an individual level and actively design for the given individual when working with near-perfect automated systems.
Application
Multiple operator-level measures should be collected in real-time in order to monitor an operator’s state and leverage real-time, individualized assistance.
BagheriN.JamiesonG. A. (2004). The impact of context-related reliability on automation failure detection and scanning behaviour. 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583), 1, 212–217.
BatesD.MächlerM.BolkerB.WalkerS. (2014). Fitting linear mixed-effects models using lme4. arXiv preprint arXiv:1406.5823.
4.
BellerJ.HeesenM.VollrathM. (2013). Improving the driver-automation interaction: An approach using automation uncertainty. Human Factors, 55, 1130–1141.doi:10.1177/001872081348232724745204
5.
BlissJ. P. (2003). Investigation of alarm-related accidents and incidents in aviation. The International Journal of Aviation Psychology, 13, 249–268.doi:10.1207/S15327108IJAP1303_04
6.
BrownC. M.NoyY. (2004). Behavioural adaptation to in-vehicle safety measures: Past ideas and future directions. InRothengatterT.HugueninR. D. (Eds.), Traffic and transport psychology: Theory and application (pp.25–46).
7.
ChanceyE. T.BlissJ. P.LiechtyM.ProapsA. B. (2015). False alarms vs. misses: Subjective trust as a mediator between reliability and alarm reaction measures. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 59, 647–651.doi:10.1177/1541931215591141
8.
ChanceyE. T.YamaniY.BrillJ. C.BlissJ. P. (2017). Effects of alarm system error bias and reliability on performance measures in a multitasking environment: Are false alarms really worse than misses?Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 61, 1621–1625.doi:10.1177/1541931213601890
CoralM. P. (2016). Analyzing cognitive workload through eye-related measurements: A meta-analysis. Wright State University.
11.
DavenportR. B.BustamanteE. A. (2010). Effects of false-alarm vs. miss-prone automation and likelihood alarm technology on trust, reliance, and compliance in a miss-prone task. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 54, 1513–1517.doi:10.1177/154193121005401933
12.
de HaasB.IakovidisA. L.SchwarzkopfD. S.GegenfurtnerK. R. (2019). Individual differences in visual salience vary along semantic dimensions. Proceedings of the National Academy of Sciences of the United States of America, 116, 11687–11692.doi:10.1073/pnas.182055311631138705
13.
DehaisF.PeysakhovichV.ScannellaS.FongueJ. (2015). “Automation surprise” in aviation: Real-time solutions[Conference session]. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems CHI 2015, Republic of Korea.
14.
DixonS. R.WickensC. D. (2006). Automation reliability in unmanned aerial vehicle control: A reliance-compliance model of automation dependence in high workload. Human Factors, 48, 474–486.doi:10.1518/00187200677860682217063963
15.
DixonS. R.WickensC. D.McCarleyJ. S. (2007). On the independence of compliance and reliance: Are automation false alarms worse than misses?Human Factors, 49, 564–572.doi:10.1518/001872007X21565617702209
16.
EllisS. R.StarkL. (1986). Statistical dependency in visual scanning. Human Factors, 28, 421–438.doi:10.1177/0018720886028004053793110
17.
EndsleyM. R. (2017). From here to autonomy: Lessons learned from human–automation research. Human Factors, 59, 5–27.
18.
EndsleyM. R.KirisE. O. (1995). The out-of-the-loop performance problem and level of control in automation. Human Factors, 37, 381–394.doi:10.1518/001872095779064555
19.
ForoughiC. K.SibleyC.BrownN. L.RoviraE.PakR.CoyneJ. T. (2019). Detecting automation failures in a simulated supervisory control environment. Ergonomics, 62, 1150–1161.doi:10.1080/00140139.2019.162963931179874
20.
GlaholtM. G. (2014). Eye tracking in the cockpit: A review of the relationships between eye movements and the aviator’s cognitive state (Report No. DRDC-RDDC-2014-R153): Defence Research and Development Canada.
21.
HancockP. A.BillingsD. R.SchaeferK. E.ChenJ. Y. C.de VisserE. J.ParasuramanR. (2011). A meta-analysis of factors affecting trust in human-robot interaction. Human Factors: The Journal of the Human Factors and Ergonomics Society, 53, 517–527.doi:10.1177/001872081141725422046724
22.
HergethS.LorenzL.VilimekR.KremsJ. F. (2016). Keep your scanners peeled: Gaze behavior as a measure of automation trust during highly automated driving. Human Factors, 58, 509–519.doi:10.1177/001872081562574426843570
23.
HoffK. A.BashirM. (2015). Trust in automation: Integrating empirical evidence on factors that influence trust. Human Factors, 57, 407–434.doi:10.1177/001872081454757025875432
24.
HolmqvistK.NyströmM.AnderssonR.DewhurstR.JarodzkaH.van de WeijerJ. (2011). Eye tracking: A comprehensive guide to methods, paradigms, and measures (2nd ed.). CreateSpace Independent Publishing Platform.
25.
JarodzkaH.ScheiterK.GerjetsP.van GogT. (2010). In the eyes of the beholder: How experts and novices interpret dynamic stimuli. Learning and Instruction, 20, 146–154.doi:10.1016/j.learninstruc.2009.02.019
26.
KruizingaA.MulderB.De WaardD. (2006). Eye scan patterns in a simulated ambulance dispatcher’s task. Shaker Publishing.
27.
LeeJ. D.MorayN. (1994). Trust, self-confidence, and operators’ adaptation to automation. International Journal of Human-Computer Studies, 40, 153–184.doi:10.1006/ijhc.1994.1007
28.
LeeJ. D.SeeK. A. (2004). Trust in automation: Designing for appropriate reliance. Human Factors, 46, 50–80.doi:10.1518/hfes.46.1.50.3039215151155
29.
LewandowskyS.MundyM.TanG. P. (2000). The dynamics of trust: Comparing humans to automation. Journal of Experimental Psychology: Applied, 6, 104.doi:10.1037//1076-898x.6.2.10410937315
30.
LiechtyJ.PietersR.WedelM. (2003). Global and local covert visual attention: Evidence from a Bayesian hidden Markov model. Psychometrika, 68, 519–541.doi:10.1007/BF02295608
31.
MasalonisA. J.ParasuramanR. (1999). Trust as a construct for evaluation of automated aids: Past and future theory and research. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 43, 184–187.doi:10.1177/154193129904300312
32.
MeyerJ.BallasE. (1997). A two-detector signal detection analysis of learning to use alarms. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 41, 186–189.doi:10.1177/107118139704100143
33.
MorayN.InagakiT. (1999). Laboratory studies of trust between humans and machines in automated systems. Transactions of the Institute of Measurement and Control, 21, 203–211.doi:10.1177/014233129902100408
34.
MracekD. L.ArsenaultM. L.DayE. A.HardyJ. H.TerryR. A. (2014). A multilevel approach to relating subjective workload to performance after shifts in task demand. Human Factors, 56, 1401–1413.doi:10.1177/001872081453396425509821
35.
MuirB. M. (1994). Trust in automation: Part I. Theoretical issues in the study of trust and human intervention in automated systems. Ergonomics, 37, 1905–1922.doi:10.1080/00140139408964957
36.
MuirB. M.MorayN. (1996). Trust in automation. Part II. Experimental studies of trust and human intervention in a process control simulation. Ergonomics, 39, 429–460.doi:10.1080/001401396089644748849495
37.
OnnaschL.WickensC. D.LiH.ManzeyD. (2014). Human performance consequences of stages and levels of automation: An integrated meta-analysis. Human Factors, 56, 476–488.doi:10.1177/001872081350154924930170
38.
ParasuramanR. (1986). Vigilance, monitoring, and search. InBoffK.KaufmanL.ThomasJ. (Eds.), Handbook of perception and human performance. Volume 2: Cognitive processes and performance (pp.43.1–43.39). Wiley.
39.
ParasuramanR.ManzeyD. H. (2010). Complacency and bias in human use of automation: An attentional integration. Human Factors, 52, 381–410.doi:10.1177/001872081037605521077562
40.
ParasuramanR.MolloyR.SinghI. L. (1993). Performance consequences of automation-induced ‘complacency’. The International Journal of Aviation Psychology, 3, 1–23.doi:10.1207/s15327108ijap0301_1
41.
ParasuramanR.RileyV. (1997). Humans and automation: Use, misuse, disuse, abuse. Human Factors, 39, 230–253.doi:10.1518/001872097778543886
42.
RaptisG.KatsiniC.BelkM.FidasC.SamarasG.AvourisN. (2017). Using eye gaze data and visual activities to infer human cognitive styles: Method and feasibility studies[Conference session]. Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization, pp. 164–173.doi:https://doi.org/10.1145/3079628.3079690
43.
RehderB.HoffmanA. B. (2003). Eyetracking and selective attention in category learning. Proceedings of the Annual Meeting of the Cognitive Science Behaviour, 276–281.doi:https://doi.org/ISBN%20978-0-9768318-8-4
44.
RoviraE.McGarryK.ParasuramanR. (2007). Effects of imperfect automation on decision making in a simulated command and control task. Human Factors, 49, 76–87.doi:10.1518/00187200777959808217315845
45.
RoviraE.PakR.McLaughlinA. (2017). Effects of individual differences in working memory on performance and trust with various degrees of automation. Theoretical Issues in Ergonomics Science, 18, 573–591.doi:10.1080/1463922X.2016.1252806
46.
SarterN. B.MumawR. J.WickensC. D. (2007). Pilots’ monitoring strategies and performance on automated flight decks: An empirical study combining behavioral and eye-tracking data. Human Factors, 49, 347–357.doi:10.1518/001872007X19668517552302
47.
ShicF.ChawarskaK.BradshawJ.ScassellatiB. (2008). Autism, eye-tracking, entropy. 7th IEEE International Conference on Development and Learning, ICDL, 73–78.doi:https://doi.org/10.1109/DEVLRN.2008.4640808
48.
ShiferawB.DowneyL.CrewtherD. (2019). A review of gaze entropy as a measure of visual scanning efficiency. Neuroscience & Biobehavioral Reviews, 96, 353–366.doi:10.1016/j.neubiorev.2018.12.00730621861
49.
ShiferawB. A.DowneyL. A.WestlakeJ.StevensB.RajaratnamS. M. W.BerlowitzD. J.SwannP.HowardM. E. (2018). Stationary gaze entropy predicts Lane departure events in sleep-deprived drivers. Scientific Reports, 8, 2220.doi:10.1038/s41598-018-20588-729396509
50.
SibleyC.CoyneJ.ThomasJ. (2016). Demonstrating the supervisory control operations user testbed (SCOUT). Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 60, 1324–1328.doi:10.1177/1541931213601306
51.
SmithT. J.SmithK. U. (1987). Feedback control mechanisms of human behavior. InSalvendyG. (Ed.), Handbook of human factors (pp.251–293). Wiley.
52.
ThomasL. C.WickensC. D. (2004). Eye-tracking and individual differences in off-normal event detection when flying with a synthetic vision system display. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 48, 223–227.doi:10.1177/154193120404800148
53.
TsukaharaJ. S.HarrisonT. L.EngleR. W. (2016). The relationship between baseline pupil size and intelligence. Cognitive Psychology, 91, 109–123.doi:10.1016/j.cogpsych.2016.10.00127821254
54.
United States Department of Defense. (2017). Unmanned systems integrated roadmap FY2017-2042.
55.
VictorT. W.TivestenE.GustavssonP.JohanssonJ.SangbergF.Ljung AustM. (2018). Automation expectation mismatch: Incorrect prediction despite eyes on threat and hands on wheel. Human Factors, 60, 1095–1116.doi:10.1177/001872081878816430096002
56.
WarmJ. S.ParasuramanR.MatthewsG. (2008). Vigilance requires hard mental work and is stressful. Human Factors, 50, 433–441.doi:10.1518/001872008X31215218689050
WickensC. D.XuX. (2002). Automation trust, reliability and attention HMI 02-03.
59.
WiegmannD. A.RichA.ZhangH. (2001). Automated diagnostic aids: The effects of aid reliability on users’ trust and reliance. Theoretical Issues in Ergonomics Science, 2, 352–367.doi:10.1080/14639220110110306
60.
WinnefeldJ. A.KendallF. (2013). Unmanned systems integrated roadmap FY 2013-2038. Approved for Open Publication Reference, 0553.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
