Sage Journals: Discover world-class research

Abstract

Objective

Examine (1) the extent to which humans can accurately estimate automation reliability and calibrate to changes in reliability, and how this is impacted by the recent accuracy of automation; and (2) factors that impact the acceptance of automated advice, including true automation reliability, reliability perception, and the difference between an operator’s perception of automation reliability and perception of their own reliability.

Background

Existing evidence suggests humans can adapt to changes in automation reliability but generally underestimate reliability. Cognitive science indicates that humans heavily weight evidence from more recent experiences.

Method

Participants monitored the behavior of maritime vessels (contacts) in order to classify them, and then received advice from automation regarding classification. Participants were assigned to either an initially high (90%) or low (60%) automation reliability condition. After some time, reliability switched to 75% in both conditions.

Results

Participants initially underestimated automation reliability. After the change in true reliability, estimates in both conditions moved towards the common true reliability, but did not reach it. There were recency effects, with lower future reliability estimates immediately following incorrect automation advice. With lower initial reliability, automation acceptance rates tracked true reliability more closely than perceived reliability. A positive difference between participant assessments of the reliability of automation and their own reliability predicted greater automation acceptance.

Conclusion

Humans underestimate the reliability of automation, and we have demonstrated several critical factors that impact the perception of automation reliability and automation use.

Application

The findings have potential implications for training and adaptive human-automation teaming.

Keywords

human-automation teaming automation perceived automation reliability automation use

Get full access to this article

View all access options for this article.

References

Barg-Walkow

L. H.

Rogers

W. A.

(2016). The effect of incorrect reliability information on expectations, perceptions, and use of automation. Human Factors, 58(2), 242–260. https://doi.org/10.1177/0018720815610271

Bisseret

(1981). Application of signal detection theory to decision making in supervisory control the effect of the operator's experience. Ergonomics, 24(2), 81–94. https://doi.org/10.1080/00140138108924833

Brooks

M. E.

Kristensen

Benthem

K. J. v.

Magnusson

Berg

C. W.

Nielsen

Skaug

H. J.

Mächler

Bolker

B. M.

(2017). glmmTMB balances speed and flexibility among packages for zero-inflated generalized linear mixed modeling. The R Journal, 9(2), 378–400. https://doi.org/10.32614/rj-2017-066

Einhorn

H. J.

Hogarth

R. M.

(1981). Behavioral decision theory: Processes of judgement and choice. Annual Review of Psychology, 32(1), 53–88. https://doi.org/10.1146/annurev.ps.32.020181.000413

Feigh

K. M.

Dorneich

M. C.

Hayes

C. C.

(2012). Toward a characterization of adaptive systems: A framework for researchers and system designers. Human Factors, 54(6), 1008–1024. https://doi.org/10.1177/0018720812443983

Gallistel

Krishan

Liu

Miller

Latham

(2014). The perception of probability. Psychological Review, 121(1), 96–123. https://doi.org/10.1037/a0035232

Green

D. M.

Swets

J. A.

(1966). Signal detection theory and psychophysics (Vol. 1, pp. 1969–2012). Wiley.

Hoff

Bashir

(2015). Trust in automation: Integrating empirical evidence on factors that influence trust. Human Factors, 57(3), 407–434. https://doi.org/10.1177/0018720814547570

Jarvik

(1951). Probability learning and a negative recency effect in the serial anticipation of alternative symbols. Journal of Experimental Psychology, 41(4), 291–297. https://doi.org/10.1037/h0056878

10.

Gao

Lee

J. D.

(2006). Extending the decision field theory to model operators' reliance on automation in supervisory control situations. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 36(5), 943–59. https://doi.org/10.1109/tsmca.2005.855783

11.

Jones

Sieck

W. R.

(2003). Learning myopia: An adaptive recency effect in category learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 29(4), 626–640. https://doi.org/10.1037/0278-7393.29.4.626

12.

Lee

J. D.

See

K. A.

(2004). Trust in automation: Designing for appropriate reliance. Human Factors, 46(1), 50–80. https://doi.org/10.1518/hfes.46.1.50_30392

13.

Lichtenstein

Slovic

(1971). Reversals of preference between bids and choices in gambling decisions. Journal of Experimental Psychology, 89(1), 46–55. https://doi.org/10.1037/h0031207

14.

Ludwig

C. J.

Farrell

Ellis

L. A.

Hardwicke

T. E.

Gilchrist

I. D.

(2012). Context-gated statistical learning and its role in visual-saccadic decisions. Journal of Experimental Psychology: General, 141(1), 150–169. https://doi.org/10.1037/a0024916

15.

Lusted

L. B.

(1976). In DeDombal

Grevy

(Eds.), Clinical decision-making. Decision-making and Medical Care. Amsterdam: North Holland.

16.

Madhavan

Wiegmann

D. A.

(2007). Effects of information source, pedigree, and reliability on operator interaction with decision support systems. Human Factors, 49(5), 773–785. https://doi.org/10.1518/001872007X230154

17.

Madhavan

Wiegmann

D. A.

Lacson

F. C.

(2006). Automation failures on tasks easily performed by operators undermine trust in automated aids. Human Factors, 48(2), 241–256. https://doi.org/10.1518/001872006777724408

18.

Merritt

S. M.

Lee

Unnerstall

J. L.

Huber

(2015). Are well-calibrated users effective users? Associations between calibration of trust and performance on an automation-aided task. Human Factors, 57(1), 34–47. https://doi.org/10.1177/0018720814561675

19.

Moray

Inagaki

Itoh

(2000). Adaptive automation, trust, and self-confidence in fault management of time-critical tasks. Journal of Experimental Psychology: Applied, 6(1), 44–58. https://doi.org/10.1037//1076-898x.6.1.44

20.

Nicks

D. C.

(1959). Prediction of sequential two-choice decisions from event runs. Journal of Experimental Psychology, 57(2), 105–114. https://doi.org/10.1037/h0045193

21.

Nissen

M. J.

Bullemer

(1987). Attentional requirements of learning: Evidence from performance measures. Cognitive Psychology, 19(1), 1–32. https://doi.org/10.1016/0010-0285(87)90002-8

22.

Parasuraman

Riley

(1997). Humans and automation: Use, misuse, disuse, abuse. Human Factors, 39(2), 230–253. https://doi.org/10.1518/001872097778543886

23.

Pop

V. L.

Shrewsbury

Durso

F. T.

(2015). Individual differences in the calibration of trust in automation. Human Factors, 57(4), 545–556. https://doi.org/10.1177/0018720814564422

24.

Schaefer

K. E.

Chen

J. Y.

Szalma

J. L.

Hancock

P. A.

(2016). A meta-analysis of factors influencing the development of trust in automation: Implications for understanding autonomy in future systems. Human Factors, 58(3), 377–400. https://doi.org/10.1177/0018720816634228

25.

Singmann

Kellen

(2019). An introduction to mixed models for experimental psychology. In New methods in cognitive psychology (pp. 4–31). Routledge. https://doi.org/10.4324/9780429318405-2

26.

Sood

Forehand

(2005). On self-referencing differences in judgment and choice. Organizational Behavior and Human Decision Processes, 98(2), 144–154. https://doi.org/10.1016/j.obhdp.2005.05.005

27.

Speekenbrink

Shanks

D. R.

(2010). Learning in a changing environment. Journal of Experimental Psychology: General, 139(2), 266–298. https://doi.org/10.1037/a0018620

28.

Tversky

Kahneman

(1974). Judgment under uncertainty: Heuristics and biases. Science, 185(4157), 1124–1131. https://doi.org/10.1126/science.185.4157.1124

29.

Vagia

Transeth

Fjerdingen

(2016). A literature review on the levels of automation during the years. What are the different taxonomies that have been proposed? Applied Ergonomics, 53(Part A), 190–202. https://doi.org/10.1016/j.apergo.2015.09.013

30.

Wickens

C. D.

Hollands

J. G.

Banbury

Parasuraman

(2013). Engineering psychology and human performance. Psychology Press.

31.

Wiegmann

D. A.

Rich

Zhang

(2001). Automated diagnostic aids: The effects of aid reliability on users' trust and reliance. Theoretical Issues in Ergonomics Science, 2(4), 352–367. https://doi.org/10.1080/14639220110110306.

32.

Yang

X. J.

Unhelkar

V. V.

Shah

J. A.

(2017) Evaluating effects of user experience and system transparency on trust in automation. In 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Vienna, Austria, 6-9 March 2017 (pp. 408–416). IEEE.

33.

Yang

X. J.

Wickens

C. D.

Hölttä-Otto

(2016). How users adjust trust in automation: Contrast effect and hindsight bias. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 60(1), 196–200. https://doi.org/10.1177/1541931213601044

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.13 MB

The Perception of Automation Reliability and Acceptance of Automated Advice

Abstract

Objective

Background

Method

Results

Conclusion

Application

Keywords

Get full access to this article

References

Supplementary Material