Automation Failures on Tasks Easily Performed by Operators Undermines Trust in Automated Aids

Abstract

Automation users often distrust diagnostic aids that are imperfectly reliable. The extent to which users' trust in automation is influenced by the simplicity of automation errors was explored. Participants (n = 30) performed 200 trials of a target-detection task using an imperfectly reliable automated aid. For the “easy-miss” group (n = 15) the automated aid missed targets only on easy trials and was accurate on difficult trials. For the “difficult-miss” group (n = 15) the aid missed targets only on difficult trials and was accurate on easy trials. A control group (n = 15) performed the task unaided. The easy-miss group trusted the aid less, was less accurate, and contradicted the aid more than the difficult-miss group, even on trials when the easy-miss aid was more reliable than the difficult-miss aid. Results suggest that the “easiness” of the aid's errors for the easy-miss group undermined automation trust and reliance. Potential future directions include examining whether easy false alarms affect user trust in a manner akin to easy misses.

Get full access to this article

View all access options for this article.

References

Breznitz

(1983). Cry-wolf: The Psychology of False Alarms. Mahwah, NJ: Erlbaum.

Dzindolet

M. T.

Pierce

L. G.

Beck

H. P.

Dawe

L. A.

(2002). Perceived utility of human and automated aids in a visual detection task. Human Factors, 44(1), 79–94.

Dzindolet

M. T.

Pierce

L. G.

Beck

H. P.

Dawe

L. A.

Anderson

B. W.

(2001). Predicting misuse and disuse of combat identification systems. Military Psychology, 13(3), 147–164.

Dzindolet

M. T.

Peterson

S. A.

Pomranky

R. A.

Pierce

L. G.

Beck

H. P.

(in press). The role of trust in automation reliance. International Journal of Human-Computer Studies..

Elvers

G. C.

Elrif

(1997). The effects of correlation and response bias in alerted monitor displays. Human Factors, 39 (4), 570–580.

Getty

D. J.

Swets

J. A.

Pickett

R. M.

Gonthier

(1995). System operator response to warnings of danger: A laboratory investigation of the effects of the predictive value of a warning on human response time. Journal of Experimental Psychology: Applied, 1 (1), 19–33.

Green

D. M.

Swets

J. A.

(1988). Signal Detection Theory and Psychophysics. New York: Wiley.

Lee

J. D.

Moray

(1992). Trust, control strategies and allocation of function in human machine systems. Ergonomics, 22 (6), 671–691.

Maltz

Meyer

(2001). Use of warnings in an attentionally demanding detection task. Human Factors, 43 (2), 217–225.

10.

Meyer

(2001). Effects of warning validity and proximity on responses to warnings. Human Factors, 43 (4), 563–572.

11.

Parasuraman

Riley

(1997). Humans and automation use: Use, misuse, disuse, abuse. Human Factors, 39 (2), 230–253.

12.

Ruble

D. N.

Stangor

(1986). Stalking the elusive schema: Insights from developmental and social-psychological analyses of gender schemas. Social Cognition, 4 (2), 227–261.

13.

Sheridan (2002). Human performance in relation to automation. In Sheridan

T. B.

(Ed.), Human and Automation: System Design and Research Issues, (pp. 69–89). Santa Monica, CA: John Wiley.

14.

Sheridan

Farrell

(1974). Man-machine Systems: Information, Control, and Decision Models of Human Performance. Cambridge, MA: MIT Press.

15.

Smith

D. A.

Graesser

A. C.

(1981). Memory for actions in scripted activities as a function of typicality, retention interval, and retrieval task. Memory and Cognition, 9 (6), 550–559.

16.

Swets

J. A.

Pickett

R. M.

(1982). Evaluation of Diagnostic Systems: Methods from Signal Detection Theory. New York: Academic Press.

17.

Wickens

C. D.

Hollands

J. G.

(2000). Engineering Psychology and Human Performance (3rd ed.). Upper Saddle River, NJ: Prentice Hall.

18.

Wiegmann

D. A.

(2002). Agreeing with automated diagnostic aids: A study of users' concurrence strategies. Human Factors, 44(1), 44–50.

19.

Wiegmann

D. A.

Rich

Zhang

(2001). Automated diagnostic aids: The effects of aid reliability on users' trust and reliance. Theoretical Issues in Ergonomics Science, 2(4), 352–367.