Objective: We tested the hypothesis that automation errors on tasks easily performed by humans undermine trust in automation. Background: Research has revealed that the reliability of imperfect automation is frequently misperceived. We examined the manner in which the easiness and type of imperfect automation errors affect trust and dependence. Method: Participants performed a target detection task utilizing an automated aid. In Study 1, the aid missed targets either on easy trials (easy miss group) or on difficult trials (difficult miss group). In Study 2, we manipulated both easiness and type of error (miss vs. false alarm). The aid erred on either difficult trials alone (difficult errors group) or on difficult and easy trials (easy miss group; easy false alarm group). Results: In both experiments, easy errors led to participants mistrusting and disagreeing more with the aid on difficult trials, as compared with those using aids that generated only difficult errors. This resulted in a downward shift in decision criterion for the former, leading to poorer overall performance. Misses and false alarms led to similar effects. Conclusion: Automation errors on tasks that appear “easy” to the operator severely degrade trust and reliance. Application: Potential applications include the implementation of system design solutions that circumvent the negative effects of easy automation errors.
Get full access to this article
View all access options for this article.
References
1.
Bliss, J. P., Gilson, R., & Deaton, I. (1995). Human probability matching behaviour in response to alarms of varying reliability. Ergonomics, 38, 2500–2312.
2.
Breznitz, S. (1983). Cry-wolf: The psychology of false alarms. Mahwah, NJ: Erlbaum.
3.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.
4.
Cotte, N., Meyer, J., & Coughlin, J. F. (2001). Older and younger drivers' reliance on collision warning systems. In Proceedings of the Human Factors and Ergonomics Society 45th Annual Meeting (pp. 277–280). Santa Monica, CA: Human Factors and Ergonomics Society.
5.
Dunning, D., Johnson, K., Ehrlinger, J., & Kruger, J. (2003). Why people fail to recognize their own incompetence. Current Directions in Psychological Science, 12, 83–87.
6.
Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. (2003). The role of trust in automation reliance. International Journal of Human-Computer Studies, 58, 697–718.
7.
Dzindolet, M. T., Pierce, L. G., Beck, H. P., & Dawe, L. A. (2002). The perceived utility of human and automated aids in a visual detection task. Human Factors, 44, 79–94.
8.
Dzindolet, M. T., Pierce, L. G., Beck, H. P., Dawe, L. A., & Anderson, B. W. (2001). Predicting misuse and disuse of combat identification systems. Military Psychology, 13(3), 147–164.
9.
Elvers, G. C., & Elrif, P. (1997). The effects of correlation and response bias in alerted monitor displays. Human Factors, 39, 570–580.
10.
Getty, D. J., Swets, J. A., Pickett, R. M., & Gonthier, D. (1995). System operator response to warnings of danger: A laboratory investigation of the effects of the predictive value of a warning on human response time. Journal of Experimental Psychology: Applied, 1, 19–33.
11.
Green, D. M., & Swets, J. A. (1988). Signal detection theory and psychophysics. New York: Wiley.
12.
Gupta, N., Bisantz, A. M., & Singh, T. (2001). Investigation of factors affecting driver performance using adverse condition warning systems. In Proceedings of the Human Factors and Ergonomics Society 45th Annual Meeting (pp. 1699–1703). Santa Monica, CA: Human Factors and Ergonomics Society.
13.
Lee, J. D., & Moray, N. (1992). Trust, control strategies and allocation of function in human machine systems. Ergonomics, 22, 671–691.
14.
Lee, J. D., & Moray, N. (1994). Trust, self-confidence, and operators' adaptation to automation. International Journal of Human-Computer Studies, 40, 153–184.
15.
Lee, J. D., & See, K. A. (2004). Trust in automation: Designing for appropriate reliance. Human Factors, 46, 50–80.
16.
Lerch, F. J., Prietula, M. J., & Kulik, C. T. (1997). The Turing effect: The nature of trust in expert system advice. In P. J. Feltovich & K. M. Ford, (Eds.), Expertise in context: Human and machine. (pp. 417–448). Cambridge, MA: MIT Press.
17.
Madhavan, P., & Wiegmann, D. A. (in press). A review of operator trust in automated aids: Is trust in machines comparable to trust in humans?Theoretical Issues in Ergonomics Science.
18.
Maltz, M., & Meyer, J. (2001). Use of warnings in an attentionally demanding detection task. Human Factors, 43, 217–225.
19.
Meyer, J. (2001). Effects of warning validity and proximity on responses to warnings. Human Factors, 43, 563–572.
20.
Meyer. J. (2004). Conceptual issues in the study of dynamic hazard warnings. Human Factors, 46, 196–204.
21.
Meyer, J., & Bitan, Y. (2002). Why better operators receive worse warnings. Human Factors, 44, 343–353.
22.
Milton, J. S., & Corbet, J. J. (1979). Applied statistics with probability. New York: Van Nostrand.
23.
Muir, B. M. (1987). Trust between humans and machines, and the design of decision aids. International Journal of Man-Machine Studies, 27, 527–539.
24.
Muir, B. M. (1994). Trust in automation: Part I. Theoretical issues in the study of trust and human intervention in automated systems, Ergonomics, 37, 1905–1922.
25.
Oesch, J. M., & Murnighan, J. K. (2003). Egocentric perceptions of relationships, competence, and trustworthiness in salary allocation choices. Social Justice Research, 16, 53–78.
26.
Parasuraman, R., Hancock, P. A., & Olofinboba, O. (1997). Alarm effectiveness in driver-centered collision-warning systems. Ergonomics, 40, 390–399.
27.
Parasuraman, R., & Riley, V. (1997). Humans and automation use: Use, misuse, disuse, abuse. Human Factors, 39, 230–253.
28.
Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000). A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics—Part A: Systems and Humans, 30, 286–297.
29.
Robinson, D. E., & Sorkin, R. D. (1985). A contingent criterion model of computer assisted detection. In R. Eberts & C. G. Eberts (Eds.), Trends in ergonomics/human factors (Vol. 2, pp. 75–82). Amsterdam: North Holland.
30.
Ruble, D. N., & Stangor, C. (1986). Stalking the elusive schema: Insights from developmental and social-psychological analyses of gender schemas. Social Cognition, 4, 227–261.
31.
Sheridan, T. B. (2002). Human performance in relation to automation. In T. B. Sheridan (Ed.), Humans and automation: System design and research issues (pp. 69–89). Santa Monica, CA: Human Factors and Ergonomics Society.
32.
Sheridan, T. B., & Ferrell, W. (1974). Man-machine systems: Information, control, and decision models of human performance. Cambridge, MA: MIT Press.
33.
Smith, D. A., & Graesser, A. C. (1981). Memory for actions in scripted activities as a function of typicality, retention interval, and retrieval task. Memory and Cognition, 9, 550–559.
34.
Swets, J. A., & Pickett, R. M. (1982). Evaluation of diagnostic systems: Methods from signal detection theory. New York: Academic Press.
35.
Thomas, L. C., Wickens, C. D., & Rantanen, E. M. (2003). Imperfect automation in aviation traffic alerts: A review of conflict detection algorithms and their implications for human factors research. In Proceedings of the Human Factors and Ergonomics Society 47th Annual Meeting (pp. 344–348). Santa Monica, CA: Human Factors and Ergonomics Society.
36.
Wickens, C. D., & Hollands, J. G. (2000). Engineering psychology and human performance (3rd ed.). Upper Saddle River, NJ: Prentice Hall.
37.
Wiegmann, D. A. (2002). Agreeing with automated diagnostic aids: A study of users' concurrence strategies. Human Factors, 44, 44–50.
38.
Wiegmann, D. A., Rich, A., & Zhang, H. (2001). Automated diagnostic aids: The effects of aid reliability on users' trust and reliance. Theoretical Issues in Ergonomics Science, 2, 352–367.
