Background: It was proposed that misuse and disuse often occur because operators (a) cannot determine if automation or a nonautomated alternative maximizes the likelihood of task success (appraisal errors) or (b) know the utilities of the options but disregard this information when deciding to use automation (intent errors). Objective: This investigation assessed the effectiveness of performance feedback, a procedure developed to attenuate appraisal errors, and scenario training, an intervention designed to decrease intent errors. Methods: Operators given feedback were told how many errors they and an automated device made on a target detection task. Scenario training took operators through the thought processes of optimal decision makers after the utilities of the automated and nonautomated alternatives had been determined. Following 200 training trials, participants chose between relying on their observations or an automated device. Results: There was little misuse, but disuse rates were high (84%) among operators receiving neither feedback nor scenario training. Operators paired with a more accurate machine and given feedback made approximately twice as many errors as the automated device. Nevertheless, intent errors were commonplace; 55% of the operators who received feedback without scenario training did not rely on automation. Feedback effectiveness was enhanced when used in conjunction with scenario training; the disuse rate decreased to 29%. Conclusion: A combination of feedback and scenario training was more effective in mitigating disuse than either intervention used in isolation. Application: An important application of these findings is that operator training programs should incorporate techniques to control intent and appraisal errors.
Get full access to this article
View all access options for this article.
References
1.
Beck, H.P., Dzindolet, M.T., & Pierce, L.G. (2002). Applying a decision-making model to understand misuse, disuse and appropriate automation use. In E. Salas (Ed.), Advances in human factors and cognitive engineering: Automation (Vol. 2, pp. 37-78). Amsterdam : JAI.
2.
Beck, H.P., & Pierce, L.G. (1996). The impact of selected group processes on the coordination and motivation of Army teams (ARL-CR-292). Aberdeen Proving Ground, MD: U.S. Army Research Laboratory .
3.
Dzindolet, M.T., Petersen, S.A., Pomranky, R.A., Pierce, L.G., & Beck, H.P. (2003). The role of trust in automation reliance. International Journal of Human and Computer Studies, 58, 697-718.
4.
Dzindolet, M.T., Pierce, L.G., Beck, H.P., & Dawe, L.A. (2002). The perceived utility of human and automated aids in a visual detection task. Human Factors, 44, 79-94.
Gurner, J. (2001). Casey's last ride. Retrieved March 3, 2006, from http://www.watervalley.net/users/caseyjones/cj~long.htm
7.
Halpin, S.M., Johnson, E.M., & Thornberry, J.A. (1973). Cognitive reliability in manned systems. IEEE Transactions on Reliability, R-22, 165-170.
8.
Heimberg, R.G., & Becker, R.E. (2002). Cognitive-behavioral group therapy for social phobia: Basic mechanisms and clinical strategies. New York : Guilford Press.
9.
Kantowitz, B.H., Hanowski, R.J., & Kantowitz, S.C. (1997). Driver acceptance of unreliable traffic information in familiar and unfamiliar settings. Human Factors, 39, 164-176.
10.
Kozlowski, S.W.J., Toney, R.J., Mullins, M.E., Weissbein, D.A., Brown, K.G., & Bell, B.S. (2001). Developing adaptability: A theory for the design of integrated-embedded training systems. In E. Salas (Ed.), Advances in human factors and cognitive engineering: Automation, (Vol. 1, pp. 59-123). Amsterdam : JAI.
11.
Langer, E.J., & Roth, J. (1975). Heads I win, tails it's chance: The illusion of control as a function of the sequence of outcomes in a purely chance task . Journal of Personality and Social Psychology, 32, 951-955.
12.
Lee, J.D., & Moray, N. (1992). Trust, control strategies and allocation of function in human-machine systems. Ergonomics, 35, 1243-1270.
13.
Lee, J.D., & See, K.A. (2004). Trust in automation: Designing systems for appropriate reliance. Human Factors, 46, 50-80.
14.
Meichenbaum, D., & Cameron, R. (1974). The clinical potential of modifying what clients say to themselves. Psychotherapy: Psychotherapy: Theory, Research and Practice, 11, 103-117.
15.
Metzger, U., & Parasuraman, R. (2005). Automation in future air traffic management: Effects of decision aid reliability on controller performance and mental workload. Human Factors, 47, 35-49.
16.
Moray, N., Inagaki, T., & Itoh, M. (2000). Adaptive automation, trust, and self-confidence in fault management of time-critical tasks. Journal of Experimental Psychology: Applied, 6, 44-58.
17.
Morrissey, R.B. (1979, September 8). Three Mile Island: Sifting out the facts. America, 141, 91-93.
18.
National Transportation Safety Board. (1996 ). Collision of Washington Metropolitan Transit Authority Train T111 with standing train at Shady Grove Station, near Gaithersburg, MD, January 6, 1996 (Rep. PB 96-916304). Washington, DC: Author.
19.
Nikolic, M.I., & Sarter, N.B. (2001). Peripheral visual feedback:Apowerful means of supporting effective attention allocation in event-driven, data-rich environments . Human Factors, 43, 30-38.
20.
Parasuraman, R., & Miller, C.A. (2004). Trust and etiquette in highcriticality automated systems. Communications of the ACM, 47, 51-55.
21.
Parasuraman, R., & Riley, V. (1997). Humans and automation: Use, misuse, disuse, abuse . Human Factors, 39, 230-253.
22.
Rodin, J., & Langer, E.J. (1980). Aging labels: The decline of control and the fall of self-esteem. Journal of Social Issues, 36, 12-29.
23.
Savidge, M. (2001, March 20). Greeneville skipper takes the stand . Retrieved March 3, 2006, from http://archives.cnn.com/2001/US/03/20/sub.collision.02/index.html
24.
Scaturo, D.J. (2005). Balancing directive and nondirective elements of treatment. In D. J. Scaturo (Ed.), Clinical dilemmas in psychotherapy: A transtheoretical approach to psychotherapy integration (pp. 67-81). Washington, DC: American PsychologicalAssociation.
25.
Singh, I.L., Deaton, J.E., & Parasuraman, R. (2001). Development of a scale to assess pilot attitudes to towards cockpit automation. Indian Academy of Applied Psychology, 27, 205-211.
26.
Singh, I.L., Hilburn, B., & Parasuraman, R. (1999). Effect of feedback on adaptive automation. Journal of the Indian Academy of Applied Psychology , 25, 157-165.
27.
Singh, I.L., Molloy, R., & Parasuraman, R. (1993). Automation-induced "complacency": Development of the complacency-potential rating scale. International Journal of Aviation Psychology, 3, 111-122.
28.
Singh, I.L., Sharma, H.O., & Parasuraman, R. (2001). Effects of manual training and automation reliability on automation induced complacency in a flight simulation task. Psychological Studies, 46, 21-27.
29.
Sklar, A.E., & Sarter, N.B. (1999). Good vibrations: Tactile feedback in support of attention allocation and human-automation coordination in event-driven domains . Human Factors, 41, 543-552.
30.
Taylor, S.E. (1989). Positive illusions: Creative self-deception and the healthy mind. New York: Basic Books.
31.
Thompson, S.C. (1981). Will it hurt less if I can control it? A complex answer to a simple question. Psychological Bulletin, 90, 89-101.
32.
Weil, M.M., & Rosen, L.D. (1995). The psychological impact of technology from a global perspective: A study of technological sophistication and technophobia in university students from twenty-three countries. Computers in Human Behavior, 11, 95-133.
33.
Wiegmann, D.A. (2002). Agreeing with automated diagnostic aids: A study of users' concurrence strategies. Human Factors, 44, 44-50.
34.
Wiegmann, D.A., & Cristina, F.J. (2000). Effects of feedback lag variability on the choice of an automated diagnostics aid: A preliminary predictive model. Theoretical Issues in Ergonomics Science, 1, 139-156.
35.
Wiegmann, D.A., Rich, A., & Zhang, H. (2001). Automated diagnostic aids: The effects of aid reliability on user's trust and reliance. Theoretical Issues in Ergonomics Science, 2, 352-367.
