Objective: Effects of four types of automation support and two levels of automation reliability were examined. The objective was to examine the differential impact of information and decision automation and to investigate the costs of automation unreliability. Background: Research has shown that imperfect automation can lead to differential effects of stages and levels of automation on human performance. Method: Eighteen participants performed a “sensor to shooter” targeting simulation of command and control. Dependent variables included accuracy and response time of target engagement decisions, secondary task performance, and subjective ratings of mental workload, trust, and self-confidence. Results: Compared with manual performance, reliable automation significantly reduced decision times. Unreliable automation led to greater cost in decision-making accuracy under the higher automation reliability condition for three different forms of decision automation relative to information automation. At low automation reliability, however, there was a cost in performance for both information and decision automation. Conclusion: The results are consistent with a model of human-automation interaction that requires evaluation of the different stages of information processing to which automation support can be applied. Application: If fully reliable decision automation cannot be guaranteed, designers should provide users with information automation support or other tools that allow for inspection and analysis of raw data.
Get full access to this article
View all access options for this article.
References
1.
Adams, T.K. (2001). Future warfare and the decline of human decision-making . Parameters (Winter 2001-2002 ), 57-71.
2.
Barnes, M.J. (2003). The human dimension of battlespace visualization: Research and design issues (Tech. Rep. ARL-TR-2885). Aberdeen Proving Ground, MD: Army Research Lab.
3.
Billings, C. (1997). Aviation automation: The search for a human-centered approach. Hillsdale, NJ: Erlbaum .
4.
Crocoll, W.M., & Coury, B.G. (1990). Status or recommendation: Selecting the type of information for decision aiding. In Proceedings of the Human Factors Society 34th Annual Meeting (pp. 1524-1528). Santa Monica, CA: Human Factors and Ergonomics Society.
5.
Davison, H., & Wickens, C.D. (1999). Rotorcraft hazard cueing: The effects on attention and trust (Tech. Rep. ARL-99-5/NASA-99-1). Savoy: University of Illinois, Aviation Research Lab.
6.
Department of the Army. (2003). THAAD theatre high altitude area defense missile system, USA. Retrieved November 17, 2003, from http://www.army-technology.com/projects/thaad/
7.
Dzindolet, M.T., Pierce, L.G., Beck, H.P., & Dawe, L.A. (2001). A framework of automation use (Tech. Rep. ARL-CR-2412) . Aberdeen Proving Ground, MD: Army Research Lab.
8.
Endsley, M.R., & Kaber, D.B. (1999). Level of automation effects on performance, situation awareness and workload in a dynamic control task. Ergonomics , 42, 462-292.
9.
Galster, S.M., Bolia, R.S., & Parasuraman, R. (2002). Effects of information and decision-aiding cueing on action implementation in a visual search task . In Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting (pp. 438-442). Santa Monica, CA: Human Factors and Ergonomics Society .
10.
Hart, S.G., & Staveland, L.E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In P. A. Hancock & N. Meshkati (Eds.), Human mental workload (pp. 139-183). Amsterdam: Elsevier Science.
11.
Jones, P.M., Wilkins, D.C., Bargar, R., Sniezek, J., Asaro, P., Danner, N., et al. (2000). CoRAVEN: Knowledge-based support for intelligent analysis. In Federated Laboratory 4th Annual Symposium: Advanced Displays and Interactive Displays Consortium (pp. 89-94). University Park, MD: U.S. Army Research Lab.
12.
Lee, J.D., & Moray, N. (1994). Trust, self-confidence, and operators' adaptation to automation. International Journal of Human-Computer Studies , 40, 153-184.
13.
Lee, J.D., & Sanquist, T.F. (1996). Maritime automation. In R. Parasuraman & M. Mouloua (Eds.), Human performance in automated systems: Recent research and trends (pp. 365-384). Hillsdale, NJ: Erlbaum.
14.
Lee, J., & See, J. (2004). Trust in automation and technology: Designing for appropriate reliance. Human Factors, 46, 50-80.
15.
Lorenz, B., Di Nocera, F., Röttger, S., & Parasuraman, R. (2002). Automated fault management in a simulated space flight micro-world. Aviation, Space, and Environmental Medicine, 73, 886-897.
16.
May, P.A., Molloy, R., & Parasuraman, R. (1993, October). Effects of automation reliability and failure rate on monitoring performance in a multi-task environment . Paper presented at the Human Factors and Ergonomics Society 37th Annual Meeting, Seattle, WA.
17.
Merlo, J.L., Wickens, C., & Yeh, M. (2000). Effect of reliability on cue effectiveness and display signaling. In Proceedings of the 4th Annual Army Federated Laboratory Symposium (pp. 27-31). College Park, MD: Army Research Laboratory.
18.
Molloy, R., & Parasuraman, R. (1994). Automation-induced monitoring inefficiency: The role of display integration and redundant color coding. In M. Mouloua & R. Parasuraman (Eds.), Human performance in automated systems: Recent research and trends (pp. 224-228). Hillsdale, NJ: Erlbaum.
19.
Ott, R.L. (1993). An introduction to statistical methods and data analysis. Belmont, CA: Wadsworth .
20.
Parasuraman, R., Molloy, R., & Singh, I.L. (1993). Performance consequences of automation-induced "complacency."International Journal of Aviation Psychology , 3, 1-23.
21.
Parasuraman, R., & Riley, V. (1997). Humans and automation: Use, misuse, disuse, abuse . Human Factors, 39, 230-253.
22.
Parasuraman, R., Sheridan, T.B., & Wickens, C.D. (2000). A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 30, 286-297.
23.
Sarter, N., & Schroeder, B. (2001). Supporting decision making and action selection under time pressure and uncertainty: The case of in-flight icing. Human Factors, 43, 573-583.
24.
Sheridan, T.B., & Verplank, W.L. (1978). Human and computer control of undersea teleoperators. Cambridge: Massachusetts Institute of Technology, Man Machine Systems Laboratory.
25.
Stanton, N.A., Young, M., & McCaulder, B. (1997). Drive-by-wire: The case of mental workload and the ability of the driver to reclaim control. Safety Science , 27, 149-159.
26.
Wickens, C.D. (2000). Imperfect and unreliable automation and its implications for attention allocation, information access and situation awareness (ARL-00-10/NASA-00-2) . Moffett Field, CA: NASAAmes Research Center.
27.
Wickens, C.D., & Dixon, S.R. (2005). Is there a magic number 7 (to the minus 1)? The benefits of imperfect diagnostic automation: A synthesis of the literature (Tech. Rep. AHFD-05-01/MAAD-05-1). Savoy: University of Illinois, Aviation Research Lab.
28.
Wickens, C.D., Gempler, K., & Morphew, M.E. (2000). Workload and reliability of traffic displays in aircraft traffic avoidance. Transportation Human Factors Journal , 2, 99-126.
29.
Wickens, C.D., Helleberg, J., & Xu, X. (2002). Pilot maneuver, choice, and workload in free flight . Human Factors, 44, 171-188.
30.
Wickens, C.D., & Xu, X. (2002). Automation trust, reliability and attention (Tech. Rep. AHFD-02-14/MAAD-02-2). Savoy: University of Illinois, Aviation Research Lab.
31.
Wiegmann, D.A., Rich, A., & Zhang, H. (2001). Automated diagnostic aids: The effects of aid reliability on users' trust and reliance. Theoretical Issues in Ergonomics Science, 2, 352-367.
