The out-of-the-loop performance problem, a major potential consequence of automation, leaves operators of automated systems handicapped in their ability to take over manual operations in the event of automation failure. This is attributed to a possible loss of skills and of situation awareness (SA) arising from vigilance and complacency problems, a shift from active to passive information processing, and change in feedback provided to the operator. We studied the automation of a navigation task using an expert system and demonstrated that low SA corresponded with out-of-the-loop performance decrements in decision time following a failure of the expert system. Level of operator control in interacting with automation is a major factor in moderating this loss of SA. Results indicated that the shift from active to passive processing was most likely responsible for decreased SA under automated conditions.
Get full access to this article
View all access options for this article.
References
1.
Bainbridge, L. (1981). Mathematical equations or processing routines? In J. Rasmussen and W. B. Rouse (Eds.), Human detection and diagnosis of systems failures (pp. 259–286). New York: Plenum.
2.
Bainbridge, L. (1983). Ironies of automation. Automatica, 19, 775–779.
3.
Bergeron, H. P. (1981). Single pilot IFR autopilot complexity/benefit trade-off study. Journal of Aircraft, 18, 705–706.
4.
Billings, C. E. (1991). Human-centered aircraft automation: A concept and guidelines (NASA Tech. Memorandum 103885). Moffet Field, CA: NASA Ames Research Center.
5.
Carmody, M. A., and Gluckman, J. P. (1993). Task specific effects of automation and automation failure on performance, workload and situational awareness. In R. S. Jensen and D. Neumeister (Eds.), Proceedings of the Seventh International Symposium on Aviation Psychology (pp. 167–171). Columbus, OH: Ohio State University, Department of Aviation.
6.
Cowan, N. (1988). Evolving conceptions of memory storage, selective attention, and their mutual constraints within the human information processing system. Psychological Bulletin, 104(2), 163–191.
7.
Curry, R. E., and Ephrath, A. R. (1977). Monitoring and control of unreliable systems. In T. B. Sheridan and G. Johannsen (Eds.), Monitoring behavior and supervisory control (pp. 193–203). New York: Plenum.
8.
Danaher, J. W. (1980). Human error in ATC system operations. Human Factors, 22, 535–545.
9.
Endsley, M. (1987). The application of human factors to the development of expert systems for advanced cockpits. In Proceedings of the Human Factors Society 31st Annual Meeting (pp. 1388–1392). Santa Monica, CA: Human Factors and Ergonomics Society.
10.
Endsley, M. R. (1988). Design and evaluation for situation awareness enhancement. In Proceedings of the Human Factors Society 32nd Annual Meeting (pp. 97–101). Santa Monica, CA: Human Factors and Ergonomics Society.
11.
Endsley, M. R. (1992, June). Situation awareness: A fundamental factor underlying the successful implementation of AI in the air traffic control system. Paper presented at the NASA/FAA Workshop on Artificial Intelligence and Human Factors in Air Traffic Control and Aviation Maintenance, Daytona Beach, FL.
12.
Endsley, M. R. (1993a). Situation awareness and workload: Flip sides of the same coin. In R. S. Jensen and D. Neumeister (Eds.), Proceedings of the Seventh International Symposium on Aviation Psychology (pp. 906–911). Columbus, OH: Ohio State University, Department of Aviation.
13.
Endsley, M. R. (1993b, February). Situation awareness in dynamic human decision making: Theory. Paper presented at the First International Conference on Situational Awareness in Complex Systems, Orlando, FL.
14.
Ephrath, A. R., and Young, L. R. (1981). Monitoring vs. man-in-the-loop detection of aircraft control failures. In J. Rasmussen and W. B. Rouse (Eds.), Human detection and diagnosis of system failures (pp. 143–154). New York: Plenum.
15.
Hart, S. G., and Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In P. A. Hancock and N. Meshkati (Eds.), Human mental workload (pp. 139–183). Amsterdam: North-Holland.
16.
Ignizio, J. P. (1991). An introduction to expert systems. New York: McGraw-Hill.
17.
Kessel, C. J., and Wickens, C. D. (1982). The transfer of failure detection skills between monitoring and controlling dynamic systems. Human Factors, 24, 49–60.
18.
Kiris, E. O., and Endsley, M. R. (1993). Information presentation for expert systems in future aircraft cockpits. In R. S. Jensen and D. Neumeister (Eds.), Proceedings of the Seventh International Symposium on Aviation Psychology (pp. 27–32). Columbus, OH: Ohio State University, Department of Aviation.
19.
Lee, J., and Moray, N. (1992). Trust, control strategies and allocation of function in human-machine systems. Ergonomics, 35, 1243–1270.
20.
Moray, N. (1986). Monitoring behavior and supervisory control. In K. Boff (Ed.), Handbook of perception and human performance (pp. 40/1–40/51). New York: Wiley.
21.
Norman, D. A. (1989). The problem of automation: Inappropriate feedback and interaction not overautomation (ICS Report 8904). La Jolla, CA: University of California, San Diego, Institute for Cognitive Science.
22.
Orlady, H. W., and Wheeler, W. A. (1989). Training for advanced cockpit technology aircraft. In R. S. Jensen (Ed.), Proceedings of the Fifth International Symposium on Aviation Psychology (pp. 91–96). Columbus, OH: Ohio State University, Department of Aviation.
23.
Parasuraman, R. (1987). Human-computer monitoring. Human Factors, 29, 695–706.
24.
Parasuraman, R. (1992, June). Adaptive function allocation effects on pilot performance. Paper presented at the NASA/FAA Workshop on Artificial Intelligence and Human Factors in Air Traffic Control and Aviation Maintenance, Daytona Beach, FL.
25.
Parasuraman, R., Molloy, R., and Singh, I. L. (1993). Performance consequences of automation-induced complacency. International Journal of Aviation Psychology, 3, 1–23.
26.
Rasmussen, J. (1981). Models of mental strategies in process plant diagnosis. In J. Rasmussen and W. B. Rouse (Eds.), Human detection and diagnosis of system failures (pp. 241–258). New York: Plenum.
27.
Rasmussen, J. (1983). Skills, rules, and knowledge: Signals, signs and symbols and other distinctions in human performance models. IEEE Transactions on Systems, Man and Cybernetics, SMC-13, 257–266.
28.
Selcon, S. J. (1990). Decision support in the cockpit: Probably a good thing? In Proceedings of the Human Factors Society 34th Annual Meeting (pp. 46–50). Santa Monica, CA: Human Factors and Ergonomics Society.
29.
Sharit, J. (1985). Supervisory control of a flexible manufacturing system. Human Factors, 27, 47–59.
30.
Shiff, B. (1983) An experimental study of the human-computer interface in process control. Unpublished thesis, University of Toronto, Department of Industrial Engineering, Canada.
31.
Singh, I. L., Molloy, R., and Parasuraman, R. (1993). Automation-induced complacency: Development of the complacency-potential rating scale. International Journal of Aviation Psychology, 3, 111–122.
32.
Slamecka, N. J., and Graf, P. (1978). The generation effect: Delineation of a phenomenon. Journal of Experimental Psychology: Human Learning and Memory, 4, 592–604.
33.
Thackray, R. I., and Touchstone, R. M. (1989). Detection efficiency on an air traffic control monitoring task with and without computer aiding. Aviation, Space and Environmental Medicine, 60, 744–748.
34.
Waterman, D. A. (1986). A guide to expert systems. Reading, MA: Addison-Wesley.
35.
Wempe, T. (1965). Effects of gust-induced and maneuvering acceleration stress on pilot-vehicle performance. Aerospace Medicine, 36, 246–255.
36.
Wickens, C. D. (1992). Engineering psychology and human performance. (2nd ed.). New York: HarperCollins.
37.
Wickens, C. D., and Kessel, C. (1979). The effect of participatory mode and task workload on the detection of dynamic system failures. IEEE Transactions on Systems, Man and Cybernetics, SMC-9, 24–34.
38.
Wiener, E. L. (1985). Cockpit automation: In need of a philosophy. In Proceedings of the 1985 Behavioral Engineering Conference (pp. 369–375). Warrendale, PA: Society of Automotive Engineers.
39.
Wiener, E. L. (1988). Cockpit automation. In E. L. Wiener and D. C. Nagel (Eds.), Human factors in aviation (pp. 433–461). San Diego: Academic.
40.
Wiener, E. L. (1992, June). The impact of automation on aviation human factors. Paper presented at the NASA/FAA Workshop on Artificial Intelligence and Human Factors in Air Traffic Control and Aviation Maintenance, Daytona Beach, FL.
41.
Wiener, E. L., and Curry, R. E. (1980). Flight deck automation: Promises and problems. Ergonomics, 23, 995–1011.
42.
Young, L. R. A. (1969). On adaptive manual control. Ergonomics, 12, 635–657.
