We investigated the effects of automation as applied to different stages of information processing on team performance in a complex decision-making task. Forty teams of 2 individuals performed a simulated Theater Defense Task. Four automation conditions were simulated with computer assistance applied to realistic combinations of information acquisition, information analysis, and decision selection functions across two levels of task difficulty. Multiple measures of team effectiveness and team coordination were used. Results indicated different forms of automation have different effects on teamwork. Compared with a baseline condition, an increase in automation of information acquisition led to an increase in the ratio of information transferred to information requested; an increase in automation of information analysis resulted in higher team coordination ratings; and automation of decision selection led to better team effectiveness under low levels of task difficulty but at the cost of higher workload. The results support the use of early and intermediate forms of automation related to acquisition and analysis of information in the design of team tasks. Decision-making automation may provide benefits in more limited contexts. Applications of this research include the design and evaluation of automation in team environments.
Get full access to this article
View all access options for this article.
References
1.
Bainbridge, L. (1987). Ironies of automation. In J. Rasmussen, K. Duncan, & J. Leplat (Eds.), New technology and human error (pp. 271-283). New York: Wiley.
2.
Bolstad, C. A., & Endsley, M.R. (1999). Shared mental models and shared displays: An empirical evaluation of team performance. In Proceedings of the Human Factors and Ergonomics Society 43rd Annual Meeting (pp. 213-217). Santa Monica, CA: Human Factors and Ergonomics Society.
3.
Bolstad, C. A., & Endsley, M. R. (2000). The effect of task load and shared displays on team situation awareness. In Proceedings of the XIVth Triennial Congress of the International Ergonomics Association and 44th Annual Meeting of the Human Factors and Ergonomics Society (pp. 1.189–1.192). Santa Monica, CA: Human Factors and Ergonomics Society.
4.
Bowers, C. A., Deaton, J., Oser, R., Prince C., & Kolb, M. (1993). The impact of automation on crew communication and performance. In Proceedings of the Seventh International Symposium on Aviation Psychology (pp. 758–761). Columbus: Ohio State University.
5.
Bowers, C. A., Jentsch, F., & Salas, E. (1994). Studying automation in the lab - Can you? Should you? In N. McDonald, N. Johnston, & R. Fuller (Eds.), Applications of psychology to aviation systems (Vol. 3, pp. 281–286). Avebury, UK: Ashgate.
6.
Bowers, C. A., Morgan, B. B., Salas, E., & Prince, C. (1993). Assessment of coordination demand for aircrew coordination and training. Military Psychology, 5, 95–112.
7.
Bowers, C. A., Oser, R. L., Salas, E., & Cannon-Bowers, J. A. (1996). Team performance in automated systems. In R. Parasuraman, & M. Mouloua (Eds.), Automation and human performance: Theory and applications (pp. 243–263). Mahwah, NJ: Erlbaum.
8.
Brannick, M. T., Prince, A., Prince, C., & Salas, E. (1995). The measurement of team process. Human Factors, 37, 641–651.
9.
Brannick, M. T., Roach, P. M., & Salas, E. (1993). Understanding team performance: A multimethod study. Human Performance, 6, 287–308.
10.
Clamann, M. P., Wright, M. C., & Kaber, D. B. (2002). Comparison of performance effects of adaptive automation applied to various stages of human-machine system information processing. In Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting (pp. 342–346). Santa Monica, CA: Human Factors and Ergonomics Society.
11.
Clothier, C. (1991). Behavioral interactions across various aircraft types: Results of systematic observations of line operations and simulations. In Proceedings of the Sixth International Symposium on Aviation Psychology (pp. 332–337). Columbus: Ohio State University.
12.
Cooke, N., Kiekel, P. A., & Bell, B. (2002). Addressing limitations in the measurement of team cognition. In Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting (pp. 403–407). Santa Monica, CA: Human Factors and Ergonomics Society.
13.
Costley, J., Johnson, D., & Lawson, D. (1989). A comparison of cockpit communication B737-B757. In Proceedings of the Fifth International Symposium on Aviation Psychology (pp. 413-418). Columbus: Ohio State University.
14.
Coury, B. G., & Semmel R. D. (1996). Supervisory control and the design of intelligent user interfaces. In R. Parasuraman, & M. Mouloua (Eds.), Automation and human performance: Theory and applications (pp. 201–219). Mahwah, NJ: Erlbaum.
15.
Dickinson, T. L., & McIntyre, R. M. (1997). A conceptual framework for teamwork measurement. In M. T. Brannick, E. Salas, & C. Prince (Eds.), Team performance assessment and measurement: Theory, methods and applications (pp. 19–45). Mahwah, NJ: Erlbaum.
16.
Endsley, M. R. (1995). Measurement of situation awareness in dynamic systems. Human Factors, 37, 65–84.
17.
Endsley, M. R. (1997). Level of automation: Integrating humans and automated systems. In Proceedings of the Human Factors and Ergonomics Society 41st Annual Meeting (pp. 200–204). Santa Monica, CA: Human Factors and Ergonomics Society.
18.
Endsley, M. R., & Kaber, D. B. (1999). Level of automation effects on performance, situation awareness and workload in a dynamic control task. Ergonomics, 42, 462–492.
19.
Endsley, M. R., & Kiris, E. O. (1995). The out-of-the-loop performance problem and level of control in automation. Human Factors, 37, 381–394.
20.
Foushee, H. C., Lauber, J. K., Baetge, M. M., & Acomb, D. B. (1986). Crew factors in flight operations III: The operational significance of exposure to short-haul air transport operations (NASA Tech. Memorandum 88322). Moffett Field, CA: NASA-Ames Research Center.
21.
Galster, S. M., Bolia, R. S., & Parasuraman, R. (2002). Effects of information automation and decision-aiding cuing on action implementation in a visual search task. In Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting (pp. 438–442). Santa Monica, CA: Human Factors and Ergonomics Society.
22.
Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In P. A. Hancock, & N. Meshkati (Eds.), Human mental workload (pp. 139–178). Amsterdam: Elsevier Science.
23.
Jentsch, F., & Bowers, C. (1996). Automation and crew performance: The importance of who and what. In Proceedings of the Human Factors and Ergonomics Society 40th Annual Meeting (pp. 49–53). Santa Monica, CA: Human Factors and Ergonomics Society.
24.
Jentsch, F., Sellin-Wolters, S., Bowers, C. A., & Salas, E. (1995). Crew coordination behaviors as predictors of problem detection and decision making times. In Proceedings of the Human Factors and Ergonomics Society 39th Annual Meeting (pp. 1350–1353). Santa Monica, CA: Human Factors and Ergonomics Society.
25.
Johannesen, L. J., Cook, R. I., & Woods, D. D. (1994). Cooperative communications in dynamic fault management. In Proceedings of the Human Factors and Ergonomics Society 38th Annual Meeting (pp. 225–229). Santa Monica, CA: Human Factors and Ergonomics Society.
26.
Kaber, D. B., & Endsley, M. R. (1997). The combined effect of level of automation and adaptive automation on human performance with complex, dynamic control systems. In Proceedings of the Human Factors and Ergonomics Society 41st Annual Meeting (pp. 205–209). Santa Monica, CA: Human Factors and Ergonomics Society.
27.
Kaber, D. B., Onal, E., & Endsley, M. R. (2000). Design of automation for telerobots and the effect on performance, operator situation awareness and subjective workload. Human Factors and Ergonomics in Manufacturing, 10, 409–430.
28.
Kaber, D. B., & Riley, J. R. (1999). Adaptive automation of a dynamic control task based on secondary task workload measurement. International Journal of Cognitive Ergonomics, 3, 169–187.
29.
Kaber, D. B., Wright, M. C., & Clamann, M. P. (2002). Human response to adaptive automation of early sensory/information acquisition functions and later stages of information processing in complex system operations (Final Report, NASA Langley Research Center Grant #NAG-1--01039). Hampton, VA: NASA Langley Research Center.
30.
Laois, L., & Giannacourou, M. (1995). Perceived effects of advanced ATC functions on human activities: Results of a survey on controllers and experts. In Proceedings of the Eighth International Symposium on Aviation Psychology (pp. 392–397). Columbus: Ohio State University.
31.
MacMillan, J., Entin, E., & Serfaty, D. (2002). From team structure to team performance: A framework. In Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting (pp. 408–412). Santa Monica, CA: Human Factors and Ergonomics Society.
32.
Orasanu, J. M. (1990). Shared mental models and crew decision making (Cognitive Science Laboratory Report No. 46). Princeton, NJ: Princeton University.
33.
Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000). A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 30, 286–297.
34.
Rovira, E., Zinni, M., & Parasuraman, R. (2002). Effects of information and decision automation on multi-task performance. In Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting (pp. 327–331). Santa Monica, CA: Human Factors and Ergonomics Society.
35.
Ruff, H., Draper, M. H., & Narayanan, S. (2000). The effect of automation level and decision aid fidelity on the control of multiple remotely operated vehicles. In D. B. Kaber, & M. R. Endsley (Eds.), Human performance, situation awareness and automation: User-centered design for the new millennium (pp. 70–75). Madison, WI: Omnipress.
36.
Sarter, N. B., & Schroeder, B. (2001). Supporting decision making and action selection under time pressure and uncertainty: The case of in-flight icing. Human Factors, 43, 573–583.
37.
Scerbo, M. W. (1996). Theoretical perspectives on adaptive automation. In R. Parasuraman, & M. Mouloua (Eds.), Automation and human performance: Theory and applications (pp. 37–63). Mahwah, NJ: Erlbaum.
38.
Serfaty, D., Entin, E. E., & Johnston, J. (1998). Team coordination training. In J. A. Cannon-Bowers, & E. Salas (Eds.), Making decisions under stress (pp. 221–245). Washington, DC: American Psychological Association.
39.
Sheridan, T. B., & Verplanck, W. L. (1978). Human and computer control of undersea teleoperators (Tech. Report). Cambridge, MA: MIT Man-Machine Laboratory.
40.
Travillian, K. K., Volpe, C. E., Cannon-Bowers, J. A., & Salas, E. (1993). Cross-training highly interdependent teams: Effects on team processes and team performance. In Proceedings of the Human Factors and Ergonomics Society 37th Annual Meeting (pp. 1243–1247). Santa Monica, CA: Human Factors and Ergonomics Society.
41.
Urban, J. M., Bowers, C. A., Monday, S. D., & Morgan, B. B. (1993). Effects of workload on communication processes in decision-making teams: An empirical study with implications for training. In Proceedings of the Human Factors and Ergonomics Society 37th Annual Meeting (pp. 1233–1237). Santa Monica, CA: Human Factors and Ergonomics Society.
42.
Volpe, C. E., Cannon-Bowers, J. A., Salas, E., & Spector, P. E. (1996). The impact of cross-training on team functioning: An empirical investigation. Human Factors, 38, 87–100.
43.
Wickens, C. D., Mavor, A. S., Parasuraman, R., & McGee, J. P. (Eds.). (1998). The future of air traffic control: Human operators and automation.Washington, DC: National Academy.
44.
Wiener, E. L. (1993). Crew coordination and training in the advanced-technology cockpit. In E. L. Wiener, B. G. Kanki, & R. L. Helmreich (Eds.), Cockpit resource management (pp. 199–266). San Diego, CA: Academic.
45.
Wise, J. A., Guide, P. C., Abbott, D. W., & Ryan, L. (1992). Human factors in aviation safety: The effects of automation on corporate pilots (Interim Tech. Report CAAR-15406--92--1). Daytona Beach, FL: Embry-Riddle Aeronautical University.
46.
Woods, D. D. (1996). Decomposing automation: Apparent simplicity, real complexity. In R. Parasuraman, & M. Mouloua (Eds.), Automation and human performance: Theory and applications (pp. 3–16). Mahwah, NJ: Erlbaum.
