This article discusses issues involved with the computer simulation of experimental tasks and settings used in team research with human participants. First, the purpose and components of simulation methodology are discussed along with the notion of simulation validity. Second, the common “external validity” criticism often leveled at experiments is considered, with particular attention given to the fidelity and realism questions researchers face when choosing or developing simulated team-performance environments. Third, the advantages and disadvantages of the current state of computerized team simulations are examined. The article concludes with considerations for future team research using computer simulation methodology.
Get full access to this article
View all access options for this article.
References
1.
Ancona, D. G. , & Caldwell, D. F. (1992). Bridging the boundary: External activity and performance in organizational teams. Administrative Science Quarterly, 37, 634-665.
2.
Berkowitz, L. , & Donnerstein, E. (1982). External validity is more than skin deep: Some answers to criticisms of laboratory experiments. American Psychologist, 37(3), 245-257.
3.
Bowers, C. A. , Salas, E., Prince, C., & Brannick, M. (1992). Games teams play: A method for investigating team coordination and performance. Behavior Research Methods, Instruments, & Computers, 24, 503-506.
4.
Campion, M. A. , Medsker, G. J., & Higgs, A. C. (1993). Relations between work group characteristics and effectiveness: Implications for designing effective work groups. Personnel Psychology, 46, 823-850.
5.
Cook, T. D. , & Campbell, D. T. (1979). Quasi-experimentation: Design & analysis issues for field settings. Boston: Houghton Mifflin.
6.
Driskell, J. E. , & Salas, E. (1992). Can you study real teams in contrived settings? The value of small group research to understanding teams. In R. W. Swezey & E. Salas (Eds.), Teams: Their training and performance (pp. 101-124). Norwood, NJ: Ablex.
7.
Dwyer, D. J. , Fowlkes, J. E., Oser, R. L., Salas, E., & Lane, N. E. (1997). Team performance measurement in distributed environments: The TARGETS methodology. In M. T. Brannick, E. Salas, and C. Prince (Eds.), Team performance and measurement: Theory, methods, and applications. Mahwah, NJ: Lawrence Erlbaum.
8.
Dyer, J. (1984). Team research and team training: State-of-the-art review. In F. A. Muckler (Ed.), Human factors review (pp. 285-323). Santa Monica, CA: Human Factors Society.
9.
Gladstein, D. (1984). Groups in context: A model of task group effectiveness. Administrative Science Quarterly, 29, 499-517.
10.
Gray, W. (1999, June). Experimental vs. mundane realism. Presentation at the Scaled Worlds Conference in Athens, GA.
11.
Hollenbeck, J. R. , Colquitt, J. A., Ilgen, D. R., LePine, J. A., & Hedlund, J. (1998). Accuracy decomposition and team decision making: Testing theoretical boundary conditions. Journal of Applied Psychology, 83, 494-500.
12.
Hollenbeck, J. R. , Ilgen, D. R., Sego, D. J., & Hedlund, J. (1995). Multilevel theory of team decision making: Decision performance in teams incorporating distributed expertise. Journal of Applied Psychology, 80, 292-331.
13.
Hollenbeck, J. R. , Ilgen, D. R., Tuttle, D. B., & Sego, D. J. (1995). Team performance on monitoring tasks: An examination of decision errors in contexts requiring sustained attention. Journal of Applied Psychology, 80, 685-696.
14.
Huey, B. M. , & Wickens, C. D. (1993). Workload transition: Implications for individual and team performance. Washington, DC: National Academy Press.
15.
Hyatt, D. E. , & Ruddy, T. M. (1997). An examination of the relationship between work group characteristics and performance: Once more into the breech. Personnel Psychology, 50, 553-585.
16.
Jentsch, F. , & Bowers, C. A. (1998). Evidence for the validity of PC-based simulations in studying aircrew coordination. The International Journal of Aviation Psychology, 8(3), 243-260.
17.
Keys, B. , & Wolfe, J. (1999). The role of management games and simulations in education and research. Retrieved December 1999 from the World Wide Web: www.gasou.edu/facstaff/cml/article2.html
18.
Koonce, J. M. , & Bramble, Jr., W. J. (1998). Personal computer-based flight training devices. The International Journal of Aviation Psychology, 8(3), 277-292.
19.
Lindsley, D. H. , Brass, D. J., & Thomas, J. B. (1994). Efficacy-performance spirals: A multilevel perspective. Academy of Management Journal, 20, 645-678.
20.
Mathieu, J. E. , & Button, S. B. (1992). An examination of the relative impact of normative information and self-efficacy on personal goals and performance over time. Journal of Applied Social Psychology, 22, 1758-1775.
21.
Mathieu, J. E. , Zaccaro, S. J., & Marks, M. A. (1999, April). Multi-team simulation using the ACES environment. Paper presented at the annual meetings of the Society for Industrial and Organizational Psychology, Atlanta, GA.
22.
McGrath, J. E. (1991). Time, interaction, and performance (TIP): A theory of groups. Small Group Research, 22, 147-174.
23.
McGrath, J. E. (1993). Introduction: The JEMCO workshop—Description of a longitudinal study. Small Group Research, 24, 285-306.
24.
McGrath, J. E. (1997). Small group research, that once and future field: An interpretation of the past with an eye to the future. Group Dynamics: Theory, Research, and Practice, 1, 7-27.
25.
Peters, V. , Vissers, G., & Heijne, G. (1998). The validity of games. Simulation and Gaming, 29(1), 20-30.
26.
Raser, J. C. (1969). Simulations and society: An exploration of scientific gaming. Boston: Allyn & Bacon.
27.
Rogelberg, S. (1999, April). Simulations for the study of teams. Symposium conducted at the meeting of the Society for Industrial and Organizational Psychology, Atlanta, GA.
28.
Salas, E. , Bowers, R., & Rhodenizer, L. (1998). It is not how much you have but how you use it: Toward a rational use of simulation to support aviation training. The International Journal of Aviation Psychology, 8(3), 197-208.
29.
Salas, E. , Dickinson, T. L., Converse, S. A., & Tannenbaum, S. I. (1992). Toward an understanding of team performance and training. In R. W. Swezey & E. Salas (Eds.), Teams: Their training and performance (pp. 3-30). Norwood, NJ: Ablex.
30.
Swezey, R. W. , & Llaneres, R. W. (1997). Models in training and instruction. In G. Salvendy (Ed.), Handbook of human factors and ergonomics (2nd ed.). New York: John Wiley.
31.
Swezey, R. W. , Streufert, S., Satish, U., & Siem, F. M. (1998). Preliminary development of a computer-based team performance assessment simulation. International Journal of Cognitive Ergonomics, 2(3), 163-179.
32.
Tesluk, P. E. , & Mathieu, J. E. (1999). Overcoming roadblocks to effectiveness: Incorporating management of performance barriers into models of work group effectiveness. Journal of Applied Psychology, 84, 200-217.
33.
Weaver, J. L. , Bowers, C. A., Salas, E., & Cannon-Bowers, J. A. (1995). Networked simulations: New paradigms for team performance research. Behavior Research Methods, Instruments, & Computers, 27(1), 12-24.
34.
Wolfe, J. , & Roge, J. N. (1997). Computerized general management games as strategic management learning environments. Simulation and Gaming, 28, 423-441.
35.
Yeo, G. K. , & Tan, S. T. (1999). Toward a multilingual, experiential environment for learning decision technology. Simulation and Gaming, 30(1), 70-82.
36.
Zaheer, S. , Albert, S., & Zaheer, A. (1999). Time scales and organizational theory. Academy of Management Review, 24, 725-741.
