Evaluations of gaming simulations and business games as teaching devices are typically end-state driven. This emphasis fails to detect how the simulation being evaluated does or does not bring about its desired consequences. This paper advances the use of a logic model approach, which possesses a holistic perspective that aims at including all elements associated with the situation created by a game. The use of the logic model approach is illustrated as applied to SIMGAME, a board game created for secondary school level business education in six European Union countries.
Anderson, P.H., & Lawton, L. (1992). A survey of methods used for evaluating student performance on business simulations. Simulation & Gaming , 23, 490-498.
2.
Atkinson, J.W. (1974). Motivational determinants of intellectual performance and cumulative achievement. In J.W. Atkinson & J. Raynor (Eds.), Achievement and performance (pp. 389-410). Washington, DC: Winston.
3.
Berliner, D. (1984). The half-full glass: A review of research on teaching . In P. Hosford (Ed.), Using what we know about teaching. Alexandria, VA: ASCD.
4.
Bloom, B.S. (1976). Human characteristics and school learning. New York: McGraw-Hill.
5.
Brophy, J.E., & Good, T.L. (1986). Teacher behavior and student achievement. In M. C. Wittrock (Ed.), Handbook of research on teaching (3rd ed., pp. 328-375). New York: Macmillan.
6.
Brown, J.S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning . Educational Researcher, 18, 32-42.
7.
Brunner, C., Bennett, D., & Honey, M. (1998). Girl games and technological desire. In J. Cassell & H. Jenkins , (Eds.), From Barbie to Mortal Combat: Gender and computer games (pp. 72-89). Boston: MIT Press .
8.
Carr, D. (2005). Contexts, gaming pleasures, and gendered preferences . Simulation & Gaming, 36, 464-482.
9.
Chelimsky, E., & Shadish, W. R. (Eds.). (1997). Evaluation for the 21st century. A handbook. Thousand Oaks, CA: Sage.
10.
Chen, H.T. (1990). Theory-driven evaluations. Newbury Park, CA: Sage.
11.
Chen, H.T. (2005). Practical program evaluation. Thousand Oaks, CA: Sage.
12.
Chen, H.T., & Rossi, P.H. (1983). Evaluating with sense: The theory-driven approach . Evaluation Review, 7, 283-302.
13.
Cohen, E.G. (1994). Restructuring the classroom: Conditions for productive small groups. Review of Educational Research, 64, 1-35.
14.
Cronbach, L.J., Ambron, S.R., Dornbusch, S.M., Hess, R.D., Hornik, R.C., Phillips, D.C., et al. (1980). Toward reform of program evaluation. San Francisco: Jossey-Bass.
15.
Cronbach, L., & Snow, R. (1977). Aptitudes and instructional methods: A handbook for research on interactions. New York: Irvington.
16.
Davidson, E.J. (2000). Ascertaining causality in theory-based evaluation . New Directions for Evaluation, 87, 5-13.
17.
Ditton, H. (2002). Unterrichtsqualität. Konzeptionen, methodische Überlegungen und Perspektiven [Quality of classroom learning and teaching. Concepts, methodological considerations and perspectives]. Unterrichtswissenschaft, 30, 197-212.
18.
Donaldson, S.I., & Gooler, L.E. (2003). Theory-driven evaluation in action: Lessons from a $20 million statewide Work and Health Initiative. Evaluation and Program Planning, 26, 355-366.
19.
Dorn, D.S. (1989). Simulation games: One more tool on the pedagogical shelf. Teaching Sociology, 17, 1-18.
20.
Faria, A.J. (2001). The changing nature of business simulation/gaming research: A brief history. Simulation & Gaming, 32, 97-110.
Friedrich, H.F., Hron, A., & Hesse, F.W. (2001). A framework for designing and evaluating virtual seminars. European Journal of Education, 36, 157-174.
23.
Garris, R., Ahlers, R., & Driskell, J.E. (2002). Games, motivation, and learning: A research and practice model. Simulation & Gaming, 33, 441-467.
24.
Gosen, J., & Washbush, J. (2004). A review of scholarship on assessing experiential learning effectiveness. Simulation & Gaming, 35(2), 270-293.
25.
Greenblat, C.S. (1989). Extending the range of experience. In D. Crookall & D. Saunders (Eds.), Communication and simulation: From two fields to one theme (pp. 269-283). Clevendon, England: Multilingual Matters.
26.
Greenlaw, P.S., & Wyman, F.P. (1973). The teaching effectiveness of games in collegiate business courses. Simulation & Games, 4(3), 259-294.
27.
Helmke, A., & Weinert, F.E. (1997). Bedingungsfaktoren schulischer Leistungen [Determinants of school performance]. In F. E. Weinert (Ed.), Psychologie des Unterrichts und der Schule (Enzyklopädie der Psychologie, Pädagogische Psychologie, Bd. 3) [Psychology of teaching and school psychology (Encyclopedia of psychology, educational psychology, Vol. 3)] (pp. 71-176). Göttingen, Germany: Hogrefe.
28.
Hennessy, M., & Greenberg, J. (1999). Bringing it all together: Modeling intervention processes using structural equation modeling. American Journal of Evaluation, 20, 471-480.
29.
Hense, J., Mandl, H., & Gräsel, C. (2001). Problemorientiertes Lernen. Warum der Unterricht mit Neuen Medien mehr sein muss als Unterrichten mit neuen Medien [Problem oriented learning. Why learning with new media has to be more than instructing with new media]. Computer und Unterricht, 11(4), 66-82.
Hindle, K. (2002). A grounded theory for teaching entrepreneurship using simulation games. Simulation & Gaming, 33, 236-241.
32.
Hox, J. (1995). Applied multilevel analysis. Amsterdam: TT-Publikaties.
33.
Huebner, T.A. (2000). Theory-based evaluation: Gaining a shared understanding between school staff and evaluators. New Directions for Evaluation , 87, 79-89.
34.
Keys, J.B., & Wolfe, J. (1990). The role of management games and simulations in education and research. Yearly Review of Management, 16(2), 307-336.
35.
Kolb, D.A. (1984). Experiential learning: Experience as the source of learning and development. Englewood Cliffs, NJ: Prentice Hall.
36.
Kriz, W.C. (2003). Creating effective learning environments and learning organizations through gaming simulation design. Simulation & Gaming, 34, 495-511.
37.
Leeuw, F.L. (2003). Reconstructing program theories. Methods available and problems to be solved. American Journal of Evaluation , 24, 5-20.
38.
Mark, M.M., Hoffman, D.A., & Reichardt, C.S. (1992). Testing theories in theory-driven evaluations: (Tests of) moderation in all things. In H. T. Chen & P. H. Rossi (Eds.), Using theory to improve program and policy evaluations (pp. 71-84). New York: Greenwood Press.
39.
Pekrun, R., & Schiefele, U. (1996). Emotions und motivationspsychologische Bedingungen der Lernleistungen [Emotional and motivational determinants of learning performance] . In F. E. Weinert (Ed.), Psychologie des Lernens und der Instruktion (pp. 153-180). Göttingen, Germany: Hogrefe.
40.
Peters, V.A.M., & Vissers, G.A.N. (2004). A simple classification model for debriefing simulation games. Simulation & Gaming , 35(1), 70-84.
41.
Petrosino, A. (2000). Whether and why? The potential benefits of including program theory evaluation in meta-analysis. New Directions for Evaluation, 87, 59-69.
42.
Raths, J.D., & McAninch A.C. (1999). What counts as knowledge in teacher education? In J. D. Raths & A. C. McAninch (Eds.), Advances in Teacher Education (vol. 5). Stanford, CT: Ablex.
43.
Remus, W.E. (1981). Experimental designs for analyzing data on games: Or, even the best statistical methods do not replace good experimental control . Simulation & Games, 12, 3-14.
44.
Reynolds, A.J. (1998). Confirmatory program evaluation: A method for strengthening causal inference. American Journal of Evaluation , 19, 203-221.
45.
Rogers, P.J., Petrosino, A., Huebner, T.A., & Hacsi, T.A. (2000). Program theory evaluation: Practice, promise, and problems. New Directions for Evaluation, 87, 5-13.
46.
Rossi, P., Lipsey, M.W., & Freeman, H. (2004). Evaluation. A systematic approach. Thousand Oaks, CA: Sage.
47.
Salomon, G., & Globerson, T. (1989). When teams do not function the way they ought to. International Journal of Educational Research, 13, 89-99.
48.
Scriven, M. (1973). Goal-free evaluation. In E. R. House (Ed.), School evaluation: The politics and process (pp. 319-328). Berkeley, CA: McCutchan.
49.
Shadish, W.R., Cook, T.D., & Leviton, L.C. (1991). Foundations of program evaluation. Theories of practice. Newbury Park, CA: Sage .
50.
Slavin, R.E. (1990). Cooperative learning: Theory, research, and practice . Englewood Cliffs, NJ: Prentice Hall.
51.
Slavin, R.E. (1994). Educational psychology. Theory and practice. Boston: Allyn & Bacon.
52.
Spector, J.M. (2000). System dynamics and interactive learning environments: Lessons learned and implications for the future. Simulation & Gaming, 31, 528-535.
53.
Spector, J.M.Christensen, D.L., Sioutine, A.V., & McCormack, D. (2001). Models and simulations for learning in complex domains: Using causal loop diagrams for assessment and evaluation. Computers in Human Behaviour, 14, 517-545.
54.
Stame, N. (2004). Theory-based evaluation and types of complexity . Evaluation, 10, 58-76.
55.
Stufflebeam, D. L., Madaus G. F., & Kellaghan T. (Eds.). (2000). Evaluation models-viewpoints on educational and human services evaluation. Boston: Kluwer.
56.
Thiagarajan, S. (1992). Using games for debriefing. Simulation & Gaming, 23(2), 161-173.
57.
Wang, M.C., Haertel, G.D., & Walberg, H.J. (1993). Toward a knowledge base for school learning. Review of Educational Research, 63, 249-294.
58.
Weiss, C.H. (1997). Theory-based evaluation: Past, present, and future . New Directions for Evaluation, 76, 41-56.
59.
Weiss, C.H. (2000). Which links in which theories shall we evaluate ? New Directions for Evaluation, 87, 35-45.
60.
Wholey, J.S. (1979). Evaluation. Promise and performance. Washington, DC: The Urban Institute.
61.
W.K. Kellogg Foundation. (2004a). W. K. Kellogg Foundation evaluation handbook. Battle Creek, MI: W.K. Kellogg Foundation.
62.
W. K. Kellogg Foundation. (2004b). W. K. Kellogg Foundation logic model development guide. Battle Creek, MI: W. K. Kellogg Foundation.
63.
Wolfe, J. (1990). The evaluation of computer-based business gaming . In J. W. Gentry (Ed.), Guide to business gaming and experiential learning (pp. 279-300). London: Kogan Page.
64.
Wolfe, J. (1997). The effectiveness of business games in strategic management course work. Simulation & Gaming, 28, 360-376.
65.
Wolfe, J., & Crookall, D. (1998). Developing a scientific knowledge of simulation/gaming . Simulation & Gaming, 29, 7-19.
66.
Wright, S.P., Horn, S.P., & Sanders, W.L. (1997). Teacher and classroom context effects on student achievement: Implications for teacher evaluation. Journal of Personnel Evaluation in Education, 11, 57-67.
67.
Yampolskaya, S., Nesman, T.M., Hernandez, M., & Koch, D. (2004). Using concept mapping to develop a logic model and articulate a program theory: A case example. American Journal of Evaluation, 25, 191-207.
