Planned Variation evaluations implement and compare two or more promising variations of an educational or social program. In this paper, I argue that Planned Variation evaluations may be especially attractive to teachers and program operators, as well as university-based researchers. The establishment of university-based centers for evaluation could support wide-scale implementation of Planned Variation projects. These projects could facilitate the substitution of effective program variants for ineffective variants, bringing Campbell’s (1984) vision of the experimenting society to reality and fostering the use of research results. This paper contrasts the Planned Variation approach with participatory, empowerment, collaborative, utilization-focused, and black-box experimental approaches. The Planned Variation Cross-Validation Model is described and contrasted with alternative models for fostering improvement in social programs.
Get full access to this article
View all access options for this article.
References
1.
Berman, P., & Pauley, E. W. (1975). Federal programs supporting educational change, Volume 2: Factors affecting change agent projects (Report). Santa Monica, CA: RAND.
2.
Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of the Learning Sciences, 2(2), 141-178.
3.
Campbell, D. T. (1984). Can we be scientific in applied social science? In R. F. Conner, D. G. Altman, & C. Jackson (Eds.), Evaluation studies review annual, Vol. 9 (pp. 26-48). Beverly Hills, CA: Sage.
4.
Campbell, D. T. (1988). Methodology and epistemology for social science: Selected papers. Chicago: University of Chicago Press.
5.
Campbell, D. T. (1998). The experimenting society. In W. N. Dunn (Ed.), The experimenting society: Essays in honor of Donald T. Campbell (pp. 35-68). New Brunswick: Transaction Publishers.
6.
Chen, H. (1990). Theory-driven evaluations. Newbury Park, CA: Sage.
7.
Chubb, J. E., & Moe, T. M. (1990). America’s public schools: Choice is a panacea. The Brookings Review, 8(3), 4-12.
8.
Cousins, J. B., & Earl, L. M. (Eds.). (1995). Participatory evaluation in education: Studies in evaluation use and organizational learning. London: Falmer Press.
9.
Dalton, B., Morocco, C. C., Tivnan, T., & Rawson-Mead, P. L. (1997). Supported inquiry science: Teaching for conceptual change in urban and suburban science classrooms. Journal of Learning Disabilities, 30(6), 670-684.
10.
Fairweather, G. W., Sanders, D. H., & Tornatzky, L. G. (1974). Creating change in mental health organizations. New York: Pergamon Press.
11.
Fetterman, D. M., Kaftarian, A. J., & Wandersman, A. (Eds.). (1996). Empowerment evaluation: Knowledge and tools for self-assessment and accountability. Newbury Park, CA: Sage.
12.
Garvin, D. A. (1993). Building a learning organization. Harvard Business Review, 71(4), 78-91.
13.
Guskey, T. R. (1986). Staff development and the process of teacher change. Educational Researcher, 15(5), 5-12.
14.
Hamilton, G., Brock, T., Farrell, M., Friedlander, D., Harknett, K., Hunter-Manns, J., Walter, J., & Weisman, J. (1997). Evaluating two welfare-to-work program approaches: Two-year findings on the labor force attachment and human capital development programs in three sites. New York: Manpower Demonstration Research Corporation.
15.
Hopfenberg, W. S., Levin, H. M., Chase, C., Christensen, S. G., Moore, M., Soler, P., Brunner, I., Keller, B., & Rodriguez, G. (1993). The accelerated schools resource guide. San Francisco: Jossey-Bass.
16.
McDermott, K. A. (2000). Barriers to large-scale success of models for urban school reform. Educational Evaluation and Policy Analysis, 22(1), 83-89.
17.
Palumbo, D., Musheno, M., & Maynard-Moody, S. (1986). Public sector entrepreneurs: The shakers and doers of program innovation. In J. S. Wholey, M. A. Abramson, & C. Bellavita (Eds.), Performance and credibility: Developing excellence in public and nonprofit organizations (pp. 69-82). Lexington, MA: Lexington Books.
18.
Parker, L. A. (1973). A new strategy for improving education: Comparative perspectives on the Institute for Learning and Teaching. Boston: Institute for Learning and Teaching.
19.
Patton, M. Q. (1978). Utilization-focused evaluation. Beverly Hills, CA: Sage.
20.
Rogers, E. M. (1995). Diffusion of innovations (4th ed.). New York: The Free Press.
21.
Rogers, E. M., & Shoemaker, F. F. (1971). Communication of innovations: A cross-cultural approach (2nd ed.). New York: The Free Press.
22.
Rossi, P. H. (1985, April 17). The Iron Law of evaluation and other metallic rules. Albany, NY: Rockefeller College, SUNY-Albany.
23.
Rothman, J. (1980). Social R&D: Research and development in the human services. Englewood Cliffs, NJ: Prentice-Hall.
24.
Schmuck, R. (1968). Social psychological factors in knowledge utilization. In T. L. Eidell & J. M. Kitchel (Eds.), Knowledge production and utilization in educational administration (pp. 143-173). Columbus, OH: University Council for Educational Administration.
25.
Schon, D. (1996). Organizational learning II. Reading, MA: Addison-Wesley.
26.
Schroeder, R. G., Van de Ven, A. H., Scudder, G. D., & Polley, D. (1986). Managing innovation and change processes: Findings from the Minnesota Innovation Research Program. Agribusiness Management, 2(40), 501-523.
27.
Shadish, W. R., Jr. (1987). Program micro- and macrotheories: A guide for social change. In L. Bickman (Ed.), Using program theory in evaluation. New Directions for Program Evaluation, 33(pp. 93-109). San Francisco, CA: Jossey Bass.
28.
Shadish, W. R. J., Cook, T. D., & Leviton, L. C. (1991). Foundations of program evaluation: Theories of practice. Newbury Park, CA: Sage.
29.
Smith, M., & O’Day, J. (1991). Systemic school reform. In S. Fuhrman & B. Malen (Eds.), The politics of curriculum and testing (pp. 233-267). Bristol, PA: Falmer.
30.
U.S. Department of Education. (1998). Guidance on the comprehensive school reform demonstration program. Available: http://www.ed.gov/offices/OESE/compreform/csrdgui.html. Accessed March 12, 1998.
31.
U.S. Department of Education. (1999). Appalachia Educational Laboratory (AEL). Available: http://www.ed.gov/prog_info/Labs/Profiles/appalach.html. Accessed April 9, 1999.
32.
Weiss, C. H. (1972). Evaluation research: Methods of assessing program effectiveness. Englewood Cliffs, NJ: Prentice-Hall.
33.
Weiss, C. H., with Bucuvalas, M. J. (1980). Social science research and decision-making. New York: Columbia University Press.
34.
Wholey, J. S. (1979). Evaluation: Promise and performance. Washington, DC: The Urban Institute.
35.
Yeh, S. S. (1998). Empowering education: Teaching argumentative writing to cultural minority middle school students. Research in the Teaching of English, 33(1), 49-83.
36.
Yeh, S. S. (2000). Building the knowledge base for improving educational and social programs through Planned Variation evaluations. American Journal of Evaluation, 21(1), 27-40.
