In this paper I argue that research, development, and evaluation of social and educational programs should routinely be integrated through a planned variation approach. This approach typically involves the design, implementation, and evaluation of an enhanced, as well as standard, variation of a program. In contrast to conventional black box evaluations, I argue that a planned variation approach is better suited to the needs of program operators, offers methodological advantages, tests program theory, helps to build the base of knowledge for constructing better programs, and would foster greater utilization of evaluation findings. I discuss implications regarding the role and preparation of evaluators and issues in conducting planned variation evaluations. I compare the planned variation approach to other types of theory-based evaluation and discuss when and why it may be preferable.
Get full access to this article
View all access options for this article.
References
1.
Bissell, J. S. (1971). Implementation of planned variation in Head Start. Washington, DC: United States Department of Health, Education, and Welfare, Office of Child Development.
2.
Bryk, A. S., & Raudenbush, S. W. (1992). Hierarchical linear models: Applications and data analysis methods. Newbury Park, CA: Sage.
3.
Connell, D., Yeh, S. S., & Barnes, H. (1999). Designing a quality enhancement study in Head Start. Cambridge, MA: Abt Associates.
4.
Crain, R. L., & Carsrud, K. B. (1985). The role of the social sciences in school desegregation policy. In R. L. Shotland & M. M. Mark (Eds.), Social science and social policy (pp. 219-236). Beverly Hills, CA: Sage.
5.
Dalton, B., Morocco, C. C., Tivnan, T., & Rawson-Mead, P. L. (1997). Supported inquiry science: Teaching for conceptual change in urban and suburban science classrooms. Journal of Learning Disabilities, 30(6), 670-684.
6.
Datta, L., McHale, C., & Mitchell, S. (1976). The effects of Head Start classroom experiences on some aspects of child development: A summary report of national evaluations, 1966-1969 (DHEW Publication No. OHD-76-30088). Washington, DC: U.S. Government Printing Office.
7.
Glennan, T. K., Jr. (1998). New American Schools after six years. Santa Monica, CA: RAND.
8.
Griffin, G. A., Barnes, S., O’Neal, S., Edwards, S. A., Defino, M. E., & Hukill, H. (1983). Changing teacher practice: Final report of an experimental study (Report No. 9052). Austin, TX: University of Texas at Austin, Research and Development Center for Teacher Education (ERIC Document Reproduction Service No. ED 240 110).
9.
Karnes, M. B. (1969). Research and development program on preschool disadvantaged children: Investigations of classroom at-home interventions (Final Report). Urbana, IL: University of Illinois.
10.
Leftover money for welfare baffles or inspires states. (1999, August 29). The New York Times, pp. 1, 20-21.
11.
Lipton, D. S. (1992). How to maximize utilization of evaluation research by policymakers. The Annals of the American Academy of Political and Social Science, 521(May), 175-188.
12.
Mann, J. (1972). The outcome of evaluative research. In C. H. Weiss (Ed.), Evaluating action programs: Readings in social action and education (pp. 267-282). Boston: Allyn and Bacon.
13.
McGuinness, C., McGuinness, D., & McGuinness, G. D. J. (1996). Phono-Graphix: A new method for remediating reading difficulties. Annals of Dyslexia, 46, 73-96.
14.
McGuinness, D. (1997). Why our children can’t read and what we can do about it: A scientific revolution in reading. New York: Simon and Schuster.
15.
Miller, B., & Dyer, J. (1975). Four preschool programs: Their dimensions and effects. Monographs of the Society for Research in Child Development, 40(162), 5-6.
16.
Patton, M. Q. (1978). Utilization-focused evaluation. Beverly Hills, CA: Sage.
17.
Pawson, R., & Tilley, N. (1997). Realistic evaluation. Thousand Oaks, CA: Sage.
18.
Rivlin, A. M., & Timpane, P. M. (Eds.). (1975). Planned variation in education: Should we give up or try harder?Washington, DC: Brookings Institution.
19.
Rothman, J. (1980). Social R &D: Research and development in the human services. Englewood Cliffs, NJ: Prentice-Hall, Inc.
20.
Ruopp, R., Travers, J., Glantz, F., & Coelen, C. (1979). Children at the center: Summary findings and their implications. Cambridge, MA: Abt Associates.
21.
Scriven, M. S. (1981). Product evaluation. In N. L. Smith (Ed.), New techniques for evaluation (pp. 121-166). Beverly Hills, CA: Sage.
22.
Scriven, M. S. (1991). Beyond formative and summative evaluation. In M. W. McLaughlin & D. C. Phillips (Eds.), Evaluation and education: At quarter century (pp. 19-64). Chicago: University of Chicago Press.
23.
Slavin, R. E., Madden, N. A., Dolan, L. J., Wasik, B. A., Ross, S., Smith, L., & Dianda, M. (1996). Success For All: A summary of research. Journal of Education for Students Placed At-Risk, 1(1), 41-76.
24.
St. Pierre, R. G., Layzer, J. I., Goodson, B. D., & Bernstein, L. S. (1997). National impact evaluation of the comprehensive child development program: Final report. Cambridge, MA: Abt Associates.
25.
Stanford Research Institute. (1971). Implementation of planned variation in Head Start: Preliminary evaluation of planned variation in Head Start according to Follow Through approaches (1969- 1970). Menlo Park, CA.
26.
States declining to draw billions in welfare money. (1999, February 8). The New York Times, pp. A1, A19.
27.
Stebbins, L. B., St. Pierre, R. G., Proper, E. C., Anderson, R. B., & Cerva, T. R. (1977). Education as experimentation: A planned variation model (Report 76-196A). Cambridge, MA: Abt Associates, Inc., Volume IV-A: An evaluation of Follow Through
28.
U.S. Congress. (1993). Government Performance and Results Act of 1993. Available: http://www.financenet.gov/financenet/fed/cfo/gpra/gpralaw.htm.
29.
U.S. Department of Education. (1999, May 3). U.S. Department of Education FY 2000 annual plan, volume 2. Washington, DC. Available: http://www.ed.gov/pubs/AnnualPlan/vol2.pdf.
30.
U.S. Department of Health and Human Services. (1999a, June). The Administration for Children and Families. Washington, DC. Available: http://www.acf.dhhs.gov/programs/opa/facts/major.htm.
31.
U.S. Department of Health and Human Services. (1999b, June). Health Resources and Services Administration: FY 1999 House Mark. Washington, DC. Available: http://www.hrsa.dhhs.gov/Budget/99housmk.pdf.
32.
U.S. Department of Justice. (1999, June). Office of Justice Programs program plan. Washington, DC. Available: http://www.ojp.usdoj.gov/99progplan/99prog.txt.
33.
Weikart, D. P., Epstein, A. S., Schweinhart, L., & Bond, J. T. (1978). The Ypsilanti preschool curriculum demonstration project: Preschool years and longitudinal results. Ypsilanti, MI: High/Scope Educational Research Foundation.
34.
Weiss, C. H. (1972). Evaluation research: Methods of assessing program effectiveness. Englewood Cliffs, NJ: Prentice-Hall.
35.
Weiss, C. H. (1978). Improving the linkage between social research and public policy. In L. E. Lynn, Jr. (Ed.), Knowledge and policy: The uncertain connection (pp. 23-81). Washington, DC: National Academy of Sciences.
36.
Weiss, C. H. (1995). Nothing as practical as good theory: Exploring theory-based evaluation for comprehensive community initiatives for children and families. In J. P. Connell, A. C. Kubisch, L. B. Schorr, & C. H. Weiss (Eds.), New approaches to evaluating community initiatives (pp. 65-92). Washington, DC: Aspen Institute.
37.
Weiss, C. H., & Bucuvalas, M. J. (1980). Social science research and decision-making. New York: Columbia University Press.
38.
Wholey, J. S. (1979). Evaluation: Promise and performance. Washington, DC: The Urban Institute.
39.
Wong, K. K., & Meyer, S. J. (1998). Title I schoolwide programs: A synthesis of findings from recent evaluation. Educational Evaluation and Policy Analysis, 20(2), 115-136.
40.
Yeh, S. S. (1998). Empowering education: Teaching argumentative writing to cultural minority middle school students. Research in the Teaching of English, 33(1), 49-83.
