Confirmatory program evaluation is discussed as a method for conducting theorydriven evaluations. It is an impact assessment that emphasizes the explication and testing of a priori program theories within a broad assessment of program effectiveness. Greater emphasis is given to identifying the causal mechanisms or active ingredients of intervention effects. Confirmatory evaluation approaches are applied to the field of early childhood intervention. Although significant progress in understanding the effects of early childhood interventions has occurred over the last four decades, questions remain about the causal mechanisms of change, who benefits most from which program components, and the reliability of effects for large-scale programs. Examples from the Chicago Longitudinal Study are highlighted to show how confirmatory evaluation can help validate the effects of social interventions. Studies of the Chicago Child-Parent Centers are described to emphasize how the causal criteria of coherence, specificity, and within- and between-study consistency can strengthen causal inference and generalizability.
Get full access to this article
View all access options for this article.
References
1.
AndersonS., AuquierA., HauckW. W., OakesD., VandaeleW., and WeisbergH. I. (1980). Statistical methods for comparative studies: Techniques for bias reduction. New York: Wiley.
2.
BarnettW. S. (1995). Long-term effects of early childhood programs on cognitive and school outcomes. Future of Children, 5(3), 25—50.
3.
BarnettW. S., and BoocockS. S. (Eds.). (1998). Early care and education for children in poverty. Albany: State University of New York Press.
4.
BarnettW. S., YoungJ. W., and SchweinhartL. J. (1998). How preschool education influences long-term cognitive development and school success: A causal model. In BarnettW.S., and BoocockS.S. (Eds.), Early care and education for children in poverty (pp. 167—184). Albany: State University of New York Press.
5.
Berrueta-ClementJ. R., SchweinhartL. J., BarnettW. S., EpsteinA. S., and WeikartD. P. (1984). Changed lives: The effects of the Perry Preschool Program on youths through age 19. Y psil-anti, MI: High/Scope Educational Research Foundation.
6.
BoruchR., PetrosinoA., and ChalmersI. (1999, July 15). The Campbell Collaborative: A proposal for systematic, multi-national, and continuous review of evidence. Background paper, School of Public Policy, University College, London.
7.
BronfenbrennerU. (1975). Is early intervention effective? In GuttentagM., and StrueningE. (Eds.), Handbook of evaluation research (Vol. 2, pp. 519—603). Beverly Hills, CA: Sage.
8.
BronfenbrennerU., and MorrisP. (1998). Ecological processes of development. In and and DamonW. (Ed.), Handbook of child psychology: Theoretical issues (Vol. 1, pp. 993—1028). New York: Wiley.
9.
BrownC. H., MrazekP. J., and HosmanC. M. H. (1999, June 13). An international classification system of prevention trials. Washington, DC: Society for Prevention Research.
10.
CampbellF. A., and RameyC. T. (1994). Effects of early intervention on intellectual and academic achievement: A follow-up study of children from low-income families. Child Development, 65, 684—698.
11.
CampbellF. A., and RameyC. T. (1995). Cognitive and school outcomes for high risk African-American students at middle adolescence: Positive effects of early intervention. American Educational Research Journal, 32, 743—772.
12.
CampbellD. T., and StanleyJ. C. (1966). Experimental and quasi-experimental designs for research. Boston: Houghton Mifflin.
13.
ChenH. T. (1990). Theory-driven evaluations. Newbury Park, CA: Sage.
14.
ChenH., and RossiP. H. (Eds.). (1992). Using theory to improve program and policy evaluations. New York: Greenwood Press.
15.
Chicago Board of Education. (1988). Chicago EARLY: Instructional activities for ages 3 to 6. Vernon Hills, IL: ETA.
16.
Chicago Longitudinal Study. (1999). Chicago Longitudinal Study: User's guide (Vol. 6). Madison: Waisman Center, University of Wisconsin.
17.
ClarkeS. H., and CampbellF. A. (1998). Does intervention early prevent crime later?Early Childhood Research Quarterly, 13, 319—343.
18.
CoieJ. D., WattN. F., WestS. G., HawkinsJ. D., AsarnowJ. R., and MarkmanH. J.. (1993). The science of prevention: A conceptual framework and some directions for a national research program. American Psychologist, 48, 1013—1022.
19.
Committee for Economic Development. (2002). Preschool for all: Investing in a productive and just society. New York: Research and Policy Committee.
20.
Consortium for Longitudinal Studies. (1983). As the twig is bent: Lasting effects of preschool programs. Hillsdale, NJ: Erlbaum.
21.
CookT. D., and CampbellD. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Chicago: Rand McNally.
22.
CronbachL. J. (1982). Designing evaluations for educational and social programs. San Francisco: Jossey-Bass.
23.
CurrieJ. (2001). Early childhood education programs. Journal of Economic Perspectives, 15, 213—238.
24.
CurrieJ., and ThomasD. (1995). Does Head Start make a difference?American Economic Review, 85, 341—364.
25.
CurrieJ., and ThomasD. (2000). School quality and the longer-term effects of Head Start. Journal of Human Resources, 35, 755—774.
26.
GoryK. M. (2001). Early childhood education: A meta-analytic affirmation of the short- and long-term benefits of educational opportunity. School Psychology Quarterly, 16, 9—30.
KarolyL. A., GreenwoodP. W., EveringhamS. S., HoubeJ., KilburnM. R., and RydellC. P.. (1998). Investing in our children: What we know and don't know about the costs and benefits of early childhood interventions. Santa Monica, CA: RAND.
32.
LallyJ. R., MangioneP. L., HonigA. S., and WittnerD. S. (1988, April). More pride, less delinquency: Findings from the ten-year follow-up study of the Syracuse University Family Development Research Program. Zero to Three, 13—18.
33.
LambM. E. (1998). Nonparental child care: Context, quality, correlates, and consequences. In and and DamonW. (Ed.), Handbook of child psychology: Child psychology in practice (Vol. 4, pp. 73—133). New York: Wiley.
34.
LazarI., DarlingtonR. B., MurrayH. W., and SnipperA. S. (1982). Lasting effects of early education: A report from the consortium for longitudinal studies. Monographs of the Society for Research in Child Development, 47, (2/3, Serial No. 195).
35.
LeeV. E., and LoebS. (1995). Where do Head Start attendees end up?. One reason why preschool effects fade out. Educational Evaluation and Policy Analysis, 17, 62—82.
36.
LipseyM. W. (1993). Theory as method: Small theories as treatments. In SechrestL., and ScottA. (Eds.), Understanding causes and generalizing about them. New directions for program evaluation, (No. 57, pp. 5–38). San Francisco: Jossey-Bass.
37.
LipseyM. W., and WilsonD. B. (1993). The efficacy of psychological, educational, and behavioral treatment. American Psychologist, 48, 1181—1209.
38.
MasseL. N., and BarnettW. S. (2002). A benefit-cost analysis of the Abecedarian early childhood intervention. New Brunswick, NJ: National Institute of Early Education Research.
39.
NaisbittN. (1968). Child-Parent Education Centers, ESEA Title I, Activity I. Unpublished report, Chicago, IL.
40.
National Science and Technology Council. (1997, April). Investing in our future: A national research initiative for America's children for the 21st century. Washington, DC: Executive Office of the President, Office of Science and Technology Policy, Committee on Fundamental Science, and the Committee on Health, Safety, and Food.
41.
OvermanE. S. (Ed.). (1988). Methodology and epistemology for social science: Select papers of Donald T. Campbell. Chicago: University of Chicago Press.
42.
OuS. (2003). Mechanisms of effects of an early childhood intervention on educational attainment. Unpublished doctoral dissertation, University of Wisconsin–Madison.
43.
PopperK. (1956). The nature of scientific discovery. New York: Basic Books.
44.
PowellD. R. (1999). Early childhood development. In ReynoldsA.J., WalbergH.J., and WeissbergR.P. (Eds.), Promoting positive outcomes: Issues in children's and families’ lives. Washington, DC: Child Welfare League of America.
45.
ProctorE. K. (2003). Research to inform the development of social work interventions. Social Work Research, 27(1), 3—5.
46.
RameyS. L, and RameyC. T. (1992). Early educational intervention with disadvantaged children: To what effect?Applied & Preventive Psychology, 1, 131—140.
47.
RameyC. T., and RameyS. L. (1998). Early intervention and early experience. American Psychologist, 53, 109—120.
48.
RameyC. T., BryantD. M., WasikB. H., SparlingJ. J., FendtK. H., and LaVangeL. M. (1992). Infant Health and Development Program for low birth weight, premature infants: Program elements, family participation, and child intelligence. Pediatrics, 89, 454—465.
49.
ReynoldsA. J. (1991). Early schooling of children at risk. American Educational Research Journal, 28, 392—422.
50.
ReynoldsA. J. (1994). Effects of a preschool plus follow-on intervention for children at risk. Developmental Psychology, 30, 787—804.
51.
ReynoldsA. J. (1995). One year of preschool or two: Does it matter?Early Childhood Research Quarterly, 10, 1—31.
52.
ReynoldsA. J. (1998). Confirmatory program evaluation: A method for strengthening causal inference. American Journal of Evaluation, 17, 21—35.
53.
ReynoldsA. J. (2000). Success in early intervention: The Chicago Child-Parent Centers. Lincoln: University of Nebraska Press.
54.
ReynoldsA. J., ChangH., and TempleJ. A. (1998). Early childhood intervention and juvenile delinquency: An exploratory analysis of the Child-Parent Centers. Evaluation Review, 22, 341—372.
55.
ReynoldsA. J., MavrogenesN. A., BezruczkoN., and HagemannM. (1996). Cognitive and family-support mediators of preschool effectiveness: A confirmatory analysis. Child Development, 67, 1119—1140.
56.
ReynoldsA. J., and OuS. (2004). Alterable predictors of child well-being in the Chicago Longitudinal Study. Children and Youth Services Review, 26, 1—14.
57.
ReynoldsA. J., OuS., and TopitzesJ. W. (2004). Paths of effects of early childhood intervention on educational attainment and delinquency: A confirmatory analysis of the Chicago Child-Parent Centers. Child Development, 75, 1299—1328.
58.
ReynoldsA. J., and RobertsonD. L. (2003). School-based early children intervention and later maltreatment in the Chicago Longitudinal Study. Child Development, 74, 3—26.
59.
ReynoldsA. J., and TempleJ. A. (1995). Quasi-experimental estimates of the effects of a preschool intervention: Psychometric and econometric comparisons. Evaluation Review, 19, 347—373.
60.
ReynoldsA. J., and TempleJ. A. (1998). Extended early childhood intervention and school achievement: Age 13 findings from the Chicago Longitudinal Study. Child Development, 69, 231—246.
61.
ReynoldsA. J., TempleJ. A., RobertsonD. L., and MannE. A. (2001). Long-term effects of an early childhood intervention on educational achievement and juvenile arrest: A 15-year follow-up of low-income children in public schools. Journal of the American Medical Association, 285(18), 2339—2346.
62.
ReynoldsA. J., TempleJ. A., RobertsonD. L., and MannE. A. (2002). Age 21 cost-benefit analysis of the Title I Chicago Child-Parent Centers. Educational Evaluation and Policy Analysis, 24, 267—303.
63.
ReynoldsA. J., WangM. C., and WalbergH. J. (Eds.). (2003). Early childhood programs for a new century. Washington, DC: CWLA Press.
64.
RosenbaumP. (1995). Observational studies. New York: Springer-Verlag.
65.
RosenbaumP., and RubinD. (1984). From association to causation in observational studies: The role of tests of strongly ignorable treatment assignment. Journal of the American Statistical Association, 79, 41—48.
66.
SchweinhartL. J., BarnesH. V., and WeikartD. P. (1993). Significant benefits: The High/Scope Perry Preschool study through age 27. Ypsilanti, MI: High/Scope Educational Research Foundation.
67.
SchweinhartL. J., and WeikartD. P. (1980). Young children grow up: The effects of the Perry Preschool Program on youths through age 15. Ypsilanti, MI: High/Scope Educational Research Foundation.
68.
SchuermanJ., SoydanH., MacdonaldG., ForslundM., de MoyaD., and BoruchR. (2002). The Campbell Collaboration. Research on Social Work Practice, 12, 309—317.
69.
SeitzV. (1990). Intervention programs for impoverished children: A comparison of educational and family support models. Annals of Child Development, 7, 73—103.
70.
ShadishW. R., and RagsdaleK. (1996). Random versus nonrandom assignment in controlled experiments: Do you get the same answer?Journal of Consulting and Clinical Psychology, 64, 1290—1305.
71.
ShonkoffJ., and PhillipsD. (Eds.). (2000). From neurons to neighborhoods: The science of early childhood development. Washington, DC: National Academy of Sciences Press.
72.
SullivanL. M. (1971). Let us not underestimate the children. Glenview, IL: Scott, Foreman and Company.
73.
SusserM. (1973). Causal thinking in the health sciences, concepts and strategies of epidemiology. New York: Oxford University Press.
74.
TempleJ. A., and ReynoldsA. J. (in press). Economic returns of investments in preschool education. In ZiglerE., GilliamW., and JonesS. (Eds.), A vision of universal prekindergarten. New York: Cambridge University Press.
75.
TempleJ. A., ReynoldsA. J., and MiedelW. T. (2000). Can early intervention prevent high school dropout: Evidence from the Chicago Longitudinal Study. Urban Education, 35, 31—56.
76.
U.S. Department of Health, Education, and Welfare. (1964). Smoking and health: Report of the advisory committee to the surgeon-general of the Public Health Service (Public Health Service Publication No. 1103). Washington, DC: Author.
77.
U.S. General Accounting Office. (1997). Head Start: Research provides little information on impact of current program (Report no. GAO/HEHS-97-59). Washington, DC: Author.
78.
YoshikawaH. (1995). Long-term effects of early childhood programs on social outcomes and delinquency. Future of Children, 5(3), 51—75.
79.
WeissbergR. P., and GreenbergM. T. (1998). School and community competence-enhancement and prevention programs. In and and DamonW. (Ed.), Handbook of child psychology: Child psychology in practice (Vol. 4, pp. 877—954). New York: Wiley.
80.
ZiglerE. (1994). Reshaping early childhood intervention to be a more effective weapon against poverty. American Journal of Community Psychology, 22, 37—48.
81.
ZiglerE., AbelsonW., TrickettP. K., and SeitzV. (1982). Is an intervention program necessary in order to improve economically disadvantaged children's IQ scores?Child Development, 53, 340—348.
82.
ZiglerE., and ButterfieldE. C. (1968). Motivational aspects of change in IQ test performance of culturally deprived nursery school children. Child Development, 39, 1—14.
83.
ZiglerE., and StyfcoS. (1993). Head Start and beyond: A national plan for extended childhood intervention. New Haven, CT: Yale University Press.
84.
ZiglerE., PiotrkowskiC. S., and CollinsR. (1994). Health services in Head Start. Annual Review of Public Health, 15, 511—534.
85.
ZiglerE., TaussigC., and BlackK. (1992). Early childhood intervention: A promising preventive for juvenile delinquency. American Psychologist, 47, 997—1006.
