Estimates of program effects from a compensatory preschool intervention were investigated
using several contemporary techniques of bias reduction. These included econometric simulta
neous modeling, latent-variable structural modeling, and ordinary least squares regression.
These techniques were applied to longitudinal data from Chicago's Child Parent Center
preschool program, a Head Start-type intervention. Analyses of 806 Black children followed
from kindergarten to Grade 6 indicated that the techniques, by and large, produced similar
estimates of program impact on school achievement test scores. Effect sizes ranged from .43 to
.67 at school entry to .24 to .30 at Grade 6 (7 years postprogram). These findings were robust
across different model specifications and similar to those of randomized studies. The only
differences among methods occurred at Grade 3. Findings suggest that different quasi-experi
mental estimation methods that attempt to control for selection bias can generate similar
estimates of the effects of an educational treatment.
Get full access to this article
View all access options for this article.
References
1.
Anderson, S., A. Auquier, W.W. Hauck, D. Oakes, W. Vandaele, and H.I. Weisberg.1980. Statistical methods for comparative studies: Techniques for bias reductionNew York: John Wiley.
2.
Barnett, W.S.1992. Benefits of compensatory preschool educationJournal of Human Resources27:279-312.
3.
Barnow, B.1987. The impact of CETA programs on earnings: A review of the literatureJournal of Human Resources22:157-93.
4.
Barnow, B.S., G.G. Cain, and A.S. Goldberger.1980. Issues in the analysis of selectivity bias In Evaluation studies review annual , vol. 5, edited by E. W. Stromsdorfer and G. Farkas, 43-59. Beverly Hills, CA: Sage.
5.
Benasich, A.A., J. Brooks-Gunn, and B.C. Clewell.1992. How do mothers benefit from early intervention programs?Journal of Applied Developmental Psychology13:311-62.
6.
Bentler, P.1980. Multivariate analysis with latent variablesAnnual Review of Psychology31:419-56.
7.
Bentler, P., and J.A. Woodward.1978. A Head Start reevaluation: Positive effects are not yet demonstrableEvaluation Quarterly2:493-510.
8.
Berk, R.A.1988. Causal inference for sociological data In The handbook of sociology, edited by N. J. Smelser, 155-72. Newbury Park, CA: Sage.
9.
Campbell, D.T., and A. Erlebacher.1970. How regression artifacts in quasi-experimental evaluations can mistakenly make compensatory education look harmful In Compensatory education: A national debate, vol. 3, edited by J. Helmuth, 185-210. New York: Brunner/ Mazel.
10.
Cicirelli, V.G., and associates.1969. The impact of Head StartAthens: Westinghouse Learning Corporation and Ohio University.
11.
Cohen, J., and P. Cohen.1983. Applied multiple regression/correlation analysis for the behavioral sciences2ded. Hillsdale, NJ: Lawrence Erlbaum.
12.
Consortium for Longitudinal Studies.1983. As the twig is bent: Lasting effects of early intervention . Hillsdale, NJ: Lawrence Erlbaum.
13.
Cook, T.D., and W.R. Shadish.1994. Social experiments: Some developments over the past fifteen yearsAnnual Review of Psychology45:545-80.
—. 1993. Econometric analysis2ded. New York: Macmillan.
16.
Haskins, R.1989. Beyond metaphor: The efficacy of early childhood educationAmerican Psychologist44:274-82.
17.
Hayduk, L.A.1987. LISRELBaltimore: Johns Hopkins University Press.
18.
Heckman, J.J.1979. Sample selection bias as a specification errorEconometrica47:153-61
19.
. Heckman, J.J., and V.J. Hotz.1989. Choosing among alternative nonexperimental methods for estimating the impact of social programs: The case of manpower trainingJournal of the American Statistical Association84:862-74
20.
. Heckman, J.J., V.J. Hotz, and M. Dabos.1987. Do we need experimental data to evaluate the impact of manpower training on earnings?Evaluation Review11:395-427.
21.
Hedges, L.V., and I. Olkin.1985. Statistical methods for meta-analysisOrlando, FL: Academic Press.
22.
Hieronymus, A.N., and H.D. Hoover.1990. Iowa Tests of Basic Skills: Manual for school administrators. SupplementChicago: Riverside.
23.
Jöreskog, K.G.1978. Structural analyses of covariance and correlation matricesPsychometrica43:443-77.
24.
Jöreskog, K.G., and D. Sörbom.1993. LISREL 8: The SIMPLIS command languageHillsdale, NJ: Lawrence Erlbaum.
25.
LaLonde, R.1986. Evaluating the econometric evaluations of training programs with experimental dataAmerican Economic Review76:604-20
26.
. Lee, L.F.1983. Generalized econometric models with selectivityEconometrica51:507-12.
27.
Lewis-Beck, M.S.1980. Applied regression: An introductionBeverly Hills, CA: Sage.
28.
Lipsey, M.W., and D.B. Wilson.1993. The efficacy of psychological, educational, and behavioral treatmentAmerican Psychologist48:1181-209.
29.
Maddala, G.S.1983. Limited dependent and qualitative variables in econometricsCambridge, UK: Cambridge University Press.
30.
Magidson, J., and D. Sörbom.1982. Adjusting for confounding factors in quasi-experiments: Another reanalysis of the Westinghouse Head Start evaluationEducational Evaluation and Policy Analysis4:321-9.
31.
Manski, C.F.1975. Maximum score estimation of the stochastic utility modelJournal of Econometrics3:205-28.
32.
—. 1993. Identification problems in the social sciences In Sociological Methodology, vol. 23, edited by P. Marsden, 1-56. Oxford: Blackwell.
33.
Manski, C.F., S.S. McLanahan, D. Powers, and G.D. Sandefur.1992. Alternative estimates of the effects of family structure during adolescence on high school graduationJournal of the American Statistical Association54:25-37.
34.
McKey, R.H., L. Condelli, H. Ganson, B.J. Barrett, C. McConkey, and M.C. Plantz.1985. The impact of Head Start on children, families, and communities. DHHS Publication No. OHDS 85-31193Washington, DC: U.S. Government Printing Office.
35.
Norusis, M.J., and Spss.1990 . SPSS/PC+ 4.0 base manualChicago: SPSS.
36.
Reynolds, A.J.1994. Effects of preschool plus follow-on intervention for children at riskDevelopmental Psychology30:787-804.
37.
—. 1995. One year of preschool intervention or two: Does it matter?Early Childhood Research Quarterly10:1-33.
38.
Reynolds, A.J., and N. Bezruczko.1993. Early schooling of children at risk through fourth gradeMerrill-Palmer Quarterly39:457-80.
39.
Reynolds, A.J., N. Bezruczko, N.A. Marvrogenes, and M. Hagemann.1994. User's guide: Longitudinal Study of Children at Risk Version 3. Chicago: Chicago Public Schools, Department of Research, Evaluation and Planning.
40.
Rindskoph, D.1992. The importance of theory in selection modeling: Incorrect assumptions means biased results In Using theory to improve program and policy evaluations, edited by H. Chen and P. H. Rossi, 179-91. New York: Greenwood.
41.
Rosenbaum, P.R., and D. B. Rubin.1983. The central role of the propensity score in observational studies for causal effectsBiometrica70:41-55.
42.
Rosenthal, R.1991. Meta-analytic procedures for social research. Rev. edNewbury Park, CA: Sage.
43.
Rossi, P.H., and W. Freeman.1989. Evaluation: A systematic approach3d ed. Newbury Park, CA: Sage.
44.
Schweinhart, L.J., and D.P. Weikart.1988. The High/Scope Perry Preschool Program In 14 ounces of prevention: A casebookfor practitioners , edited by R. H. Price, E. L. Cowen, R. P. Lorion, and J. Ramos-McKay, 53-65. Washington, DC: American Psychological Association.
45.
Sörbom, D.1978. An alternative to the methodology for factor analysisPsychometrica43:381-96.
46.
Wainer, H.1989. Eelworms, bullet holes, and Geraldine Ferraro: Some problems with statistical adjustment and some solutionsJournal of Educational Statistics14:121-40.
47.
White, K.R.1985. Efficacy of early interventionJournal of Special Education19:401-16.
48.
Winship, C., and R.D. Mare.1992. Models of sample selection biasAnnual Review of Sociology18:327-50.
49.
Zigler, E., and S. Muenchow.1992. Head Start: The inside story of America's most successful educational experimentNew York: Basic Books.
