Evaluators often utilize ANCOVA-type techniques to assess the effects of innovative
programs implemented in naturalistic settings. In this paper, design, analysis, and
reporting considerations for the application of ANCOVA-type techniques in educational
settings are described. Numerous examples are drawnfrom the national Follow Through
evaluation, and suggestions for improving reports using ANCOVA-type techniques are
presented. The overall perspective is that evaluation reports must be more precise and
must indicate the limitations of the methodology used. In doing so, a more balanced
description of a program and its effects is presented to the decision maker and to other
stakeholders.
Get full access to this article
View all access options for this article.
References
1.
Campbell, D.T. and A.E. Erlebacher (1970) "How regression artifacts in quasi-experimental evaluations can mistakenly make compensatory education look harmful," pp. 185-210 in J. Hellmuth (ed.) The Disadvantaged Child, Vol. 3. New York: Brunner/ Mazel.
2.
Cohen, D.D. and M.S. Garet (1975) "Reforming educational policy with applied research ." Harvard Educational Rev.45, 1: 17-43.
3.
Coleman, J.S., E.Q. Campbell, C.J. Hobson, J. McPartland, A.M. Mood, F.D. Weinfeld, and R.L. York (1966) Equality of Educational Opportunity. Washington, DC: Government Printing Office.
4.
Cooley, W.W. ( 1978) "Explanatory observational studies." Presented at the annual meeting of the American Educational Research Association, Toronto, Canada.
5.
— and G. Leinhardt (1975a) "The application of a modelfor investigating classroom processes." Pittsburgh: University of Pittsburgh, Learning Research and Development Center.
6.
— ( 1975b) "Design for the individualized instruction study: final report." Pittsburgh: University of Pittsburgh, Learning Research and Development Center [NIE Contract 400-75-0071].
7.
Cooley, W.W. and P.R. Lohnes (1976) Evaluation Research in Education. New York: Irvington.
8.
Cox, D.R. (1957) "The use of a concomitant variable in selecting an experimental design." Biometrika44: 150-158.
9.
Egbert, R.L. (1973) "Planned variation in Follow Through." Prepared for the Brookings Panel on Social Experimentation, Washington, DC.
10.
Eichelberger, R.T., J.L. D, and Evaluation Staff (1975) "Evaluation of the LRDC individualized programs in the Pittsburgh Public Schools." Pittsburgh : University of Pittsburgh, Learning Research and Development Center.
11.
Elashoff, J.D. (1969) "Analysis of covariance: A delicate instrument ." Amer. Educational Research J.6, 3: 383-401.
12.
Emrick, J.A., P.H. Sorensen, and M.S. Stearns (1973) "Interim evaluation of the National Follow Through Program 1969-1971: a technical report." Menlo Park, CA : StanfordResearch Institute [USOE Contract OEC-0-8-522480-4633(100)].
13.
Federal Register (1975) "Follow Through Program." (April 21): 40, 77, Part II. Washington, DC: Government Printing Office.
14.
Glass, G.V., P.D. Peckham, and J.R. Sanders (1972) "Consequences of failure to meet assumptions underlying the fixed effects analysis of variance and covariance." Rev. of Educational Research42, 3: 237-288.
15.
Haney, W. (1977) "The Follow Through planned variation experiment ." Volume V: A technical history of the National Follow Through evaluation. Cambridge, MA : Huron Institute [HEW Contract OEC-0-74-0394].
16.
Johnson, Lyndon B. (1967) "Special message to the Congress recommending a 12-point program for America's children and youth." Public Papers of the Presidents (February): 150-160.
17.
Lord, F.M. (1962) "Elementary models for measuring change," pp. 21-38 in C. Harris (ed.) Problems in Measuring Change. Madison: Univ. of Wisconsin Press.
18.
Sorensen, P.H. and W.G. Madow (1969) "A proposal for research: longitudinal evaluation of the National Follow Through Program, 1969-70." Menlo Park, CA: Stanford Research Institute.
19.
Stallings, J.A. (1973) "Follow Through Program classroom observation evaluation 1971-72." Menlo Park, CA: Stanford Research Institute.
20.
Stebbins, L. B. [ed.] (1976) " Education as experimentation: a planned variation model, Vol. 3." Cambridge, MA: Abt Associates Inc., [USOE Contract 300-75-0134].
21.
Stufflebeam, D.G., W.J. Foley, W.J. Gephart, E.G. Guba, R.L. Ham-Mond, H.O. Merriman, and M.M. Provus (1971) Educational Evaluation and Decision Making. Bloomington, IN: Phi Delta Kappa.
22.
Tukey, J. (1954) "Unsolved problems of experimental statistics." J. of the American Statistical Association49: 706-731.
23.
Wold, H. (1956) "Causal inference from observational data: a review of ends and means." J. of the Royal Statistical Society, Series A, 119: 351-390.
24.
Wolff, M. and A. Stein (1966) " Six months later: a comparison of children who had Head Start, Summer 1965, with their classmates in kindergarten." New York: Ferkhauf Graduate School of Education, Yeshiva University.
