Practices in educational program evaluation were examined through analyses of reports
submitted to the U.S. Department of Education's Joint Dissemination Review Panel
(JDRP) during the years 1980-1983. Through its review process, the JDRPjudges whether
educational programs have demonstrated effectiveness in attaining stated goals. Content
analyses of 232 JDRP reports yielded descriptive profiles of the educational programs and
their evaluations. Comparisons between JDRP-approved and nonapproved programs
were made. Regression analyses were used to relate differences in evaluation methodology
to differences in the size of educational effects detected by the programs.
Get full access to this article
View all access options for this article.
References
1.
Ball, S. (1981) "Outcomes, the size of the impacts, and program evaluation." New Directions for Program Evaluation9: 71-86.
2.
Bernstein, I.N. and H.E. Freeman (1975) Academic and Entrepreneurial Research. New York: Russell Sage.
3.
Boruch, R.F. and D.S. Cordray (1980) An Appraisal of Educational Program Evaluations: Federal, State, and Local Agencies. Contract No. 300-79-0467. Evanston, IL: Northwestern University.
4.
Boruch, R.F. and M.W. McLaughlin (1982) "Boruch and McLaughlin debate mandated experimental methods." Evaluation News3 (1): 11-20.
5.
Bryant, F.D. (1982) "Desegregation and black student achievement: improving meta-analysis of quasi-experiments." Presented at the Joint Annual Evaluation Research Society/ Evaluation Network Conference , Baltimore, MD, October.
6.
Campbell, D.T. and R. Boruch (1975) "Making the case for randomized assignment to treatments by considering the alternatives: six ways in which quasi-experimental evaluations in compensatory education tend to underestimate effects," pp. 195-296 in C. A. Bennett and A. A. Lumsdaine (eds.) Evaluation and Experiment. New York: Academic Press.
7.
Campbell, D.T. and J.C. Stanley (1963) Experimental and Quasi-Experimental Designs for Research. Chicago: Rand McNally .
8.
Cook, T.D. and D.T. Campbell (1979) Quasi-Experimentation: Design and Analysis Issues for Field Settings. Chicago: Rand McNally.
9.
Cook, T.D. and C.L. Gruder (1978) "Metaevaluation research." Evaluation Q.2: 5-51.
10.
Datta, L. (1977) "There's more to effectiveness than achievement: and some other answers about the Joint Dissemination Review Panel." AERA/SIG Educational Research and Development Evaluators Newsletter.
11.
Fang, W.L. (1981) "The Joint Dissemination Review Panel: can approved submittals be distinguished from rejected ones on the basis of presented evidence of effectiveness related to cognitive objectives?" Doctoral dissertation, University of Virginia, Charlottesville.
12.
Gilbert, J.P., R.J. Light, and F. Mosteller (1975) "Assessing social innovations: an empirical base for policy," pp. 39-193 in C. A. Bennett and A. A. Lumsdaine (eds.) Evaluation and experiment. New York: Academic Press.
13.
Glass, G.V. (1976) "Primary, secondary and meta-analysis of research ." Educ. Researcher5 (10): 3-8.
14.
Glass, G.V. (1977) "Integrating findings: the meta-analysis of research ." Rev. of Research in Education5: 351-379.
15.
Glass, G.V. (1980) Summarizing effect sizes. New Directions for Methodology of Social and Behavioral Sciences5: 13-31.
16.
Glass, G.V., B. McGaw, and M.L. Smith (1981) Meta-analysis in social research. Beverly Hills, CA: Sage.
17.
Gordon, G. and E.V. Morse (1975) "Evaluation research." Annual Rev. of Sociology1: 339-361.
18.
Hall, J.A. (1980) "Gender differences in nonverbal communication skills." New Directions for Methodology of Social and Behavioral Sciences5: 63-77.
19.
Hamilton, J.A. and A.M. Mitchell (1979) "Identifying and approving career education activities for national dissemination." Vocational Guidance Q.28: 71-81.
20.
Haney, W. (1978) "The Follow Through planned variation experiment: statistical conclusion validity." Presented at the Annual Meeting of the American Educational Research Association, Toronto, Canada, March.
21.
Hirschi, T. and H.C. Selvin (1967) Delinquency research: an appraisal of analytic methods. New York: Free Press.
22.
Kennedy, M.M. (1981) "The role of experiments in improving education ," pp. 67-77 in C. B. Aslanian (ed.) Improving Educational Evaluation Methods: Impact on Policy. Beverly Hills, CA: Sage.
23.
Lipsey, M.L., S. Crosse, J. Dunkle, J. Pollard, and G. Stobart (1985) "Evaluation: the state of the art and the sorry state of the science."New Directions for Program Evaluation27: 7-28.
24.
Lipsey, M.W. (1983) "The wrong stuff: a methodological review of published evaluation studies." Presented at the Evaluation Research Society Annual Meeting, Chicago, October.
25.
McGaw, B. and G.V. Glass (1980) "Choice of the metric for effect size in meta-analysis ." Amer. Educ. Research J. 17: 325-337.
26.
McTavish D. G., E.E. Brent, Jr., J.D. Cleary, and K.R. Knudsen (1975) The systematic assessment and prediction of research methodology, Vol. 1: Advisory report. Final report on Grant OEO 005-P-20-2-74 . Minneapolis, MN: Minnesota Systems Research.
27.
Raizen, S. A. and P. H. Rossi [Eds.] (1981) Program Evaluation in Education: When? How? To What Ends? Washington, DC: National Academy Press.
28.
Rossi, P.H. (1979) "Past, present, and future prospects of evaluation research," pp. 17-33 in L. Datta and R. Perloff (eds.) Improving Evaluations. Beverly Hills, CA: Sage.
29.
Sanders, J.R. and D.H. Nafziger (1976) A Basis for Determining the Adequacy of Evaluation Designs. Occasional Paper 6. Kalamazoo: Western Michigan University Evaluation Center, College of Education.
30.
Sechrest, L. and W.H. Yeaton (1981) "Assessing the effectiveness of social programs: methodological and conceptual issues." New Directions for Program Evaluation9: 41-56.
31.
Sechrest, L. and W.H. Yeaton (1982) "Magnitudes of experimental effects in social science research." Evaluation Rev.6: 579-600.
32.
Smith, N.L. (1979) "Requirements for a discipline of evaluation." Studies in Educ. Evaluation5: 5-12.
33.
Tallmadge, G.K. (1977) Ideabook: The Joint Dissemination Review Panel . Washington, DC: U. S. Office of Education.
34.
Tallmadge, G.K. and C.T. Wood (1978) User's guide: ESEA Title I Evaluation and Reporting System (rev. ed.) Mountain View, CA: RMC Research Corporation.
35.
The Network (1978) Program Validation: Four Case Studies. Andover, MA: Author.
