Randomized Experiments in Educational Policy Research: A Critical Examination of the Reasons the Educational Evaluation Community has Offered for not Doing Them

Abstract

This article notes the paucity with which reform efforts in education have been evaluated experimentally, despite well-nigh universal acknowledgement that experiments provide the best justification for causal conclusions. And of the few experiments that have been completed, most were done by groups outside of the community of educational evaluators working in schools and colleges of education. The reasons educational evaluators cite for not doing experiments are critically appraised. Some are shown to be wrong or over-generalized; and others point to the need for future experiments that depend more heavily on program theory and the assessment of implementation and other intervening processes. The alternatives to experiments that educational evaluators prefer are briefly appraised and found wanting whenever a high standard is needed for justifying causal conclusions.