Why have Educational Evaluators Chosen Not to Do Randomized Experiments?

Abstract

This article analyzes the reasons that have been adduced within the community of educational evaluators for not doing randomized experiments. The objections vary in cogency. Those that have most substance are not insurmountable, however, and strategies are mentioned for dealing with them. However, the objections are serious enough, and the remedies partial enough, that it seems hardly warranted to call experiments the “gold standard” of causal inference. Yet even if they are not perfect in research practice, this article shows how they are logically and empirically superior to all currently known alternatives. The article particularly addresses the objection that school personnel will not accept experiments. It shows that hundreds of them have been done there by researchers with backgrounds in psychology and public health who study the prevention of unhealthy behaviors. But experiments are much rarer among researchers trained in education who study changing academic performance. Reasons are adduced for this difference in academic culture within school-based research.