Common Problems of Design and Analysis in Evaluative Research

Abstract

The national evaluation of an early elementary compensatory education program, Follow Through, is used as the context for discussion of design and analysis problems common to many evaluations. Although experiments have well-known advantages over quasi-experiments, the latter are likely to persist for the foreseeable future. It is argued that more careful planning could avoid many unnecessary weaknesses of past quasi-experiments, and that quasi-experiments can provide valuable information for evaluations. The strengths and weaknesses of several competing analysis strategies for quasi-experiments are considered. The conclusion is that no one strategy is satisfactory. A multiple analysis approach is recommended instead which attempts to capitalize on the unique strengths and control the unique weaknesses of several analysis strategies.