Implications of What Works Clearinghouse Guidelines on Single-Case Design: An Investigation of Empty Training Phases

Abstract

The What Works Clearinghouse quality standards provide guidance regarding studies capable of supporting evidence-based practices. Standards concerning single-case designs have been extensively revised to accommodate new evaluation methods, such as the design comparable effect size. These designs often omit data in which children and other participants receive training. Recently, the What Works Clearinghouse suggested studies with such “empty training phases” do not meet minimum standards of evidence. However, evidence regarding the effect of empty training phases on results is limited. This study used a subset of single-case design data from a recent meta-analysis to simulate studies with empty training phases. We calculated design-comparable effect sizes, evaluated differences between simulated and unsimulated data, and conducted random effects meta-analyses. Effects of simulated studies with artificially designed empty training phases were nearly four times as large as effects of the original, unaltered data. Guidance for intervention researchers follows a description of findings.