Using “Short” Interrupted Time-Series Analysis To Measure The Impacts Of Whole-School Reforms

Abstract

The present article introduces a new approach for measuring the impacts of whole-school reforms. The approach is based on “short” interrupted time-series analysis, which has been used to evaluate programs in many fields. The approach is used to measure impacts on three facets of student performance: (a) average (mean) test scores, which summarize impacts on total performance; (b) the distribution of scores across specific ranges, which helps to identify where in the distribution of student performance impacts were experienced; and (c) the variation (standard deviation) of scores, which indicates how the disparity in student performance was affected. To help researchers use the approach, the article lays out its conceptual rationale, describes its statistical procedures, explains how to interpret its findings, indicates its strengths and limitations, and illustrates how it was used to evaluate a major whole-school reform—Accelerated Schools.