Overcoming the Volatility in School-Level Gain Scores: A New Approach to Identifying Value Added With Cross-Sectional Data

Abstract

Traditionally, state accountability systems have measured school-level achievement gains using cross-sectional data, for example, by comparing scores of one year’s eighth graders to scores of the next year’s eighth graders. This approach produces extremely volatile estimates of value added from year to year. This volatility suggests that the traditional use of cross-sectional data cannot reliably estimate the production of achievement by schools, and therefore schools may be unfairly sanctioned under such a system. In this analysis, the authors consider an alternative use of cross-sectional data, identifying differences in relative subject matter performance within schools. They illustrate this approach using data on public middle schools in Wisconsin during the years 1998 to 2001. Compared to school-level gain scores, relative subject matter performance is much more stable from year to year. The authors conclude that in lieu of more reliable measures of value added, state educational agencies should consider alternative uses of standardized test data.