Estimating Trends From Censored Assessment Data Under No Child Left Behind

Abstract

Under the No Child Left Behind Act, large-scale test score trend analyses are widespread. These analyses often gloss over interesting changes in test score distributions and involve unrealistic assumptions. Further complications arise from analyses of unanchored, censored assessment data, or proportions of students lying within performance levels defined by unspecified cut scores. This article introduces ‘‘shift models,’’ particularly the ‘‘normal-shift’’ model, to summarize the limited information available in censored data and to support distribution-wide trend analyses. A simulation study exploring this model’s estimation procedure—an expectation-maximization algorithm for maximum likelihood estimates (MLEs) of normally distributed censored data—found that the MLEs exhibit little to no bias over a range of sample sizes and cut scores. The normal-shift model was applied to two full state data sets and performed well in recovering effect size estimates in censored scenarios, except when censoring occurred at cut scores that generated particularly unrepresentative trends. The normal-shift model facilitates effect size estimation for unanchored, censored assessment data and, when applied to a large, cross-state data set, revealed significant positive trends from 2003-2005 and 2005-2007 for most states.