Effects of Item Parameter Drift on Vertical Scaling With the Nonequivalent Groups With Anchor Test (NEAT) Design

Abstract

The authors explored the effects of drifting common items on vertical scaling within the higher order framework of item parameter drift (IPD). The results showed that if IPD occurred between a pair of test levels, the scaling performance started to deviate from the ideal state, as indicated by bias of scaling. When there were two items drifting with 0.5 logits, IPD could have a substantial effect on vertical scaling. Although IPD had little impact on the recoveries of the parameters on the whole developmental scale, its effects on the recoveries of some parameters by separate grade and grade pair were manifest. Specifically, the mean achievement estimates became worse conditional on the pair of test levels between which IPD occurred, and the estimations of grade-to-grade growth and effect size were distorted for the grade pair corresponding to the test pair that involved IPD. Neither the estimation of standard deviation of the achievement nor the grade-to-grade variability was influenced by IPD.