Modeling Item Position Effects Using Generalized Linear Mixed Models

Abstract

Item position effects can seriously bias analyses in educational measurement, especially when multiple matrix sampling designs are deployed. In such designs, item position effects may easily occur if not explicitly controlled for. Still, in practice it usually turns out to be rather difficult—or even impossible—to completely control for effects due to the position of items. The objectives of this article are to show how item position effects can be modeled using the linear logistic test model with additional error term (LLTM +ε) in the framework of generalized linear mixed models (GLMMs), to explore in a simulation study how well the LLTM +ε holds the nominal Type I risk threshold, to conduct power analysis for this model, and to examine the sensitivity of the LLTM +ε to designs that are not completely balanced concerning item position. Overall, the LLTM +ε proved suitable for modeling item position effects when a balanced design is used. With decreasing balance, the model tends to be more conservative in the sense that true item position effects are more unlikely to be detected. Implications for linking and equating procedures which use common items are discussed.