Methods for detecting item parameter drift may be inadequate when every exposed item is at risk for drift. To address this scenario, a strategy for detecting item parameter drift is proposed that uses only unexposed items deployed in a stratified random method within an experimental design. The proposed method is illustrated by investigating unexpected score increases on a high-stakes licensure exam. Results for this example were suggestive of item parameter drift but not significant at the .05 level.
BaldwinP.ClauserB. E. (2022). Historical perspectives on score comparability issues raised by innovations in testing. Journal of Educational Measurement, 59(2), 140–160. https://doi.org/10.1111/jedm.12318
DonoghueJ. R.IshamS. P. (1998). A comparison of procedures to detect item parameter drift. Applied Psychological Measurement, 22(1), 33–51. https://doi.org/10.1177/01466216980221002
4.
ErnstM. D. (2004). Permutation methods: A basis for exact inference. Statistical Science, 19, 676–685.
5.
GoldsteinH. (1983). Measuring changes in educational attainment over time: Problems and possibilities. Journal of Educational Measurement, 20(4), 369–377. https://doi.org/10.1111/j.1745-3984.1983.tb00214.x
6.
HambletonR. K.SwaminathanH. (1985). Item response theory: Principles and applications. Kluwer-Nijhoff.
7.
HambletonR. K.SwaminathanH.RogersH. J. (1991). Fundamentals of item response theory. Sage Publications.
8.
KelleyP. R.SchumacherC. F. (1984). The Rasch model: Its use by the national board of medical examiners. Evaluation & the Health Professions, 7(4), 443–454. https://doi.org/10.1177/016327878400700405
9.
KolenM. J.BrennanR. L. (2004). Test equating, scaling, and linking: Methods and practices (2nd ed.): Springer.
10.
LordF. M. (1980). Applications of item response theory to practical testing problems: Erlbaum.
RajuN. (1990). Determining the significance of estimated signed and unsigned areas between two item response functions. Applied Psychological Measurement, 14(2), 197–207. https://doi.org/10.1177/014662169001400208
14.
RuppA. A.ZumboB. D. (2006). Understanding parameter invariance in unidimensional IRT models. Educational and Psychological Measurement, 66(1), 63–84. https://doi.org/10.1177/0013164404273942
