Expected Equating Error Resulting From Incorrect Handling of Item Parameter Drift Among the Common Items

Abstract

Incorrect handling of item parameter drift during the equating process can result in equating error. If the item parameter drift is due to construct-irrelevant factors, then inclusion of these items in the estimation of the equating constants can be expected to result in equating error. On the other hand, if the item parameter drift is related to the construct being measured, then removal of those items from the estimation of the equating constants can be expected to result in equating error. The effect of incorrect handling of item parameter drift on equating has been examined via simulation studies and other methods by a number of authors with mixed results. In this article, this effect is assessed by estimating the expected equating error under the assumptions that the three-parameter item response theory model is used for calibration—equating.

Keywords

equating item response theory common-item nonequivalent groups design item parameter drift

Get full access to this article

View all access options for this article.

References

Bock, R. , Muraki, E. , & Pfeiffenberger, W. (1988). Item pool maintenance in the presence of item parameter drift. Journal of Educational Measurement, 25, 275-285.

Clauser, B. , & Mazor, K. (1998). Using statistical procedures to identify differentially functioning test items. Educational Measurement: Issues and Practice, 17(1), 31-44.

Cook, L. , & Eignor, D. (1991). IRT equating methods. Instructional Topics in Educational Measurement Series (ITEMS), Module 10, 191-199.

Goldstein, H. (1983). Measuring changes in education attainment over time: Problems and possibilities. Journal of Educational Measurement , 20, 369-377.

Haebara, T. (1980). Equating logistic ability scales by a weighted least squares method. Japanese Psychological Research, 22, 144-149.

Holland, P. , & Thayer, D. (1988). Differential item performance and the Mantel-Haenszel procedure. In H. Wainer & H. Braun (Eds.), Test validity (pp. 129-145). Hillsdale, NJ: Lawrence Erlbaum.

Huang, C. , & Shyu, C. (2003, April). The impact of item parameter drift on equating . Paper presented at the annual meeting of the National Council on Measurement in Education, Chicago.

Kolen, M. , & Brennan, R. (1995). Test equating methods and practices. New York: Springer-Verlag.

Lord, F. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.

10.

Meyers, J. , Miller, G.E. , & Way, W.D. (2006, April). Item position and item difficulty change in an IRT-based common item equating design. Paper presented at the annual meeting of the American Educational Research Association , San Francisco.

11.

Rupp, A. , & Zumbo, B. (2003a, April). Bias coefficients for lack of invariance in unidimensional IRT models. Paper presented at the annual meeting of the National Council on Measurement in Education, Chicago.

12.

Rupp, A. , & Zumbo, B. (2003b). Which model is best? Robustness properties to justify model choice among unidimensional IRT models under item parameter drift. Alberta Journal of Educational Research, 49, 264-276.

13.

Shealy, R. , & Stout, W. (1993). A model-based standardization approach that separates true bias/DIF from group differences and detects test bias/DTF as well as item bias/DIF. Psychometrika, 58, 159-194.

14.

Stahl, J.A. , Bergstrom, B. , & Shneyderman, O. (2002, April). Impact of item drift on test-taker measurement. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.

15.

Stocking, M. , & Lord, F. (1983). Developing a common metric in item response theory . Applied Psychological Measurement, 7, 201-210.

16.

Wells, C. , Subkoviak, M. , & Serlin, R. (2002). The effect of item parameter drift on examinee ability estimates. Applied Psychological Measurement, 26(1), 77-87.

17.

Wollack, J. , Sung, H. , & Kang, T. (2005, April). Longitudinal effects of item parameter drift. Paper presented at the annual meeting of the National Council on Measurement in Education, Montreal .