It is shown that deviations of estimated from true values of item difficulty parameters, caused for example by item calibration errors, the neglect of randomness of item difficulty parameters, testlet effects, or rule-based item generation, can lead to systematic bias in point estimation of person parameters in the context of adaptive testing. This effect occurs even when the errors of the item difficulty parameters are themselves unbiased. Analytical calculations as well as simulation studies are discussed.
HollingH.BertlingJ. P.ZeuchN. (2009). Automatic item generation of probability word problems. Studies in Educational Evaluation, 35, 71-76.
8.
IpE. H. (2010). Interpretation of the three-parameter testlet response model and information function. Applied Psychological Measurement, 34, 467-482.
9.
MagisD.RaîcheG. (2011). catR: An R package for computerized adaptive testing. Applied Psychological Measurement, 35, 576-577.
10.
OsterlindS.EversonH. (2009). Differential item functioning. Newbury Park, CA: Sage.
11.
R Development Core Team. (2009). R: A language and environment for statistical computing. ISBN 3-900051-07-0. Vienna, Austria: R Foundation for Statistical Computing. Available from http://www.R-project.org
12.
RijmenF.De BoeckP. (2002). The random weights linear logistic test model. Applied Psychological Measurement, 26, 271-285.
13.
ScottS.IpE. (2002). Empirical Bayes and item-clustering effects in a latent variable hierarchical model. Journal of the American Statistical Association, 97, 409-419.
14.
SireciS.ThissenD.WainerH. (1991). On the reliability of testlet-based tests. Journal of Educational Measurement, 28, 237-247.
15.
van der LindenW. (2010). Elements of adaptive testing. Springer, NY; Dordrecht Heidelberg, London.
16.
van der LindenW.GlasC. (2000). Capitalization on item calibration error in adaptive testing. Applied Measurement in Education, 13, 35-53.
17.
WainerH.BradlowE. T.WangX. (2007). Testlet response theory and its applications. Berlin, Germany: Cambridge University Press.
18.
WainerH.KielyG. (1987). Item clusters and computerized adaptive testing: A case for testlets. Journal of Educational Measurement, 24, 185-201.
19.
WangW.WilsonM. (2005). Exploring local item dependence using a random-effects facet model. Applied Psychological Measurement, 29, 296-318.
20.
WarmT. A. (1989). Weighted likelihood estimation of ability in item response models. Psychometrika, 54, 427-450.
21.
ZumboB. (2007). Three generations of DIF analyses: Considering where it has been, where it is now, and where it is going. Language Assessment Quarterly, 4, 223-233.
