The Bayesian way of accounting for the effects of error in the ability and item parameters in adaptive testing is through the joint posterior distribution of all parameters. An optimized Markov chain Monte Carlo algorithm for adaptive testing is presented, which samples this distribution in real time to score the examinee’s ability and optimally select the items. Thanks to extremely rapid convergence of the Markov chain and simple posterior calculations, the algorithm is ready for use in real-world adaptive testing with running times fully comparable with algorithms that fix all parameters at point estimates during testing.
ChangH.-H.YingZ. (2009). Nonlinear sequential design for logistic item response models with applications to computerized adaptive tests. The Annals of Statistics, 37, 1466–1488.
2.
ChengY.PattonJ. M.ShaoC. (2015). α-stratified computerized adaptive testing in the presence of calibration error. Educational and Psychological Measurement, 75, 260–283.
3.
GelmanA.CarlinJ. B.SternH.DunsonD. B.VehtariA.RubinD. B. (2014) Bayesian data analysis (3rd ed.). Boca Raton, FL: Chapman & Hall/CRC.
4.
GelmanA.RubinD. B. (1992). Inference from iterative simulation using multiple sequences. Statistical Science, 7, 457–511.
5.
GilksW. R.RichardsonS.SpiegelhalterD. J. (1996). Introducing Markov chain Monte Carlo. In GilksW. R.RichardsonS.SpiegelhalterD. J. (Eds.), Markov chain Monte Carlo in practice (pp. 1–19). London, England: Chapman & Hall.
6.
HambletonR. K.JonesR. W. (1994). Item parameter estimation errors and their impact on test information functions. Applied Measurement in Education, 7, 171–186.
7.
HambletonR. K.JonesR. W.RogersH. J. (1993). Influence of parameter estimation errors in test development. Journal of Educational Measurement, 30, 143–155.
OwenR. J. (1969). A Bayesian approach to tailored testing(Research bulletin 69-92). Princeton, NJ: Educational Testing Service.
10.
OwenR. J. (1975). A Bayesian sequential procedure for quantal response in the context of adaptive mental testing. Journal of the American Statistical Association, 70, 351–356.
11.
PattonJ.ChengY.YuanK. -H.DiaoQ. (2013). The influence of item calibration error on variable-length computerized adaptive testing. Applied Psychological Measurement, 37, 24–40.
StockingM. L. (1994). Three practical issues for modern adaptive testing item pools(Research report 94-5). Princeton, NJ: Educational Testing Service.
14.
TsutakawaR. K.JohnsonJ. C. (1990). The effect of the uncertainty of item parameter estimation on ability estimates. Psychometrika, 55, 371–390.
15.
TsutakawaR. K.SoltysM. J. (1988) Approximation for Bayesian ability estimation. Journal of Educational and Behavioral Statistics, 13, 117–130.
16.
van der LindenW. J. (2018). Adaptive testing. In van der LindenW. J. (Ed.), Handbook of item response theory. Volume 3: Applications (pp. 197–227). Boca Raton, FL: Chapman & Hall/CRC.
17.
van der LindenW. J.GlasC. A. W. (2000). Capitalization on item calibration error in adaptive testing. Applied Measurement in Education, 12, 35–53.
18.
van der LindenW. J.GlasC. A. W. (2001) Cross-validating item parameter estimation in computerize adaptive testing. In BoomsmaA.van DuijnM. A. J.SnijdersT. A. M. (Eds.), Essays on item response theory (pp. 205–219). New York, NY: Springer.
19.
van der LindenW. J.PashleyP. J. (2010). Item selection and ability estimation adaptive testing. In van der LindenW. J.GlasC. A. W. (Eds.), Elements of adaptive testing (pp. 3–30). New York, NY: Springer.
20.
van der LindenW. J.RenH. (2015). Optimal Bayesian adaptive design for test-item calibration. Psychometrika, 80, 263–288.
