The recent surge in computerized testing brings challenges in the analysis of testing data with classic item response theory (IRT) models. To handle individually varying and irregularly spaced longitudinal dichotomous responses, we adopt a dynamic IRT model framework and then extend the model to link with individual characteristics at a hierarchical level. Further, we have developed an algorithm to select important characteristics of individuals that can capture the growth changes of one’s ability under this multi-level dynamic IRT model, where we can compute the Bayes factor of the proposed model including different covariates using a single Markov chain Monte Carlo output from the full model. In addition, we have shown the model selection consistency under the modified Zellner–Siow prior, and we have conducted simulations to illustrate the properties of the model selection consistency in finite samples. Finally, we have applied our proposed model and computational algorithms to a real data application, called EdSphere dataset, in educational testing.
AlbertJ. H.ChibS. (1993). Bayesian analysis of binary and polychotomous response data. Journal of the American statistical Association, 88(422), 669–679.
2.
BergerJ. O.PericchiL. R. (1996). The intrinsic Bayes factor for model selection and prediction. Journal of the American Statistical Association, 91(433), 109–122.
3.
BirnbaumA. L. (1968). Some latent trait models and their use in inferring an examinee’s ability. In LordF. M.NovickM. R. (Eds.), Statistical theories of mental test scores (pp. 397–479). Addison-Wesley, Reading.
4.
BockR. D.MislevyR. J. (1989). A hierarchical item response model for educational testing. In BockR. Darrell (Ed.), Multilevel analysis of educational data (pp. 57–74). Academic Press, San Diego.
5.
CastellanosM. E.Garcia-DonatoG.CabrasS. (2021). A model selection approach for variable selection with censored data. Bayesian Analysis, 16(1), 271–300.
6.
ChenM.-H. (2005). Computing marginal likelihoods from a single MCMC output. Statistica Neerlandica, 59(1), 16–29.
7.
ChenY.Silva FilhoT.PrudencioR. B.DietheT.FlachP. (2019). —irt: A new item response model and its applications. In The 22nd international conference on artificial intelligence and statistics, PMLR: Volume 89, Naha, Okinawa, Japan (pp. 1013–1021).
8.
ChevallierA.FearnheadP.SuttonM. (2022). Reversible jump PDMP samplers for variable selection. Journal of the American Statistical Association, 18, 1–31.
9.
ChibS. (1995). Marginal likelihood from the Gibbs output. Journal of the American Statistical Association, 90(432), 1313–1321.
10.
DickeyJ. M.LientzB. (1970). The weighted likelihood ratio, sharp hypotheses about chances, the order of a Markov chain. The Annals of Mathematical Statistics, 41, 214–226.
11.
FernandezC.LeyE.SteelM. F. (2001). Benchmark priors for Bayesian model averaging. Journal of Econometrics, 100(2), 381–427.
12.
FoxJ.-P. (2005). Multilevel IRT using dichotomous and polytomous response data. British Journal of Mathematical and Statistical Psychology, 58(1), 145–172.
13.
GewekeJ. (1992). Evaluating the accuracy of sampling-based approaches to the calculations of posterior moments. Bayesian Statistics, 4, 641–649.
14.
HamzaM. A.HassineS. H.AbunadiI.Al-WesabiF. N.AlsolaiH.HilalA. M.MotwakelA. (2022). Feature selection with optimal stacked sparse autoencoder for data mining. Computers, Materials & Continua, 72(2), 2581–2596.
15.
HeckD. W.BoehmU.Böing-MessingF.BürknerP.-C.DerksK.DienesZ.FuQ.GuX.KarimovaD.KiersH.KlugkistI.KuiperR. M.LeeM. D.LeendersR.LeplaaH. J.LindeM.LyA.MeijerinkM.MoerbeekM.…HoijtinkH. (2022). A review of applications of the Bayes factor in psychological research. Psychological Methods, 28(3), 558–579.
16.
HeidelbergerP.WelchP. D. (1983). Simulation run length control in the presence of an initial transient. Operations Research, 31(6), 1109–1144.
17.
IrwingP.BoothT.HughesD. J. (2018). The Wiley handbook of psychometric testing: A multidisciplinary reference on survey, scale and test development. John Wiley & Sons.
18.
KassR. E.RafteryA. E. (1995). Bayes factors. Journal of the American Statistical Association, 90(430), 773–795.
19.
KoopG.KorobilisD. (2018). Bayesian dynamic variable selection in high dimensions. arXiv preprint arXiv:1809.03031.
20.
LiY.ClydeM. A. (2018). Mixtures of g-priors in generalized linear models. Journal of the American Statistical Association, 113(524), 1828–1845.
21.
LiangF.PauloR.MolinaG.ClydeM. A.BergerJ. O. (2008). Mixtures of g priors for Bayesian variable selection. Journal of the American Statistical Association, 103(481), 410–423.
22.
LiuF.WangX.HancockR.ChenM.-H. (2022). Bayesian model assessment for jointly modeling multidimensional response data with application to computerized testing. Psychometrika, 87(4), 1290–1317.
23.
LiuY.HuG.CaoL.WangX.ChenM.-H. (2019). A comparison of Monte Carlo methods for computing marginal likelihoods of item response theory models. Journal of the Korean Statistical Society, 48(4), 503–512.
24.
LiuY.WangX. (2020). Bayesian nonparametric monotone regression of dynamic latent traits in item response theory models. Journal of Educational and Behavioral Statistics, 45(3), 274–296.
25.
LordF. M. (1980). Applications of item response theory to practical testing problems. Lawrence Erlbaum Associates.
26.
MuttakinF.WangJ.-T.MulyantoM.LeuJ.-S. (2021). Evaluation of feature selection methods on psychosocial education data using additive ratio assessment. Electronics, 11(1), 114.
27.
NewtonM. A.RafteryA. E. (1994). Approximate Bayesian inference by the weighted likelihood Bootstrap. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 56, 3–48.
28.
ParkT.CasellaG. (2008). The Bayesian lasso. Journal of the American Statistical Association, 103(482), 681–686.
29.
PanT.YinY. (2017). Using the Bayes factors to evaluate person fit in the item response theory. Applied Measurement in Education, 30(3), 213–227.
30.
PericchiL. R. (2005). Model selection and hypothesis testing based on objective probabilities and bayes factors. Handbook of Statistics, 25, 115–149.
31.
RahmanL.SetiawanN. A.PermanasariA. E. (2017). Feature selection methods in improving accuracy of classifying students’ academic performance. In 2017 2nd international conferences on information technology, information systems and electrical engineering (ICITISEE), IEEE, Yogyakarta, Indonesia, pp. 267–271.
32.
RaschG. (1960). Studies in mathematical psychology: I. probabilistic models for some intelligence and attainment tests. Danish Institute for Educational Research.
33.
RočkováV.GeorgeE. I. (2018). The spike-and-slab lasso. Journal of the American Statistical Association, 113(521), 431–444.
34.
ShimH.BonifayW.WiedermannW. (2022). Parsimonious asymmetric item response theory modeling with the complementary log-log link. Behavior Research Methods, 55, 200–219.
35.
SmithB. J. (2007). boa: An r package for MCMC output convergence assessment and posterior inference. Journal of Statistical Software, 21(11), 1–37.
36.
SpiegelhalterD. J.BestN. G.CarlinB. P.Van Der LindeA. (2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 64(4), 583–639.
37.
StennerA. (2010). Using technology to merge assessment and instruction. In 2nd international conference for teaching and learning with technology. Available from MetaMetrics, Inc.
38.
SwartzC. W.HanlonS. T.ChildressE. L.StennerA. J. (2016). An approach to design-based implementation research to inform development of edsphere®: A brief history about the evolution of one personalized learning platform. In RosenY.FerraraS.MosharrafM. (Eds.) Handbook of research on technology tools for real-world skill development (pp. 284–318). IGI Global Scientific Publishing, Pennsylvania, PA.
39.
SwartzC. W.HanlonS. T.StennerA. J.BurdickH.BurdickD. S.EmersonC. (2011). Edsphere™: Using technology to enhance literacy through deliberate practice. Metametrics Research Brief.
40.
TijmstraJ.BolsinovaM. (2019). Bayes factors for evaluating latent monotonicity in polytomous item response theory models. Psychometrika, 84(3), 846–869.
41.
TutzG. (2022). Flexible item response models for count data: The count thresholds model. Applied Psychological Measurement, 46, 643–661.
42.
Van der LindenW. J. (2018). Handbook of item response theory: Three volume set. CRC Press.
43.
VerdinelliI.WassermanL. (1995). Computing Bayes factors using a generalization of the savage-dickey density ratio. Journal of the American Statistical Association, 90(430), 614–618.
44.
WangX.BergerJ. O.BurdickD. S. (2013). Bayesian analysis of dynamic item response models in educational testing. The Annals of Applied Statistics, 7(1), 126–153.
45.
WatanabeS. (2010). Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory. Journal of Machine Learning Research, 11, 3571–3594.
46.
WuH.-H.FerreiraM. A.GompperM. E. (2016). Consistency of hyper-g-prior-based bayesian variable selection for generalized linear models. Brazilian Journal of Probability and Statistics, 30(4), 691–709.
47.
XieW.LewisP. O.FanY.KuoL.ChenM.-H. (2011). Improving marginal likelihood estimation for Bayesian phylogenetic model selection. Systematic Biology, 60(2), 150–160.
48.
ZellnerA. (1986). On assessing prior distributions and Bayesian regression analysis with g-prior distributions. In GoelP.Zellner AA. (Eds.), Bayesian inference and decision techniques (pp. 233-243). Elsevier Science Publishers, Inc., New York, NY.
49.
ZellnerA.SiowA. (1980). Posterior odds ratios for selected regression hypotheses. Trabajos de estadística y de investigación operativa, 31(1), 585–603.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
