In this article, we propose a variational Bayes (VB) inference method for the deterministic input noisy AND gate model of cognitive diagnostic assessment. The proposed method, which applies the iterative algorithm for optimization, is derived based on the optimal variational posteriors of the model parameters. The proposed VB inference enables much faster computation than the existing Markov chain Monte Carlo (MCMC) method, while still offering the benefits of a full Bayesian framework. A simulation study revealed that the proposed VB estimation adequately recovered the parameter values. Moreover, an example using real data revealed that the proposed VB inference method provided similar estimates to MCMC estimation with much faster computation.
BealM. J. (2003). Variational algorithms for approximate Bayesian inference [Unpublished doctoral dissertation]. University College London, London. https://www.cse.buffalo.edu/faculty/mbeal/thesis/
BleiD. M.KucukelbirA.McAuliffeJ. D. (2017). Variational inference: A review for statisticians. Journal of the American Statistical Association, 112, 859–877. https://doi.org/10.1080/01621459.2017.1285773
4.
ChenJ.de la TorreJ. (2014). A procedure for diagnostically modeling extant large-scale assessment data: The case of the Programme for International Student Assessment in Reading. Psychology, 5, 1967–1978. https://doi.org/10.4236/psych.2014.518200
ChenY.CulpepperS. A.WangS.DouglasJ. (2018). A hidden Markov model for learning trajectories in cognitive diagnosis with application to spatial rotation skills. Applied Psychological Measurement, 42, 5–23. https://doi.org/10.1177/0146621617721250
7.
ChiuC. Y.DouglasJ. (2013). A nonparametric approach to cognitive diagnosis by proximity to ideal response patterns. Journal of Classification, 30, 225–250. https://doi.org/10.1007/s00357-013-9132-9
8.
CulpepperS. A. (2015). Bayesian estimation of the DINA model with Gibbs sampling. Journal of Educational and Behavioral Statistics, 40, 454–476. https://doi.org/10.3102/1076998615595403
9.
CulpepperS. A. (2019). Estimating the cognitive diagnosis Q matrix with expert knowledge: Application to the fraction-subtraction dataset. Psychometrika, 84, 333–357. https://doi.org/10.1007/s11336-018-9643-8
CulpepperS. A.ChenY. (2018). Development and application of an exploratory reduced reparameterized unified model. Educational and Behavioral Statistics. https://doi.org/10.3102/1076998618791306
12.
de la TorreJ. (2009). DINA model and parameter estimation: A didactic. Journal of Educational and Behavioral Statistics, 34, 115–130. https://doi.org/10.3102/1076998607309474
DempsterA.LairdN.RubinD. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. Series B (Methodological), 39, 1–38.
16.
GelmanA.RubinD. B. (1992). Inference from iterative simulation using multiple sequences. Statistical Science, 7, 457–472. https://doi.org/10.1214/ss/1177011136
17.
GelmanA.ShaliziC. R. (2013). Philosophy and the practice of Bayesian statistics. British Journal of Mathematical and Statistical Psychology, 66, 8–38. https://doi.org/10.1111/j.2044-8317.2011.02037.x
18.
GrimmerJ. (2011). An introduction to Bayesian inference via variational approximations. Political Analysis, 19, 32–47. https://doi.org/10.1093/pan/mpq027
19.
HaertelE. H. (1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26, 301–321. https://doi.org/10.1111/j.1745-3984.1989.tb00336.x
20.
HensonR. A.TemplinJ. L.WillseJ. T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191–210. https://doi.org/10.1007/S11336-008
JangE. E. (2009). Demystifying a Q-matrix for making diagnostic inferences about L2 reading skills. Language Assessment Quarterly, 6, 210–238. https://doi.org/10.1080/15434300903071817
JeonM.RijmenF.Rabe-HeskethS. (2017). A variational maximization–maximization algorithm for generalized linear mixed models with crossed random effects. Psychometrika, 82, 693–716. https://doi.org/10.1007/s11336-017-9555-z
25.
JunkerB. W.SijtsmaK. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258–272. https://doi.org/10.1177/01466210122032064
26.
KayaY.LeiteW. L. (2017). Assessing change in latent skills across time with longitudinal cognitive diagnosis modeling: An evaluation of model performance. Educational and Psychological Measurement, 77, 369–388. https://doi.org/10.1177/0013164416659314
27.
KimA. A. (2015). Exploring ways to provide diagnostic feedback with an ESL placement test: Cognitive diagnostic assessment of L2 reading ability. Language Testing, 32, 227–258. https://doi.org/10.1177/0265532214558457
28.
KöhnH. F.ChiuC. Y. (2016). A proof of the duality of the DINA model and the DINO model. Journal of Classification, 33, 171–184. https://doi.org/10.1007/s00357-016-9202-x
LeeY.-W.SawakiY. (2009). Cognitive diagnosis approaches to language assessment: An overview. Language Assessment Quarterly, 6, 172–189. https://doi.org/10.1080/15434300902985108
31.
LeightonJ. P.GierlM. J. (Eds.). (2007). Cognitive diagnostic assessment for education: Theory and applications. Cambridge University Press.
32.
LiF.CohenA.BottgeB.TemplinJ. (2016). A latent transition analysis model for assessing change in cognitive skills. Educational and Psychological Measurement, 76, 181–204. https://doi.org/10.1177/0013164415588946
33.
LiH. (2011). A cognitive diagnostic analysis of the MELAB reading test. Spaan Fellow Working Papers in Second or Foreign Language Assessment, 9, 17–46.
NatesanP.NandakumarR.MinkaT.RubrightJ. D. (2016). Bayesian prior choice in IRT estimation using MCMC and variational Bayes. Frontiers in Psychology, 7, 1–11. https://doi.org/10.3389/fpsyg.2016.01422
PhilippM.StroblC.de la TorreJ.ZeileisA. (2018). On the estimation of standard errors in cognitive diagnosis models. Journal of Educational and Behavioral Statistics, 43, 88–115. https://doi.org/10.3102/1076998617719728
42.
PlummerM.BestN.CowlesK.VinesK. (2006). CODA: Convergence diagnosis and output analysis for MCMC. R News, 6, 7–11.
43.
R Development Core Team. (2018). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.r-project.org/
44.
RijmenF.JeonM. (2013). Fitting an item response theory model with random item effects across groups by a variational approximation method. Annals of Operations Research, 206, 647–662. https://doi.org/10.1007/s10479-012-1181-7
45.
RijmenF.JeonM.Rabe-HeskethS. (2016). Variational approximation methods. In van der LindenW. J. (Ed.), Handbook of item response theory: Statistical tools (Vol. 2, pp. 259–270). CRC Press.
46.
RuppA. A.TemplinJ. L. (2008). Unique characteristics of diagnostic classification models: A comprehensive review of the current state-of-the-art. Measurement: Interdisciplinary Research & Perspective, 6, 219–262. https://doi.org/10.1080/15366360802490866
47.
RuppA. A.TemplinJ. L.HensonR. A. (2010). Diagnostic measurement: Theory, methods and applications. Guilford Press.
48.
SorrelM. A.OleaJ.AbadF. J.de la TorreJ.AguadoD.LievensF. (2016). Validity and reliability of situational judgement test scores: A new approach based on cognitive diagnosis models. Organizational Research Methods, 19, 506–532. https://doi.org/10.1177/1094428116630065
49.
TatsuokaC. (2002). Data analytic methods for latent partially ordered classification models. Journal of the Royal Statistical Society. Series C: Applied Statistics, 51, 337–350. https://doi.org/10.1111/1467-9876.00272
50.
TatsuokaK. K. (1983). Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20, 345–354. https://doi.org/10.1111/j.1745-3984.1983.tb00212.x
51.
TatsuokaK. K.CorterJ. E.TatsuokaC. (2004). Patterns of diagnosed mathematical content and process skills in TIMSS-R across a sample of 20 countries. American Educational Research Journal, 41, 901–926. https://doi.org/10.3102/00028312041004901
52.
TemplinJ. L.HensonR. A. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods, 11, 287–305. https://doi.org/10.1037/1082-989X.11.3.287
53.
TzikasD. G.LikasA. C.GalatsanosN. P. (2008). The variational approximation for Bayesian inference. IEEE Signal Processing Magazine, 25, 131–146. https://doi.org/10.1109/MSP.2008.929620
54.
von DavierM. (2008). A general diagnostic model applied to language testing data. The British Journal of Mathematical and Statistical Psychology, 61, 287–307. https://doi.org/10.1348/000711007X193957
55.
WangS.YangY.CulpepperS. A.DouglasJ. A. (2018). Tracking skill acquisition with cognitive diagnosis models: A higher-order, hidden Markov model with covariates. Journal of Educational and Behavioral Statistics, 43, 57–87. https://doi.org/10.3102/1076998617719727
56.
WangS.ZhangS.DouglasJ.CulpepperS.WangS. (2018). Using response times to assess learning progress: A joint model for responses and response times. Measurement: Interdisciplinary Research and Perspectives, 16, 45–58. https://doi.org/10.1080/15366367.2018.1435105
YamaguchiK.OkadaK. (2018). Comparison among cognitive diagnostic models for the TIMSS 2007 fourth grade mathematics assessment. PLOS One, 13, e0188691. https://doi.org/10.1371/journal.pone.0188691
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
