In this article, we systematically introduce the just another Gibbs sampler (JAGS) software program to fit common Bayesian cognitive diagnosis models (CDMs) including the deterministic inputs, noisy “and” gate model; the deterministic inputs, noisy “or” gate model; the linear logistic model; the reduced reparameterized unified model; and the log-linear CDM (LCDM). Further, we introduce the unstructured latent structural model and the higher order latent structural model. We also show how to extend these models to consider polytomous attributes, the testlet effect, and longitudinal diagnosis. Finally, we present an empirical example as a tutorial to illustrate how to use JAGS codes in R.
AkaikeH. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19, 716–723. doi:10.1109/TAC.1974.1100705
2.
BianconciniS. (2012). A general multivariate latent growth model with applications to student achievement. Journal of Educational and Behavioral Statistics, 37, 339–364.
3.
BoltD.ChenH.DiBelloL.HartzS.HensonR.RoussosL.…TemplinJ. (2008). The Arpeggio Suite: Software for cognitive skills diagnostic assessment[Computer software and Manual]. St. Paul, MN: Assessment Systems.
4.
BoxG. E. P.TiaoG. C. (1973). Bayesian inference in statistical analysis. Reading, MA: Addison-Wesley.
5.
BradshawL.TemplinJ. (2014). Combining item response theory and diagnostic classification models: A psychometric model for scaling ability and diagnosing misconceptions. Psychometrika, 79, 403–425.
6.
BrooksS. P.GelmanA. (1998). General methods for monitoring convergence of iterative simulations. Journal of Computational and Graphical Statistics, 7, 434–455. doi:10.2307/1390675
7.
CaiL. (2017). flexMIRT® version 3.51: Flexible multilevel multidimensional item analysis and test scoring[Computer software]. Chapel Hill, NC: Vector Psychometric Group.
8.
CarpenterB.GelmanA.HoffmanM.LeeD.GoodrichB.BetancourtM.…RiddellA. (2016). Stan: A probabilistic programming language. Journal of Statistical Software, 20, 1–37.
9.
ChenJ.de la TorreJ. (2013). A general cognitive diagnosis model for expert-defined polytomous attributes. Applied Psychological Measurement, 37, 419–437.
10.
ChenY.CulpepperS. A.ChenY.DouglasJ. (2018). Bayesian estimation of the DINA Q matrix. Psychometrika, 83, 89–108.
11.
ChenY.CulpepperS. A.WangS.DouglasJ. (2018). A hidden Markov model for learning trajectories in cognitive diagnosis with application to spatial rotation skills. Applied Psychological Measurement, 42, 5–23.
12.
CongdonP. (2003). Applied Bayesian modelling. New York, NY: John Wiley.
13.
ChungM.JohnsonM. (2018). An MCMC algorithm for estimation the Q-matrix in a Bayesian framework. arXiv preprint arXiv:1802.02286. Retrieved fromhttps://arxiv.org/abs/1802.02286
14.
CulpepperS. A. (2015a). Bayesian estimation of the DINA model with Gibbs sampling. Journal of Educational and Behavioral Statistics, 40, 454–476.
CulpepperS. A.HudsonA. (2018). An improved strategy for Bayesian estimation of the reduced reparameterized unified model. Applied Psychological Measurement, 42, 99–115.
17.
CurtisS. M. (2010). BUGS code for item response theory. Journal of Statistical Software, 36, 1–34.
18.
DeCarloL. T. (2012). Recognizing uncertainty in the Q-matrix via a Bayesian extension of the DINA model. Applied Psychological Measurement, 36, 447–468. doi:10.1177/0146621612449069
19.
de la TorreJ. (2009). DINA model and parameter estimation: A didactic. Journal of Educational and Behavioral Statistics, 34, 115–130.
20.
de la TorreJ. (2011). The generalized DINA model framework. Psychometrika, 76, 179–199. doi:10.1007/s11336-011-9207-7
21.
de la TorreJ.DouglasJ. (2004). Higher-order latent trait models for cognitive diagnosis. Psychometrika, 69, 333–353. doi:10.1007/BF02295640
22.
GelmanA.CarlinJ. B.SternH. S.DunsonD. B.VehtariA.RubinD. B. (2014). Bayesian data analysis (3rd ed.). Boca Raton, FL: CRC Press.
GeorgeA. C.RobitzschA.KieferT.GrossJ.UenlueA. (2016). The R package CDM for cognitive diagnosis models. Journal of Statistical Software, 74, 1–24.
25.
HaertelE. H. (1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26, 301–321. doi:10.1111/j.1745-3984.1989.tb00336.x
26.
HansenM.CaiL.MonroeS.LiZ. (2016). Limited-information goodness-of-fit testing of diagnostic classification item response models. British Journal of Mathematical and Statistical Psychology, 69, 225–252. doi:10.1111/bmsp.12074
27.
HartzS. M. (2002). A Bayesian framework for the unified model for assessing cognitive abilities: Blending theory with practicality. (Unpublished doctoral dissertation). University of Illinois at Urbana-Champaign, Champaign, IL.
28.
HensonR.TemplinJ.WillseJ. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191–210.
29.
HoffmanM. D.GelmanA. (2014). The No-U-turn sampler: Adaptively setting path lengths in Hamiltonian Monte Carlo. Journal of Machine Learning Research, 15, 1593–1623.
30.
HuangH.-Y.WangW.-C. (2014). The random-effect DINA model. Journal of Educational Measurement, 51, 75–97.
31.
JiangH. (1996). Applications of computational statistics in cognitive diagnosis and IRT modeling. (Unpublished doctoral dissertation). University of Illinois at Urbana-Champaign, Champaign, IL.
32.
JunkerB. W.SijtsmaK. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258–272.
33.
KarelitzT. M. (2004). Ordered category attribute coding framework for cognitive assessments. (Unpublished doctoral dissertation). University of Illinois at Urbana-Champaign, Champaign, IL.
34.
KellnerK. (2017). jagsUI: A wrapper around “rjags” to streamline “JAGS” analyses (R package version 1.4.9). Retrieved fromhttp://CRAN.R-project.org/package=jagsUI
35.
KöhnH.-F.ChiuC.-Y. (2016). A proof of the duality of the DINA model and the DINO model. Journal of Classification, 33, 171–184.
LiF.CohenA.BottgeB.TemplinJ. (2016). A latent transition analysis model for assessing change in cognitive skills. Educational and Psychological Measurement, 76, 181–204.
39.
LiX. M.WangW.-C. (2015). Assessment of differential item functioning under cognitive diagnosis models: The DINA model example. Journal of Educational Measurement, 52, 28–54.
40.
LiaoM.JiaoH. (2016, April). A combination of diagnostic classification model and item response theory model with testlet effect. Paper presented at the International Objective Measurement Workshop, Washington, DC.
41.
LunnD. J.ThomasA.BestN.SpiegelhalterD. (2000). WinBUGS-a Bayesian modelling framework: Concepts, structure, and extensibility. Statistics and Computing, 10, 325–337.
MaW.de la TorreJ. (2016b). A sequential cognitive diagnosis model for polytomous responses. British Journal of Mathematical and Statistical Psychology, 69, 253–275. doi:10.1111/bmsp.12070
44.
MacreadyG. B.DaytonC. M. (1977). The use of probabilistic models in the assessment of mastery. Journal of Educational and Behavioral Statistics, 2, 99–120.
45.
MartinA. D.QuinnK. M.ParkJ. H. (2011). Mcmcpack: Markov chain Monte Carlo in R. Journal of Statistical Software, 42, 1–21.
PlummerM.StukalovA.DenwoodM. (2016). rjags: Bayesian graphical models using MCMC (R package version 4-6). Retrieved fromhttp://CRAN.R-project.org/package=rjags
51.
R Core Team. (2016). R: A language and environment for statistical computing. Vienna, Austria: The R Foundation for Statistical Computing.
52.
RuppA.TemplinJ.HensonR. (2010). Diagnostic measurement: Theory, methods, and applications. New York, NY: Guilford Press.
53.
SchwarzG. (1978). Estimating the dimension of a model. The Annals of Statistics, 6, 461–464.
54.
SinharayS.AlmondR. G. (2007). Assessing fit of cognitive diagnostic models: A case study. Educational and Psychological Measurement, 67, 239–257.
55.
SpiegelhalterD. J.BestN. G.CarlinB. P.van der LindeA. (2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society, Series B, 64, 583–616.
56.
SpiegelhalterD.ThomasA.BestN.LunnD. (2014). OpenBUGS Version 3.2.3 user manual. Helsinki, Finland. Retrieved fromhttp://www.openbugs.net/w/Manuals/
TatsuokaK. K. (1983). Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20, 345–354.
59.
TatsuokaK. K. (1990). Toward an integration of item-response theory and cognitive error diagnosis. In FrederiksenN.GlaserR.LesgoldA.SaftoM. (Eds.), Monitoring skills and knowledge acquisition (pp. 453–488). Hillsdale, NJ: Erlbaum.
60.
TemplinJ.HensonR. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods, 11, 287–305.
61.
TemplinJ.HoffmanL. (2013). Obtaining diagnostic classification model estimates using Mplus. Educational Measurement: Issues and Practice, 32, 37–50.
62.
VermuntJ. K.TranB.MagidsonJ. (2008). Latent class models in longitudinal research. In MenardS. (Ed.), Handbook of longitudinal research: Design, measurement, and analysis (pp. 373–385). Burlington, MA: Elsevier.
63.
von DavierM. (2005). mdltm: Software for the general diagnostic model and for estimating mixtures of multidimensional discrete latent traits models[Computer software]. Princeton, NJ: ETS.
64.
von DavierM. (2008). A general diagnostic model applied to language testing data. British Journal of Mathematical and Statistical Psychology, 61, 287–307.
65.
WagenmakersE.-J.LeeM. D.LodewyckxT.IversonG. (2008). Bayesian versus frequentist inference. In HoijtinkH.KlugkistI.BoelenP. A. (Eds.), Bayesian evaluation of informative hypotheses (pp. 181–207). New York, NY: Springer.
66.
WainerH.BradlowE. T.WangX. (2007). Testlet response theory and its applications. New York, NY: Cambridge University Press.
67.
WainerH.KielyG. (1987). Item clusters and computerized adaptive testing: A case for testlets. Journal of Educational Measurement, 24, 185–202.
68.
WangS.YangY.CulpepperS. A.DouglasJ. A. (2018). Tracking skill acquisition with cognitive diagnosis models: A higher-order, hidden Markov model with covariates. Journal of Educational and Behavioral Statistics, 43, 57–87.
69.
YanD.MislevyR. J.AlmondR. G. (2003). Design and analysis in a cognitive assessment(ETS Research Report Series, RR-03-32). Princeton, NJ: ETS.
70.
ZhanP.BianY.WangL. (2016). Factors affecting the classification accuracy of reparametrized diagnostic classification models for expert-defined polytomous attributes. Acta Psychologica Sinica, 48, 318–330.
71.
ZhanP.JiaoH.LiaoD. (2018). Cognitive diagnosis modelling incorporating item response times. British Journal of Mathematical and Statistical Psychology, 71, 262–286.
ZhanP.LiaoM.BianY. (2018). Joint testlet cognitive diagnosis modeling for paired local item dependence in response times and response accuracy. Frontiers in Psychology, 9, 607.
