This study proposes a Bayesian method for diagnostic classification models (DCMs) for a partially known Q-matrix setting between exploratory and confirmatory DCMs. This Q-matrix setting is practical and useful because test experts have pre-knowledge of the Q-matrix but cannot readily specify it completely. The proposed method employs priors for the Bayesian variable selection to simultaneously estimate the effects of active and nonactive attributes, and the simulations lead to appropriate attribute recovery rates. Furthermore, the proposed method recovers the attribute mastery of individuals at the same as for a fully known Q-matrix. In addition, the proposed methods can be used to estimate the unknown Q-matrix part. A real data example indicates that the proposed Bayesian estimation method for the partially known Q-matrix fits better than a fully specified Q-matrix. Finally, extensions and future research directions are discussed.
BaghaeiP.HohensinnC. (2017). A method of Q-matrix validation for the linear logistic test model. Frontiers in Psychology, 8, 897. https://doi.org/10.3389/fpsyg.2017.00897
2.
BalamutaJ. J.CulpepperS. A. (2022). Exploratory restricted latent class models with monotonicity requirements under Pólya–Gamma data augmentation. Psychometrika, 87(3), 903–945. https://doi.org/10.1007/s11336-021-09815-9
CarpenterB.GelmanA.HoffmanM. D.LeeD.GoodrichB.BetancourtM.BrubakerM. A.GuoJ.LiP.RiddellA. (2017). Stan: A probabilistic programming language. Journal of Statistical Software, 76, 1–32. https://doi.org/10.18637/jss.v076.i01
5.
CarvalhoC. M.PolsonN. G.ScottJ. G. (2010). The horseshoe estimator for sparse signals. Biometrika, 97(2), 465–480. https://doi.org/https://doi.org/10.1093/biomet/asq017
6.
ChenJ. (2017). A residual-based approach to validate Q-matrix specifications. Applied Psychological Measurement, 41(4), 277–293. https://doi.org/10.1177/0146621616686021
7.
ChenJ.de la TorreJ. (2014). A procedure for diagnostically modeling extant large-scale assessment data: The case of the programme for international student assessment in reading. Psychology, 5(18), 1967–1978. https://doi.org/10.4236/psych.2014.518200
ChenY.LiX.LiuJ.YingZ. (2017). Regularized latent class analysis with application in cognitive diagnosis. Psychometrika, 82, 660–692. https://doi.org/10.1007/s11336-016-9545-6
11.
ChenY.LiuJ.XuG.YingZ. (2015). Statistical analysis of Q-matrix based diagnostic classification models. Journal of the American Statistical Association, 110(510), 850–866. https://doi.org/10.1080/01621459.2014.934827
12.
ChiuC.-Y. (2013). Statistical refinement of the Q-matrix in cognitive diagnosis. Applied Psychological Measurement, 37(8), 598–618. https://doi.org/10.1177/0146621613488436
13.
ChiuC.-Y.DouglasJ. (2013). A nonparametric approach to cognitive diagnosis by proximity to ideal response patterns. Journal of Classification, 30, 225–250. https://doi.org/10.1007/s00357-013-9132-9
14.
ChungM. (2019). A Gibbs sampling algorithm that estimates the Q-matrix for the DINA model. Journal of Mathematical Psychology, 93, 102275. https://doi.org/10.1016/j.jmp.2019.07.002
15.
CulpepperS. A. (2019a). Estimating the cognitive diagnosis Q-matrix with expert knowledge: Application to the fraction-subtraction dataset. Psychometrika, 84(2), 333–357. https://doi.org/10.1007/s11336-018-9643-8
16.
CulpepperS. A. (2019b). An exploratory diagnostic model for ordinal responses with binary attributes: Identifiability and estimation. Psychometrika, 84(4), 921–940. https://doi.org/10.1007/s11336-019-09683-4
17.
de la TorreJ. (2008). An empirically based method of Q-matrix validation for the DINA model: Development and applications. Journal of Educational Measurement, 45(4), 343–362. https://doi.org/10.1111/j.1745-3984.2008.00069.x
de la TorreJ.DouglasJ. A. (2004). Higher-order latent trait models for cognitive diagnosis. Psychometrika, 69(3), 333–353. https://doi.org/https://doi.org/10.1007/BF02295640
21.
de la TorreJ.QiuX.-L.SantosK. C. (2022). An empirical Q-Matrix validation method for the polytomous G-DINA model. Psychometrika, 87(2), 693–724. https://doi.org/10.1007/s11336-021-09821-x
22.
DeCarloL. T. (2012). Recognizing uncertainty in the Q-matrix via a Bayesian extension of the DINA model. Applied Psychological Measurement, 36(6), 447–468. https://doi.org/10.1177/0146621612449069
23.
DeCarloL. T. (2019). Insights from reparameterized DINA and beyond. In von DavierM.LeeY.-S. (Eds.), Handbook of diagnostic classification models: Models and model extensions, applications, software packages (pp. 223–243). Springer International Publishing. https://doi.org/10.1007/978-3-030-05584-4_11
24.
EndersC. K. (2022). Applied missing data analysis (2nd ed.). The Guilford Press.
HensonR. A.TemplinJ. L.WillseJ. T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191–210. https://doi.org/10.1007/s11336-008-9089-5
28.
IshwaranH.RaoJ. S. (2005). Spike and slab variable selection: Frequentist and Bayesian strategies. The Annals of Statistics, 33(2), 730–773. https://doi.org/10.1214/009053604000001147
29.
JunkerB. W.SijtsmaK. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25(3), 258–272. https://doi.org/10.1177/01466210122032064
30.
KuoL.MallickB. (1998). Variable selection for regression models. Sankhyā: The Indian Journal of Statistics, Series B, 60, 65–81. https://www.jstor.org/stable/25053023
31.
LiuC.-W.AnderssonB.SkrondalA. (2020). A constrained Metropolis–Hastings Robbins–Monro algorithm for Q matrix estimation in DINA models. Psychometrika, 85(2), 322–357. https://doi.org/10.1007/s11336-020-09707-4
LiuJ.XuG.YingZ. (2013). Theory of the self-learning Q-matrix. Bernoulli: Official Journal of the Bernoulli Society for Mathematical Statistics and Probability, 19(5A), 1790. https://doi.org/10.3150/12-BEJ430
34.
LunnD. J.ThomasA.BestN.SpiegelhalterD. (2000). WinBUGS – A Bayesian modelling framework: Concepts, structure, and extensibility. Statistics and Computing, 10, 325–337. https://doi.org/10.1023/A:1008929526011
35.
MaW.de la TorreJ. (2020a). GDINA: An R package for cognitive diagnosis modeling. Journal of Statistical Software, 93(14), 1–26. https://doi.org/10.18637/jss.v093.i14
36.
MaW.delaTorreJ. (2020b). An empirical Q-matrix validation method for the sequential generalized DINA model. British Journal of Mathematical and Statistical Psychology, 73(1), 142–163. https://doi.org/10.1111/bmsp.12156
37.
MacreadyG. B.DaytonC. M. (1977). The use of probabilistic models in the assessment of mastery. Journal of Educational Statistics, 2(2), 99–120. https://doi.org/10.3102/10769986002002099
38.
MadisonM. J.BradshawL. P. (2018). Assessing growth in a diagnostic classification model framework. Psychometrika, 83, 963–990. https://doi.org/https://doi.org/10.1007/s11336-018-9638-5
39.
MadisonM. J.ChungS.KimJ.BradshawL. P. (2024). Approaches to estimating longitudinal diagnostic classification models. Behaviormetrika, 51, 7–19. https://doi.org/10.1007/s41237-023-00202-5
40.
MakalicE.SchmidtD. F. (2016). A simple sampler for the horseshoe estimator. IEEE Signal Processing Letters, 23(1), 179–182. https://doi.org/10.1109/LSP.2015.2503725
NájeraP.SorrelM. A.AbadF. J. (2019). Reconsidering cutoff points in the general method of empirical Q-matrix validation. Educational and Psychological Measurement, 79(4), 727–753. https://doi.org/10.1177/0013164418822700
44.
NájeraP.SorrelM. A.de la TorreJ.AbadF. J. (2020). Improving robustness in Q-matrix validation using an iterative and dynamic procedure. Applied Psychological Measurement, 44(6), 431–446. https://doi.org/10.1177/0146621620909904
45.
NájeraP.SorrelM. A.de la TorreJ.AbadF. J. (2021). Balancing fit and parsimony to improve Q-matrix validation. British Journal of Mathematical and Statistical Psychology, 74, 110–130. https://doi.org/10.1111/bmsp.12228
46.
O’haraR. B.SillanpääM. J. (2009). A review of Bayesian variable selection methods: What, how and which. Bayesian Analysis, 4(1), 85–117. https://doi.org/10.1214/09-BA403
47.
OkaM.OkadaK. (2023). Scalable Bayesian approach for the DINA Q-matrix estimation combining stochastic optimization and variational inference. Psychometrika, 88(1), 302–331. https://doi.org/10.1007/s11336-022-09884-4
48.
PlummerM. (2003). JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling. Proceedings of the 3rd international workshop on distributed statistical computing, Vienna, 20–22 March 2003, 1–10.
RuppA. A.TemplinJ. (2008). The effects of Q-matrix misspecification on parameter estimates and classification accuracy in the DINA model. Educational and Psychological Measurement, 68(1), 78–96. https://doi.org/10.1177/0013164407301545
52.
RuppA. A.TemplinJ.HensonR. A. (2010). Diagnostic measurement: Theory, methods, and applications. Guilford Press.
53.
SenS.CohenA. S. (2021). Sample size requirements for applying diagnostic classification models. Frontiers in Psychology, 11, 621251. https://doi.org/10.3389/fpsyg.2020.621251
54.
ShiQ.MaW.RobitzschA.SorrelM. A.ManK. (2021). Cognitively diagnostic analysis using the G-DINA model in R. Psych, 3(4), 812–835. https://doi.org/https://doi.org/10.3390/psych3040052
55.
SorrelM. A.AbadF. J.OleaJ.de la TorreJ.BarradaJ. R. (2017). Inferential item-fit evaluation in cognitive diagnosis modeling. Applied Psychological Measurement, 41(8), 614–631. https://doi.org/10.1177/0146621617707510
SunY.YeS.SuG.SunY. (2016). Q-matrix learning and DINA model parameter estimation. 2016 International conference on behavioral, economic and socio-cultural computing (BESC) [Conference session]. pp. 1–6. https://doi.org/10.1109/BESC.2016.7804471
58.
TadesseM. G.VannucciM. (2021). Handbook of Bayesian variable selection. CRC Press.
59.
TatsuokaK. K. (1985). A probabilistic model for diagnosing misconceptions by the pattern classification approach. Journal of Educational Statistics, 10(1), 55–73. https://doi.org/10.3102/10769986010001055
60.
TemplinJ. L.HensonR. A. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods, 11(3), 287. https://doi.org/10.1037/1082-989X.11.3.287
61.
TjoeH.de la TorreJ. (2013). Designing cognitively-based proportional reasoning problems as an application of modern psychological measurement models. Journal of Mathematics Education, 6(2), 17–26. https://api.semanticscholar.org/CorpusID:54598161
62.
TjoeH.de la TorreJ. (2014). The identification and validation process of proportional reasoning attributes: An application of a cognitive diagnosis modeling framework. Mathematics Education Research Journal, 26, 237–255. https://doi.org/10.1007/s13394-013-0090-7
63.
TuD.ChiuJ.MaW.WangD.CaiY.OuyangX. (2023). A multiple logistic regression-based (MLR-B) Q-matrix validation method for cognitive diagnosis models: A confirmatory approach. Behavior Research Methods, 55(4), 2080–2092. https://doi.org/10.3758/s13428-022-01880-x
64.
van ErpS.OberskiD. L.MulderJ. (2019). Shrinkage priors for Bayesian penalized regression. Journal of Mathematical Psychology, 89, 31–50. https://doi.org/10.1016/j.jmp.2018.12.004
von DavierM. (2008). A general diagnostic model applied to language testing data. British Journal of Mathematical and Statistical Psychology, 61(2), 287–307. https://doi.org/https://doi.org/10.1348/000711007X193957
67.
von DavierM. (2014a). The DINA model as a constrained general diagnostic model: Two variants of a model equivalency. British Journal of Mathematical and Statistical Psychology, 67(1), 49–71. https://doi.org/10.1111/bmsp.12003
68.
von DavierM. (2014b). The log-linear cognitive diagnostic model (LCDM) as a special case of the general diagnostic model (GDM). ETS Research Report Series, 2014(2), 1–13. https://doi.org/10.1002/ets2.12043
69.
von DavierM.LeeY.-S. (2019). Handbook of diagnostic classification models. Springer International Publishing.
70.
WangD.CaiY.TuD. (2020). Q-Matrix estimation methods for cognitive diagnosis models: Based on partial known Q-matrix. Multivariate Behavioral Research, 1–13. https://doi.org/10.1080/00273171.2020.1746901
71.
WangW.SongL.DingS.MengY.CaoC.JieY. (2018). An EM-based method for Q-matrix validation. Applied Psychological Measurement, 42(6), 446–459. https://doi.org/10.1177/0146621617752991
72.
WangW.SongL.DingS.WangT.GaoP.XiongJ. (2020). A semi-supervised learning method for Q-matrix specification under the DINA and DINO model with independent structure. Frontiers in Psychology, 11, 524203. https://doi.org/10.3389/fpsyg.2020.02120
73.
WatanabeS. (2010). Asymptotic equivalence of bayes cross validation and widely applicable information criterion in singular learning theory. Journal of Machine Learning Research, 11, 3571–3594. https://www.jmlr.org/papers/volume11/watanabe10a/watanabe10a.pdf
74.
WatanabeS. (2018). Mathematical theory of Bayesian statistics. CRC Press.
75.
XiongJ.LuoZ.LuoG.YuX. (2022). Data-driven Q-matrix learning based on Boolean matrix factorization in cognitive diagnostic assessment. British Journal of Mathematical and Statistical Psychology, 75(3), 638–667. https://doi.org/10.1111/bmsp.12271
76.
XuG.ShangZ. (2018). Identifying latent structures in restricted latent class models. Journal of the American Statistical Association, 113(523), 1284–1295. https://doi.org/10.1080/01621459.2017.1340889
77.
XuX.de la TorreJ.ZhangJ.GuoJ.ShiN. (2020). Estimating CDMs using the slice-within-Gibbs sampler. Frontiers in Psychology, 11, 2260. https://doi.org/10.3389/fpsyg.2020.02260
78.
YamaguchiK.TemplinJ. (2022). A Gibbs sampling algorithm with monotonicity constraints for diagnostic classification models. Journal of Classification, 39(1), 24–54. https://doi.org/10.1007/s00357-021-09392-7
79.
YamaguchiK.ZhangJ. (2023). Fully Gibbs sampling algorithms for Bayesian variable selection in latent regression models. Journal of Educational Measurement, 60(2), 202–234. https://doi.org/10.1111/jedm.12348
80.
YuX.ChengY. (2020). Data-driven Q-matrix validation using a residual-based statistic in cognitive diagnostic assessment. British Journal of Mathematical and Statistical Psychology, 73, 145–179. https://doi.org/10.1111/bmsp.12191
81.
ZhanP.JiaoH.ManK.WangL. (2019). Using JAGS for Bayesian cognitive diagnosis modeling: A tutorial. Journal of Educational and Behavioral Statistics, 44(4), 473–503. https://doi.org/10.3102/1076998619826040
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
