Assessment based on fine-grained latent traits can provide more detailed information about the subjects. Cognitive diagnostic assessment (CDA) is a framework of education and psychological measurement that is grounded in the assessment of fine-grained latent traits. The -matrix, defining the relationship between items and attributes, is the basis for CDA. Data-driven -matrix estimation has become a research hotspot due to its high efficiency and objectivity. However, the existing method for -matrix estimation primarily focuses on scenarios with low or middle correlations between attributes (latent traits), and they point out that the accuracy of -matrix estimation significantly declines in situations with high attribute correlations. To address the limitations of existing methods in scenarios with high attribute correlation. This paper proposes a majority-class symmetric undersampling (MCSU) method tailored for CDA. To evaluate its performance, two simulation studies are conducted. The simulation results under a wide variety of conditions show that the MCSU can improve the estimation accuracy of attribute number and -matrix in highly correlated scenarios. Finally, a real dataset of the Examination for the Certificate of Proficiency in English is analyzed using the proposed method.
BalamutaJ. J.CulpepperS. A. (2022). Exploratory restricted latent class models with monotonicity requirements under Pólya-gamma data augmentation. Psychometrika, 87, 903–945. https://doi.org/10.1007/s11336-021-09815-9
2.
ChawlaN. V.BowyerK. W.HallL. O.KegelmeyerW. P. (2002). SMOTE: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16, 321–357.
3.
ChenJ. (2017). A residual-based approach to validate q-matrix specifications. Applied Psychological Measurement, 41(4), 277–293. https://doi.org/10.1177/0146621616686021
ChenY.CulpepperS. A.ChenY. (2023). Bayesian inference for an unknown number of attributes in restricted latent class models. Psychometrika, 88, 613–635. https://doi.org/10.1007/s11336-022-09900-7
ChenY.LiuJ.XuG.YingZ. (2015). Statistical analysis of Q-matrix based diagnostic classification models. Journal of the American Statistical Association, 110(510), 850–866. https://doi.org/10.1080/01621459.2014.934827
8.
ChenY.LiuY.CulpepperS. A.ChenY. (2021). Inferring the number of attributes for the exploratory DINA model. Psychometrika, 86(1), 30–64. https://doi.org/10.1007/s11336-021-09750-9
9.
ChiuC.-Y. (2013). Statistical refinement of the Q-Matrix in cognitive diagnosis. Applied Psychological Measurement, 37(8), 598–618. https://doi.org/10.1177/0146621613488436
10.
ChungM. (2019). A Gibbs sampling algorithm that estimates the Q-matrix for the DINA model. Journal of Mathematical Psychology, 93, 102275. https://doi.org/10.1016/j.jmp.2019.07.002
11.
CulpepperS. A. (2019). An exploratory diagnostic model for ordinal responses with binary attributes: identifiability and estimation. Psychometrika, 84(4), 921–940. https://doi.org/10.1007/s11336-019-09683-4
12.
CulpepperS. A.ChenY. (2019). Development and application of an exploratory reduced reparameterized unified model. Journal of Educational and Behavioral Statistics, 44(1), 3–24. https://doi.org/10.3102/1076998618791306
13.
de la TorreJ. (2008). An empirically based method of Q-Matrix validation for the DINA model: Development and applications. Journal of Educational Measurement, 45(4), 343–362. https://doi.org/10.1111/j.1745-3984.2008.00069.x
de la TorreJ.DouglasJ. A. (2004). Higher-order latent trait models for cognitive diagnosis. Psychometrika, 69(3), 333–353.
17.
de la TorreJ.QiuX.-L.SantosK. C. (2022). An empirical q-matrix validation method for the polytomous G-DINA model. Psychometrika, 87(2), 693–724. https://doi.org/10.1007/s11336-021-09821-x
18.
DeCarloL. T. (2012). Recognizing uncertainty in the Q-matrix via a Bayesian extension of the DINA model. Applied Psychological Measurement, 36(6), 447–468. https://doi.org/10.1177/0146621612449069
19.
FengY.HabingB.HuebnerA. (2014). Parameter estimation of the reduced RUM using the EM algorithm. Applied Psychological Measurement, 38, 137–150.
20.
GierlM. J.LeightonJ. P. (2007). Directions for future research in cognitive diagnostic assessment. In LeightonJ.GierlM. (Eds.), Cognitive diagnostic assessment for education (pp. 341–352). Cambridge University Press.
21.
HartzS. (2002). A Bayesian framework for the unified model for assessing cognitive abilities: Blending theory with practicality [Unpublished doctoral dissertation] University of Illinois at Urbana, Champaign, IL.
22.
HeH.GarciaE. A. (2009). Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering, 21(9), 1263–1284.
23.
HensonR. A.TemplinJ. L.WillseJ. T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74(2), 191–210.
24.
JapkowiczN. (2000). The class imbalance problem: Significance and strategies. In Proceedings of the 2000 International Conference on Artificial Intelligence (IC-AI’2000): Special Track on Inductive Learning, Las Vegas, NV, United States.
25.
JunkerB. W.SijtsmaK. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25(3), 258–272.
26.
KangC.YangY.ZengP. (2019). Q-Matrix refinement based on item fit statistic RMSEA. Applied Psychological Measurement, 43(7), 527–542. https://doi.org/10.1177/0146621618813104
27.
LeeD. D.SeungH. S. (1999). Learning the parts of objects by non-negative matrix factorization. Nature, 401, 788–791.
28.
LeightonJ. P.GierlM. J. (Eds.). (2007). Cognitive diagnostic assessment for education: Theory and applications. Cambridge University Press. https://doi.org/10.1017/CBO9780511611186
LiuC.-W.AnderssonB.SkrondalA. (2020). A constrained Metropolis–Hastings Robbins–Monro algorithm for Q matrix estimation in DINA models. Psychometrika, 85(2), 322–357. https://doi.org/10.1007/s11336-020-09707-4
MaC.OuyangJ.XuG. (2023). Learning latent and hierarchical structures in cognitive diagnosis models. Psychometrika, 88(1), 175–207. https://doi.org/10.1007/s11336-022-09867-5
33.
MaW.de la TorreJ. (2020). An empirical Q-matrix validation method for the sequential generalized DINA model. British Journal of Mathematical and Statistical Psychology, 73(1), 142–163. https://doi.org/10.1111/bmsp.12156
34.
NájeraP.AbadF. J.SorrelM. A. (2021). Determining the number of attributes in cognitive diagnosis modeling. Frontiers in Psychology, 12, 614470. https://doi.org/10.3389/fpsyg.2021.614470
35.
NájeraP.SorrelM. A.AbadF. J. (2019). Reconsidering cutoff points in the general method of empirical Q-Matrix validation. Educational and Psychological Measurement, 79(4), 727–753. https://doi.org/10.1177/0013164418822700
36.
NájeraP.SorrelM. A.de la TorreJ.AbadF. J. (2020). Improving robustness in Q-Matrix validation using an iterative and dynamic procedure. Applied Psychological Measurement, 44(6), 431–446. https://doi.org/10.1177/0146621620909904
37.
NájeraP.SorrelM. A.TorreJ.AbadF. J. (2021). Balancing fit and parsimony to improve Q-matrix validation. British Journal of Mathematical and Statistical Psychology, 74(S1), 110–130. https://doi.org/10.1111/bmsp.12228
38.
RuppA.TemplinJ.HensonR. A. (2010). Diagnostic measurement: Theory, methods, and applications. Guilford Press, 2010.
39.
SessomsJ.HensonR. A. (2018). Applications of diagnostic classification models: A literature review and critical commentary. Measurement: Interdisciplinary Research and Perspectives, 16, 1–17.
40.
TatsuokaK. K. (1983). Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20(4), 345–354.
41.
TatsuokaK. K. (Eds.) (2009). Cognitive assessment: An introduction to the rule space method. Routledge.
42.
TemplinJ.BradshawL. (2014). Hierarchical diagnostic classification models: A family of models for estimating and testing attribute hierarchies. Psychometrika, 79, 317–339.
43.
TemplinJ.HoffmanL. (2013). Obtaining diagnostic classification model estimates using Mplus. Educational Measurement: Issues and Practice, 32, 37–50.
44.
TuD.ChiuJ.MaW.WangD.CaiY.OuyangX. (2023). A multiple logistic regression-based (MLR-B) Q-matrix validation method for cognitive diagnosis models: A confirmatory approach. Behavior Research Methods, 55, 2080–2092. https://doi.org/10.3758/s13428-022-01880-x
45.
von DavierM.LeeY.-S. (2019). Handbook of diagnostic classification models. Springer International Publishing.
46.
WangD.CaiY.TuD. (2020). Q-Matrix estimation methods for cognitive diagnosis models: Based on partial known Q-matrix. Multivariate Behavioral Research, 56, 514–526. https://doi.org/10.1080/00273171.2020.1746901
47.
WangW.SongL.DingS. (2018). An exploratory discrete factor loading method for Q-Matrix specification in cognitive diagnostic models. In: WibergM.CulpepperS.JanssenR.GonzálezJ.MolenaarD. (Eds.) Quantitative psychology. IMPS 2017. Springer Proceedings in Mathematics & Statistics (vol 233, pp. 351–332). Springer. https://doi.org/10.1007/978-3-319-77249-3_2
48.
XiongJ.LuoZ.LuoG.YuX. (2022). Data-driven Q-matrix learning based on Boolean matrix factorization in cognitive diagnostic assessment. The British Journal of Mathematical and Statistical Psychology, 75(3), 638–667. https://doi.org/10.1111/bmsp.12271
49.
XiongJ.LuoZ.LuoG.YuXLiY. (2024). An exploratory Q-matrix estimation method based on sparse non-negative matrix factorization. Behavior Research Methods, 56, 7647–7673. https://doi.org/10.3758/s13428-024-02442-z
50.
XuG.ShangZ. (2018). Identifying latent structures in restricted latent class models. Journal of the American Statistical Association, 113(523), 1284–1295. https://doi.org/10.1080/01621459.2017.1340889
51.
YuX.ChengY. (2020). Data-driven Q-matrix validation using a residual-based statistic in cognitive diagnostic assessment. British Journal of Mathematical and Statistical Psychology, 73(S1), 145–179. https://doi.org/10.1111/bmsp.12191
52.
ZhengY.CheonH.KatzC. M. (2020). Using machine learning methods to develop a short tree-based adaptive classification test: Case study with a high-dimensional item pool and imbalanced data. Applied Psychological Measurement, 44(7–8), 499–514.
53.
ZhouT.JiaoH. (2023). Exploration of the stacking ensemble machine learning algorithm for cheating detection in large-scale assessment. Educational and Psychological Measurement, 83, 831–854. https://doi.org/10.1177/00131644221117193
54.
ZhouZ. (2016). Machine Learning. Tsinghua University Press.
