In cognitive diagnosis, attribute hierarchies are considered important structural features of cognitive diagnostic models, as they provide auxiliary information about the nature of attributes. In this article, the idea of ordering theory is applied to cognitive diagnosis, and a new approach to identify attribute hierarchy based on the attribute correlation intensity matrix is proposed. This approach attempts to identify attribute hierarchy in data with a small sample size while ensuring a high accuracy rate. The results of simulation studies and empirical data analysis show that the proposed approach can be used to identify attribute hierarchy in diagnostic tests, especially in small samples, making it worth popularizing.
AirasianP. W.BartW. M. (1973). Ordering theory: A new and useful measurement model. Journal of Educational Technology, 5, 56–60.
2.
ChangS. Y.LinY. H. (2007). Application of weighted polytomous ordering theory in concepts structure analysis of fraction addition for pupils [Paper presentation]. Meeting of Proceedings of the 7th WSEAS International Conference on Simulation, Modelling and Optimization, Beijing, China.
3.
ChenH.ChenJ. (2016). Retrofitting non-cognitive-diagnostic reading assessment under the generalized DINA model framework. Language Assessment Quarterly, 13(3), 218–230.
4.
ChenY.LiX.LiuJ.YingZ. (2017). Regularized latent class analysis with application in cognitive diagnosis. Psychometrika, 82(3), 660–692.
5.
ChickeringD. M. (2002). Learning equivalence classes of Bayesian-network structures. Journal of Machine Learning Research, 2, 445–498.
6.
ChiuC.-Y.DouglasJ. (2013). A nonparametric approach to cognitive diagnosis by proximity to ideal response patterns. Journal of Classification, 30(2), 225–250.
7.
DevichiC.MunierV. (2013). About the concept of angle in elementary school: Misconceptions and teaching sequences. The Journal of Mathematical Behavior, 32, 1–19.
8.
DingS. L.LuoF. (2013). An efficient algorithm of deriving Hasse diagram from the reachibility matrix of a partial order relation—Together with its application to cognitive diagnosis. Journal of Jiangxi Normal University (Natural Science), 37(5), 441–444.
9.
DingS. L.LuoF.WangW. Y.YuX. F.XiongJ. H. (2019). The augment algorithm and its role in cognitive diagnosis. Chinese/English Journal of Educational Measurement and Evaluation, 3, 1–12.
10.
FarinG. E.HansfordD. (2004). Practical linear algebra: A geometry toolbox. AK Peters Ltd.
11.
FuD. X.QinC. Y.LuoZ. S.LiY. J.YuX. F.YeZ. Y. (2024). Using regularized methods to validate Q-matrix in cognitive diagnostic assessment. Journal of Educational and Behavioral Statistics. Advance online publication. https://doi.org/10.3102/10769986241240084.
12.
HattieJ. (1985). Methodology review: Assessing unidimensionality of tests and items. Applied Psychological Measurement, 9(2), 139–164.
13.
HernandezM. I.CousoD.PintoR. (2015). Analyzing students’ learning progressions throughout a teaching sequence on acoustic properties of materials with a model-based inquiry approach. Journal of Science Educational and Technology, 24, 356–377.
14.
HuB.TemplinJ. (2020). Using diagnostic classification models to validate attribute hierarchies and evaluate model fit in Bayesian networks. Multivariate Behavioral Research, 55(2), 300–311.
15.
KarelitzT. M. (2004). Ordered category attribute coding framework for cognitive assessments [Unpublished doctoral dissertation]. University of Illinois at Urbana-Champaign.
16.
KranchD. A. (2012). Teaching the novice programmer: A study of instructional sequences and perception. Education and Information Technologies, 17, 291–313.
17.
LeightonJ. P.GierlM.HunkaS. M. (2004). The attribute hierarchy method for cognitive assessment: A variation on Tatsuoka’s rule-space approach. Journal of Educational Measurement, 41(3), 205–236.
18.
LiuH. C. (2012). Ideal item ordering matrix and Liu’s ordering theory. New item response theories. Asia University.
19.
LiuH. C.ChenW. S.TsaiH. C. (2012). Polytomous item relational structure theory based on empirical distribution critical value. Advanced Materials Research, 472–475, 1329–1332.
20.
LiuH. C.JengB. C.TsaiH. C.YuY. K.LuC. Y. (2013). Liu’s polytomous item relational structure theory based on Q-matrix theory. Applied Mechanics and Materials, 311, 93–98.
21.
LiuH. C.ShiaB. C.ChengD. Y. (2014). An improved Liu’s ordering theory based on empirical distributive critical value [Paper presentation]. Meeting of the 27th of International Conference on Industrial, Engineering & Other Applications of Applied Intelligent Systems, Kaohsiung, Taiwan.
22.
LiuH. C.ShiaB. C.JuJ. M.WangT. L.SuC. H.LinY. T. (2016). A novel generalized item relational structure theory based on Liu’s normalization and consistency criteria. ICIC Express Letters, 10(12), 2957–2962.
23.
LiuH. C.WuS. N.ChenC. C. (2011). Item relational structure algorithm based on empirical distribution critical value. Journal of Software, 6(11), 2106–2113.
24.
MaC.XuG. (2021a). Learning latent and hierarchical structures in cognitive diagnosis models. arXiv preprint arXiv: 2104.02143.
25.
MaC.XuG. (2021b). Hypothesis testing for hierarchical structures in cognitive diagnosis models. arXiv preprint arXiv: 2106.03218.
26.
MaY. F. (2021). Attribute hierarchical structure estimation based on three structure learning algorithms [Master’s thesis, Jiangxi Normal University].
27.
QinC. Y.DongS. H.YuX. F. (2024). Exploration of polytomous-attribute Q-matrix validation in cognitive diagnostic assessment. Knowledge-Based Systems, 292, 111577.
28.
Rupp, A. A., Templin, J., & Henson, R. (2010). Diagnostic measurement: Theory, methods, and applications. Guiford.
29.
SinharayS. (2016). The choice of the ability estimate with asymptotically correct standardized person-fit statistics. British Journal of Mathematical and Statistical Psychology, 69(2), 175–193.
30.
TakeyaM. (1980). Construction and utilization of item relational structure graphs for use in test analysis. Journal of Educational Technology, 5, 93–103.
31.
TatsuokaK. K. (Ed.). (1984). Analysis of errors in fraction addition and subtraction problems (Final Report for Grant NIE-G-2000). University of illinois, Computer-based Education Research Laboratory.
32.
TatsuokaK. K. (1995). Architecture of knowledge structure and cognitive diagnosis: A statistical pattern recognition and classification approach. In NicholsP. D.ChipmanS. F.BrennanR. L. (Eds.), Cognitively diagnostic assessment (pp. 327–361). Lawrence Erlbaum Associates.
33.
TatsuokaK. K. (2009). Cognitive assessment: An introduction of the rule space method. Routledge; Taylor & Francis Group.
34.
TemplinJ.BradshawL. (2014). Hierarchical diagnostic classification models: A family of models for estimating and testing attribute hierarchies. Psychometrika, 79(2), 317–339.
35.
TsamardinosI.BrownL. E.AliferisC. F. (2006). The max-min hill-climbing Bayesian network structure learning algorithm. Machine Learning, 65(1), 31–78.
36.
von DavierM.LeeY.-S. (2019). Handbook of diagnostic classification models. Springer International Publishing.
37.
WangC.LuJ. (2020). Learning attribute hierarchies from data: Two exploratory approaches. Journal of Educational and Behavioral Statistics, 46(1), 58–84.
38.
XuG.ShangZ. (2017). Identifying latent structures in restricted latent class models. Journal of the American Statistical Association, 113(523), 1284–1295. https://doi.org/10.1080/01621459.2017.1340889
39.
YuX. F.ChengY. (2019). Data driven Q matrix validation using a residual based statistic in cognitive diagnostic assessment. British Journal of Mathematical and Statistical Psychology, 73, 145–179.
40.
YuX. F.ChengY. (2024). An iterative two-step online item calibration method in CD-CAT, Behavior Research Methods, 56, 233–257.
41.
YuX. F.DingS. L.QinC. Y.LuY. N. (2011). Application of Bayesian networks to identify hierarchical relation among attributes in cognitive diagnosis. Acta Psychologica Sinica, 43(3), 338–346.
42.
YuX. F.MaY. F.LuoZ. S.QinC. Y. (2021). The attribute hierarchical structure learning based on K2 algorithm. Journal of Jiangxi Normal University (Natural Science), 45(4), 376–383.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
