In practice, constructed-response items are commonly used to diagnose students’ performance using polytomous scores. Existing longitudinal cognitive diagnosis models (CDMs) primarily focus on dichotomizing the data, unsuitable for polytomous scores. This article introduces a longitudinal CDM for handling polytomous responses over time. The proposed model expands the capabilities of learning models to handle polytomous data and account for hierarchies among attributes within the CDMs. For estimation, a Gibbs formulation was proposed to estimate parameters in the measurement part, while a Metropolis-Hastings sampler was employed for the transition part. An empirical study was conducted to showcase the practical application and advantages of the proposed model. Additionally, two simulation studies demonstrated that parameters can be well recovered under various conditions.
AkbayL.de la TorreJ. (2020). Estimation approaches in cognitive diagnosis modeling when attributes are hierarchically structured. Psicothema, 32(1), 122–129.
2.
BandalosD. L. (2018). Measurement theory and applications for the social sciences. Guilford Publications.
3.
BirenbaumM.TatsuokaK. K.GutvirtzY. (1992). Effects of response format on diagnostic assessment of scholastic achievement. Applied Psychological Measurement, 16(4), 353–363.
4.
ChangH.-H.WangC.ZhangS. (2021). Statistical applications in educational measurement. Annual Review of Statistics and Its Application, 8, 439–461.
5.
ChenJ.de la TorreJ.ZhangZ. (2013). Relative and absolute fit evaluation in cognitive diagnosis modeling. Journal of Educational Measurement, 50(2), 123–140.
6.
ChenY.CulpepperS. A.WangS.DouglasJ. (2018). A hidden Markov model for learning trajectories in cognitive diagnosis with application to spatial rotation skills. Applied Psychological Measurement, 42(1), 5–23.
7.
CuiY.ZhaoG.ZhangD. (2022). Improving students’ inquiry learning in web-based environments by providing structure: Does the teacher matter or platform matter?British Journal of Educational Technology, 53(4), 1049–1068.
8.
CulpepperS. A. (2015). Bayesian estimation of the DINA model with Gibbs sampling. Journal of Educational and Behavioral Statistics, 40(5), 454–476.
9.
CulpepperS. A.BalamutaJ. J. (2021). Inferring latent structure in polytomous data with a higher-order diagnostic model. Multivariate Behavioral Research, 58(2), 368–386.
10.
FischerG. H. (1995). Some neglected problems in IRT. Psychometrika, 60(4), 459–487.
11.
GaoX.MaW.WangD.CaiY.TuD. (2021). A class of cognitive diagnosis models for polytomous data. Journal of Educational and Behavioral Statistics, 46(3), 297–322.
12.
GelmanA.HwangJ.VehtariA. (2014). Understanding predictive information criteria for Bayesian models. Statistics and Computing, 24, 997–1016.
13.
GuY. (2023). Generic identifiability of the DINA model and blessing of latent dependence. Psychometrika, 88(1), 117–131.
14.
GuY.XuG. (2020). Partial identifiability of restricted latent class models. The Annals of Statistics, 48(4), 2082–2107.
15.
GuoJ.XuX.TaoJ.ShiN. (2019). A method of q-matrix validation based symmetrised Kullback–Leibler divergence for the DINA model. Statistics and Its Interface, 12(4), 537–547.
16.
HansenM. P. (2013). Hierarchical item response models for cognitive diagnosis. University of California, Los Angeles.
17.
HelmC.WarwasJ.SchirmerH. (2022). Cognitive diagnosis models of students’ skill profiles as a basis for adaptive teaching: An example from introductory accounting classes. Empirical Research in Vocational Education and Training, 14(1), 1–30.
18.
IaconangeloC. J.PhelpsG.GitomerD. H. (2022). Dimensionality and validity of the content knowledge for teaching construct using cognitive diagnostic modeling and known groups comparisons. Teaching and Teacher Education, 114, 103690.
19.
JiaoH.LiuJ.HaynieK.WooA.GorhamJ. (2012). Comparison between dichotomous and polytomous scoring of innovative items in a large-scale computerized adaptive test. Educational and Psychological Measurement, 72(3), 493–509.
20.
JunkerB. W.SijtsmaK. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25(3), 258–272.
21.
KarelitzT. M. (2004). Ordered category attribute coding framework for cognitive assessments. University of Illinois at Urbana-Champaign.
22.
KayaY.LeiteW. L. (2017). Assessing change in latent skills across time with longitudinal cognitive diagnosis modeling: An evaluation of model performance. Educational and Psychological Measurement, 77(3), 369–388.
23.
LeightonJ.GierlM. (2007). Cognitive diagnostic assessment for education: Theory and applications. Cambridge University Press.
24.
LeightonJ. P.GierlM. J.HunkaS. M. (2004). The attribute hierarchy method for cognitive assessment: A variation on Tatsuoka’s rule-space approach. Journal of Educational Measurement, 41(3), 205–237.
25.
LewisE. L.SternJ. L.LinnM. C. (1993). The effect of computer simulations on introductory thermodynamics understanding. Educational Technology, 33(1), 45–58.
26.
LiF.CohenA.BottgeB.TemplinJ. (2016). A latent transition analysis model for assessing change in cognitive skills. Educational and Psychological Measurement, 76(2), 181–204.
27.
LiangQ.la TorreJ. d.LawN. (2023). Latent transition cognitive diagnosis model with covariates: A three-step approach. Journal of Educational and Behavioral Statistics, 48(6), 690–718.
28.
LiuO. L.LeeH.-S.HofstetterC.LinnM. C. (2008). Assessing knowledge integration in science: Construct, measures, and evidence. Educational Assessment, 13(1), 33–55.
29.
LiuO. L.RyooK.LinnM. C.SatoE.SvihlaV. (2015). Measuring knowledge integration learning of energy topics: A two-year longitudinal study. International Journal of Science Education, 37(7), 1044–1066.
30.
LiuY.ZhangT.WangX.YuG.LiT. (2023). New development of cognitive diagnosis models. Frontiers of Computer Science, 17(1), 1–13.
31.
MaC.OuyangJ.XuG. (2022). Learning latent and hierarchical structures in cognitive diagnosis models. Psychometrika, 88(1), 175–207.
32.
MaW.de la TorreJ. (2016). A sequential cognitive diagnosis model for polytomous responses. British Journal of Mathematical and Statistical Psychology, 69(3), 253–275.
33.
MadisonM. J. (2019). Reliably assessing growth with longitudinal diagnostic classification models. Educational Measurement: Issues and Practice, 38(2), 68–78.
34.
MadisonM. J.BradshawL. P. (2018). Assessing growth in a diagnostic classification model framework. Psychometrika, 83(4), 963–990.
35.
PanQ.QinL.KingstonN. (2020). Growth modeling in a diagnostic classification model (DCM) framework–a multivariate longitudinal diagnostic classification model. Frontiers in Psychology, 11, 1714.
36.
PearsallN. R.SkipperJ. E. J.MintzesJ. J. (1997). Knowledge restructuring in the life sciences: A longitudinal study of conceptual change in biology. Science Education, 81(2), 193–215.
37.
Pei-TaD.Yan-FangP.Fei-MingL. (2021). A review of longitudinal cognitive diagnosis models for learning tracking. Journal of Psychological Science, 44(1), 214–222.
38.
SessomsJ.HensonR. A. (2018). Applications of diagnostic classification models: A literature review and critical commentary. Measurement: Interdisciplinary Research and Perspectives, 16(1), 1–17.
39.
TatsuokaK. K. (1983). Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20(4), 345–354.
40.
TemplinJ. L.HensonR. A. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods, 11(3), 287.
41.
TuD.ZhengC.CaiY.GaoX.WangD. (2018). A polytomous model of cognitive diagnostic assessment for graded data. International Journal of Testing, 18(3), 231–252.
Von DavierM.XuX.CarstensenC. H. (2011). Measuring growth in a longitudinal large-scale assessment with a general latent variable model. Psychometrika, 76, 318–336.
44.
WangS.YangY.CulpepperS. A.DouglasJ. A. (2018). Tracking skill acquisition with cognitive diagnosis models: A higher-order, hidden Markov Model with covariates. Journal of Educational and Behavioral Statistics, 43(1), 57–87.
45.
XuX.de la TorreJ.ZhangJ.GuoJ.ShiN. (2020). Estimating CDMS using the slice-within-Gibbs sampler. Frontiers in psychology, 11, 2260.
46.
XuX.FangG.GuoJ.YingZ.ZhangS. (2024). Diagnostic classification models for testlets: Methods and theory. Psychometrika, 89, 851–876.
47.
XuX.RenS.ZhangD. (2023). Knowledge integration in science learning: Tracking students’ knowledge development and skill acquisition with cognitive diagnosis models. Educational Measurement: Issues and Practice, 43(1), 66–82.
48.
YigitH. D.DouglasJ. A. (2021). First-order learning models with the gdina: Estimation with the em algorithm and applications. Applied Psychological Measurement, 45(3), 143–158.
49.
ZhanP. (2020). A markov estimation strategy for longitudinal learning diagnosis: Providing timely diagnostic feedback. Educational and Psychological Measurement, 80(6), 1145–1167.
50.
ZhanP. (2021). Refined learning tracking with a longitudinal probabilistic diagnostic model. Educational Measurement: Issues and Practice, 40(1), 44–58.
51.
ZhanP.HeK. (2021). A longitudinal diagnostic model with hierarchical learning trajectories. Educational Measurement: Issues and Practice, 40(3), 18–30.
52.
ZhanP.JiaoH.LiaoD. (2018). Cognitive diagnosis modelling incorporating item response times. British Journal of Mathematical and Statistical Psychology, 71(2), 262–286.
53.
ZhanP.JiaoH.LiaoD.LiF. (2019). A longitudinal higher-order diagnostic classification model. Journal of Educational and Behavioral Statistics, 44(3), 251–281.
54.
ZhanP.LiF.JiaoH. (2021). Cognitive diagnostic assessment for learning. Frontiers in Psychology, 12, 806636.
55.
ZhanP.WangW.-C.LiX. (2020). A partial mastery, higher-order latent structural model for polytomous attributes in cognitive diagnostic assessments. Journal of Classification, 37(2), 328–351.
56.
ZhangS.ChangH.-H. (2020). A multilevel logistic hidden markov model for learning under cognitive diagnosis. Behavior Research Methods, 52, 408–421.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
