A Polytomous Extension of the Higher-Order,Hidden Markov Model With Covariates and Hierarchical Learning Trajectories

Abstract

In practice, constructed-response items are commonly used to diagnose students’ performance using polytomous scores. Existing longitudinal cognitive diagnosis models (CDMs) primarily focus on dichotomizing the data, unsuitable for polytomous scores. This article introduces a longitudinal CDM for handling polytomous responses over time. The proposed model expands the capabilities of learning models to handle polytomous data and account for hierarchies among attributes within the CDMs. For estimation, a Gibbs formulation was proposed to estimate parameters in the measurement part, while a Metropolis-Hastings sampler was employed for the transition part. An empirical study was conducted to showcase the practical application and advantages of the proposed model. Additionally, two simulation studies demonstrated that parameters can be well recovered under various conditions.

Keywords

cognitive diagnosis models a hidden Markov model polytomous responses tracking skill acquisition hierarchical structure

Get full access to this article

View all access options for this article.

References

Akbay

de la Torre

(2020). Estimation approaches in cognitive diagnosis modeling when attributes are hierarchically structured. Psicothema, 32(1), 122–129.

Bandalos

D. L.

(2018). Measurement theory and applications for the social sciences. Guilford Publications.

Birenbaum

Tatsuoka

K. K.

Gutvirtz

(1992). Effects of response format on diagnostic assessment of scholastic achievement. Applied Psychological Measurement, 16(4), 353–363.

Chang

H.-H.

Wang

Zhang

(2021). Statistical applications in educational measurement. Annual Review of Statistics and Its Application, 8, 439–461.

Chen

de la Torre

Zhang

(2013). Relative and absolute fit evaluation in cognitive diagnosis modeling. Journal of Educational Measurement, 50(2), 123–140.

Chen

Culpepper

S. A.

Wang

Douglas

(2018). A hidden Markov model for learning trajectories in cognitive diagnosis with application to spatial rotation skills. Applied Psychological Measurement, 42(1), 5–23.

Cui

Zhao

Zhang

(2022). Improving students’ inquiry learning in web-based environments by providing structure: Does the teacher matter or platform matter? British Journal of Educational Technology, 53(4), 1049–1068.

Culpepper

S. A.

(2015). Bayesian estimation of the DINA model with Gibbs sampling. Journal of Educational and Behavioral Statistics, 40(5), 454–476.

Culpepper

S. A.

Balamuta

J. J.

(2021). Inferring latent structure in polytomous data with a higher-order diagnostic model. Multivariate Behavioral Research, 58(2), 368–386.

10.

Fischer

G. H.

(1995). Some neglected problems in IRT. Psychometrika, 60(4), 459–487.

11.

Gao

Wang

Cai

(2021). A class of cognitive diagnosis models for polytomous data. Journal of Educational and Behavioral Statistics, 46(3), 297–322.

12.

Gelman

Hwang

Vehtari

(2014). Understanding predictive information criteria for Bayesian models. Statistics and Computing, 24, 997–1016.

13.

(2023). Generic identifiability of the DINA model and blessing of latent dependence. Psychometrika, 88(1), 117–131.

14.

(2020). Partial identifiability of restricted latent class models. The Annals of Statistics, 48(4), 2082–2107.

15.

Guo

Tao

Shi

(2019). A method of q-matrix validation based symmetrised Kullback–Leibler divergence for the DINA model. Statistics and Its Interface, 12(4), 537–547.

16.

Hansen

M. P.

(2013). Hierarchical item response models for cognitive diagnosis. University of California, Los Angeles.

17.

Helm

Warwas

Schirmer

(2022). Cognitive diagnosis models of students’ skill profiles as a basis for adaptive teaching: An example from introductory accounting classes. Empirical Research in Vocational Education and Training, 14(1), 1–30.

18.

Iaconangelo

C. J.

Phelps

Gitomer

D. H.

(2022). Dimensionality and validity of the content knowledge for teaching construct using cognitive diagnostic modeling and known groups comparisons. Teaching and Teacher Education, 114, 103690.

19.

Jiao

Liu

Haynie

Woo

Gorham

(2012). Comparison between dichotomous and polytomous scoring of innovative items in a large-scale computerized adaptive test. Educational and Psychological Measurement, 72(3), 493–509.

20.

Junker

B. W.

Sijtsma

(2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25(3), 258–272.

21.

Karelitz

T. M.

(2004). Ordered category attribute coding framework for cognitive assessments. University of Illinois at Urbana-Champaign.

22.

Kaya

Leite

W. L.

(2017). Assessing change in latent skills across time with longitudinal cognitive diagnosis modeling: An evaluation of model performance. Educational and Psychological Measurement, 77(3), 369–388.

23.

Leighton

Gierl

(2007). Cognitive diagnostic assessment for education: Theory and applications. Cambridge University Press.

24.

Leighton

J. P.

Gierl

M. J.

Hunka

S. M.

(2004). The attribute hierarchy method for cognitive assessment: A variation on Tatsuoka’s rule-space approach. Journal of Educational Measurement, 41(3), 205–237.

25.

Lewis

E. L.

Stern

J. L.

Linn

M. C.

(1993). The effect of computer simulations on introductory thermodynamics understanding. Educational Technology, 33(1), 45–58.

26.

Cohen

Bottge

Templin

(2016). A latent transition analysis model for assessing change in cognitive skills. Educational and Psychological Measurement, 76(2), 181–204.

27.

Liang

la Torre

J. d.

Law

(2023). Latent transition cognitive diagnosis model with covariates: A three-step approach. Journal of Educational and Behavioral Statistics, 48(6), 690–718.

28.

Liu

O. L.

Lee

H.-S.

Hofstetter

Linn

M. C.

(2008). Assessing knowledge integration in science: Construct, measures, and evidence. Educational Assessment, 13(1), 33–55.

29.

Liu

O. L.

Ryoo

Linn

M. C.

Sato

Svihla

(2015). Measuring knowledge integration learning of energy topics: A two-year longitudinal study. International Journal of Science Education, 37(7), 1044–1066.

30.

Liu

Zhang

Wang

(2023). New development of cognitive diagnosis models. Frontiers of Computer Science, 17(1), 1–13.

31.

Ouyang

(2022). Learning latent and hierarchical structures in cognitive diagnosis models. Psychometrika, 88(1), 175–207.

32.

de la Torre

(2016). A sequential cognitive diagnosis model for polytomous responses. British Journal of Mathematical and Statistical Psychology, 69(3), 253–275.

33.

Madison

M. J.

(2019). Reliably assessing growth with longitudinal diagnostic classification models. Educational Measurement: Issues and Practice, 38(2), 68–78.

34.

Madison

M. J.

Bradshaw

L. P.

(2018). Assessing growth in a diagnostic classification model framework. Psychometrika, 83(4), 963–990.

35.

Pan

Qin

Kingston

(2020). Growth modeling in a diagnostic classification model (DCM) framework–a multivariate longitudinal diagnostic classification model. Frontiers in Psychology, 11, 1714.

36.

Pearsall

N. R.

Skipper

J. E. J.

Mintzes

J. J.

(1997). Knowledge restructuring in the life sciences: A longitudinal study of conceptual change in biology. Science Education, 81(2), 193–215.

37.

Pei-Ta

Yan-Fang

Fei-Ming

(2021). A review of longitudinal cognitive diagnosis models for learning tracking. Journal of Psychological Science, 44(1), 214–222.

38.

Sessoms

Henson

R. A.

(2018). Applications of diagnostic classification models: A literature review and critical commentary. Measurement: Interdisciplinary Research and Perspectives, 16(1), 1–17.

39.

Tatsuoka

K. K.

(1983). Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20(4), 345–354.

40.

Templin

J. L.

Henson

R. A.

(2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods, 11(3), 287.

41.

Zheng

Cai

Gao

Wang

(2018). A polytomous model of cognitive diagnostic assessment for graded data. International Journal of Testing, 18(3), 231–252.

42.

D.-B.

Cai

Dai Hai-Qi

(2010). A polytomous cognitive diagnosis model: P-dina model. Acta Psychologica Sinica, 42(10), 1011.

43.

Von Davier

Carstensen

C. H.

(2011). Measuring growth in a longitudinal large-scale assessment with a general latent variable model. Psychometrika, 76, 318–336.

44.

Wang

Yang

Culpepper

S. A.

Douglas

J. A.

(2018). Tracking skill acquisition with cognitive diagnosis models: A higher-order, hidden Markov Model with covariates. Journal of Educational and Behavioral Statistics, 43(1), 57–87.

45.

de la Torre

Zhang

Guo

Shi

(2020). Estimating CDMS using the slice-within-Gibbs sampler. Frontiers in psychology, 11, 2260.

46.

Fang

Guo

Ying

Zhang

(2024). Diagnostic classification models for testlets: Methods and theory. Psychometrika, 89, 851–876.

47.

Ren

Zhang

(2023). Knowledge integration in science learning: Tracking students’ knowledge development and skill acquisition with cognitive diagnosis models. Educational Measurement: Issues and Practice, 43(1), 66–82.

48.

Yigit

H. D.

Douglas

J. A.

(2021). First-order learning models with the gdina: Estimation with the em algorithm and applications. Applied Psychological Measurement, 45(3), 143–158.

49.

Zhan

(2020). A markov estimation strategy for longitudinal learning diagnosis: Providing timely diagnostic feedback. Educational and Psychological Measurement, 80(6), 1145–1167.

50.

Zhan

(2021). Refined learning tracking with a longitudinal probabilistic diagnostic model. Educational Measurement: Issues and Practice, 40(1), 44–58.

51.

Zhan

(2021). A longitudinal diagnostic model with hierarchical learning trajectories. Educational Measurement: Issues and Practice, 40(3), 18–30.

52.

Zhan

Jiao

Liao

(2018). Cognitive diagnosis modelling incorporating item response times. British Journal of Mathematical and Statistical Psychology, 71(2), 262–286.

53.

Zhan

Jiao

Liao

(2019). A longitudinal higher-order diagnostic classification model. Journal of Educational and Behavioral Statistics, 44(3), 251–281.

54.

Zhan

Jiao

(2021). Cognitive diagnostic assessment for learning. Frontiers in Psychology, 12, 806636.

55.

Zhan

Wang

W.-C.

(2020). A partial mastery, higher-order latent structural model for polytomous attributes in cognitive diagnostic assessments. Journal of Classification, 37(2), 328–351.

56.

Zhang

Chang

H.-H.

(2020). A multilevel logistic hidden markov model for learning under cognitive diagnosis. Behavior Research Methods, 52, 408–421.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.94 MB