Sage Journals: Discover world-class research

Abstract

At present, there are only a limited number of studies examining how to optimally construct cognitive diagnostic tests. The cognitive diagnostic index (CDI) and attribute-level discrimination index (ADI) have been proposed to assemble such tests. The CDI and ADI have been shown to be instrumental in constructing cognitive diagnostic tests when the attribute relationships are assumed to be nonhierarchical. For greater generality when designing cognitive diagnostic assessment, attribute hierarchy and the ratio of test length to the number of attributes (RTA) are two important factors to be considered. This article proposes modified indices that take into account attribute hierarchy and RTA. Simulation studies show that, under the deterministic input, noisy, “and” gate model (DINA) and the reduced version of the reparameterized unified model (rRUM), the proposed indices provide higher attribute and attribute pattern correct classification rates than the original indices.

Keywords

cognitive diagnosis test construction attribute hierarchy cognitive diagnosis index attribute-level discrimination index

Get full access to this article

View all access options for this article.

References

Chang

Ying

(1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20, 213-229.

Cheng

(2010). Improving cognitive diagnosis computerized adaptive testing by balancing attribute coverage: The modified maximum global discrimination index method. Educational and Psychological Measurement, 70, 902-913.

de la Torre

(2011). The generalized DINA model framework. Psychometrika, 76, 179-199.

de la Torre

Douglas

(2004). Higher-order latent trait models for cognitive diagnosis. Psychometrika, 69, 333-353.

de la Torre

Douglas

(2008). Model evaluation and multiple strategies in cognitive diagnosis: An analysis of fraction subtraction data. Psychometrika, 73, 595-624.

de la Torre

Hong

Deng

(2010). Factors affecting the item parameter estimation and classification accuracy of the DINA model. Journal of Educational Measurement, 47, 227-249.

Finkelman

Kim

Roussos

L. A.

(2009). Automated test assembly for cognitive diagnosis models using a genetic algorithm. Journal of Educational Measurement, 46, 273-292.

Finkelman

Kim

Roussos

Verschoor

(2010). A binary programming approach to automated test assembly for cognitive diagnosis models. Applied Psychological Measurement, 34, 310-326.

Haertel

E. H.

(1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26, 333-352.

10.

Henson

R. A.

Douglas

(2005). Test construction for cognitive diagnostics. Applied Psychological Measurement, 29, 262-277.

11.

Henson

R. A.

Roussos

Douglas

(2008). Cognitive diagnostic attribute-level discrimination indices. Applied Psychological Measurement, 32, 275-288.

12.

Henson

R. A.

Templin

J. L.

Willse

J. T.

(2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191-210.

13.

Junker

B. W.

Sijtsma

(2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258-272.

14.

Kuo

B. C.

H. M.

Shih

S. C.

(2012, July). The validity of Q-matrix design for DINA model: A practical perspective. Paper presented at the 77th Annual Meeting of the Psychometric Society, Lincoln, NE.

15.

Leighton

J. P.

Gierl

M. J.

Hunka

(2004). The attribute hierarchy model: An approach for integrating cognitive theory with assessment practice. Journal of Educational Measurement, 41, 205-236.

16.

Liu

J. C.

(2013, July). The self-learning Q-matrix—Theory and applications. Paper presented at the 78th Annual Meeting of the Psychometric Society, Arnhem, The Netherlands.

17.

Maris

(1999). Estimating multiple classification latent class models. Psychometrika, 64, 87-212.

18.

Pai

H. S.

Kuo

B. C.

Chen

C. H.

(2012, July). Cognitive Diagnostic Indices with pattern and attribute level adjustment. Paper presented at the 77th Annual Meeting of the Psychometric Society, Lincoln, NE.

19.

Roussos

L. A.

DiBello

L. V.

Stout

Hartz

S. M.

Henson

R. A.

Templin

J. L.

(2007). The fusion model skills diagnosis system. In Leighton

J. P.

Gierl

M. J.

(Eds.), Cognitive diagnostic assessment for education: Theory and applications (pp. 275-318). Cambridge, UK: Cambridge University Press.

20.

Tatsuoka

K. K.

(1983). Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20, 345-354.

21.

von Davier

(2008). A general diagnosis model applied to language testing data. British Journal of Mathematical and Statistical Psychology, 61, 287-307.

Modified Cognitive Diagnostic Index and Modified Attribute-Level Discrimination Index for Test Construction

Abstract

Keywords

Get full access to this article

References