Cognitive diagnosis models have received much attention in the recent psychometric literature because of their potential to provide examinees with information regarding multiple fine-grained discretely defined skills, or attributes. This article discusses the issue of methods of examinee classification for cognitive diagnosis models, which are special cases of restricted latent class models. Specifically, the maximum likelihood estimation and maximum a posteriori classification methods are compared with the expected a posteriori method. A simulation study using the Deterministic Input, Noisy-And model is used to assess the classification accuracy of the methods using various criteria.
Cheng, Y. ( 2009). When cognitive diagnosis meets computerized adaptive testing: CD-CAT. Psychometrika, 74, 619-632.
2.
Cohen, J. ( 1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37-46.
3.
de la Torre, J. ( 2009). DINA model and parameter estimation: A didactic. Journal of Educational and Behavioral Statistics, 34, 115-130.
4.
de la Torre, J., & Douglas, J. ( 2004). Higher-order latent trait models for cognitive diagnosis . Psychometrika, 69, 333-353.
5.
de la Torre, J., & Douglas, J. ( 2008). Model evaluation and multiple strategies in cognitive diagnosis: An analysis of fraction subtraction data. Psychometrika , 73, 595-624.
6.
de la Torre, J., Hong, Y., & Deng, W. ( 2010). Factors affecting the item parameter estimation and classification accuracy of the DINA model. Journal of Educational Measurement , 47, 227-249.
7.
DiBello, L., Roussos, L., & Stout, W. ( 2007). Review of cognitively diagnostic assessment and a summary of psychometric models. In C. R. Rao & S. Sinharay (Eds.), Handbook of statistics 26: Psychometrics (pp. 979-1030). Amsterdam, Netherlands: Elsevier.
8.
Embretson, S., & Reise, S. ( 2000). Item response theory for psychologists. Mahwah, NJ: Erlbaum.
9.
Hartz, S. ( 2002). A Bayesian framework for the unified model for assessing cognitive abilities: Blending theory with practice (Unpublished doctoral dissertation) . University of Illinois at Urbana-Champaign, Champaign.
10.
Henson, R., & Douglas, J. ( 2005). Test construction for cognitive diagnosis. Applied Psychological Measurement, 29, 262-277.
Henson, R., Templin, J., & Willse, J. ( 2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191-210.
13.
Jang, E. ( 2008). A framework for cognitive diagnostic assessment. In C. A. Chapelle, Y.-R. Chung, & J. Xu (Eds.), Towards adaptive CALL:Natural language processing for diagnostic language assessment (pp. 117-131). Ames: Iowa State University Press.
14.
Junker, B., & Sijtsma, K. ( 2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258-272.
15.
Maris, E. ( 1999). Estimating multiple classification latent class models . Psychometrika, 64, 187-212.
16.
No Child Left Behind Act of 2001, Pub. L. No. 1-7-110 (2001).
17.
R Development Core Team. (2010). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing . Retrieved from http://www.R-project.org
18.
Rupp, A., & Templin, J. ( 2008). Unique characteristics of diagnostic classification models: A comprehensive review of the current state-of-the-art. Measurement , 6, 219-262.
19.
Tatsuoka, K. ( 1990). Toward an integration of item-response theory and cognitive error diagnosis. In N. Frederiksen, R. Glaser, A. Lesgold, & M. Safto (Eds.), Monitoring skills and knowledge acquisition (pp. 453-488). Hillsdale, NJ: Erlbaum.
20.
von Davier, M. ( 2005). A general diagnostic model applied to language testing data (ETS Research ReportRR-05-16). Princeton, NJ: Educational Testing Service.
21.
Xu, X., Chang, H., & Douglas, J. ( 2003, April). A simulation study to compare CAT strategies for cognitive diagnosis. Paper presented at the annual meeting of the American Educational Research Association, Chicago.
