This article proposes a new item selection method, namely, the modified maximum global discrimination index (MMGDI) method, for cognitive diagnostic computerized adaptive testing (CD-CAT). The new method captures two aspects of the appeal of an item: (a) the amount of contribution it can make toward adequate coverage of every attribute and (b) the amount of contribution it can make toward recovering the latent cognitive profile. A simulation study shows that the new method ensures adequate coverage of every attribute, which improves the validity of the test scores, and defensibility of the proposed uses of the test. Furthermore, compared with the original global discrimination index method, the MMGDI method improves the recovery rate of each attribute and of the entire cognitive profile, especially the latter. Therefore, the new method improves both the validity and reliability of the test scores from a CD-CAT program.
Chang, H., & Ying, Z. ( 1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20, 213-229.
2.
Chen, S., & Ankenmann, R.D. (2004). Effects of practical constraints on item selection rules at the early stages of computerized adaptive testing . Journal of Educational Measurement, 41, 149-174.
3.
Cheng, Y. ( 2009). When cognitive diagnosis meets computerized adaptive testing: CD-CAT. Psychometrika, 74, 619-632.
Cover, T.M., & Thomas. J.A. ( 1991). Elements of information theory. New York, NY: John Wiley.
6.
Doignon, J. P., & Falmagne, J. C. (Eds.). (1999). Knowledge spaces. New York, NY: Springer.
7.
Haertel, E.H. ( 1984). An application of latent class models to assessment data . Applied Psychological Measurement, 8, 333-346.
8.
Haertel, E.H. ( 1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement , 26, 333-352.
9.
Haertel, E.H., & Wiley, D.E. ( 1993). Presentations of ability structures: Implications for testing. In N. Frederiksen, R. J. Mislevey, & I. I. Bejar (Eds.), Test theory for a new generation of tests (pp. 359-384). Hillsdale, NJ: Erlbaum.
10.
Hambleton, R.K. ( 2005). Preface. In W. J. van der Linden (Ed.), Linear models to optimal test design (p. xi). New York, NY: Springer.
11.
Hartz, S. ( 2002). A Bayesian framework for the unified model for assessing cognitive abilities: Blending theory with practice (Unpublished doctoral thesis) , University of Illinois at Urbana-Champaign, Champaign.
12.
Hartz, S., Roussos, L., & Stout, W. ( 2002). Skill diagnosis: Theory and practice [Computer software user manual for Arpeggio software]. Princeton, NJ: Educational Testing Service.
13.
Junker, B.W., & Sijtsma, K. ( 2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258-272.
14.
Kingsbury, G.G., & Zara, A.R. ( 1991). A comparison of procedures for content-sensitive item selection in computerized adaptive tests. Applied Measurement in Education, 4, 241-261.
15.
Lord, F.M. ( 1980). Applications of item response theory to practical testing problems. Mahwah, NJ: Erlbaum.
16.
Macready, G.B., & Dayton, C.M. ( 1977). The use of probablistic models in the assessment of mastery . Journal of Educational Statistics, 33, 379-416.
17.
Maris, E. ( 1999). Estimating multiple classification latent class models . Psychometrika, 64, 187-212.
18.
McGlohen, M., & Chang, H. ( 2008). Combining computer adaptive testing technology with cognitively diagnostic assessment. Behavior Research Methods, 40, 808-821.
19.
Mislevy, R.J., Almond, R.G., Yan, D., & Steinberg, L.S. (1999). Bayes nets in educational assessment: Where do the numbers come from? In K. B. Laskey & H. Prade (Eds.), Proceedings of the fifteenth conference on uncertainty in artificial intelligence (pp. 437-446). San Francisco, CA: Morgan Kaufmann.
20.
No Child Left Behind Act of 2001, Pub. L. No. 1-7-110 (2001).
21.
Tatsuoka, K.K. ( 1983). Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement , 20, 345-354.
22.
Tatsuoka, K.K. ( 1995). Architecture of knowledge structures and cognitive diagnosis: A statistical pattern recognition and classification approach. In R. D. Nichols, S. F. Chipman , & R. L. Brennan (Eds.), Cognitively diagnostic assessment (pp. 327-359). Hillsdale, NJ: Erlbaum.
23.
Thissen, D., & Mislevy, R.J. ( 2000). Testing algorithms. In H. Wainer (Ed.), Computerized adaptive testing: A primer (pp. 101-133). Hillsdale, NJ: Erlbaum.
24.
Weiss, D.J. ( 1982). Improving measurement quality and efficiency with adaptive testing. Applied Psychological Measurement, 6, 473-492.
25.
Weissman, A. ( 2007). Mutual information item selection in adaptive classification testing. Educational and Psychological Measurement, 67, 41-58.
26.
Xu, X., Chang, H., & Douglas, J. ( 2003). A simulation study to compare CAT strategies for cognitive diagnosis. Paper presented at the annual meeting of National Council on Measurement in Education, Montreal, Quebec, Canada.
27.
Xu, X., & Douglas, J. ( 2006). Computerized adaptive testing under nonparametric IRT models. Psychometrika, 71, 121-137.
