Sage Journals: Discover world-class research

Abstract

Multidimensional computerized adaptive testing (MCAT) has been developed over the past decades, and most of them can only deal with dichotomously scored items. However, polytomously scored items have been broadly used in a variety of tests for their advantages of providing more information and testing complicated abilities and skills. The purpose of this study is to discuss the item selection algorithms used in MCAT with polytomously scored items (PMCAT). Several promising item selection algorithms used in MCAT are extended to PMCAT, and two new item selection methods are proposed to improve the existing selection strategies. Two simulation studies are conducted to demonstrate the feasibility of the extended and proposed methods. The simulation results show that most of the extended item selection methods for PMCAT are feasible and the new proposed item selection methods perform well. Combined with the security of the pool, when two dimensions are considered (Study 1), the proposed modified continuous entropy method (MCEM) is the ideal of all in that it gains the lowest item exposure rate and has a relatively high accuracy. As for high dimensions (Study 2), results show that mutual information (MUI) and MCEM keep relatively high estimation accuracy, and the item exposure rates decrease as the correlation increases.

Keywords

multidimensional computerized adaptive test multidimensional graded response model Fisher information Kullback–Leibler information polytomous items

Get full access to this article

View all access options for this article.

References

Chang

H. H.

Ying

(1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20, 213-229. doi:10.1177/014662169602000303

Chang

H. H.

Ying

(1999). A-stratified multistage computerized adaptive testing. Applied Psychological Measurement, 23, 211-222. doi:10.1177/01466219922031338

Chen

S. Y.

Ankenmann

R. D.

Spray

J. A.

(2003). The relationship between item exposure and test overlap in computerized adaptive testing. Journal of Educational Measurement, 40, 129-145. doi:10.1111/j.1745-3984.2003.tb01100.x

De Ayala

R. J

. (1992). The nominal response model in computerized adaptive testing. Applied Psychological Measurement, 16, 327-343. doi:10.1177/014662169201600403

Donoghue

J. R.

(1994). An empirical examination of the IRT information function of polytomously scored reading items under the generalized partial credit model. Journal of Educational Measurement, 31, 295-311.

Finkelman

Nering

M. L.

Roussos

L. A.

(2009). A conditional exposure control method for multidimensional adaptive testing. Journal of Educational Measurement, 46, 84-103. doi:10.1111/j.1745-3984.2009.01070.x

Flens

Smits

Terwee

C. B.

Dekker

Huijbrechts

de Beurs

(2017). Development of a Computer Adaptive Test for depression based on the Dutch-Flemish Version of the PROMIS Item Bank. Evaluation & the Health Professions, 40, 79-105. doi:10.1177/0163278716684168

Gardner

Shear

Kelleher

K. J.

Pajer

K. A.

Mammen

Buysse

Frank

(2004). Computerized adaptive measurement of depression: A simulation study. BMC Psychiatry, 4, Article 13. doi:10.1186/1471-244X-4-13

Glas

C. A. W.

(1992). A Rasch model with a multivariate distribution of ability. In Objective measurement: Theory into practice (Vol. 1, pp. 236-258). Norwood, NJ: Ablex.

10.

Lin

(2012). Item selection methods in multidimensional computerized adaptive testing adopting polytomously-scored items under multidimensional generalized partial credit model (Doctoral dissertation). University of Illinois at Urbana–Champaign.

11.

Luecht

R. M.

(1996). Multidimensional computerized adaptive testing in a certification or licensure context. Applied Psychological Measurement, 20, 389-404. doi:10.1177/014662169602000406

12.

Mulder

van der Linden

W. J.

(2009). Multidimensional adaptive testing with optimal design criteria for item selection. Psychometrika, 74, 273-296. doi:10.1007/s11336-008-9097-5

13.

Mulder

van der Linden

W. J.

(2010). Multidimensional adaptive testing with Kullback-Leibler information item selection. In van der Linden

W. J.

Glas

C. A. W.

(Eds.), Elements of adaptive testing, statistics for social and behavioral sciences (pp. 77-101). doi:10.1007/978-0-387-85461-8

14.

Muraki

Carlson

J. E.

(1995). Full-information factor analysis for polytomous item responses. Applied Psychological Measurement, 19, 73-90. doi:10.1177/014662169501900109

15.

Reckase

M. D.

(2009). Multidimensional item response theory. New York. NY: Springer. doi:10.1007/978-0-387-89976-3

16.

Robbins

(1985). An empirical Bayes estimation problem. In Lai

T. L.

Siegmund

(Eds.), Herbert Robbins selected papers (pp. 72-73). New York, NY: Springer. doi:10.1007/978-1-4612-5110-1_6

17.

Samejima

(1976). Graded response model of the latent trait theory and tailored testing. In Clark

C. K.

(Ed.), Proceedings of the first Conference on Computerized Adaptive Testing (pp. 5-17). Washington, DC: U.S. Government Printing Office.

18.

Segall

D. O.

(1996). Multidimensional adaptive testing. Psychometrika, 61, 331-354. doi:10.1007/BF02294343

19.

Smits

Cuijpers

van Straten

(2011). Applying computerized adaptive testing to the CES-D scale: A simulation study. Psychiatry Research, 188, 147-155. doi:10.1016/j.psychres.2010.12.001

20.

van der Linden

W. J

. (1999). Multidimensional adaptive testing with a minimum error-variance criterion. Journal of Educational and Behavioral Statistics, 24, 398-412. doi:10.3102/10769986024004398

21.

Veldkamp

B. P.

van der Linden

W. J.

(2002). Multidimensional adaptive testing with constraints on test content. Psychometrika, 67, 575-588. doi:10.1007/BF02295132

22.

Wang

Chang

H. H.

(2011). Item selection in multidimensional computerized adaptive testing-gaining information from different angles. Psychometrika, 76, 363-384. doi:10.1007/s11336-011-9215-7

23.

Wang

Chang

H. H.

Boughton

K. A.

(2011). Kullback-Leibler information and its applications in multidimensional adaptive testing. Psychometrika, 76, 13-39. doi:10.1007/s11336-010-9186-0

24.

Yao

Schwarz

R. D.

(2006). A multidimensional partial credit model with associated item and test statistics: An application to mixed-format tests. Applied Psychological Measurement, 30, 469-492. doi:10.1177/0146621605284537

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.07 MB

Item Selection Methods in Multidimensional Computerized Adaptive Testing With Polytomously Scored Items

Abstract

Keywords

Get full access to this article

References

Supplementary Material