Wald’s (1947) sequential probability ratio test can be implemented as an adaptive test for classifying examinees into categories. However, current implementations use an item selection method that is either random or based on Fisher information (FI), a criterion related to optimized examinee trait estimates. In this study, a method based on Kullback-Leibler information (KLI) was evaluated. Simulation studies were conducted for two- and three-category classifications in which item selection methods based on FI and KLI were compared. Results showed that testing algorithms using KLI-based item selection performed better than or as well as those using FI-based item selection.
Get full access to this article
View all access options for this article.
References
1.
Armitage, P. (1950). Sequential analysis with more than two alternative hypotheses, and its relation to discriminant function analysis. Journal of the Royal Statistical Society, B, 12, 137–144.
2.
Chang, H.-H. , & Ying, Z. (1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20, 213–229.
3.
Cover, T. M. , & Thomas, J. A. (1991). Elements of information theory. New York: Wiley.
4.
Eggen, T. J. H. M , & Straetmans, G. J. J. M. (1996). Computerized adaptive testing for classifying examinees into three categories (Measurement and Research Department Rep. 96-3). Arnhem, The Netherlands: Cito.
5.
Govindarajulu, Z. (1987). The sequential statistical analysis of hypothesis testing, point and interval estimation, and decision theory. Columbus OH: American Sciences Press.
6.
Kingsbury, G. G. , & Zara, A. R. (1991). A comparison of procedures for content-sensitive item selection in computerized adaptive tests. Applied Measurement in Education, 4, 241–261.
7.
Lewis, C. , & Sheenan, K. (1990). Using Bayesian decision theory to design a computerized mastery test. Applied Psychological Measurement, 14, 376–386.
8.
Reckase, M. D. (1983). A procedure for decision making using tailored testing. In D. J. Weiss (Ed.), New horizons in testing: Latent trait test theory and computerized adaptive testing (pp. 237–255). New York: Academic Press.
9.
Sobel, M. , & Wald, A. (1949). A sequential decision procedure for choosing one of three hypotheses concerning the unknown mean of a normal distribution. Annals of Mathematical Statistics, 20, 502–522.
10.
Spray, J. A. (1993). Multiple-category classification using a sequential probability ratio test (Research report 93-7). Iowa City IA: ACT, Inc.
11.
Spray, J. A. , & Reckase, M. D. (1994, April). The selection of test items for decision making with a computer adaptive test. Paper presented at the annual meeting of the National Council on Measurement in Education, New Orleans LA.
12.
Spray, J. A. , & Reckase, M. D. (1996). Comparison of SPRT and sequential Bayes procedures for classifying examinees into two categories using a computerized test. Journal of Educational and Behavioral Statistics, 21, 405–414.
13.
Verhelst, N. D. , Glas, C. A. W., & Verstralen, H. H. F.M. (1995). One-parameter logistic model [Computer software]. Arnhem, The Netherlands: Cito.
14.
Wald, A. (1947). Sequential analysis. New York: Wiley.
