Sage Journals: Discover world-class research

Abstract

The objective of this study was to explore the possibilities for using computerized adaptive testing in situations in which examinees are to be classified into one of three categories. Testing algorithms with two different statistical computation procedures are described and evaluated. The first computation procedure is based on statistical testing and the other on statistical estimation. Item selection methods based on maximum information (MI) considering content and exposure control are considered. The measurement quality of the proposed testing algorithms is reported. The results of the study are that a reduction of at least 22% in the mean number of items can be expected in a computerized adaptive test (CAT) compared to an existing paper-and-pencil placement test. Furthermore, statistical testing is a promising alternative to statistical estimation. Finally, it is concluded that imposing constraints on the MI selection strategy does not negatively affect the quality of the testing algorithms.

Get full access to this article

View all access options for this article.

References

Armitage, P. (1950). Sequential analysis with more than two alternative hypotheses, and its relation to discriminant function analysis. Journal of the Royal Statistical Society, B(12), 137-144.

Kingsbury, G. G. , & Weiss, D. J. (1983). A comparison of IRT-based adaptive mastery testing and a sequential mastery testing procedure. In D. J. Weiss (Ed.), New horizons in testing (pp. 257-286). New York: Academic Press.

Kingsbury, G. G. , & Zara, A. R. (1989). Procedures for selecting items for computerized adaptive testing. Applied Measurement in Education, 2, 359-375.

Kingsbury, G. G. , & Zara, A. R. (1991). A comparison of procedures for content-sensitive item selection in computerized adaptive tests. Applied Measurement in Education, 4, 241-261.

Lewis, C. , & Sheenan, K. (1990). Using Bayesian decision theory to design a computerized mastery test. Applied Psychological Measurement, 14, 376-386.

Lord, F. M. (1971). A theoretical study of two-stage testing. Psychometrika, 36, 227-242.

Reckase, M. D. (1983). A procedure for decision making using tailored testing. In D. J. Weiss (Ed.), New horizons in testing (pp. 237-255). New York: Academic Press.

Sobel, M. , & Wald, A. (1949). A sequential decision procedure for choosing one of three hypotheses concerning the unknown mean of a normal distribution. Annals of Mathematical Statistics, 20, 502-522.

Spray, J. A. (1993). Multiple-category classification using a sequential probability ratio test (ACT Research Report Series No. 93-7). Iowa City, IA: American College Testing.

10.

Spray, J. A. , & Reckase, M. D. (1994, April). The selection of test items for decision making with a computer adaptive test. Paper presented at the annual meeting of the National Council on Measurement in Education, New Orleans.

11.

Spray, J. A. , & Reckase, M. D. (1996). Comparison of SPRT and sequential Bayes procedures for classifying examinees into two categories using a computerized test. Journal of Educational and Behavioral Statistics, 21, 405-414.

12.

Sympson, J. B. , & Hetter, R. D. (1985). Controlling item exposure rates in computerized adaptive testing. Proceedings of 27th annual meeting of the Military Testing Association (pp. 973-977). San Diego, CA: Navy Personnel Research and Development Center.

13.

Thissen, D. , & Mislevy, R. J. (1990). Testing algorithms. In H. Wainer , N. J. Dorans , R. Flaugher , B. F. Green , R. J. Mislevy , L. Steinberg , & D. Thissen (Eds.), Computerized adaptive testing: A primer (pp. 103-134). Hillsdale, NJ: Lawrence Erlbaum.

14.

Verhelst, N. D. , & Glas, C.A.W. (1995). The one-parameter logistic model. In G. H. Fischer & I. W. Molenaar (Eds.), Rasch models: Foundations, recent developments, and applications (pp. 215-237). New York: Springer-Verlag.

15.

Verhelst, N. D. , Glas, C.A.W. , & Verstralen, H.H.F.M. (1995). One-parameter logistic model (OPLM). Arnhem, the Netherlands: Cito.

16.

Wainer, H. (1990). Introduction and history. In H. Wainer , N. J. Dorans , R. Flaugher , B. F. Green , R. J. Mislevy , L. Steinberg , & D. Thissen (Eds.), Computerized adaptive testing: A primer (pp. 1-21). Hillsdale, NJ: Lawrence Erlbaum.

17.

Wald, A. (1947). Sequential analysis. New York: John Wiley.

18.

Warm, T. A. (1989). Weighted maximum likelihood estimation of ability in item response theory. Psychometrika, 54, 427-450.

19.

Weiss, D. J. , & Kingsbury, G. G. (1984). Application of computerized adaptive testing to educational problems. Journal of Educational Measurement, 21, 361-375.

Computerized Adaptive Testing for Classifying Examinees into three Categories

Abstract

Get full access to this article

References