Sage Journals: Discover world-class research

Abstract

A well-known stopping rule in adaptive mastery testing is to terminate the assessment once the examinee’s ability confidence interval lies entirely above or below the cut-off score. This article proposes new procedures that seek to improve such a variable-length stopping rule by coupling it with curtailment and stochastic curtailment. Under the new procedures, test termination can occur earlier if the probability is high enough that the current classification decision remains the same should the test continue. Computation of this probability utilizes normality of an asymptotically equivalent version of the maximum likelihood ability estimate. In two simulation sets, the new procedures showed a substantial reduction in average test length while maintaining similar classification accuracy to the original method.

Keywords

item response theory adaptive mastery testing stochastic curtailment ability confidence interval

Get full access to this article

View all access options for this article.

References

Bartroff

Finkelman

Lai

T. L.

(2008). Modern sequential analysis and its applications to computerized adaptive testing. Psychometrika, 73, 473-486.

Chang

H.-H.

Stout

(1993). The asymptotic posterior normality of the latent trait in an IRT model. Psychometrika, 58, 37-52.

Chang

H.-H.

Ying

(1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20, 213-229.

Chang

H.-H.

Ying

(2009). Nonlinear sequential designs for logistic item response theory models with applications to computerized adaptive tests. The Annals of Statistics, 37, 1466-1488.

Chang

Y.-C. I.

(2005). Application of sequential interval estimation to adaptive mastery testing. Psychometrika, 70, 685-713.

Eisenberg

Ghosh

B. K.

(1980). Curtailed and uniformly most powerful sequential tests. The Annals of Statistics, 8, 1123-1131.

Finkelman

M. D.

(2008). On using stochastic curtailment to shorten the SPRT in sequential mastery testing. Journal of Educational and Behavioral Statistics, 33, 442-463.

Finkelman

M. D.

(2010). Variations on stochastic curtailment in sequential mastery testing. Applied Psychological Measurement, 34, 27-45.

Hambleton

R. K.

Swaminathan

Rogers

H. J.

(1991). Fundamentals of item response theory. Newbury Park, CA: Sage.

10.

Huebner

A. R.

Fina

A. D.

(2014). The stochastically curtailed generalized likelihood ratio: A new termination criterion for variable-length computerized classification tests. Behavior Research Methods. Advance online publication. doi:10.3758/s13428-014-0490-y

11.

Lan

K. K. G.

Simon

Halperin

(1982). Stochastically curtailed tests in long-term clinical trials. Communications in Statistics, Part C: Sequential Analysis, 1, 207-219.

12.

Reckase

M. D.

Spray

J. A.

(1994, April). The selection of test items for decision making with a computer adaptive test. Paper presented at the annual meeting of the National Council on Measurement in Education, New Orleans, LA.

13.

Spray

J. A.

(1993). Multiple-category classification using a sequential probability ratio test (ACT Research Report Series No. 93–7). Iowa City, IA: American College Testing.

14.

Spray

J. A.

Reckase

M. D.

(1996). Comparison of SPRT and sequential Bayes procedure for classifying examinees into two categories using a computerized test. Journal of Educational and Behavioral Statistics, 21, 405-414.

15.

Thompson

N. A.

(2007). A practitioner’s guide for variable-length computerized classification testing. Practical Assessment, Research & Evaluation, 12(1). Retrieved from http://pareonline.net/getvn.asp?v=12&n=1

16.

Thompson

N. A.

(2011). Termination criteria for computerized classification testing. Practical Assessment, Research & Evaluation, 16(4). Retrieved from http://pareonline.net/getvn.asp?v=16&n=4

17.

Vos

H. J.

(2000). A Bayesian procedure in the context of sequential mastery testing. Psicolõgica, 21, 191-211.

18.

Wald

(1947). Sequential analysis. New York, NY: John Wiley.

19.

Weiss

D. J.

(Ed.). (1983). New horizons in testing: Latent trait test theory and computerized adaptive testing. New York, NY: Academic Press.

20.

Weiss

D. J.

Kingsbury

G. G.

(1984). Application of computerized adaptive testing to educational problems. Journal of Educational Measurement, 21, 361-375.

21.

Weissman

(2007). Mutual information item selection in adaptive classification testing. Educational and Psychological Measurement, 67, 41-58.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.19 MB

Stochastic Curtailment in Adaptive Mastery Testing

Abstract

Keywords

Get full access to this article

References

Supplementary Material