Sage Journals: Discover world-class research

Abstract

The purpose of this study is to compare the level of confidence in pass/fail decisions obtained with computer adaptive tests and pencil and paper tests. Examinees took a variablelength computer adaptive test and two fixedlength pencil and paper tests. The computer adaptive test was stopped when the examinee ability estimate was either 1.3 times the standard error of measurement above or below the pass/fail point (one-tailed 90% confidence interval) or when a maximum test length was reached. Results show that greater confidence in the accuracy of the pass/fail decisions is obtained for more examinees when the computer adaptive test implements a 90% confidence stopping rule than with paper and pencil tests of comparable test length.

Get full access to this article

View all access options for this article.

References

Gershon, R. C. (1989). CATADMINISTRATOR [Computer Program]. Chicago: Micro Connections.

Green, B. F. , Bock, R. D. , Humphreys, L. G. , Linn, R. L. , & Reckase, M. D. (1984). Technical guidelines for assessing computerized adaptive tests. Journal of Educational Measurement, 21(4), 347-360.

Kingsbury, G. G. , & Houser, R. L. (1990, April). Assessing the utility of item response models: Computerized adaptive testing. Paper presented to the annual meeting of the National Council on Measurement in Education, Boston, MA.

Lord, F. M. , & Novick, M. R. (1968). Statistical theories of mental test score. Reading, MA: Addison-Wesley.

Maurelli, V. A. , & Weiss, D. J. (1983). Factors influencing the psychometric characteristics of an adaptive testing strategy for test batteries (Research Rep. No. 81-4). Minneapolis: University of Minnesota, Department of Psychology, Psychometric Methods Program, Computerized Adaptive Testing laboratory.

McKinley, R. L. , & Reckase, M. D. (1980). Computer applications to ability testing.Association for Educational Data Systems Journal, 13, 193-203.

McKinley, R. L. , & Reckase, M. D. (1984, March). Implementing an adaptive testing program in an instructional program environment. Paper presented at the meeting of the American Educational Research Association, New Orleans.

Olsen, J. B. , Maynes, D. D. , Slawson, D. , & Ho, K. (1986, April). Comparison and equating of paper-administered, computer-administered and computerized adaptive tests of achievement. Paper presented at the meeting of the American Educational Research Association, San Francisco, CA.

Reckase, M. D. (1989). Adaptive testing: The evolution of a good idea. Educational Measurement: Issues and Practice, 8, 3-3.

10.

Weiss, D. J. (1983). New horizons in testing: Latent trait test theory and computerized adaptive testing. New York: Academic Press.

11.

Weiss, D. J. (1985). Final report: Computerized adaptive measurement of achievement and ability (Project No. 150-433, N00014-79-C0172). Minneapolis, MN: University of Minnesota.

12.

Weiss, D. J. , & Kingsbury, G. G. (1984). Application of computerized adaptive testing to educational problems. Journal of Educational Measurement, 21(4), 361-375.

13.

Wright, B. D. , Linacre, J. M. , & Schultz, M. (1989). BIGSCALE [Computer Program]. Chicago: University of Chicago.

14.

Wright, B. D. , & Stone, M. H. (1979). Best test design. Chicago: MESA Press.

Confidence in Pass/Fail Decisions for Computer Adaptive and Paper and Pencil Examinations

Abstract

Get full access to this article

References