Sage Journals: Discover world-class research

Abstract

Three reliability estimates are derived for the Bayes modal estimate (BME) and the maximum likelihood estimate (MLE) of θin computerized adaptive tests (CAT). Each reliability estimate is a function of test information. Two of the estimates are shown to be upper bounds to true reliability. The three reliability estimates and the true reliabilities of both MLE and BME were computed for seven simulated CATs. Results showed that the true reliabilities for MLE and BME were nearly identical in all seven tests. The three reliability estimates never differed from the true reliabilities by more than .02 (.01 in most cases). A simple implementation of one reliability estimate was found to accurately estimate reliability in CATs.

Get full access to this article

View all access options for this article.

References

Department of Defense (1994). Technical manual for ASVAB 18-19 career exploration program. Chicago: Headquarters, U.S. Military Entrance Processing Command.

Green, B. F. , Bock, R. D. , Humphreys, L. G. , Linn, R. L. , Lord, F. M. ,& Reckase, M. D. (1983). Technical guidelines for assessing computerized adaptive tests. Journal of Educational Measurement, 21, 347–360.

Hetter, R. D. , & Sympson, J. B. (1997). Item exposure control in CAT-ASVAB. In W. A. Sands , B. K. Waters , & J. R. McBride (Eds). Computerized adaptive testing: From inquiry to operation. Washington DC: American Psychological Association.

Kim, J. K. , & Nicewander, W. A. (1993). Theta estimation for conventional tests. Psychometrika, 54, 587–599.

Lord, F. M. ,& Novick, M. R. (1968). Statistical theories of mental test scores. Reading MA: Addison-Wesley.

McBride, J. R. (1997). Technical perspectives. In W. A. Sands , B. K. Waters , & J. R. McBride (Eds). Computerized adaptive testing: From inquiry to operation. Washington DC: American Psychological Association.

Mellenberg, G. J. (1996). Measurement precision in test score and item response models. Psychological Methods, 1, 293–299.

Mislevy, R. J., & Bock, R. D. (1990). BILOG 3:Item analysis and test scoring with binary logistic models [Computer software]. Mooresville IN: Scientific Software.

Moreno, K. E. , Wetzel, C. D. , McBride, J. R. , & Weiss, D. J (1984). Relationship between corresponding Armed Services Vocational Aptitude Battery (ASVAB) and computerized adaptive testing (CAT) subtests. Applied Psychological Measurement, 8, 155–163.

10.

Rao, C. R. (1965). Linear statistical inference and its application. New York: Wiley.

11.

Samejima, F. (1994). Estimation of reliability coefficients using the test information function and its modifications. Applied Psychological Measurement, 18, 229–244.

12.

Segall, D. O. , Moreno, K. E. , & Hetter, R. D. (1997). Item pool development and evaluation. In W. A. Sands , B. K. Waters ,& J. R. McBride (Eds). Computerized adaptive testing: From inquiry to operation. Washington DC: American Psychological Association.

13.

Thissen, D. (1990). Reliability and measurement precision. In H. Wainer (Ed.), Computerized adaptive testing: A primer. Hillsdale NJ: Erlbaum.

Some Reliability Estimates for Computerized Adaptive Tests

Abstract

Get full access to this article

References