Test Equating Under the Multiple-Choice Model

Abstract

This article presents a characteristic curve procedure for computing transformations of the item response theory (IRT) ability scale assuming the multiple-choice model. The multiple-choice model provides a realistic and informative approach to analyzing multiple-choice items in two important ways. First, the probability of guessing is a decreasing function of proficiency rather than a constant across different proficiency levels as in the three-parameter logistic model. Second, the model utilizes information from incorrect answers as well as from correct answers. The multiple-choice model includes many well-known IRT models as special cases, such as Bock’s nominal response model. Formulas needed to implement a characteristic curve method for scale transformation are presented for the multiple-choice model. The use of the characteristic curve method for the multiple-choice model is illustrated in an example equating American College Testing mathematics tests. In the process of deriving the scale transformation procedure for the multiplechoice model, corrections were made in some of the formulas presented by Baker for computing a scale transformation for the nominal response model.

Get full access to this article

View all access options for this article.

References

Baker, F. B. (1992). Equating tests under the graded response model. Applied Psychological Measurement, 16, 87-96.

Baker, F. B. (1993). Equating tests under the nominal response model. Applied Psychological Measurement, 17, 239-251.

Bock, R. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37, 29-51.

Dennis, J. E. , & Schnabel, R. B. (1996). Numerical methods for unconstrained optimization and nonlinear equations. Philadelphia: Society for Industrial and Applied Mathematics.

Haebara, T. (1980). Equating logistic ability scales by a weighted least squares method. Japanese Psychological Research, 22, 144-149.

Holland, P. W. , & Rubin, D. (1982). Test equating. New York: Academic Press.

Kim, J.-S. , & Hanson, B. A. (2000). Test equating under the multiple-choice model. Paper presented at the annual meeting of American Educational Research Association, New Orleans, LA. Available from http://www.b-a-h.com/papers/paper0001.html

Kolen, M. J. , & Brennan, R. L. (1995). Test equating: Methods and practices. New York: Springer.

Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.

10.

Loyd, B. H. , & Hoover, H. D. (1980). Vertical equating using the Rasch model. Journal of Educational Measurement, 17, 179-193.

11.

Marco, G. L. (1977). Item characteristic curve solutions to three intractable testing problems. Journal of Educational Measurement, 14, 139-160.

12.

Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph, No. 17.

13.

Stocking, M. L. , & Lord, F. M. (1983). Developing a common metric in item response theory. Applied Psychological Measurement, 7, 201-210.

14.

Thissen, D. (1991). MULTILOG user’s guide: Multiple, categorical item analysis and test scoring using item response theory [Computer program]. Chicago, IL: Scientific Software International.

15.

Thissen, D. , & Steinberg, L. (1984). A response model for multiple choice items. Psychometrika, 49, 501-519.

16.

Thissen, D. , & Steinberg, L. (1997). A response model for multiple-choice items. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 51-65). New York: Springer.

17.

Thissen, D. , Steinberg, L. , & Fitzpatrick, A. R. (1989). Multiple-choice models: The distractors are also part of the item. Journal of Educational Measurement, 26, 161-176.