Further Investigation of Nonfunctioning Options in Multiple-Choice Test Items

Abstract

This study addressed the hypothesis that, after the systematic elimination of nonfunctioning options, four-option test items would perform as well as five-option test items having one or more dysfunctional distracters. The study consisted of two investigations involving an examination administered to 700 candidates for certification in a medical specialty. In the first investigation, it was found that content experts exhibited a high degree of accuracy in identifying nonfunctioning options where the criterion was empirical item analysis data. The second phase of the study compared five-option versions of multiple-choice items with four-option versions in which a nonfunctioning option had been removed. Results indicated that (a) removal of a nonfunctioning option resulted in a slight, non significant overall increase in item difficulty and no significant differences in item discrimination, (b) a test consisting of items with a nonfunctioning option removed was nearly equally reliable compared with a set of the same items in a five-option format, and (c) the use of empirical or judgmental methods of identifying nonfunctioning options was not related to changes in item performance. Implications, cautions, and suggestions for future research are provided.

Get full access to this article

View all access options for this article.

References

Buckingham, B. R. (1926). Research for teachers. New York: Silver Burdett.

Costin, F. (1970). The optimal number of alternatives in multiple-choice achievement tests: Some empirical evidence for a mathematical proof. Educational and Psychological Measurement, 30(2), 353-358.

Costin, F. (1972). Three-choice versus four-choice items: Implications for reliability and validity of objective achievement tests. Educational and Psychological Measurement, 32(4), 1035-1038.

Ebel, R. L. (1969). Expected reliability as a function of choices per item. Educational and Psychological Measurement, 29(2), 565-570.

Grier, J. B. (1975). The number of alternatives for optimum test reliability. Journal of Educational Measurement, 12(2), 109-113.

Grier, B. (1976). The optimal number of alternatives at a choice point with travel time considered. Journal of Mathematical Psychology, 14(1), 91-97.

Haladyna, T. M. , & Downing, S. M. (1985, April). A quantitative review of research on multiple-choice item writing. Paper presented at the annual meeting of the American Educational Research Association, Chicago.

Haladyna, T. M. , & Downing, S. M. (1988, April). Functional distractors: Implications for test-item writing and test design. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.

Haladyna, T. M. , & Downing, S. M. (1989). Validity of a taxonomy of multiple-choice item-writing rules. Applied Measurement in Education, 2(1), 51-78.

10.

Lord, F. M. (1977). Optimal number of choices per item: A comparison of four approaches. Journal of Educational Measurement, 14(1), 33-38.

11.

Trevisan, M. S. , Sax, G. , & Michael, W. B. (1991). The effects of the number of options per item and student ability on test validity and reliability. Educational and Psychological Measurement, 51(4), 829-837.

12.

Tversky, A. (1964). On the optimal number of alternatives at a choice point. Journal of Mathematical Psychology, 1(2), 386-391.

13.

Williams, B. J. , & Ebel, R. L. (1957). The effect of varying the number of alternatives per item on multiple-choice vocabulary test items. In E. M. Huddleston (Ed.), The fourteenth yearbook of the National Council on Measurements Used in Education (pp. 63-65). Princeton, NJ: Educational Testing Service.