The Effects of the Number of Options per Item and Student Ability on Test Validity and Reliability

Abstract

Reliability and validity of multiple-choice examinations were computed as a function of the number of options per item and student ability for junior class parochial high school students administered the verbal section of the Washington Pre-College Test Battery. The least discriminating options were deleted to create 3- and 4-option test formats from the original 5-option item test. Students were placed into ability groups by using noncontiguous grade point average (GPA) cutoffs. The GPAs were the criteria for the validity coefficients. Significant differences (p c 0.05) were found between reliability coefficients for low ability students. The optimum number of options was three when the ability groups were combined. None of the validity coefficients followed the hypothesized trend. These results are part of the mounting evidence that suggests the efficacy of the 3-option item. An explanation is provided.

Get full access to this article

View all access options for this article.

References

Costin, F. (1970). The optimal number of alternatives in multiple-choice tests: Some empirical evidence for a mathematical proof. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 309, 353-358.

Costin, F. (1972). Three-choice versus four-choice items: Implications for reliability and validity of objective achievement tests. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 32, 1035-1038.

Ebel, R. (1969). Expected reliability as a function of choices per item. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 29, 565-570.

Educational Testing Service (1948-1990a). College Board Scholastic Aptitude Test. Princeton, NJ: Author.

Educational Testing Service (1948-1990b). Graduate Record Examinations. Princeton, NJ: Author.

Green, K. , Sax, G. , and Michael, W. B. (1982). Validity and reliability of tests having different numbers of options for students of differing levels of ability. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 42, 239-245.

Grier, J. B. (1975). The number of alternatives for optimum test reliability. Journal of Educational Measurement, 12, 109-113.

Hakstian, A. R. and Whalen, T. E. (1976). A k-sample significance test for independent alpha coefficients. Psychometrika, 41, 219-231.

Levine, M. V. and Drasgow, F. (1983). The relation between incorrect option choice and estimated ability. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 43, 675-685.

10.

Lord, R. M. (1977). Optimal number of choices per item—A comparison of four approaches. Journal of Educational Measurement, 14, 33-38.

11.

Marascuilo, L. (1966). Large-sample multiple comparisons. Psychological Bulletin, 65, 280-290.

12.

Owen, W. V. and Froman, R. D. (1987). What's wrong with three-option multiple choice items? EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 47, 513-522.

13.

Straton, R. G. and Catts, R. M. (1980). A comparison of two, three, four-choice item tests given a fixed total number of choices. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 40, 357-365.

14.

Tversky, A. (1964). On the optimal number of alternatives at a choice point. Journal of Mathematical Psychology, 1, 386-391.

15.

University of Washington (1989). Scorepak: Item Analysis. Seattle: University of Washington, Educational Assessment Center.

16.

University of Washington (1983). Washington Pre-College Admissions Test Battery. Seattle: University of Washington, Washington Pre-College Department.

17.

Weber, M. B. (1978). The effect of choice format on internal consistency. Paper presented at the National Council on Measurement in Education Annual Meeting, Toronto, Canada.