Students from two consecutive semesters were given multiple-choice tests over five units of an undergraduate course in psychology. During the first semester, students were given five 50-question 4-option multiple-choice tests, and during the second semester students were given five 50-question 3-option multiple-choice tests. One-hundred and forty-four (57.6%) of the questions were identical between semesters except for second semester test items having only 3 options. Results indicate that students performed significantly better on 3-option items than on 4-option items (corrected for chance guessing), and that this improvement may be due to improved validity of the test items.
Get full access to this article
View all access options for this article.
References
1.
Boring, E. G. (1950). A history of experimental psychology (2nd ed.). Englewood Cliffs, NJ: Prentice-Hall.
2.
Costin, F. (1970). The optimal number of alternatives in multiple-choice achievement tests: Some empirical evidence for a mathematical proof.Educational and Psychological Measurement, 30, 353-358.
3.
Costin, F. (1972). Three-choice versus four-choice items: Implications for reliability and validity of objective achievement tests.Educational and Psychological Measurement, 32, 1035-1038.
4.
Ebel, R. L. (1969). Expected reliability as a function of choices per item.Educational and Psychological Measurement, 29, 565-570.
5.
Grier, J. B. (1975). The number of alternatives for optimum test reliability.Journal of Educational Measurement, 12, 109-112.
6.
Hogben, D. (1973). The reliability, discrimination and difficulty of word-knowledge tests employing multiple choice items containing three, four, or five alternatives.The Australian Journal of Education, 17, 63-68.
7.
Lord, F. M. (1977). Optimal number of choices per item-a comparison of four approaches.Journal of Educational Measurement,, 14, 33-38.
8.
Mattson, D. (1965). The effects of guessing on the standard error of measurement and the reliability of test scores.Educational and Psychological Measurement, 25, 727-730.
9.
Owen, S. V. and Froman, R. D. (1987). What's wrong with three-option multiple choice items?Educational and Psychological Measurement, 47, 513-522.
10.
Ruch, G. M. and Charles, J. W. (1928). A comparison of five types of objective tests in elementary psychology.Journal of Applied Psychology, 12, 398-404.
11.
Ruch, G. M. , DeGraff, M. H., and Gordon, W. E. (1926). Objective examination methods in the social studies. New York: Scott Foresman.
12.
Ruch. G. M. and Stoddard, G. D. (1925). Comparative reliabilities of five types of objective examinations.Journal of Educational Psychology, 16, 89-103.
13.
Ruch, G. M. and Stoddard, G. D. (1927). Tests and measurements in high school instruction. Chicago: World Book.
14.
Stratton, R. G. and Catts, R. M. (1980). A comparison of two, three and four-choice item tests given a fixed total number of choices.Educational and Psychological Measurement, 40, 357-365.
15.
Toops, H. (1921). Trade tests in education. In Teachers College Contributions to Education (No. 115). New York: Columbia University.
16.
Trevisan, M. S. , Sax, G., and Michael, W. B. (1991). The effects of the number of options per item and student ability on test validity and reliability.Educational and Psychological Measurement, 51, 829-837.
17.
Tversky, A. (1964). On the optimal number of alternatives at a choice point.Journal of Mathematical Psychology, 1, 386-391.
18.
Williams, B. J. and Ebel, R. L. (1957). The effect of varying the number of alternatives per item on multiple-choice vocabulary test items. In The Fourteenth Yearbook of the National Council on Measurements Used in Education (pp. 63-65). East Lansing, MI: Michigan State University.
