This study examines the question of the optimal number of choices on a multiple-choice test from an information theory perspective. Results are compared to addressing this question using more traditional statistical approaches. Based on information theory, the study reveals that, in general, three choices to a multiple-choice test item seem optimal. This finding verifies what other researchers have found from statistical and observational (item analysis) approaches.
Get full access to this article
View all access options for this article.
References
1.
Andres, A. M. , & del Castillo, L. (1990). Multiple-choice tests: Power, length, and optimal number choices per item. British Journal of Mathematical and Statistical Psychology, 43(1), 57-71.
2.
Barner, R. , & Bruno, J. E. (1991). Formative evaluation and mathematics attainment of inner city students. Urban Review, 23(4), 251-269.
3.
Baxter, J. B. (1991, March). Information referenced testing for formative evaluation and self-assessment to support the elaboration and advancement of student thinking and learning. Paper presented at the Eighth International Conference on Technology and Education, Toronto, Canada.
4.
Bruno, J. E. (1987). Admissible probability measurement in instructional management. Journal of Computer Based Instruction, 14(1), 95-107.
5.
Bruno, J. E. (1989). Developing a professional standard of care for academic advisors of student athletes. Academic Athletic Journal, 3(4), 17-30.
6.
Bruno, J. E. (1992). Using testing to provide feedback to support instruction: A reassessment of the role of assessment in educational organizations. In D. Leclercq & J. E. Bruno (Eds.), Item banking: Interactive testing and self assessment (pp. 190-209). Berlin: Springer-Verlag.
7.
Costin, F. (1970). The optimal number of alternatives in multiple-choice achievement tests: Some empirical evidence for a mathematical proof. Educational and Psychological Measurement, 30, 353-358.
8.
Cox, E. P. (1980). The optimal number of response alternatives for a scale: A review. Journal of Marketing Research, 17, 407-422.
9.
Dirkzwager, A. (1975). Computer-based testing with automatic scoring based on subjective probabilities. In O. Lecarne & R. Lewis (Eds.), Computers in education (pp. 305-311). Amsterdam: IFIP, North Holland Publishing.
10.
Dirkzwager, A. (1992). A computer environment to develop valid and realistic predictions and self assessment of knowledge and personal probabilities. In D. Leclercq & J. E. Bruno (Eds.), Item banking: Interactive testing and self assessment (pp. 146-166). Berlin: Springer-Verlag.
11.
Duncan, E. (1983). An appropriate number of multiple-choice item alternatives: A difference of opinion. Measurement and Evaluations in Guidance, 15(4), 283-292.
12.
Ebel, R. L. (1969). Expected reliability as a function of choices per item. Educational and Psychological Measurement, 29, 565-570.
13.
Grier, J. B. (1975). The number of alternatives for optimal test reliability. Journal of Educational Measurement, 12, 109-113.
14.
Grier, J. B. (1976). The optimal number of alternatives at a choice point with travel time considered. Journal of Mathematical Psychology, 14, 91-97.
15.
Guest, L. (1965). A comparison of two choice and four choice questions. Journal of Advertising Research, 11(3), 32-34.
16.
Hunt, D. (1992). Human self assessment: Theory and applications to learning and testing. In D. Leclercq & J. E. Bruno (Eds.), Item banking: Interactive testing and self assessment (pp. 177-199). Berlin: Springer-Verlag.
17.
Jacoby, J. , & Matell, M. (1971). Three point Likert scales are good enough. Journal of Marketing Research, VIII, 495-500.
18.
Klentschy, M. L. (1992). Designing instructional support and decision making systems to service accelerated learning environments in large urban elementary schools. Unpublished doctoral dissertation, University of California, Los Angeles.
19.
Komoritu, S. S. , & Graham, W. (1965). The number of scale points and the reliability of scales. Educational and Psychological Measurement, 25(4), 987-995.
20.
Leclercq, D. (1992). Validity, reliability, and acuity of self assessment in educational testing. In D. Leclercq & J. E. Bruno (Eds.), Item banking: Interactive testing and self assessment (pp. 1-131). Berlin: Springer-Verlag.
21.
Leclercq, D. , & Bruno, J. E. (Eds.). (1993). Item banking: Interactive testing and self assessment: Proceedings of the NATO Advanced Science Institute Series, Advanced Research Workshop on Item Banking and Self Assessment, Liege, Belgium, October 27-31, 1992.Berlin: Springer-Verlag.
22.
Lord, F. (1944). Reliability of multiple-choice tests as a function of choices per item. Journal of Educational Psychology, 35, 175-180.
23.
Lord, F. (1977). Optimal number of choices per item. A comparison of four approaches. Journal of Educational Measurement, 14, 33-38.
24.
Lord, F (1980). Applications of item response theory to practical test problems. Hillsdale, NJ: Lawrence Erlbaum.
25.
Matell, M. S. , & Jacoby, J. (1971). Is there an optimal number of alternatives for Likert scale items?Educational and Psychological Measurement, 31, 657-674.
26.
Mathewson, S. (1990). Designing human resource and staff development programs in information dependent organizations: An application of periodic information audits or critical care nursing personnel. Unpublished doctoral dissertation, University of California, Los Angeles.
27.
Oaster, T.R.E. (1989). Number of alternatives per choice point and stability of Likert-type scales. Perceptual and Motor Skills, 68, 549-550.
28.
Ruch, G. M. , & Stoddard, G. D. (1927). Tests and measurements in high school instruction. Chicago: World Book.
29.
Shuford, E. H. (1992). In pursuit of the fallacy: Resurrecting the penalty. In D. Leclercq & J. E. Bruno (Eds.), Item banking: Interactive testing and self assessment (pp. 76-98). Berlin: Springer-Verlag.
30.
Shuford, E. , & Brown, T. (1975). Elicitation of personal probabilities and their assessment. Instructional Science, 4(2), 137-188.
31.
Straton, R. G. , & Catts, R. M. (1980). A comparison of 2,3, and 4 choice item tests given a fixed number of choices. Educational and Psychological Measurement, 23, 357-365.
32.
Torabi-Parizi, R. , & Campbell, N. J. (1982). Classroom test writing: Effects of item format on test quality. Elementary School Journal, 83(2), 155-160.
33.
Tversky, A. (1964). On the optimal number of alternatives at a choice point. Journal of Mathematical Psychology, I(1), 386-391.
