This study examined the validity of two item-writing rules in the design of test items: (a) the desirable number of options for a multiple-choice test item and (b) use of the inclusive none of these option. An experimental repeated measures design found that items with three options were more difficult than those with four options and items employing the none of these option were more difficult than those not using this inclusive option format. Neither format manipulation affected item discrimination. Therefore, evidence allows no recommendation for the none of these option but suggests an advantage for multiple-choice items with fewer than the traditional four or five options.
Get full access to this article
View all access options for this article.
References
1.
Budescu, D. V. and Nevo, B. (1985). Optimal number of options: An investigation of the assumption of proportionality. Journal of Educational Measurement, 22, 183-196.
2.
Clemans, W. V. (1958). An index of item-criterion relationship. Educational and Psychological Measurement, 18, 167-172.
3.
Crehan, K. D. , Gross, L. J., Koehler, R. A., and Slakter, M. J. (1978). Developmental aspects of test-wiseness. Educational Research Quarterly, 3, 40-44.
4.
Crehan, K. D. and Haladyna, T. M. (1991). The validity of two item-writing rules. Journal of Experimental Education, 59, 183-192.
5.
Guttman, L. (1941). The quantification of class attributes: A theory and method of scale construction. In P. Horst (Ed.). The prediction of personal adjustment. New York: Social Science Research Council.
6.
Haladyna, T. M. and Downing, S. M. (1989a). A taxonomy of multiple-choice item writing rules. Applied Measurement in Education, 1, 37-50.
7.
Haladyna, T. M. and Downing, S. M. (1989b). The validity of a taxonomy of multiple-choice item writing rules. Applied Measurement in Education, 1, 51-77.
8.
Haladyna, T. M. and Downing, S. M. (1991). How many options is enough for a multiple-choice test item? Research monograph 91.1. Phoenix, AZ: Institute for Research and Evaluation Studies at Arizona State University West.
9.
Lord, F. M. (1952). The relationship of the reliability of multiple choice items to the distribution of item difficulties. Psychometrika, 18, 181-194.
10.
Myers, D. (1986). Psychology. New York, N.Y.: Worth Publishers.
11.
Owen, S. V. and Froman, R. D. (1987). What's wrong with three option multiple-choice items?Educational and Psychological Measurement, 47, 513-521.
12.
Rich, C. E. and Johanson, G. (1990). An item level analysis of"none of the above". Paper presented at the annual meeting of the American Educational Research Association, Boston, MA.
13.
Trevisan, M. S. , Sax, G., and Michael, W. B. (1991). The effects of the number of options per item and student ability on test validity and reliability. Educational and Psychological Measurement, 51, (4), 829-837.
14.
Trevisan, M. S. , Sax, G., and Michael, W. B. (in press). Estimation the optimal number of options per item using an incremental option paradigm. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT.
