A Meta-Analytic Review of Item Discrimination and Difficulty in Multiple-Choice Items Using "None-Of-The-Above"

Abstract

The research on using "none-of-the-above" (N-O-T-A) as a test item option continues to find contradictory results on item difficulty and item discrimination. The argument for not using N-O-T-A is that this option is considered more difficult and less discriminating than items with one correct format (regular).The purpose of this study was to conduct a meta-analysis on the difficulty and discrimination of the N-O-T-A test option. A total of twelve articles yielding 20 effect sizes was examined for difficulty, and a total of seven studies yielding eleven effect sizes was examined for discrimination. The meta-analysis indicated nonsignificant effect sizes of .01 for discrimination and -.17 for item difficulty. These findings indicate that using "none-of-the-above"as a test item option does not result in items of lesser quality than in items not using this option.

Get full access to this article

View all access options for this article.

References

Aamodt, M. G. and McShane, T. D. (in press). A meta-analytic investigation of various test item characteristics on test scores and test completion times. Public Personnel Management.

Crehan, K. and Haladyna, T. M. (1989). The validity of two item-writing. Unpublished manuscript.

Dudycha, A. L. and Carpenter, J. B. (1973). Effects of item format on item discrimination and difficulty. Journal of Applied Psychology, 58(1), 116-121.

Forsyth, R. A. and Spratt, K. F. (1980). Measuring problem solving ability in mathematics with multiple-choice items: The effect of item format on selected item and test characteristics. Journal of Educational Measurement, 17(1), 31-43.

Frary, R. B. (1991). The None-of-the-above option: An empirical study. Applied Measurement in Education, 4(2), 115-124.

Haladyna, T. M. and Downing, S. M. (1989). The validity of a taxonomy of multiple-choice item writing rules. Applied Measurement in Education, 1, 51-78.

Hughes, H. H. , and Trimble, W. E. (1965). The use of complex alternatives in multiple choice items. Educational and Psychological Measurement, 25(1), 117-126.

Hunter, J. E. and Schmidt, F. L. (1990). Methods of meta-analysis: Correcting error and bias in research findings. Newbury Park, CA: Sage.

Knowles, S. L. and Welch, C. A. (1990). Item discrimination and difficulty in multiple-choice items using none-of-the-above. Unpublished manuscript, Radford University.

10.

Qosterhof, A. C. and Coats, P. K. (1984). Comparison of difficulties and reliabilities of quantitative word problems in completion and multiple choice item formats. Applied Psychological Measurement, 8(3), 287-294.

11.

Rimland, B. (1960). The effects of varying time limits and of using "right answer not given" in experimental forms of the U.S. Navy arithmetic test. Educational and Psychological Measurement, 20(3), 533-539.

12.

Tollefson, N. (1987). A comparison of the item difficulty and item discrimination of multiple-choice items using the "none of the above" and one correct response options. Educational and Psychological Measurement, 47, 377-383.

13.

Tollefson, N. and Tripp, A. (1983, April). The effect of item format on item difficulty and item discrimination. Paper presented at the Annual meeting of the American Educational Research Association, Montreal, Quebec.

14.

Wesman, A. G. and Bennett, G. K. (1946). The use of "none of these" as an option in test construction. The Journal of Educational Psychology, 37, 541-549.

15.

Williamson, M. L. and Hopkins, K. D. (1967). The use of "none of these" versus homogenous alternatives on multiple choice tests: Experimental reliability and validity comparisons. Journal of Educational Measurement, 4(2), 53-58.