Multiple-Mark Items: An Alternative Objective Item Format?

Abstract

This study examined four threats to the validity of an alternative objective item format, the multiple-mark format. This format uses a multiple-choice item format and directs students to mark correct alternatives and leave blank incorrect alternatives. The [ILLEGIBLE] format was used on the Kansas Reading and Mathematics Assessments at three [ILLEGIBLE] levels. The four validity threats studied were students understanding directions, students omitting responses, dependency among item alternatives, and student guessing. [ILLEGIBLE] results indicated that over 95% of students followed multiple-mark directions and fewer than 1% omitted multiple-mark items. Reliability results showed some dependency among alternatives. As a result, it was recommended that scoring occur at the item level for multiple-mark items. In regard to guessing, it appeared that when students guessed, they chose to leave an alternative blank rather than mark it. Overall, the reliability and validity coefficients showed that the multiple-mark format has promise as an alternative objective format that is aligned with new curricular standards.

Get full access to this article

View all access options for this article.

References

Albanese, M. A. , Kent, T. H. , & Whitney, D. R. (1977, November). Cluing in multiple choice test items with combinations of correct responses. Paper presented at annual meeting of Research in Medical Education, Washington, DC.

Bell, R. C. , Pattison, P. E. , & Withers, G. P. (1988). Conditional independence in a clustered item test. Applied Psychological Measurement, 12(1), 15-26.

Cronbach, L. J. (1941). An experimental comparison of the multiple true-false and multiple-multiple choice test. Journal of Educational Psychology, 32, 533-543.

Dressel, P. L. , & Schmid, J. (1953). Some modifications of the multiple-choice item. Educational and Psychological Measurement, 13, 574-595.

Frisbie, D. A. (1992). The multiple true-false item format: A status review. Educational Measurement: Issues and Practice, 11(4), 21-26.

Glasnapp, D. G. , & Poggio, J. P. (1994, April). Psychometric characteristics of the multiple-correct multiple-choice item. Paper presented at annual meeting of the National Council on Measurement in Education, New Orleans.

Grandy, J. (1987). Characteristics of examinees who leave questions unanswered on the GRE General Test under rights-only scoring (ETS Research Report 87-38). Princeton, NJ: ETS.

Grosse, M. E. , & Wright, B. D. (1985). Validity and reliability of true-false tests. Educational and Psychological Measurement, 45, 1-13.

Hsu, T. , Moss, P. , & Khampalikit, C. (1984). The merits of multiple-answer items as evaluated by using six scoring formulas. Journal of Experimental Education, 52, 152-158.

10.

National Council of Teachers of Mathematics (NCTM). (1989). Curriculum and evaluation standards for school mathematics. Reston, VA: Author.

11.

Sireci, S. G. , Thissen, D. , & Wainer, H. (1991). On the reliability of testlet-based tests. Journal of Educational Measurement, 28(3), 237-247.

12.

Yen, W. M. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 8(2), 125-145.

13.

Yen, W. M. (1993). Scaling performance assessments: Strategies for managing local item dependence. Journal of Educational Measurement, 30(3), 187-214.