A Meta-Analytic Investigation of the Effect of Various Test Item Characteristics on Test Scores and Test Completion Times

Abstract

The current study reported the results of a meta-analytic investigation of the effects on test scores and test completion times of three aspects of writing test items: The number of answers in multiple-choice exams, the order of item difficulty, and the organization of items by content. The results of meta-analysis indicated that three-choice questions are slightly easier than four-choice questions (d = .90) and take significantly less time to complete (d = −.61). Exams beginning with easier items and then moving to more difficult items are slightly easier than exams with randomly ordered items (d = .11) or exams beginning with difficult items (d — .22). Exams in which the items are organized by content are slightly easier than exams containing randomly ordered items (d = .04). All of the above effect sizes are small.