Interactive Confidence Test Scoring and Interpretation

Abstract

Confidence scored tests allow subjects to respond to the options on a multiple choice test by assigning probabilities to each of the item options. Reproducing scoring functions applied to the set of response probabilities produces scores which are maximized in the long run, if and only if, probabilities assigned to correct answers are equal to the proportion of times those probabilities are assigned to correct answers. This maximizing property makes it possible to separate evidence of knowledge from confidence in knowledge. Such separation has the potential for providing more reliable and valid measures of learning as well as information about the tendency to overvalue or undervalue one's information. This paper describes an interactive computer program for administering, scoring, and interpreting the results of confidence tests.