Adjusting Confidence Tests for Realism

Abstract

Confidence tests were administered three times during a course in physical diagnosis. Conventional measures of knowledge and written case studies were examined in order to determine acceptability, reliability, and strength of association between confidence measures and other reflections of performance. Confidence testing wasfound to be acceptable, in fact, even enjoyable, to the students. Reliability was significantly improved and realism was stable across test halves and independent of knowledge, suggesting the presence of separate, measurable attributes in clinical thinking. Although statistical significance was not reached, confidence testing seemed qualitatively better atpredictingperformance on written case histories. Students, in general, improved their performance over the year in realism assessment. Educational implications and directionsforfuture research are discussed.