Improving Understanding of Diagnostic Test Outcomes

Abstract

Background. Understanding diagnostic test outcomes requires determining the positive predictive value (PPV) of the test, which most laypeople and medical professionals struggle to do. Despite advances found with frequency formats and visual aids, less than 40% of people can typically identify this value. This study tests the impact of using congruent reference classes in problem-question pairings, evaluates the role of numeracy, and assesses how diagnostic value estimates affect the reported likelihood to use the test. Method. A 3 × 2, Pairing (congruent test-focus, congruent condition-focus, incongruent) × Response Format (frequency, percentage) mixed design experiment was conducted, in which participants answered diagnostic questions about 7 medical problems presented in a format focusing either on the reference class of those who test positive or those who have the condition. Answer accuracy, numeracy, and ratings of likelihood to use estimates were assessed. Results. Focusing on the congruent test reference class allowed 87% of participants to consistently identify the PPV, and focusing on the congruent condition reference class led 63% of participants to consistently identify the sensitivity (SEN). Aligning reference classes was especially beneficial for those with lower numeracy, increasing accuracy on problems from 21% for incongruent pairings to 66% for congruent pairings. Ratings of likelihood to use the test were closely tied to participants’ estimates of diagnostic values, regardless of the accuracy of those estimates. Conclusions. Although often overlooked, a straightforward mapping of reference classes from the relevant diagnostic information to the question of interest reduces confusion and substantially increases accuracy in estimates of diagnostic values. These findings can be used to strengthen training in the assessment of uncertainties associated with medical test results.