Participatory Bullseye Toolkit Interview: Identifying Physicians’ Relative Prioritization of Decision Factors When Ordering Radiologic Imaging in a Hospital Setting

Abstract

Critical Decision Method (CDM), a popular cognitive task analysis (CTA) method, is an in-depth retrospective interview that uses a historical non-routine incident to identify experts’ decision-making factors in complex socio-technical settings with high consequences for failure. However, it is challenging to use CDM to make comparisons, including those between experts and trainees. We describe an alternative CTA method used to study physicians’ decision making for ordering diagnostic imaging. After being primed with 11 simulated patient scenarios, nine attending and 11 resident physicians were asked to map out and present their decision-making process with a bullseye participatory design toolkit. Interviews were analyzed qualitatively, revealing four common decision factors: diagnostic efficacy, patient safety, organizational constraints, and patient comfort. The bullseye maps were used to quantitatively measure priority differences between these decision factors. Attending and resident physicians both prioritized diagnostic efficacy over the other factors (2.38 vs. 3.71, p <.01, and 2.59 vs. 3.52, p<.01, respectively), but attending physicians’ decisions had a higher proportion of non-diagnostic items (65% vs. 50%, p = .008). Our results demonstrate the usefulness of this method in eliciting decision factors for a complex, face-valid task and for identifying differences due to levels of expertise and training.