Sage Journals: Discover world-class research

Abstract

French

Purpose:

Radiologists work primarily in collaboration with other healthcare professionals. As such, these stakeholder perspectives are of value to the development and assessment of educational outcomes during the transition to competency-based medical education. Our aim in this study was to determine which aspects of the Royal College CanMEDS competencies for diagnostic radiology are considered most important by future referring physicians.

Methods:

Institutional ethics approval was obtained. After pilot testing, an anonymous online survey was sent to all residents and clinical fellows at our university. Open-ended questions asked respondents to describe the aspects of radiologist service they felt were most important. Thematic analysis of the free-text responses was performed using a grounded theory approach. The resulting themes were mapped to the 2015 CanMEDS Key Competencies.

Results:

115 completed surveys were received from residents and fellows from essentially all specialties and years of training (out of 928 invited). Major themes were 1) timeliness and accessibility of service, 2) quality of reporting, and 3) acting as a valued team member. The competencies identified as important by resident physicians were largely consistent with the CanMEDS framework, although not all key competencies were covered in the responses.

Conclusions:

This study illustrates how CanMEDS roles and competencies may be exemplified in a concrete and specialty-specific manner from the perspective of key stakeholders. Our survey results provide further insight into specific objectives for teaching and assessing these competencies in radiology residency training, with the ultimate goal of improving patient care through strengthened communication and working relationships.

Keywords

competency-based medical education postgraduate training referring physicians radiologists quality value survey

Introduction

In Canada, postgraduate medical education (PGME) in diagnostic radiology is currently undergoing a significant transformation from a traditional time-based to an outcomes-based approach for teaching and assessment.¹ Competence by Design (CBD) was created by the Royal College of Physicians and Surgeons of Canada, and is part of a worldwide movement toward competency-based medical education (CBME). CBD uses the CanMEDS 2015 framework, composed of 7 roles (Medical Expert, Communicator, Collaborator, Leader, Health Advocate, Scholar, and Professional) which organize the required competencies that must be met by all resident physicians, regardless of specialty.²

To apply CBD in PGME, programs and educators must develop competency-based curricula and assessment tools, the core units of which are entrustable professional activities (EPAs) and milestones. The current Diagnostic Radiology Objectives of Training will be replaced by a Competencies document that has a similar purpose and format, but uses the CanMEDS 2015 framework.¹ The overall aim is to meet societal needs and ensure that physicians achieve the competencies needed to provide effective, safe, and high-quality care.²

Since radiologists primarily work in collaboration with other members of the healthcare team, strong communication and working relationships with referring physicians are integral to supporting optimal patient management and outcomes. As key stakeholders, these referrers can provide valuable insights into educational outcomes for radiology residents.

van der Lee and colleagues have previously explored stakeholders’ perceptions of what competencies are important in Obstetrics and Gynecology training, assessing overlap of specialty-specific competencies with the CanMEDS framework.³ However, to the best of our knowledge, no such dedicated study has been conducted with regard to stakeholders’ perceptions for other specialties, specifically Diagnostic Radiology.

Our aim in this study was to determine which aspects of the Royal College CanMEDS competencies are considered important for diagnostic radiologists (including interventional radiologists) by future referring physicians. Such stakeholder perspectives would be of potential value to the development and assessment of educational curricula and outcomes during the transition to CBME.

Methods

Design and Conceptual Framework

Our study used a similar design as Bandiera and Lendrum, applying an inductive grounded theory approach to the data analysis, but not a full grounded theory method to the entire project.⁴ We also incorporated elements of an interpretive description approach, in that the analytic framework was constructed through the experiential knowledge of the researchers in order to focus on information that would be pertinent to those using the research findings.⁵

Study Setting and Survey Design

The study was conducted at a single academic medical centre in Canada. A purposeful sampling strategy was employed to focus on physicians in fields other than radiology. An element of convenience sampling was included through our decision to survey postgraduate trainees from all specialties and levels. These included residents (as defined by Canadian postgraduate medical education departments), as well as clinical fellows. Although trainees at community sites were included, they would have been a small minority of the study population.

A survey was developed based on a review of the literature and discussions with residents from multiple specialties and schools. The draft survey was created using an online survey tool developed by the university’s Information Technology department, and piloted with several residents from different specialties to ensure clarity of questions and usability of the interface. The results of the pilot studies were not incorporated in the current study results. To render the content relevant to practice and encourage response rate, the study questions were combined with other questions related to a quality improvement initiative from the radiology department focused on improving interdepartmental communication (Supplemental Appendix 1).

Institutional research ethics board approval was obtained. The final survey and a letter of informed consent were subsequently distributed to all postgraduate medical trainees (residents and fellows) via email by the PGME office in September 2012. The information letter stated that all responses would be anonymous and participation (including responses to individual questions) would be completely voluntary. A reminder was sent 1 week after the initial invitation, and the survey was open for a month. To preserve anonymity, in cases where small numbers of residents in a specialty might have led them to become identifiable through demographic information, up to 3 specialties were combined in a single response option (Figure 1).

Figure 1.

Respondents by postgraduate medical education specialty program. OMFS indicates Oral and Maxillofacial Surgery.

The survey questions used for this study were the demographic questions (program of study and year of training) and open-ended questions as follows:

“What aspects of radiology service do you feel are important and currently well provided? In what way?”

“What aspects of radiology service do you feel are important and need to be better provided? How so?”

“Please feel free to offer any additional comments on any of the survey questions, or any other thoughts you have on this topic:”

Data Collection and Analysis

The data were collected using the university’s survey tool and subsequently extracted to Microsoft Excel (Redmond, WA, United States). Thematic analysis of the free text responses was performed manually. The resulting themes were matched to the roles and key competencies in the CanMEDS 2015 Physician Competency Framework.²

Two faculty members in the Department of Radiology (academic radiologists) independently coded the survey responses using a grounded theory approach, extracting repeating ideas that were then grouped into coherent categories and organized into themes following methods that have been detailed by Auerbach and others.^6,7,8 A common list of themes was developed, and triangulated with matching CanMEDS key competencies through discussion and consensus. Similar to Bandiera and Lendrum, we did not further map to enabling competencies, since the higher-level descriptors are how CanMEDS roles are usually described.⁴

Results

115 completed surveys were received from 928 individuals contacted, for a response rate of 12%. The study included 114 postgraduate trainees from across essentially all specialties (1 respondent did not provide their field of training) and postgraduate years of study. Figures 1 and 2 exhibit the distribution of respondents by specialty and year of study respectively.

Figure 2.

Number of respondents by postgraduate year of study.

Three major themes were identified from the survey responses and are discussed below along with their component subthemes and illustrative quotations (Supplemental Appendix 2). Our team considered that we had achieved saturation since no new ideas arose from the last question, wherein respondents were invited to share anything else they felt was important.

Timeliness and Accessibility of Service

Overall, timely access to imaging and timeliness of reports were the most commonly cited repeating ideas within the survey responses. Ease and accessibility of contacting radiologists was also frequently mentioned. The 3 subthemes under this heading were:

1. Timeliness of imaging and procedures

Although this was not strictly a physician competency, respondents placed significant value on being able to have radiological studies performed in a prompt manner. In particular, there was a desire for increased after-hours access to imaging.

2. Report turnaround time

There was specific emphasis on prompt reporting by radiologists in order to facilitate efficient and prompt delivery of patient care. In addition, respondents indicated that they especially appreciated receiving phone calls about acute or unexpected findings.

3. Accessibility of radiologists

Referring physicians appreciated radiologists who were available and accessible for in-person or phone consultations. Some issues with identifying and contacting the appropriate person were mentioned.

Quality of Reporting

Quality of reporting was also emphasized by respondents. The two subthemes that emerged were:

1. Expertise

Knowledgeable and accurate subspecialty interpretations were valued by referring physicians. There was a wish for radiologists to act as a consulting service and provide advice and recommendations on next steps. As this was an academic centre, respondents also placed importance on attending staff involvement with residents on call.

2. Report structure

Referring physicians appreciated clear reports that provided summaries and targeted differential diagnoses, and addressed the specific question that prompted the request for imaging.

Acting as a Valued Team Member

Lastly, respondents underscored the need for radiologists to act as a valued member of the healthcare team. The 3 subthemes under this heading were:

1. Collaborative relationships

Referring physicians appreciated radiologists that were approachable, friendly, and willing to take time to discuss cases. There was a wish to decrease perceived resistance to after-hours imaging requests.

2. Ownership of patient care

Respondents highlighted the value of radiologists who were team-focused, willing to help facilitate imaging studies and ensure proper handover of urgent cases.

3. Additional consultant roles

Referring physicians expressed a wish for radiologists to be better integrated with other services through participation in multidisciplinary rounds, and to provide teaching on imaging to other departments.

Comparison to CanMEDS 2015 Framework

The competencies identified to be most important by resident physicians were largely consistent with the CanMEDS framework; we were able to match key competencies to each of the themes and subthemes. The CanMEDS roles and key competencies aligning with each of the three overarching themes and their subthemes are listed in Table 1.

Table 1.

Themes Mapped to CanMEDS Roles and Key Competencies.

Themes	Subthemes	Associated CanMEDS roles
Timeliness and accessibility of service	Imaging Reports Accessing the appropriate radiologist	Medical Expert
		Plan and perform procedures and therapies for the purpose of assessment and/or management.
		Communicator
		Document and share written and electronic information about the medical encounter to optimize clinical decision-making, patient safety, confidentiality, and privacy.
		Collaborator
		Work effectively with physicians and other colleagues in the healthcare professions. Work with physicians and other colleagues in the health care professions to promote understanding, manage differences, and resolve conflicts.
		Leader
		Contribute to the improvement of health care delivery in teams, organizations, and systems. Demonstrate leadership in professional practice.
		Professional
		Demonstrate a commitment to society by recognizing and responding to societal expectations in healthcare.
Quality of reporting	Expertise Report structure	Medical Expert
		Practice medicine within their defined scope of practice and expertise. Perform a patient-centred clinical assessment and establish a management plan. Actively contribute, as an individual and as a member of a team providing care, to the continuous improvement of health care quality and patient safety.
		Communicator
		Document and share written and electronic information about the medical encounter to optimize clinical decision-making, patient safety, confidentiality, and privacy.
		Collaborator
		Work effectively with physicians and other colleagues in the healthcare professions.
		Scholar
		Engage in the continuous enhancement of their professional activities through ongoing learning. Integrate best available evidence into practice.
Acting as a valued team member	Collaborative relationships Ownership of patient responsibility Consultant role	Medical Expert
		Plan and perform procedures and therapies for the purpose of assessment and/or management. Establish plans for ongoing care and, when appropriate, timely consultation. Actively contribute, as an individual and as a member of a team providing care, to the continuous improvement of health care quality and patient safety.
		Communicator
		Document and share written and electronic information about the medical encounter to optimizing clinical decision-making, patient safety, confidentiality, and privacy.
		Collaborator
		Work effectively with physicians and other colleagues in the healthcare professions. Work with physicians and other colleagues in the health care professions to promote understanding, manage differences, and resolve conflicts. Hand over the care of a patient to another health care professional to facilitate continuity of safe patient care.
		Leader
		Engage in the stewardship of health care resources. Demonstrate leadership in professional practice.
		Scholar
		Teach students, residents, the public, and other health care professionals. Contribute to the creation and dissemination of knowledge and practices applicable to health.
		Professional
		Demonstrate a commitment to patients by applying best practices and adhering to high ethical standards. Demonstrate a commitment to society by recognizing and responding to societal expectations in health care.

The main exception is the emphasis on timely access to imaging/procedures. This is not considered a physician competency in the CanMEDS framework, but appears inseparable from radiologists’ role, and was the most frequent code in our study. Closely linked with this idea is the expressed importance of timely reporting, which was the second-most frequent code.

All roles were mentioned except Health Advocate, perhaps because the question was framed as service to other professionals rather than the community or patient population, as is covered by this role. Within Medical Expert, Collaborator, and Scholar, all key competencies were covered.

Within the Communicator role, only one key competency relating to Documentation was covered. This is because the CanMEDS 2015 Communicator role focuses exclusively on the interaction between physicians and patients, whereas communication with other health care professionals is categorized under the Collaborator role.

Within the Leader role, all competencies were covered except those relating to managing one’s career planning, finances, and health human resources in a practice.

Within the Professional role, key competencies regarding commitment to patients and society were mentioned, but not commitment to the profession (i.e. physician-led regulation) or physician health and well-being.

Discussion

In this study, we explored which aspects of the Royal College CanMEDS competencies were considered important for diagnostic radiologists by future referring physicians, with a view to informing how we teach and assess these competencies in radiology residency training. Although based at a single academic institution, survey responses were obtained from residents covering nearly all fields of generalist and specialist practice, and yielded insights into how key competencies may be exemplified in practice settings from the perspective of stakeholders.

Although most roles and key competencies were covered, a major theme was timeliness and accessibility of radiologists’ service. This is congruent with other studies in the radiology literature which have surveyed referring staff physicians from other specialties.⁹ It also supports the premise of van der Lee et al that there may be specialty-specific competency requirements which vary from the CanMEDS framework.³ In this case, it is clear that the value of radiologists’ consultation services and contributions to patient care are heavily linked to the availability of imaging resources and other logistical factors, to an extent that may not be seen in all specialties.

Similar to other studies, the Medical Expert role competencies were well covered, consistent with this being the central CanMEDS role which integrates all aspects of practice. The Collaborator role was also completely covered, which is in keeping with radiology as a primarily consultative specialty. Similar to the findings of van der Lee et al, key Professional and Leader competencies regarding commitment to patients and society were mentioned, but not those competencies regarding the profession (i.e. physician-led regulation) or physicians’ own well-being, perhaps reflecting low visibility or perceived lack of relevance to other healthcare professionals, as might be expected.³ Health Advocate role competencies were not mentioned, consistent with other studies such as Chou et al in which program directors found this role difficult to assess,¹⁰ and Verma et al which found poor awareness and understanding of this role by residents in particular.¹¹ This suggests that these competencies which are not typically the subject of external feedback will continue to merit attention through other means.

In comparison to van der Lee et al, we found more emphasis on the Scholar role, perhaps due to this being a tertiary academic centre where issues of training and feedback would be of greater salience, and subspecialized knowledge more valued. This is also in contrast to Ng et al, who found that the Scholar role was least frequently mentioned as important by family medicine residents; however, this may be accounted for by the design of that survey which asked respondents to indicate only a single most important CanMEDS role.¹²

The themes from this survey serve as an illustration of how CanMEDS roles and competencies may be exemplified in a concrete way from the perspective of those who consult radiologists on a day-to-day basis, and may have applicability to the assessment of these competencies. Challenges related to understanding and tools for evaluation of CanMEDS roles other than Medical Expert have previously been noted,⁹ and have led to the development of specific assessment tools such as a CanMEDS OSCE for radiology.¹³ Although 360 degree evaluations incorporate feedback from others in the radiology department, it is unclear how frequently feedback is solicited from others outside of the department after residents have completed the PGY1 rotations (during which they rotate through multiple other services) and are in the radiology department full-time. As the abstract nature of the general competencies renders them less visible and easily measured,¹⁴ it has been proposed that aligning these training objectives in a more tangible way with clinical practice will facilitate implementation.^3,15

Transitioning to CBME requires effectively building tools that expand the richness and frequency of resident evaluations to individualize resident feedback and training experiences. In this survey, we have identified several themes that could be useful in this realm. In particular, the ‘report quality’ and ‘valued team member’ themes could provide a basis for radiology CBME programs looking to better match evaluations with referring colleagues’ expectations of what constitutes effective and high-quality care from practicing radiologists. We hope that bridging the gap with EPAs in this way will support the overarching goals of training and mitigate tendencies toward a reductionist numerical approach to the completion of milestones.^13,16

We did not specifically perform respondent validation of the results, and there could be bias in the subset of those contacted who chose to respond to the survey, given the low response rate of 12%. However, these results are congruent with our preliminary interviews and pilot testing. Although some subthemes and examples reflect our survey population of residency trainees who may focus on their on-call experiences, our overall themes are also congruent with other studies in the radiology literature which have surveyed referring staff physicians from other specialties. For example, Johnson et al found that accuracy was considered to be the most important aspect of radiology reporting, with lack of timeliness the greatest problem area,¹⁷ Scatarlge et al found that skill in interpretation was a deciding factor among chest physicians in their decision to refer for high-resolution CT scans,¹⁸ and Lio et al found that when referring physicians were asked about the services that thoracic radiologists provided in their centre, accessibility and availability were the most important qualities.¹⁹

Although most other surveys of referring physicians have focused on preferences regarding radiology reports,^20-23 or used quantitative measures of service,^24,25 there is a growing recognition of the value existing in less easily quantifiable aspects of radiologists’ work, which may be best assessed through the viewpoint of the recipients of the service.^26,27 A qualitative study by Zafar et al found that primary care physicians were more likely to trust the interpretations and ask questions of radiologists that they personally knew, potentially affecting their decision to follow up incidental findings.²⁸ Similarly, Dickerson et al found that in-person discussions often led to alterations in surgical decision making, even if those discussions were concordant with the original finalized reports.²⁹ Others such as Lama et al have pointed out that more effective collaboration would present an opportunity to improve feedback regarding errors and foster a safety culture.³⁰ As many entrustable professional activities involve interprofessional collaboration,³¹ potential future research could include the assessment of changes in interdepartmental relationships after the implementation of targeted strategies based on these results, and could be expanded to other institutions, community settings, and physicians in independent practice.

Overall, this study suggests that medical trainees in other specialties place particular value on the timeliness and accessibility of radiologists and the consultations they provide, as well as the quality and expertise of their reporting, and their collaborative roles as members of the healthcare team. The survey results illustrate how CanMEDS competencies may be exemplified concretely in practice from the perspective of key stakeholders. It is hoped that these may aid efforts to develop and assess educational outcomes, with the ultimate goal of improving quality of care and patient safety through strengthened communication and working relationships.

Supplemental Material

Supplemental Material, sj-pdf-1-caj-10.1177_08465371211008649 - Which Aspects of the CanMEDS Competencies are Most Valued in Radiologists? Perspectives of Trainees From Other Specialties

Supplemental Material, sj-pdf-1-caj-10.1177_08465371211008649 for Which Aspects of the CanMEDS Competencies are Most Valued in Radiologists? Perspectives of Trainees From Other Specialties by Stefanie Y. Lee, Namita Sharma, Yoan K. Kagoma and Andrea P. Lum in Canadian Association of Radiologists Journal

Footnotes

Acknowledgments

The authors thank the Postgraduate Medical Education Office at Western University for assistance in distributing the survey.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Stefanie Y. Lee, MD, FRCPC

Supplemental Material

Supplemental material for this article is available online.

References

Kwan

Mbanwi

Cofie

, et al. Creating a competency-based medical education curriculum for canadian diagnostic radiology residency (queen’s fundamental innovations in residency education)—part 1: transition to discipline and foundation of discipline stages. Can Assoc of Radiol J. 2020:846537119894723.

Frank

Snell

Sherbino

. CanMEDS 2015. Royal College of Physicians and Surgeons of Canada; 2015.

van der Lee

Fokkema

Westerman

, et al. The CanMEDS framework: Relevant but not quite the whole story. Medical Teacher. 2013;35(11):949–955.

Bandiera

Lendrum

. Dispatches from the front: emergency medicine teachers’ perceptions of competency-based education. CJEM. 2011;13(3):155–161.

Thompson Burdine

Thorne

Sandhu

. Interpretive Description: a Flexible Qualitative Methodology for Medical Education Research. Medical Education; 2020.

Auerbach

Silverstein

. Qualitative Data: an Introduction to Coding and Analysis. New York University Press; 2003.

Elliot

. Thinking about the coding process in qualitative data analysis. The Qualitative Report [Internet]. Publication 2018. Accessed March 31, 2021.23(11):2850-2861. https://nsuworks.nova.edu/tqr/vol23/iss11/14

Corbin

Strauss

. Basics of Qualitative Research. Sage; 2008.

Zhang

Hefke

Figiel

, et al. Identifying radiological needs of referring clinicians. J Digit Imaging. 2013;26(3):393–401.

10.

Chou

Cole

McLaughlin

, et al. CanMEDS evaluation in Canadian postgraduate training programmes: tools used and programme director satisfaction. Med Educ. 2008;42(9):879–886.

11.

Verma

Flynn

Seguin

. Faculty’s and residents’ perceptions of teaching and evaluating the role of health advocate: a study at one canadian university. Acad Med. 2005;80(1):103–108.

12.

Burke

Narula

. Knowledge of CanMEDS-family medicine roles: survey of Canadian family medicine residents. Can Fam Physician. 2013;59(9):e428–e434.

13.

Probyn

Lang

Finlay

, et al. Assessment of the Intrinsic CanMEDS Roles in diagnostic radiology residents using an objective structured clinical assessment (OSCE). MedEdPublish. 2016;5(2):40.

14.

Ten Cate

Scheele

. Viewpoint: competency-based postgraduate training: can we bridge the gap between theory and clinical practice? Acad Med. 2007;82(6):542–547.

15.

Jippes

Van Luijk

Pols

Achterkamp

Brand

Van Engelen

. Facilitators and barriers to a nationwide implementation of competency-based postgraduate medical curricula: A qualitative study. Med Teach. 2012;34(8):e589–e602.

16.

Pinsk

Karpinski

Carlisle

. Introduction of competence by design to canadian nephrology postgraduate training. Can J Kidney Health Dis. 2018;5:205435811878697.

17.

Johnson

Ying

Swan

Williams

Applegate

Littenberg

. Improving the quality of radiology reporting: a physician survey to define the target. J Am Coll Radiol. 2004;1(7):497–505.

18.

Scatarige

Diette

Haponik

Merriman

Fishman

. Availability, requesting practices, and barriers to referral for high-resolution CT of the lungs. Acad Radiol. 2002;9(12):1370–1377.

19.

Lio

Tillack

Sachs

Restauri

Vargas

Suby-Long

. Improving radiologist accessibility and availability in a PACS environment. J Am Coll Radiol. 2015;12(10):1076–1078.

20.

Bosmans

Weyler

De Schepper

Parizel

. The radiology report as seen by radiologists and referring clinicians: results of the COVER and ROVER surveys. Radiology. 2011;259(1):184–195.

21.

Grieve

Plumb

Khan

. Radiology reporting: a general practitioner’s perspective. Br J Radiol. 2010;83(985):17–22.

22.

Gunn

Sahani

Bennett

, et al. Recent measures to improve radiology reporting: perspectives from primary care physicians. J Am Coll Radiol. 2013;10(2):122–127.

23.

Mehta

Assimacopoulos

Heiberger

, et al. Opinions, views, and expectations concerning the radiology report: a rural medicine report. Cureus. 2019;11(10):e5822.

24.

McMenamy

Rosenkrantz

Jacobs

Kim

. Use of a referring physician survey to direct and evaluate department-wide radiology quality improvement efforts. J Am Coll Radiol. 2015;12(11):1223–1225.

25.

Zhang

Hefke

Figiel

Schwarz

Rominger

Klose

. Identifying radiological needs of referring clinicians. J Digit Imaging. 2012;26(3):393–401.

26.

Tillack

Breiman

. Renegotiating expertise: an examination of pacs and the challenges to radiology using a medical anthropologic approach. J Am Coll Radiol. 2012;9(1):64–68.

27.

Larson

Hwang

. Needs of referring providers by practice type: results of a survey at an academic medical center. AJR Am J Roentgenol. 2021;216(1):216–224.

28.

Zafar

Bugos

Langlotz

Frasso

. “Chasing a ghost”: factors that influence primary care physicians to follow up on incidental imaging findings. Radiology. 2016;281(2):567–573.

29.

Dickerson

Alam

Brown

Stojanovska

Collaborative

Davenport

. In-person communication between radiologists and acute care surgeons leads to significant alterations in surgical decision making. J Am Coll Radiol. 2016;13(8):943–949.

30.

Lama

Hogg

Olson

. Perspectives from the other side of the screen: how clinicians and radiologists communicate about diagnostic errors. Diagnosis. 2020;7(1):45–53.

31.

Ten Cate

Pool

. The viability of interprofessional entrustable professional activities. Adv Health Sci Educ Theory Pract. 2020;25(5):1255–1262.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.11 MB