Preparedness of Residents and Medical Students for the Transition to Competence by Design in Diagnostic Radiology Post-Graduate Medical Education

Abstract

French

Introduction: This needs assessment evaluated residents’ and medical students’ knowledge of Competence by Design (CBD), perceived benefits of and challenges or barriers to the transition to CBD for residents, and perceived overall preparedness for the transition to CBD in diagnostic radiology. Materials and Methods: All diagnostic radiology residents and medical students in Canada were eligible to participate in this national cross-sectional, questionnaire-based needs assessment. Knowledge of CBD was evaluated through participants’ self-reported rating of their knowledge of CBD on a 5-point Likert scale. Perceived benefits of and challenges or barriers to the transition to CBD for residents were rank ordered. Participants’ overall self-reported preparedness for the transition to CBD was assessed on a 5-point Likert scale. Data were summarized by descriptive statistics and bivariate analyses were conducted as appropriate. Results: Ninety-four residents (n = 77) and medical students (n = 17) participated in this needs assessment. Participants’ mean ± standard deviation self-reported rating of their overall knowledge of CBD was 2.86 ± .94. Provision of meaningful feedback to learners and learners’ ability to identify their own educational needs were among the highest ranked perceived benefits of the transition to CBD, while demands on time and increased frequency of evaluation were among the highest ranked perceived challenges or barriers to the transition to CBD. Few participants reported being either “prepared” (4.7%) or “somewhat prepared” (14.0%) for the transition to CBD. Conclusion: Preparedness for the transition to CBD in diagnostic radiology may be improved. Targeted interventions to augment the preparedness of residents and medical students should be considered.

Keywords

Competency-Based Medical Education Competence by Design resident education medical education

Introduction

Competence by Design (CBD) is a medical education initiative developed by the Royal College of Physicians and Surgeons of Canada (RCPSC), which is intended to transform the post-graduate medical education (PGME) training of residents in Canada.^1-3 In contrast to the current longitudinal PGME paradigm, which facilitates PGME training over a dedicated period of time, CBD applies Competency-Based Medical Education (CBME) to PGME training through the integration of an organizational framework of competencies.^1-3 Common terms used in CBD are summarized in Box 1. While CBD has been implemented in the diagnostic radiology PGME program at Queen’s University since 2017 as part of an institutional initiative,^4,5 diagnostic radiology PGME programs nationally are transitioning to CBD in July 2022 for the incoming cohort.⁶

Diagnostic radiology residents (“residents”) will be key stakeholders in the transition to CBD.⁷ Not only will incoming residents be evaluated as learners but also new roles will be undertaken by senior residents, such as peer assessors during the completion of Entrustable Professional Activities (EPAs).⁸ Regardless of the role undertaken by residents upon the transition to CBD, the setting in which diagnostic radiology PGME will be undertaken in Canada will undoubtedly be changed.^1-3

Given this important medical education transition, the Canadian Association of Radiologists (CAR) Resident and Fellow Section (RFS) and the RCPSC Diagnostic Radiology Specialty Committee jointly conducted a needs assessment to evaluate the preparedness of residents and medical students for the transition to CBD. This needs assessment was intended to evaluate residents’ and medical students’ knowledge of CBD, to assess perceived benefits of and challenges or barriers to the transition to CBD for residents, and to determine residents’ and medical students’ perceived overall preparedness for the transition to CBD. Data collected from this needs assessment was further intended both to inform the development of educational resources and to determine additional evidence-informed targeted strategies to better facilitate the transition to CBD for residents.

Methods

All residents currently enrolled in a Canadian diagnostic radiology PGME program as well as medical students enrolled in a Canadian undergraduate medical education institution were eligible to participate in this national cross-sectional, questionnaire-based needs assessment. The developed questionnaire was adapted from Bogie et al.⁹ The Qualtrics survey tool was used for online data collection.¹⁰ Questionnaire distribution was facilitated through the communications channels of the CAR as well as that of various diagnostic radiology medical student interest groups at Canadian undergraduate medical education institutions. The questionnaire was open for responses over a 6-week designated period. Institutional review board approval was not required for this needs assessment as per national regulations.¹¹

Academic demographic information collected included academic level and institution of diagnostic radiology PGME program or of undergraduate medical education. Intent to pursue application to diagnostic radiology PGME programs was sought from final year medical student participants.

Participants’ perceived impact of the transition to CBD on their PGME training was assessed. Knowledge of CBD was evaluated through participants’ self-reported rating of their overall knowledge of CBD as well as understanding of major principles and terms used in CBD on a 5-point Likert scale. Knowledge of the CanMEDS roles associated with the CanMEDS 2015 Framework¹² was evaluated through participants’ self-reported rating of their knowledge of the definitions and associated expectations of each of these roles on a 5-point Likert scale. Perceived difficulty of evaluation of each of the CanMEDS roles was rank ordered.

Data on participants’ prior utilization and perceived utility of available RCPSC CBD resources was collected. Participants’ perceived benefit of further developing their knowledge of CBD was assessed using a 10-point ordinal scale. Preferred formats for subsequent CBD knowledge development were further identified.

Perceived benefits of and challenges or barriers to the transition to CBD for residents were rank ordered. Degree of confidence in the transition to CBD subsequently achieving each of its several main intended educational outcomes was assessed on a 5-point Likert scale. Participants’ self-reported overall preparedness for the transition to CBD was also assessed on a 5-point Likert scale.

Data were summarized by descriptive statistics. Bivariate analyses were conducted as appropriate. Corrections for multiple testing were applied as required. Where data points were not provided by participants the proportion of participants was calculated using the number of participants for which data was available as the denominator.

Results

Participant Characteristics

Ninety-four residents (n = 77) and medical students (n = 17) participated in this needs assessment. Participants’ academic demographics are summarized in Table 1. All final year medical students who participated in this needs assessment reported an intent to apply to diagnostic radiology PGME programs for the 2021-2022 Canadian Residency Matching Service (CaRMS) application period.

Table 1.

Participant Academic Demographics.

Participant category	Academic demographics
Residents (n = 77)	Institution, n (% of residents)	University of British Columbia	4 (5.2)
		University of Alberta	13 (16.9)
		University of Calgary	12 (15.6)
		University of Saskatchewan	5 (6.5)
		Queen’s University	2 (2.6)
		Western University	1 (1.3)
		McMaster University	8 (10.4)
		University of Toronto	6 (7.8)
		University of Ottawa	5 (6.5)
		McGill University	3 (3.9)
		University of Montreal	3 (3.9)
		Laval University	4 (5.2)
		Sherbrooke University	3 (3.9)
		Dalhousie University	7 (9.1)
		Memorial University	1 (1.3)
	Academic level, n (% of residents)	Intern (PGY 1)	18 (23.4)
		Junior resident (PGY 2 to PGY 3)	38 (49.4)
		Senior resident (PGY 4 to PGY 5)	17 (22.1)
		Fellow in PGME program (PGY 6 to PGY 7)	3 (3.9)
Medical students (n = 17)	Institution, n (% of medical students)	University of British Columbia	3 (17.6)
		University of Alberta	3 (17.6)
		University of Calgary	1 (5.9)
		University of Saskatchewan	1 (5.9)
		Northern Ontario School of Medicine	1 (5.9)
		McMaster University	3 (17.6)
		University of Toronto	2 (11.8)
		McGill University	2 (11.8)
		Sherbrooke University	1 (5.9)
	Academic level, n (% of medical students)	Year 1	3 (17.6)
		Year 2	3 (17.6)
		Year 3	6 (35.3)
		Year 4	5 (29.4)

Impact of Transition to CBD on PGME Training

Sixteen residents (20.8%) and 13 medical students (76.5%) perceived that they would be evaluated in their PGME training under CBD. Forty-two residents (54.5%) identified that as senior residents they would take on new roles, such as peer assessors during the completion of EPAs. Fourteen residents (18.2%) and 9 medical students (52.9%) identified that they intended to be involved in undertaking medical education initiatives pertaining to CBD, such as medical education research, during their PGME training. Fourteen residents (18.2%) and 1 medical student (5.9%) perceived that the transition to CBD would not impact their PGME training.

Knowledge of CBD

Participants’ self-reported overall knowledge of CBD as well as understanding of major principles and terms used in CBD are summarized in Table 2. Participants’ mean ± standard deviation (SD) self-reported rating of their overall knowledge of CBD was 2.86 ± .94. Residents’ mean ± SD self-reported rating of their overall knowledge of CBD (2.88 ± .98) was not statistically significantly different than that of medical students’ (2.76 ± .75; P > .05). Of major principles and terms used in CBD, mean ± SD self-reported understanding of EPAs was observed to be the greatest (2.97 ± 1.28), while mean ± SD self-reported understanding of Electronic Portfolios was observed to be the poorest (2.48 ± 1.04). Residents’ mean ± SD self-reported understanding of EPAs (3.03 ± 1.22) was statistically significantly greater than that of medical students (2.71 ± 1.53; P = .03).

Table 2.

Overall Knowledge of CBD as well as Understanding of Major Principles and Terms Used in CBD.

Knowledge Parameter	Overall^a	Medical Students^a	Residents^a	P value
Competence by Design	2.86 ± .94	2.76 ± .75	2.88 ± .98	P > .05
Competency-Based Medical Education	2.90 ± 1.02	2.88 ± .78	2.91 ± 1.07	P > .05
Entrustable Professional Activities	2.97 ± 1.28	2.71 ± 1.53	3.03 ± 1.22	P = .03^b
Milestones	2.76 ± 1.13	2.59 ± 1.18	2.80 ± 1.12	P > .05
Competence Continuum	2.55 ± 1.07	2.35 ± 1.06	2.59 ± 1.07	P > .05
Electronic Portfolios	2.48 ± 1.04	2.41 ± 1.12	2.50 ± 1.03	P > .05
Academic Coaching (coaching in the moment and coaching over time)	2.63 ± 1.04	2.65 ± 1.06	2.63 ± 1.04	P > .05
Competence Committees	2.67 ± 1.11	2.06 ± 1.09	2.80 ± 1.07	P > .05

^aMean ± SD is reported; Likert scale ratings were denoted as 1 = “weak,” 2 = “somewhat weak,” 3 = “average,” 4 = “somewhat strong,” and 5 = “strong”.

^bDenotes statistical significance.

Knowledge of CanMEDS Roles and Perceived Difficulty of Evaluation

Participants’ self-reported knowledge of the definitions and associated expectations of each of the CanMEDS roles and participants’ perceived difficulty of evaluation of these roles are summarized in Table 3. Participants’ mean ± SD self-reported rating of their knowledge of the definition and associated expectations of the “Communicator” role was the greatest (4.00 ± .84) and that of the “Health Advocate” role was the least (3.83 ± .93). Residents’ mean ± SD self-reported rating of their knowledge of the definition and associated expectations of the “Medical Expert” role (3.97 ± .83) was statistically significantly greater than that of medical students’ (3.79 ± 1.05; P = .04). Overall, perceived difficulty of evaluation of the “Health Advocate” role was ranked to be most challenging while perceived difficulty of evaluation of the “Medical Expert” role was ranked to be least challenging.

Table 3.

Knowledge of the Definitions and Associated Expectations of Each of the CanMEDS Roles Associated with the CanMEDS 2015 Framework and Perceived Difficulty of Evaluation of These Roles.

CanMEDS role	Overall rank ordered perceived difficulty of evaluation^a	Overall self-reported knowledge of definitions and associated expectations^b
Health Advocate	1	3.83 ± .93
Leader	2	3.91 ± .86
Collaborator	3	3.92 ± .88
Professional	4	3.99 ± .83
Communicator	5	4.00 ± .84
Scholar	6	3.92 ± .87
Medical Expert	7	3.94 ± .87

^aParticipants’ overall rank order is reported; rank order is denoted as 1 = most challenging to assesses and 7 = least challenging to assess.

^bMean ± SD is reported; Likert scale ratings were denoted as 1 = “weak,” 2 = “somewhat weak,” 3 = “average,” 4 = “somewhat strong,” and 5 = “strong”.

Prior Utilization and Perceived Utility of RCPSC CBD Resources and CBD Knowledge Development

Only 18 participants (19.1%) had accessed the RCPSC CBD resources, of which only 1 participant (5.6%) was a medical student. Of those participants who accessed the RCPSC CBD resources, most found these resources “somewhat helpful” (44.4%), “neither not helpful nor helpful” (33.3%), or “somewhat not helpful” (11.1%).

Participants’ overall mean ± SD perceived benefit of further developing their knowledge of CBD was 5.91 ± 2.51. Medical students’ mean ± SD perceived benefit of further developing their knowledge of CBD (8.12 ± 1.54) was statistically significantly greater than that of residents (5.42 ± 2.42; P = .00). Participants identified a half-day academic “retreat” (57.0%), educational lecture sessions or workshops (50.0%), and distribution of relevant educational resources (41.9%) as preferred formats for CBD knowledge development.

Benefits of and Challenges or Barriers to the Transition to CBD

Provision of meaningful feedback to learners and the ability for learners to identify their own educational needs were ranked as the greatest benefits of the transition to CBD for residents (Figure 1). Demands on time, increased frequency of evaluation, and challenges with the feasibility of direct and indirect observation were ranked as the greatest challenges or barriers of the transition to CBD for residents (Figure 2).

Figure 1.

Rank order of greatest benefits of the transition to CBD for residents.

Figure 2.

Rank order of greatest challenges or barriers to the transition to CBD for residents.

Confidence in Achievement of Intended Educational Outcomes

Most participants reported being “neither not confident nor confident” that the transition to CBD would improve either the overall educational and training experience (35.2%) or improve teaching in PGME training (47.1%). Only 27.3% and 24.1% of participants were either “somewhat confident” or “confident” that the transition to CBD would achieve the aforementioned educational outcomes, respectively. Most participants were “somewhat confident” that the transition to CBD would improve feedback in PGME training (33.3%) (Figure 3).

Figure 3.

Confidence in the transition to CBD subsequently achieving intended educational outcomes.

Overall Preparedness for the Transition to CBD

Overall, few participants reported being either “prepared” (4.7%) or “somewhat prepared” (14.0%) for the transition to CBD. Only 5.6% of residents reported being “prepared” and 13.9% of residents reported being “somewhat prepared” for the transition to CBD. No medical students reported being “prepared” and 14.3% of medical students reported being “somewhat prepared” for the transition to CBD (Figure 4).

Figure 4.

Perceived overall preparedness for the transition to CBD.

Discussion

This is the first needs assessment which nationally evaluated residents’ and medical students’ knowledge of CBD. We observed that participants’ self-reported overall knowledge of CBD as well as understanding of major principles and terms used in CBD may be improved. These observations are comparable to that of Bogie et al.,⁹ which used questions from which those in our needs assessment were adapted to evaluate knowledge of CBD among psychiatry residents and medical students. Nonetheless, given that our needs assessment relied on participants’ self-reported rating of their overall knowledge of CBD as well as understanding of major principles and terms used in CBD, knowledge of CBD may actually be poorer than that observed. While Yang et al.¹³ previously assessed knowledge of CBME among medical students at a single undergraduate medical education institution through participants’ self-reported agreement to the statement “I understand the learning approach that CBME seeks to implement,” the methodology used may not have had sufficient sensitivity to identify deficiencies in knowledge of CBD.

Evaluation of knowledge of the CanMEDs roles is important as under CBD residents will work towards the completion of EPAs, which incorporate various CanMEDS Milestones.^8,12 While participants’ self-reported rating of their knowledge of the definitions and associated expectations of each of the CanMEDS roles tended to be “average” to “somewhat strong,” it is interesting to note that the CanMEDS roles for which self-reported rating of knowledge was the least tended to be those which were perceived to be the most difficult to evaluate. Our observation that the “Medical Expert” role was perceived to be the least difficult to evaluate is in keeping with previous literature, which reported perceived difficulty among faculty in other specialties in appropriately assessing residents on those EPAs which incorporate non-“Medical Expert” CanMEDS roles.^6,14-16 Appropriate assessment of non-“Medical Expert” CanMEDS roles is integral as it has been previously demonstrated that other specialties most value aptitude in the “Communicator” and “Professional” roles among residents.¹⁷ Dedicated professional development opportunities for faculty not only to further develop assessment but also to further develop teaching of non-“Medical Expert” CanMEDS roles may thus be of benefit in the transition to CBD.^15,16

Our observations of participants’ perceived impact of the transition to CBD on their PGME training suggests additional work is required to ensure that residents are aware of their roles. We observed that approximately 1 in 5 residents perceived that they would be evaluated as learners under CBD, whereas only those medical students who will be part of the incoming cohort in July 2022 shall be evaluated as learners under CBD.^1,6 While CBD has been implemented in the diagnostic radiology PGME program at Queen’s University since 2017 as part of an institutional initiative,^4,5 only 2 participants reported being residents from this institution. Few residents may have also been in combined PGME training programs, such as diagnostic radiology and nuclear medicine, with the latter having already transitioned to CBD.⁶ Clear, broadly disseminated messaging is needed to ensure that residents are aware of the impact of the transition to CBD on their PGME training.

Participants’ reported limited access of the available RCPSC CBD resources may indicate that additional efforts are required to ensure that residents are aware of resources to further develop their knowledge of CBD. Moreover, the perceived limited utility of the available RCPSC CBD resources may suggest that development of additional diagnostic radiology-specific CBD resources may be of benefit. Medical students’ greater perceived benefit of further developing their knowledge of CBD than residents’ is most likely associated with their subsequent role as learners who will be evaluated under CBD in their PGME training. This is in keeping with student motivation theory, which describes that perceived benefit is an integral component of motivation in developing new skills.¹⁸ Additional dedicated efforts will be required to motivate residents to further develop their knowledge of CBD.

Provision of professional development opportunities to residents will be integral to facilitate the transition to CBD.^7,19-22 Participants’ demonstrated preference for a half-day “retreat” as compared to other formats of subsequent CBD knowledge development may suggest that participants favor immersive formats which may allow the opportunity to foster dialogue regarding the transition to CBD. These demonstrated preferences have helped to inform the CAR RFS’ and the RCPSC Diagnostic Radiology Specialty Committee’s ongoing efforts in assisting with the transition to CBD. In particular, a national orientation to CBD for residents was developed jointly by both organizations arising from this needs assessment. Facilitation of ongoing national professional development opportunities for residents is encouraged given considerations of comparably limited resources available to individual diagnostic radiology PGME programs.

This is the first needs assessment which nationally assessed perceived benefits of and challenges or barriers to the transition to CBD for residents. Active efforts will be required to mitigate those greatest perceived challenges or barriers to the transition to CBD for residents. Opportunities to mitigate demands on time associated with CBD for residents should be actively sought out and creative strategies should be employed. As it has been previously demonstrated that in the setting of poor faculty involvement in CBD associated tasks fall on to the responsibility of residents,⁷ ensuring active involvement of all stakeholders, such as faculty and administrative staff, in the transition to CBD shall be integral. Furthermore, while the effects on residents of the transition to CBD in other specialties have not yet been fully manifested, it has been suggested that CBD may have negative effects on the health and wellness of residents.⁷ Additional investigation to further determine the effects of CBD on the health and wellness of residents shall be required.⁷ Additional challenges or barriers to the transition to CBD have been previously summarized and include assessment, information technology infrastructure, resources, and culture.²³

While provision of meaningful feedback to learners was ranked as the greatest benefit to the transition to CBD for residents, many residents in other specialties have reported that the quality of feedback that they have received is poor.^19,21,24 Provision of professional development opportunities on the delivery of high-quality feedback should be a priority to facilitate the transition to CBD.²³

Participants’ limited perceived preparedness for the upcoming transition to CBD is concerning. As the transition to CBD is an important medical education transition, limited perceived preparedness may further exacerbate associated adjustment and uncertainty.²⁵ Of greatest priority, incoming residents will be required to understand CBD to the extent that they will be able both to effectively function as learners and to embrace learning under CBD. Evidence-informed endeavours will need to be undertaken to improve the preparedness of residents for the upcoming transition to CBD. Such endeavours will be required both to motivate residents to further develop their knowledge of CBD and to provide residents with the appropriate educational resources and professional development opportunities to do so.

Our needs assessment was limited by its cross-sectional, questionnaire-based design. Given inherent sampling bias, the generalizability of our results may be limited as it is most likely that those with a perceived interest in CBD chose to participate. Evaluation of knowledge of CBD was limited by self-assessment bias as participants provided self-reported ratings. Assessment of participants’ perceptions of the benefits of and challenges or barriers to the transition to CBD for residents and perceived overall preparedness was also limited given participants solely being limited to rank order and to select discrete categorical responses to close-ended questions, respectively. Use of open-ended questions, delivered either through a narrative questionnaire or interview, may have yielded additional insights. Participation of residents who will be impacted by CBD as learners was not feasible as at the time of this needs assessment these individuals remained medical students who had not yet been offered placement in a diagnostic radiology PGME program and thus could not be reliably identified. Finally, as the reach of the distribution of our questionnaire was not able to be determined, response rates were not able to be reliably calculated. Nonetheless, a low response rate, particularly among medical students, may demonstrate disinterest in CBD and poor “buy in,” which may further represent a challenge to the transition to CBD.

Conclusion

We identified gaps in the preparedness of residents and medical students for the transition to CBD. Specifically, we observed that knowledge of CBD may be improved and that demands on time, increased frequency of evaluation, and challenges with the feasibility of direct and indirect observation are perceived to be the greatest challenges or barriers to the transition to CBD for residents. Few participants reported being either “prepared” or “somewhat prepared” for the transition to CBD. A half-day academic “retreat,” educational lecture sessions or workshops, and distribution of relevant educational resources were identified as preferred formats for CBD knowledge development. This needs assessment may further inform the development of educational resources and determine additional evidence-informed targeted strategies to better facilitate the transition to CBD for residents. The CAR RFS and the RCPSC Diagnostic Radiology Specialty Committee remain committed to supporting residents during this important medical education transition.

Box 1. Common Terms Used in CBD.

Competency: An “observable ability” of a resident related to a specific task. As a competency is an “observable ability” it may be “measured and assessed to ensure [its] acquisition.”²⁶

Entrustable Professional Activity (EPA): An “essential task” of a specialty specific to each stage of training which a resident may be “trusted to perform.”^8,26

CanMEDS Milestones: Discrete skills which are needed to perform “essential tasks” of a specialty. Multiple CanMEDS Milestones may be incorporated in to each individual EPA as multiple discrete skills may be required to perform “essential tasks.” CanMEDS Milestones encompass CanMEDS roles, which include Medical Expert, Communicator, Collaborator, Leader, Health Advocate, Scholar and Professional.^8,26

Competency Committee: Assembled faculty in a particular PGME program involved in the training of residents who iteratively review the assessment data of each resident to collectively evaluate and provide recommendations for the progression of each resident through each stage of training.²⁷

Academic Advisor/Academic Coach: A faculty member generally not involved in the Competency Committee who develops a longitudinal “educational partnership” with a resident to iteratively provide feedback and discuss “a [resident’s] progression towards competence.”²⁸

Adapted from Bentley et al.²⁵

Footnotes

Acknowledgments

We thank Chantal Hetu-Neuman, Laura Sandvold and Julia Niles of the Canadian Association of Radiologists as well as diagnostic radiology PGME program administrators throughout Canada for their support of this important initiative.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Alanna Supersad

Hang Yu

Kiana Lebel

Priscila Crivellaro

Aline Khatchikian

Appendix

References

Probyn

Hartery

Taylor

Karpinski

. Transforming postgraduate diagnostic radiology training in Canada with competence-based medical education. Can Assoc Radiol J. 2021;72(2):324-325.

Royal College of Physicians and Surgeons of Canada . What is competence by design? 2021. https://www.royalcollege.ca/rcsite/cbd/what-is-cbd-e. Accessed December 31, 2021.

Royal College of Physicians and Surgeons of Canada . CBD resident guide, 2021. https://www.google.ca/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&cad=rja&uact=8&ved=2ahUKEwikvNXigPXyAhVwJzQIHd96CIIQFnoECAIQAQ&url=http%3A%2F%2Fwww.royalcollege.ca%2Frcsite%2Fdocuments%2Fcbd%2Fcbd-residents-guide-e.pdf&usg=AOvVaw3mq8iCB6mpaApm75j5UTCi. Accessed December 31, 2021.

Kwan

BYM

Mbanwi

Cofie

, et al. Creating a competency-based medical education curriculum for Canadian diagnostic radiology residency (queen’s fundamental innovations in residency education)—Part 1: Transition to discipline and foundation of discipline stages. Can Assoc Radiol J. 2020;72(3):372-380.

Mishra

Chung

Rogoza

, et al. Creating a competency-based medical education curriculum for Canadian diagnostic radiology residency (queen’s fundamental innovations in residency education)-Part 2: Core of discipline stage. Canadian Association of Radiologists Journal; 2021;72(4):678-685.

Royal College of Physicians and Surgeons of Canada . Competence by Design (CBD) Implementation Pulse Check, 2019. https://www.royalcollege.ca/rcsite/documents/cbd/cbd-pulse-check-report-e.pdf. Accessed September 30, 2021.

Royal College of Physicians and Surgeons of Canada . Competence by Design (CBD) Implementation Pulse Check. CBD Program Evaluations Operations Team, Fall 2020.

Royal College of Physicians and Surgeons of Canada . EPAs and CanMEDS milestones, 2021. https://www.royalcollege.ca/rcsite/cbd/implementation/cbd-milestones-epas-e. Accessed August 31, 2021.

Bogie

Payne

Harms

McConnell

Samaan

. Leading from behind: An educational intervention to address faculty and learner preparedness for competence by design in psychiatry. J Multidiscip Healthc. 2021;14:2587-2595.

10.

Qualtrics . University of British Columbia survey tool, 2021. https://it.ubc.ca/services/teaching-learning-tools/survey-tool. Accessed August 31, 2021.

11.

Tri-Council Policy Statement Ethical Conduct for Research Involving Humans–TCPS 2, 2018. https://ethics.gc.ca/eng/documents/tcps2-2018-en-interactive-final.pdf. Accessed July 13, 2022.

12.

Royal College of Physicians and Surgeons of Canada . CanMEDS interactive, 2021. http://canmeds.royalcollege.ca. Accessed August 31, 2021.

13.

Yang

Jomaa

Islam

, et al. Competency-based medical education in radiology: A survey of medical student perceptions. Can Assoc Radiol J. 2021:72(3):352-358.

14.

Tannenbaum

Furmli

Kent

, et al. Exploring faculty perceptions of competency-based medical education and assessing needs for implementation in obstetrics and gynaecology residency. J Obstet Gynaecol Can. 2020:42(6):707-717.

15.

Puddester

MacDonald

Clements

Gaffney

Wiesenfeld

. Designing faculty development to support the evaluation of resident competency in the intrinsic CanMEDS roles: practical outcomes of an assessment of program director needs. BMC Med Educ. 2015;151:1-9.

16.

Gaboury

Ouellet

Xhignesse

, et al. Strategies identified by program directors to improve adoption of the CanMEDS framework.Can Assoc Radiol J. 2018;9(1):e26.

17.

Lee

Sharma

Kagoma

Lum

. Which aspects of the CanMEDS competencies are most valued in radiologists? perspectives of trainees from other specialties. Can Med Edu J. 2021;73(1):30-37.

18.

Schunk

Zimmerman

, eds. Motivation and self-regulated learning: Theory, research, and applications. London, UK: Routledge; 2012.

19.

Tomiak

Linford

McDonald

Willms

Hammad

. Implementation of competency-based medical education in a Canadian medical oncology training program: A first year retrospective review. J Cancer Educ. 2020;37:1-5.

20.

Stockley

Egan

Van Wylick

, et al. A systems approach for institutional CBME adoption at Queen’s University. Med Teach. 2020;42:916-921.

21.

Hall

Rich

Dagnone

, et al. It’s a marathon, not a sprint: Rapid evaluation of competency-based medical education program implementation. Acad Med. 2020;95(5):786-793.

22.

Stefan

Hall

Sherbino

Chan

. Faculty development in the age of competency-based medical education: A needs assessment of Canadian emergency medicine faculty and senior trainees. Can J Emerg Med. 2019;21(4):527-534.

23.

Bentley

Darras

Forster

, et al. Review of challenges to the implementation of competence by design in post-graduate medical education: What can diagnostic radiology learn from the experience of other specialties. Acad Radiol. (In press).

24.

David

Walsh

Lockyer

, et al. Entrustable professional activities: An analysis of faculty time, trainee perspectives and actionability. Can J Gen Int Med. 2021;16(1):8-13.

25.

Hall

Oswald

Hauer

, et al. Twelve tips for learners to succeed in a CBME program. Med Teach. 2021;43(7):1-6.

26.

Englander

Frank

Carraccio

Sherbino

Ross

Snell

. Toward a shared language for competency-based medical education. Med Teach. 2017;39(6):582-587.

27.

Royal College of Physicians and Surgeons of Canada . Competence committees, 2022. https://www.royalcollege.ca/rcsite/cbd/assessment/competence-committees-e. Accessed July 25, 2022.

28.

Royal College of Physicians and Surgeons of Canada . Coaching and feedback, 2022. https://www.royalcollege.ca/rcsite/cbd/implementation/wbas/coaching-and-cbd-e. Accessed July 25, 2022.