Does One Size Truly Fit All? The COUPE Undergraduate Perspective on Competency-Based Medical Education in Psychiatry

A new era of medical education is under way, with competency-based medical education (CBME) being widely adopted. CBME is an outcomes-based approach to the design, implementation, assessment, and evaluation of a medical education program using an organizing framework of competencies. ¹

The evolution and theory of CBME have been clearly described. ^{1 –5} Driven by calls for enhanced patient outcomes and accountability, CBME has been embraced as representing a long-overdue reform in medical education, the last reform having evolved out of the Flexner Report (1910). ⁶ However, increasing discussion has raised numerous questions and concerns about CBME. ^{1,5,7 –15} The most commonly cited concerns have revolved around the intrinsically reductionist nature of the CBME model, the amount and frequency of assessment and resources required, the faculty development necessary to ensure competency of the evaluators, and the risk of overemphasizing the skills and knowledge aspects of medical education over the professionalism and integration abilities of a good physician.

It is unclear that CBME’s implementation will produce a “better doctor” and how this will be defined and measured. In addition, it remains unclear whether measures will align with real-life outcomes meaningful to the public (e.g., patient evaluations of their physicians, better patient outcomes, cost-efficient resource utilization, better patient communication, lower rates of complaints to regulating bodies).

Internationally, competency-based criteria for postgraduate medical education (PGME) have been in place for years and most recently in the United States, with the Milestones Project. In postgraduate programs across Canada, the Royal College of Physicians and Surgeons of Canada (RCPSC) is rolling out a hybrid model of CBME. This competence by design (CBD) model will blend CBME into the existing time/rotation-based structure in phases, with psychiatry scheduled for July 2018 implementation.

Pending the official implementation of CBD in Canadian PGME, CBME has yet to be formally developed for undergraduate medical education (UGME), although several medical schools across Canada have integrated CBME concepts into their curriculum (August 2017, COUPE discussion, unpublished observation). Review of existing literature on CBME in UGME demonstrates that it has been well analyzed. ^7,8,16,17 Specific to psychiatry, the American Association of Directors of Medical Student Education in Psychiatry (ADMSEP) produced “Key Diagnoses, Learning Goals and Milestones for Psychiatry in Undergraduate Medical Education” for US medical schools. ¹⁸

The Canadian Organization of Undergraduate Psychiatric Educators (COUPE) is the national group of UGME leaders in psychiatry, representing all Canadian medical programs, responsible for setting the national standards for UGME in psychiatry. COUPE aligns such standards with guidelines defined by the Medical Council of Canada (MCC) and the Association of Faculties of Medicine of Canada (AFMC).

In considering the application of CBME to undergraduate psychiatry in Canada, COUPE members reviewed the literature on CBME, with a focus on undergraduate psychiatry. COUPE would suggest that there is value to earlier consideration of CBME development in UGME, in parallel with PGME. Given UGME is the level where competencies can and should be broken down, a sequential application of CBME from UGME through to PGME enables PGME emphasis on building competencies focussed on integration. Furthermore, the common end point of UGME goals is much more uniform than in PGME, with its different specialties. ¹⁹ This article adds to the existing literature on CBME by identifying key concepts for consideration in adapting CBME specifically to undergraduate psychiatry education. In addition, the article proposes a novel, comprehensive national CBME framework for UGME in psychiatry across Canada.

Challenges of CBME

Does One Size Fit All?

CBME has been piloted primarily within the surgical/procedural specialties. Given psychiatry is more integration and interaction based, it is fair to question whether CBME is the best educational framework for this specialty—that is, does CBME apply to all specialties and clinical practices? While simulation has been a cornerstone of CBME in the procedural specialties, its role is less evident in a relationally based specialty such as psychiatry where microsystems are more variable and where there may be less observable behaviours and measurable outcomes. ⁵ Breaking down publicly valued physician attributes central to psychiatry (i.e., compassion, humanism, altruism, professionalism, and skilled communicator) will be difficult to do in objective, measurable terms, ¹¹ amenable to evaluation. Furthermore, evaluation of CBME efficacy will be challenging in nonprocedural specialties in contrast to surgical specialties, where outcomes are more tangible and outcome measurements are more achievable, even feasibly completed by machine, independent of subjective human observation.

Is CBME Too Reductionistic for Psychiatry?

One of the major criticisms of CBME is that it has a propensity to reduce learning and assessment to a series of “checkboxes” ^15,20 and that the parts will not add up to the whole of practice or ensure the capability to integrate across domains. ⁸ Acknowledging this, Holmboe et al. ²¹ suggest that integrative, multifaceted assessments begin to address this concern; multisource assessments result in a process that can synthesize the results of longitudinal and developmental assessment into a more comprehensive, holistic evaluation that is greater than the sum of its parts. Although conceptually palatable, the challenges of this may not be appreciated until in practice, as demonstrated in the American Outcome Project, which “has not resulted in widespread ‘operationalizing’ of outcomes in the evaluation of residents or in the accreditation of program…partly because of the complex nature of the competencies, which reflect high-level syntheses of more operationally measurable learning objectives.” ^22(p2) This has led to the development of “milestones,” facilitating the developmental progression of observable behaviours, ^{23 –25} although the Milestones Project has been perceived by some as idealistic and limited in its implementation considerations. ¹³

The CBME model should include explicit expectations and assessments while avoiding deconstruction into increasingly smaller competency units or focus on competencies that are easy to describe and assess. ¹⁹ Ideally, these simpler competencies would establish a solid foundation for CBME at the UGME level. This would then allow PGME to focus on the development of higher level competencies. These should, by definition, require the integration of knowledge, skills, and attitudes learned earlier, across the educational continuum. ⁸ This would be facilitated by using entrustable professional activities (EPAs), described as concrete critical clinical activities generally encompassing multiple competencies, ^19,26 demonstrated in context. ¹² In contrast to deconstructed competency components, EPAs potentially provide a more meaningful context for assessment and feedback, readily mapping onto milestones and providing a functional link between competency and performance. ^27,28

The assessment of such integration will present a complex challenge, given it will not be amenable to many of the familiar evaluation tools used in PGME. ²⁹ Moreover, complex and valued personal/professional attributes cannot be reduced to observable behaviours, thus calling into question their place as “competencies.” ^5,14 As noted in the Flexner centenary report, CBME then inadvertently risks losing humanism, accountability, and altruism, attributes that society expects of medical education. ³⁰

Will the Assessors Be Competent?

Realistically, CBME will only be as successful as the faculty who observe and evaluate the competencies of learners. ²¹ Competencies will be developed to align with 21st-century medical practices (e.g., systems functionalities and context-aware professionalism), in which most faculty have not received training. ^10,31 CBME requires not only improved multimodal assessment tools but also motivated, trained, and skilled faculty to use these tools, ²¹ in a model of criterion-based assessment. ⁷ Consequently, successful faculty development will require faculty to expand their proficiency in CanMEDS roles such as collaborator, professional, and health advocate. This is in contrast to the traditional current UGME model, which primarily emphasizes medical knowledge acquisition as the core clinical competency. ⁹ In addition, “CBME should be integral to a career that includes ongoing assessment,” ^21(p680) and thus training and assessing the assessors will be an integral but novel concept in medical training. This will be a paradigm shift from “finding the perfect tool to focusing on the faculty who use the tool.” ¹⁰

Advantages of CBME

CBME implementation will involve increased attention to the development of an explicit, structured curriculum. With this will come a standardization of the approach to teaching and assessment, contingent on clear benchmarks. The resulting faculty training necessary for providing increased direct observation and feedback may improve not only the learning experience but also the quality of teachers who will have to role model explicit competencies while demonstrating competency in assessment and feedback, a novel demand of faculty. The emphasis on developmental progression through high-quality assessment (requiring multiple components and qualified assessors, tools, and assessor training ¹² ), combined with the explicitly defined curriculum, will facilitate the identification of gaps and shift focus from the process of education to the type of doctor produced. ⁵

CBME in Psychiatry

Despite concerns about CBME, and specifically its suitability to nonprocedural specialties such as psychiatry, there is a risk in suggesting that CBME does not apply to psychiatry. Historically, psychiatry has struggled with discrimination and marginalization from both the public and the medical community, largely through stigmatizing preconceived notions and misunderstanding of the specialty. Consequently, to even suggest that CBME is not “as suitable” to psychiatry could further ostracize psychiatry from medicine. As such, it behooves the psychiatric educational community to find ways to adapt the CBME model to psychiatry and to incorporate the complexity of learning and assessment of its multifaceted skills. Developing an UGME framework in continuity with PGME begins that process of breaking down and then building up competencies to the ultimate skills of integration in the context of unsupervised practice (i.e., EPAs). This developmental progression and shift in focus will require that faculty and residents think differently about assessment, accepting that performance at a “lower” level on the milestone reporting form is acceptable and expected for an early learner. This will represent a cultural shift for learners and assessors, focussing on development, not outcome, through constant improvement and remediation, ^12,13 concepts central to lifelong learning.

COUPE Framework for CBME in Undergraduate Psychiatry Education

Medical education has embraced CBME and accepted its challenges. The question remains how to best apply CBME to psychiatry. Through consensus, COUPE produced an integrative framework that attempts to address this question, as it pertains to UGME. This framework embodies several philosophies: 1) the reductionistic nature of CBME may make it more easily applicable to UGME; 2) UGME competency building blocks enable PGME to explicitly define a higher level competency of integration; 3) a seamless approach from UGME to PGME is intrinsic to CBME, with the end point of UGME being the start point of PGME; and 4) the essentials of CBME will not only involve defining and assessing competencies but also relating competencies to real-life EPAs.

The process for this framework development involved review of existing literature and frameworks, as well as a COUPE group discussion. This led to consensus development and integration of existing frameworks into a model reflecting the current literature on CBME.

This framework has several unique features: it incorporates multiple existing frameworks (AFMC EPAs, CanMEDS competencies, ADMSEP learning goals and milestones, COUPE objectives, MCC themes, and the RCPSC collaboration regarding Mental Health Core Competencies for Physicians ³² ), delineates instructional methods and assessment tools that will be essential for CBME implementation, and describes continuity from preclerkship to clerkship.

The goals of creating such a framework are to 1) create national consensus and a guideline for Canadian accreditation standards, 2) anticipate curricular changes and create a seamless flow for CBME from UGME to PGME, and 3) incorporate existing terminologies in one framework to clearly delineate their relationships and avoid inappropriate interchange of terminologies. COUPE Framework for CBME in Undergraduate Psychiatry Education is available on-line as supplemental material.

Next Steps

To facilitate the success of CBME in psychiatry, next steps should include aligning the UGME framework with PGME to create continuity of developmental learning while developing curriculum and assessment tools. An important consideration is that CBME may be objective at the competence definition level but risks becoming subjective at the assessment level, if not well executed. Clear curricula and evaluation design are thus necessary to make the model successful. An additional consideration will be to distinguish “training” versus “education” ³³ and the possibility that a focus on training doctors versus educating physicians could contribute to loss of meaning in the interpersonal interactions, inadvertently contributing to rising rates of burnout. Historically, there has been an evolution in the role of “physician as person” through the CanMEDS frameworks, with the explicit role being lost in the iterative process. To address this, the Dutch created the role of “reflector” and placed it as the stem of the CanMEDS flower, representing a role that supports and nourishes each of the 7 CanMEDS roles. ³⁴ Further distinguishing education versus training, Morcke et al. stated, “It seems illogical to subsume personal and professional attributes that develop through education into the same process as competences that are developed by training. To do so would potentially be to deny the importance of relationships in medical practice.” ^5(p861) Given this, including identity alongside competency allows a reframing of our approaches to medical education away from an exclusive focus on “doing the work of a physician” towards a focus that includes “being a physician.” ¹⁴

The most important next steps will be prioritizing evaluative research to ensure CBME is “working” (i.e., truly producing better physicians). Strong consideration should be given to studying proposals before mandating them, ^35,36 minimizing risk of implementation without supporting evidence and of changes without meaningful outcomes. ^35,37 Research should be theoretically and methodologically diverse, ³⁸ including consideration of whether CBME is more applicable to certain domains and specialties than others.

The RCPSC states that the greater goal of CBD is “to enhance patient care by improving learning and assessment across the continuum from residency to retirement” (RCPSC website; http://www.royalcollege.ca/rcsite/documents/cbd/resident-developoment-orientation-cbd-e.pptx). This would suggest that perhaps desired outcomes should be phrased in patient-relevant terms. Research must be geared to answering the question of whether CBME produces better doctors who take good care of patients and have good outcomes in unsupervised practice. ^35,36,39

Conclusion

Clearly, there are varying perspectives on the advantages and disadvantages of CBME. If, however, the implementation of CBME results in a more structured curriculum, and the increased direct observation and feedback embedded within this lead to greater faculty engagement and accountability, then we may well be on our way to improving our accountability to the public, across all domains of medical education and practice.

Supplemental Material

Supplemental Material, COUPE_Competency_Framework.V6.submission - Does One Size Truly Fit All? The COUPE Undergraduate Perspective on Competency-Based Medical Education in Psychiatry

Supplemental Material, COUPE_Competency_Framework.V6.submission for Does One Size Truly Fit All? The COUPE Undergraduate Perspective on Competency-Based Medical Education in Psychiatry by Natasja Menezes, Raed Hawa, Ron Oswald, and Elliott Kyung Lee in The Canadian Journal of Psychiatry