Sage Journals: Discover world-class research

Abstract

Study Design:

Retrospective data analysis study.

Objective:

Attending continuing professional development (CPD) and continuing medical education (CME) activities is a necessity for practicing surgeons in most parts of the world. To enhance best practices in conducting CME/CPD, objective evaluation of these events is crucial. This article aims to evaluate one such international standardized CPD course conducted for facial surgeons across the globe. The Management of Facial Trauma course was developed by an international planning committee of experienced surgeons and has been implemented in all regions of the world.

Method:

This 2-day course is delivered using a combination of short lectures, small group discussions, and practical hands-on activities. Data collected from pre- and post-course evaluations of 86 Management of Facial Trauma courses conducted worldwide from 2017-2019 were collated and analyzed.

Results:

Participant demographics and experience levels varied slightly across the regions. Evaluation of the course effectiveness revealed overall high ratings for educational impact, content usefulness, and faculty performance.

Conclusion:

Our results indicated that this standardized course met the audience needs and enabled participants to plan changes in clinical practice. In addition, it confirmed that the course was relevant across different specialties and across different cultures and countries.

Keywords

continuing professional development continuing medical education facial trauma program evaluation competency-based curriculum

Introduction

Continuing professional development (CPD) and continuing medical education (CME) activities are essential for enhancing and supporting clinical skills of physicians and surgeons enabling them to learn the principles and the latest, evidence-based management for patients in their specialty. CME/CPD activities also contribute significantly to the delivery of high quality, safe patient care.^1,2 CPD activities across the globe show variations in their quality of planning and delivery and there is a lack of evidence regarding the optimal ways to plan and deliver CME/CPD.^3,4 Advances in best practices in CME/CPD activities require consistent evaluation of experiences and sharing of knowledge by institutions involved in CME/CPD activities across the world to benefit those who wish to emulate these efforts for practical purposes in local contexts.^5,6

The AO (Association for the study of Osteosynthesis in English) Foundation is an international not-for-profit organization established in 1958 with the mission to transform surgery and change lives.⁷ The organization is structured in clinical divisions, each of which run educational activities for their specialty.⁸ The AO CMF (craniomaxillofacial) clinical division was established in 1974. AO CMF runs specific educational activities in the form of courses, seminars, symposia, and webinar/webcasts. Up to 2014, AO CMF ran its “Principles of Fracture management course” in many countries worldwide. In the same year, the AO CMF International Board reviewed the program and noticed that the original templated program had evolved in many regions and countries and the content was no longer the same everywhere. They decided to standardize the curriculum across all regions, and to create a new course based on patient problems and the surgeon competencies required to manage them.

To develop the new structure, an educational task force was appointed, and a series of taskforce meetings and faculty retreats were held over a 2-year period. The educational taskforce consisted of 5 international expert faculty who applied a backward planning process to develop a competency-based CMF trauma curriculum.^9,10 The result was a new course format which was piloted in 2 educational events and launched worldwide in 2017 with the new name “AO CMF Course-Management of Facial Trauma” (AO CMF-MFT course). The goal of this new course is to address the core concepts and competencies necessary to manage acute facial trauma.

From 2017 to 2019, the course was delivered 106 times in 30 countries in multiple languages across all regions of the world. It utilizes a 3-step approach consisting of initial online preparation (with pre-course self-assessment and self-directed learning materials), a live face-to-face, 2-day course comprising standardized short lectures (30%), small group discussions (37%), and practical exercises (33%), and an online post-course evaluation.

This article will present an analysis of the evaluation of 86 AO CMF-MFT courses (for which both pre- and post-course data was available in our centralized information system) based on responses collected among 3,406 participants worldwide. Our purpose is to share knowledge and add to the evidence base for establishing best practices in CME/CPD in the oral and maxillofacial surgery domain. The key aims of our evaluation process are to find out: what are the variations in the target audience attending the AO CMF-MFT course across the different AO regions? is the course meeting the educational needs of surgeons in various parts of the world? and has the introduction of program modifications improved the educational impact for participants over 3 years?

Materials and Methods

Our Educational Intervention

The AO CMF-MFT course was developed for surgeons, residents, and fellows who manage patients with facial trauma. The course was scheduled in all regions of the world through an annual planning process by AO CMF. The appointed chairperson and faculty for each course received the curriculum and all the teaching materials and resources. An event team organized the venue and communication to participants (typically ranging from 24 to 64) and faculty (6 to 18). Registration was available online and participants complete a pre-course assessment (7 profiling and self-assessment questions and 12 multiple choice questions (MCQs)) and a set of preparation readings and online materials. The pre-course assessment helped the faculty by providing participant demographics, experience levels, and self-assessed educational gaps, and these data are supported by objective data from the MCQs (clinical scenarios with 4 management options). Both the assessment and evaluation system and our approach to MCQs are reported separately in the literature.^11,12

The 2-day, face-to-face course is divided into 3 modules. On day 1, the first module runs for 8 hours and focuses on mid and upper face trauma. The second module runs for 5.5 hours on day 2 and focuses on mandibular trauma. The last 1.5-hour module addresses additional topics of pediatric fractures and pan facial fractures. Content is delivered through a combination of educational activities that were selected based on the planned learning objectives: 12 lectures, 4 small group discussions (13 cases), and 6 hands-on practical exercises, with almost equal time allocation (Figure 1).

Figure 1.

Examples of lectures, small group discussions, and hands-on practical exercises.

Version 1 of the program was launched in January 2017. Content updates were made in September 2018, with text updates, image enhancements, and some content replacements in the lectures and in the case discussions based on 187 feedback comments from faculty. Version 2 was launched in January 2019 with images and text changes in most lectures and 4 cases replaced. At the end of every course, participants complete an onsite feedback form and an online post-course evaluation consisting of 8 questions and 12 MCQs.

Study Design

This study was a retrospective analysis of data collected from pre- and post-course evaluations from 86 MFT courses that occurred between January 2017 and December 2019. Ethical approval for the study was obtained from the institution of the primary author (ERC/IRB no.0011)

Data Collection

Data from the courses were gathered through an automated pre- and post-course evaluation link from SurveyMonkey that was emailed to all registered course participants. Participants were asked for their informed consent before responding to the questions. Most of the questions were multiple choice or rating questions using a Likert scale (1 = poorest to 5 = best).

All the analyzed data from the online evaluations were collected and sent in a standardized report format to each course chairperson. The chairperson reviewed them and provided feedback to the regional board and global education task force to help optimize future events. All centralized evaluations were conducted in English or Spanish.

Data Analysis

For this study, the data from 2,146 pre- and 1,726 post-course evaluation responses from the participants on 86 MFT courses was analyzed. Data analysis was performed specifically for those parts of the evaluations that were relevant to answering our research questions. Data was analyzed using Excel, R-studio, GraphPad Prism software and the Statistical Package of Social Sciences version 25.0 for Windows (IBM Corp, Armonk, NY, USA). A Fisher’s Exact Test was used to test for possible differences in course impact per group of participants with variable experience levels. Furthermore, a non-parametric Wilcoxon signed-rank test was used to observe whether the pre- and post-course motivation differed significantly from each other. The distribution of the continuous variable was checked by making a histogram of the obtained data. If the P value reached a level <0.05, the difference was considered to be statistically significant.

Results

The 86 AO CMF-MFT courses analyzed were attended by 3,406 participants across the globe over 3 years. Average response rates for the pre-course and post-course evaluations were 63% and 51% respectively. The Asia Pacific (AP) region hosted the most courses with a total of 1,242 participants (Table 1). The largest average number of participants per course was in the North America (NA) region (51) followed by Asia Pacific (AP) (43). The Middle East and Northern Africa (MENA) region hosted the fewest courses with a total of 302 participants. The average response rates for the pre- and post-course evaluations were highest in the Europe and Southern Africa region (ESA) and lowest in AP (Table 1).

Table 1.

AO CMF-MFT Courses Across the AO Regions 2017-2019 (n = 86 Courses).

	NA	LA	MENA	AP	ESA
Number of courses	17	14	10	29	16
Total number of participants	863	571	302	1,242	428
Average participants per course	51 ± 11	41 ± 10	30 ± 8	43 ± 13	27 ± 5
Response rate pre-event (responses/participants)	81 ± 23	61 ± 28	69 ± 20	54 ± 26	82 ± 18
Response rate post-event (responses/participants)	61 ± 13	44 ± 18	54 ± 9	42 ± 17	61 ± 15

Analysis of the demographic data showed that the course was attended mainly by participants who had graduated 0-5 years ago in the NA region, 3-10 years ago in ESA and LA regions, and 6-15 years ago in the MENA and AP regions (Table 2). The MENA region had the highest proportion of participants who graduated more than 15 years ago (11.76%).

Table 2.

Participant Profile (Registered Participants Who Completed the Pre-Event Questions).

	NA	LA	MENA	AP	ESA
Years since graduation from medical or dental school (%)
0–2 years	47	17	13	22	18
3–5 years	35	34	17	13	51
6–10 years	8	35	29	30	22
11–15 years	4	8	29	29	4
More than 15 years	6	6	12	6	5
Specialty (%)
Head and neck surgery	8	13	1	3	4
Oral/maxillofacial surgery	36	59	72	56	83
Plastic surgery	20	18	20	24	7
Neurosurgery	0.3	0	0	0.3	0
Ophthalmology/orbital surgery	0.3	0	1	0.2	0
Oculoplastic surgery	0	0	0	0.3	0
Ear, nose, and throat surgery	33	3	1	8	3
Radiology	0.3	0	0	0	0
Other	1.3	7	6	7	3
Practice location (%)
Level I trauma center	62	7	15	20	25
Level II trauma center	7	18	14	16	13
Local or community hospital	7	24	14	13	11
University hospital	21	37	47	40	48
Private practice	2	10	6	6	2
Other	1	4	4	5	1

In all the AO regions except NA, the participants were mostly oral and maxillofacial surgeons (Table 2). This category represented more than 80% of participants in ESA, more than 70% in MENA, and around 55% in LA and AP regions. In the NA region, Ear, Nose and Throat (ENT) surgeons and head and neck surgeons are considered to belong to the same speciality and together they were the most frequent specialty attending this course (41%). The second most frequent category were plastic surgeons in all regions except NA where this place was occupied by oral and maxillofacial surgeons.

The most frequent practice location in all AO regions except NA was a university hospital (35-48%) (Table 2), followed by Level I trauma centers in ESA (25%), AP (20%) and MENA regions (15%), and local or community hospital in LA (24%). In the NA region, the commonest practice location was level I trauma center (63%) followed by university Hospital (21%). Private practice was the least frequent category in ESA and NA regions (2.21% and 0.67%) while it represented almost 10% in the LA region.

Analysis of the course effectiveness revealed overall high ratings for educational impact, content usefulness and faculty effectiveness with some regional variations (Figure 2).

Figure 2.

Educational impact, overall content usefulness, and overall faculty effectiveness per region (each aspect rated on a scale of 1 to 5) over 3 years.

To better define the overall impact of the course, we considered 4 main aspects. First, we analyzed whether the courses met the audience needs. In all our regions more than 95% of course participants reported they had learned something new that they would use in their practice or that the course had reconfirmed that their current practice in the treatment of facial trauma was appropriate (Table 3). From over 1,726 respondents worldwide, only 2 participants reported that they had not learned anything new in the course, while 4 participants reported that they do not want to use what they learned in their practice, and 40 reported that they will not be able to use what they learned in their practice. The distribution was not influenced by the years of experience of the respondents (Table 4).

Table 3.

Averages of Participant Evaluation Data Collected Post-Course Per Region for 3 Years.

	NA	LA	MENA	AP	ESA
Educational impact (% for each answer option)
I learned something new and plan to use it in my practice	89	93	83	80	84
It reconfirmed that what I do in my practice setting is appropriate	8	6	15	16	11
I learned something new, but probably won’t be able to use it in my practice	3	1	1	3	4
I learned something new, but do not want to use it in my practice	0	0	0	0.58	0.37
I did not learn anything new	0	0	0.62	0	0.37
How well were the objectives met? (Likert scale, 1 = “Not at all met” up to 5 = “Fully met”)
C1: Diagnose facial injury through history, physical examination, and investigations	4.7	4.7	4.4	4.4	4.4
C2: Formulate a treatment plan (operative and nonoperative)	4.7	4.7	4.3	4.4	4.4
C3: Perform the specific treatment for facial trauma	4.5	4.7	4.3	4.3	4.3
C4: Modify the treatment plan when necessary	4.5	4.5	4.1	4.2	4.1
C5: Manage patient follow-up and rehabilitation	4.3	4.3	4.0	4.7	3.9
C6: Identify and manage complications	4.3	4.3	3.8	3.9	3.8
Intended practice changes (% of intended changes per competency)
C1: Diagnose facial injury through history, physical examination, and investigations	15	15	15	15	10
C2: Formulate a treatment plan	21	18	18	19	47
C3: Perform the specific treatment for facial trauma	22	21	19	20	17
C4: Modify the treatment plan when necessary	16	16	17	18	12
C5: Manage patient follow-up and rehabilitation	14	14	15	14	7
C6: Identify and manage complications	12	16	16	15	9

Table 4.

Educational Impact Over Years of Experience of the Course Participants Shows No Differences. Statistical Test Fisher’s Exact Test (P value = 0.296). N = 731.

Years of Experience	I learned something new and plan to use it in my practice	It reconfirmed that what I do in my practice setting is appropriate	I learned something new, but probably will not be able to use it in my practice	I learned something new, but do not want to use it in my practice	I did not learn anything new
0-2 years	98 (83.1%)	16 (13.6%)	4 (3.4%)	—	—
3-5 years	225 (86.9%)	31 (12.0%)	2 (0.8%)	—	1 (0.4%)
6-10 years	184 (84.0%)	29 (13.2%)	5 (2.3%)	1 (0.5%)	—
11-15 years	65 (76.5%)	14 (16.5%)	4 (4.7%)	1 (1.2%)	1 (1.2%)
>15 years	44 (88.0%)	6 (12.0%)	—	—	—

Secondly, we analyzed how well the course competencies/objectives were met. In most regions, the first 5 course competencies (C1-C5) had average ratings above 4 on a Likert scale (1 = not met at all to 5 fully met) and competency C6 showed a slightly lower score. All regions showed the same pattern with the highest ratings for competencies 1, 2, and 3, and lowest for competencies 5 and 6 (Table 3). From 1,726 responses, 21 reported C6 was not met at all, 7 participants reported the same for C5 and only 1 participant for C4.

The third aspect we analyzed was the intended changes the participants would make in their current practices, categorized by the competencies. In NA, LA, MENA, and AP regions, the intended changes were almost equally distributed among competencies with a slightly higher frequency (about 20%) related to competency C3 (Perform the specific treatment for facial trauma) while ESA had 46% of intended changes related to C2 (Formulate a treatment plan (operative and nonoperative) followed by C3 (16%) and C4 (11%) (Table 3).

Lastly, we also analyzed participants’ present and desired level of competency (the “gap score” which show the motivation to learn) for each of the 6 competencies both pre- and post-course. The gap between the present and desired level of ability for each competency was reduced post-course. This reduction in gaps was not only shown for all competencies but was consistent for all the AO regions (Figure 3).

Figure 3.

Pre- and post-course motivation to learn (difference between Present and Desired level of ability) per competency and per region. Only participants that responded to both pre- and post-questions were included. AP n = 270; ESA n = 208; LA n = 150; MENA n = 103. P value was calculated for the gap, based on a Wilcoxon signed-rank test. ***P value <0.001.

Over 3 years of course implementation, minor changes were made to improve the quality of the educational content in response to feedback from faculty and evaluation data. To evaluate the impact of these changes, we further analyzed the course impact over the years. We found that Likert scale average values for all competencies showed a tendency for improvement over the years except C5 which showed a slight decline (Figure 4).

Figure 4.

Averages from participant responses to evaluation questions “To what degree were the stated objectives met?” over 3 years of courses per competency C1 to C6 (1 = not at all met, 5 = fully met). Data from NA are not included in this analysis.

Similarly, modifications in course contents had a positive effect and content usefulness improved over the 3 years (Figure 5). The educational impact, however remained almost the same and showed slight variation over the years, including a reduction during 2019 (Figure 5).

Figure 5.

Averages from participant responses to evaluation questions “What was the overall impact of this educational event?” and “How useful was the content to your daily practice?” over 3 years of courses (5 = I learned something new and plan to use it in my practice and 5 = Extremely useful). Data from NA are not included in this analysis.

Discussion

The evaluation of specific CME/CPD activities helps determine their value for participants and helps planning committees to decide what to do differently next time.¹³. For our appraisal of the AO CMF-MFT course we employed Moore’s expanded framework of outcomes for CME activities.¹⁰

To understand the level of participation and variations within, we analyzed the demographic data of our participants. The AP region accounted for about 34% of the total courses and 37% of total participants included in this study. The fact that the AP region is home to about 60% of the world population may explain the higher need for these courses as well as higher number of participants.

In the ESA region, higher response rates were noted compared with other regions. Evaluation of participant profiles revealed that MENA and AP regions had a higher proportion of experienced participants, suggesting that the need for CPD courses is high in these regions even among qualified surgeons.^14
–16 In ESA and NA, the ratio of younger surgeons was relatively higher, showing that the course was also popular among postgraduate residents and fresh graduates.

Analysis of practice locations indicated that a majority of course participants across the AO regions were working either in teaching hospitals or Level I trauma centers with all facilities available to them. However, the LA region showed a significant number of attendees from community hospitals. A possible explanation for this could be that in Latin America, trauma care is traditionally not as concentrated in Level I trauma centers or teaching hospitals compared with other parts of the world. The care of trauma may be less organized and the region has a more diverse health coverage system (mixture of public medicine, labor insurance, and private insurance).^17,18 Overall, the participant profile is similar among regions with slight differences regarding the years of experience and the specialties represented.

Satisfaction of participants in each of the AO regions can be determined by reviewing all aspects of the course ratings: educational impact, overall content usefulness, and faculty effectiveness. The combination of these factors provides an impression regarding the delivery of each course. Averaging the values for each AO region showed high satisfaction ratings with small variations (Figure 2).

Literature suggests that measuring the outcomes of a CPD activity can be done by utilizing a number of assessment formats¹⁹ all of which contribute to its overall impact.²⁰.We determined the course impact in each AO region by evaluating 4 main aspects.

First we determined whether our course met the audience needs, and if it lead to changes in their practice.²¹ Our results showed that over 95% of course participants in all regions reported they had learned something new and would be able to apply it in their clinical practice. Only 5% of worldwide course participants did not have a “high impact” outcome.

Our second domain for determining impact was by reviewing the changes in self-assessed levels of competency (ability) for each of our 6 predefined course competencies/objectives. We further analyzed how well the competencies/objectives were met over the years. Slight improvements in the competencies could be seen (Figures 4 and Figure 5), reflecting improvements in educational content over the successive course versions.

To further analyze course impact, we reviewed the intended practice changes assigned to each of the course competencies. The intended changes did indeed show variations between the regions, with NA, LA, MENA, and AP showing almost similar results. Course participants in these regions reported that they intend to change their practice mainly for competency C3: Perform the specific treatment for facial trauma. However, almost half the participants from the ESA region intended to change their practice mainly for competency C2: Formulate a treatment plan (operative and nonoperative).

The last aspect of our evaluation was related to analyzing participants self-reported gains in ability for each competency by analyzing the present and desired levels of ability for each of the 6 objectives from both pre- and post-course evaluation. We noted a significant reduction in post course gaps, as we might expect. However, this reduction was different for each competency and we observed that competency C1 consistently had the smallest difference between pre and post gaps. The pre-course gap for C1 is the smallest in each region and indicated that participants had less motivation to learn related to this competency or were satisfied with their current ability. Competencies C3 and C4 showed the largest pre-course gaps between present and desired levels of competence indicating that course participants were motivated to learn these competencies.

Taken together, our results show that the AO CMF-MFT course is a valued pedagogical endeavor that contributes to CPD for facial surgeons across the world. With an overall similar participant’s profile, the AO CMF-MFT course scored well on its educational impact and content usefulness across the regions. The course objectives were met well, and the educational needs of most participants were met leading to application in their practice. While content revision and program modifications have improved the educational impact over the years, further revisions by incorporating blended, online learning elements will probably make the course accessible and affordable to more participants in future.

The standardization of course content has helped to ensure that the course meets the educational needs of surgeons across the world.²² The success observed with this course in terms of educational quality and acceptance encourages us to explore standardization of other AO CMF educational offerings. Our evaluation data provides evidence that using a backward planning and a competency-based approach enables the development of a CPD course that is relevant across different specialties and across different cultures and countries. Our standardized content teaches surgeons worldwide how to treat facial trauma at the highest level possible.

Footnotes

Acknowledgments

This manuscript has been reviewed and agreed on by all authors prior to submission. The authors would like to acknowledge the support of the taskforce and commissions and boards who developed and approved the curriculum (especially Warren Schubert, chair of the AO CMF International Board during the development phase), all the chairs and faculty who conducted the courses worldwide, Diana Greiner, Sonia Rebustini, Sarah Groves, and all the team in AO CMF in all regions, and Mike Cunningham, Monica Ghidinelli, and Alain Rickli from the AO Education Institute (AO EI) for their support of the curriculum, data collection, and the preparation of this manuscript. The AO Education Institute (AOEI) thanks Dr Ayesha Younas for preparing this manuscript based on analyzing the documented steps, analyzing the data, and interviewing and collaborating with the coauthors as part of an international internship in our team.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Ayesha Younas, MHPE, MPH, BDS

References

Sachdeva

. Continuing professional development in the twenty-first century. J Contin Educ Health Prof. 2016;36:S8–S13.

Sargeant

Wong

Campbell

. CPD of the future: a partnership between quality improvement and competency-based education. Med Educ. 2018;52(1):125–135.

de Andrade

Griebenow

Costello

, et al. The future of accreditation of continuing medical education (CME)-continuing professional development (CPD) in Europe: harmonisation through dialogue and consensus. J Eur C. 2018;7(1):1506202. doi:10.1080/21614083.2018.1506202

Aboulsoud

Filipe

. Opening Editorial: CPD and Lifelong Learning: a call for an evidence based discussion. MedEdPublish. 2019;8(1). doi:10.15694/mep.2019.000001.1

Filipe

Golnik

Mack

. CPD? What happened to CME? CME and beyond. Med Teach. 2018;40(9):914–916. doi:10.1080/0142159X.2018.1471200

Braido

Comaschi

Valle

, et al. Knowledge and health care resource allocation: CME/CPD course guidelines-based efficacy. Eur Ann Allergy Clin Immunol. 2012;44(5):193.

Jeannet

. A brief overview of the AO Organization. In: Leading a Surgical Revolution. Springer International Publishing; 2019:15–19. doi:10.1007/978-3-030-01980-8_3

Joeris

Höglinger

Meier

, et al. The impact of the AO Foundation on fracture care: an evaluation of 60 years AO Foundation. Injury. 2019;50(11):1868–1875. doi:10.1016/j.injury.2019.07.016

Shaye

Tollefson

Shah

, et al. Backward planning a craniomaxillofacial trauma curriculum for the surgical workforce in low-resource settings. World J Surg. 2018;42(11):3514–3519. doi:10.1007/s00268-018-4690-y

10.

Moore

Green

Gallis

. Achieving desired results and improved outcomes: integrating planning and assessment throughout learning activities. J Contin Educ Health Prof. 2009;29(1):1–15. doi:10.1002/chp.20001

11.

Ghidinelli

Cunningham

Uhlmann

Rickli

. Designing and implementing a harmonized evaluation and assessment system for educational events worldwide. J Orthop Trauma. 2021:s5–s10. Accessed March 9, 2021. https://journals.lww.com/jorthotrauma/Documents/Learning in the AO_Looking Back and Moving Forward.pdf

12.

de Boer

Buckley

Schmidt

Fox

Jupiter

. Topics in training. J Bone Jt Surg Am. 2010;92:1325–1329.

13.

Shannon

. Programme evaluation in CME. Lancet. 2003;362(9389):1084. doi:10.1016/s0140-6736(03)14439-x

14.

O’Neill

Hansen

Nyagetuba

JKM

, et al. A plan for surgical education in low- and middle-income countries. J Trauma Acute Care Surg. 2017;83(5):784–787. doi:10.1097/TA.0000000000001556

15.

Raykar

Yorlets

Liu

, et al. The how project: understanding contextual challenges to global surgical care provision in low-resource settings. BMJ Glob Heal. 2016;1(4):e000075. doi:10.1136/bmjgh-2016-000075

16.

Raykar

Yorlets

Liu

, et al. A qualitative study exploring contextual challenges to surgical care provision in 21 LMICs. Lancet. 2015;385(Suppl 2):S15. doi:10.1016/s0140-6736(15)60810-8

17.

Rojas

LGP

Cervantes

REL

Atanasio

JMP

Sánchez

Acevedo

JMG

Kojima

. Latin America trauma systems—Mexico and Brazil. Ota Int. 2019;2(S1):e020.

18.

Arreola-Risa

Mock

Padilla

Cavazos

Maier

Jurkovich

. Trauma care systems in urban Latin America: the priorities should be prehospital and emergency room management. J Trauma Acute Care Surg. 1995;39(3):457–462.

19.

Balmer

. The transformation of continuing medical education (CME) in the United States. Adv Med Educ Pract. 2013;4:171–182. doi:10.2147/amep.s35087

20.

Wallace

May

. Assessing and enhancing quality through outcomes-based continuing professional development (CPD): a review of current practice. Vet Rec. 2016;179(20):515–520. doi:10.1136/vr.103862

21.

Légaré

Borduas

MacLeod

Sketris