Sage Journals: Discover world-class research

Abstract

Program evaluation is essential for medical schools to demonstrate social accountability and identify areas for improvement in medical education (MEd). Although stakeholder engagement is crucial in program evaluation, no previous review has specifically examined the stakeholders involved in MEd program evaluation. This scoping review addresses this gap by identifying the stakeholders, their roles, and their levels of engagement in evaluating MEd programs, along with the facilitators and barriers to their participation. Through a systematic search across four databases, we identified 53 relevant studies out of 7206 screened. Our findings reveal seven primary stakeholder groups, with students and program directors being the most frequent participants. However, a significant gap exists in the representation of community members and patients, indicating a need for greater inclusion of these key stakeholders. Additionally, we found that stakeholders are primarily engaged as passive participants providing feedback rather than actively shaping the evaluation process. Facilitators and barriers to participation were identified from the participants’ perspective, highlighting the need for further research to understand the viewpoints of active stakeholders, such as faculty and administrators. Future studies should also explore the impact of different evaluation approaches on stakeholder engagement to develop more inclusive and effective MEd program evaluations.

Keywords

medical education program evaluation stakeholder engagement social accountability

In today’s dynamic healthcare landscape, medical schools face the critical challenge of providing high-quality education while remaining socially accountable to their communities. The World Health Organization (WHO) defines social accountability for medical schools as ‘the obligation to direct their education, research, and service activities towards addressing the priority health concerns of the community, region, and/or nation they have a mandate to serve’ (Boelen et al., 1995, p. 3). This concept revolves around four core values: relevance, quality, cost-effectiveness, and equity in the healthcare system. The Global Consensus for Social Accountability of Medical Schools (2010) highlighted that the 21st Century presents medical schools with unique challenges, including improving quality, equity, relevance, and effectiveness in healthcare delivery.

Program evaluation is crucial in helping medical schools address these challenges, demonstrate progress on social accountability, and identify areas for improvement. The American Evaluation Association (AEA) defines program evaluation as ‘a systematic process to determine merit, worth, value, or significance’ (Patton et al., 2014, p. 1) aiming to make judgments about the program, improve its effectiveness, and inform decisions about future programming (Moreau, 2017; Patton, 1997). Evaluation allows medical schools to assess the effectiveness of their educational programs, identify areas for improvement, and ensure that their graduates are well-equipped to address community needs. Accreditation processes further emphasize the importance of program evaluation, mandating that medical schools regularly assess their educational programs and teaching methods (Santen et al., 2019; WFME, n.d; WFME & WHO, 2005).

Role of Stakeholder Engagement in Evaluation

Engaging diverse stakeholders in evaluation, including students, faculty, patients, and community representatives, is essential to maximize the benefits of evaluation (Moreau & Eady, 2023). Moreau and Eady (2023) identify three uses of evaluation findings in medical education (hereafter MEd): instrumental (direct action), conceptual (understanding without action), and symbolic (fulfilling requirements). Emphasizing the crucial role of diverse stakeholders in evaluation is pivotal to maximizing the benefits of these uses. Research suggests involving key stakeholders leads to more comprehensive and relevant assessments, greater program utilization, and increased stakeholder buy-in. This focus on stakeholder engagement is crucial for medical schools striving to fulfill their social accountability obligations. By incorporating diverse perspectives throughout the evaluation process, medical schools can ensure that their programs effectively address the specific health needs of their communities, ultimately leading to improved healthcare outcomes and a stronger connection with the community (Berk, 2009; Committee on Accredidation of Canadian Medical Schools [CACMS], 2024; WFME, 2020).

Research Gaps and Questions

However, the literature lacks clarity regarding the specific stakeholders currently engaged and the extent of their involvement in the evaluation process of MEd programs. To address this gap, the following research questions are proposed: RQ1 What are the different roles of stakeholders identified in the MEd program evaluations?; RQ2 How are these stakeholders engaged in MEd program evaluations, and what are the facilitators and barriers to their participation? By answering these research questions, we can gain a comprehensive understanding of stakeholder engagement in evaluating MEd programs. This will enable researchers and evaluators to design effective strategies for maximizing stakeholder participation and integrating diverse perspectives. This knowledge will empower medical schools to fulfill their social accountability obligations more effectively, improving MEd and healthcare delivery.

Methods

To answer our research questions, we conducted a scoping review of stakeholders’ participation in the MEd program evaluation, adhering to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) guidelines (Peters et al., 2015; Tricco et al., 2018).

Search Strategy

We developed a comprehensive search strategy, incorporating best practices recommended by the university’s research support services. This strategy encompassed a broad range of keywords and synonyms related to stakeholder engagement in MEd program evaluation, including: stakeholder (participant, medical student, resident, teacher, patient, faculty), engagement (involvement, participation), and MEd (medicine, curriculum, program evaluation, course evaluation). We also incorporated relevant medical subject headings (MeSH terms). We searched four databases: Embase, Web of Science (WOS), CINAHL, and PsychInfo. To reflect current evaluation practices in medical schools, we limited our search to peer-reviewed studies published in English or French after 2013, excluding conference abstracts, letters, editorials, and review articles. The initial search occurred on February 17, 2023, with a final update on May 30, 2024. Appendix 1 provides detailed search equations for each database and article identification numbers. Identified articles were imported into the Covidence literature review management tool (Veritas Health Innovation) for further screening and analysis.

Study Selection

After removing duplicates, two authors independently reviewed titles, abstracts, and full-text articles based on the following inclusion criteria: focus on evaluating programs for undergraduate or graduate medical students, clearly identify the stakeholders involved in the program evaluation, and discuss methods for recruiting stakeholders, incentives for participation, and challenges faced during recruitment. We excluded studies where program evaluations involved students other than medical students (e.g., dental, osteopathic, or mixed student groups). We also excluded studies that evaluated teaching or assessment tools, as our aim was to highlight stakeholders’ participation in MEd program evaluations within faculty settings. Additionally, we excluded evaluations of programs in clinical settings to maintain a focus on faculty environments and evaluations of student performance, as these are typically assessed through standard course evaluations and do not directly address stakeholder participation in program evaluations. As a pilot, the two reviewing authors independently screened 15 randomly selected references to ensure consistent application of the eligibility criteria. This comprehensive review process aimed to ensure accuracy and minimize bias. Any disagreements regarding the inclusion or exclusion of articles were resolved through collaborative discussion between the two researchers at the end of each selection phase.

Data Extraction

The first and second authors independently extracted data from full-text articles using a standardized data extraction form within Covidence. The form developed and piloted a priori captured data on study characteristics (aim and design), program evaluation details (evaluation design, specific program evaluated, aim of the evaluation, and tools used), and stakeholder information (type and number of stakeholders engaged, recruitment methods, facilitators, and obstacles to participation). After independent extraction, the authors convened a consensus meeting to compare and discuss their findings. Any discrepancies were resolved through discussion and reference to the original articles, ensuring the accuracy and completeness of the final dataset.

Data Analysis and Presentation

We conducted a descriptive analysis, categorizing studies based on their goals, evaluation designs, and stakeholder characteristics. Given the diversity of included studies, we focused on identifying common factors that facilitated or hindered stakeholder participation in MEd program evaluations. Our findings, structured around our two main research questions, adhere to PRISMA-ScR guidelines for transparency and rigour (Tricco et al., 2018).

Results

The PRISMA diagram presents the results of the search in Figure 1. From the four databases, 7206 studies were screened, and 53 were included in the review. Most articles were excluded for not addressing the recruitment of evaluators nor how they are encouraged to participate in the evaluation and for not being concerned with a program evaluation as defined by our criteria. Specifically, 34 articles focused solely on students’ academic performance, 22 articles did not pertain to medical students, 2 articles were not in English or French, 3 were not original articles, and 1 article was excluded due to an error in the processing software.

Figure 1.

PRISMA Diagram

Studies Characteristics

Detailed study characteristics are provided in Appendix 2. Most (72%) of the 53 included studies originate from North America, primarily the United States (68%). The remaining studies come from Europe (13%), China (6%), Australia, and Africa (8% combined). Notably, there has been a recent surge in publications on MEd program evaluation, with 45% published between 2020 and 2024 and 30% between 2016 and 2019, reflecting a growing interest in this area of research. The following sections present our findings, which are organized by our two research questions. Detailed evaluation characteristics are provided in Table 1.

Table 1.

Characteristics of Evaluations Included

Study ID	Program evaluated	Aim of the evaluation	Evaluation tool	Participants	Participants contacted	Responses (%)	Recruitment method
Alerhand et al., 2021	POCUS ultrasound course	Evaluate the prevalence, characteristics and outcomes of POCUS ultrasound courses in emergency medicine residency programs	Surveys	PD	267	199 (74.5)	E-mail via a distribution list, then individual e-mail to PDs if there is no response
Alt-Epping et al., 2014	Interdisciplinary conference on death preliminary to dissection module	Evaluate the content, method and organization; perceptions and experiences on the themes addressed by the teaching	Survey	Students	224	152 (67.9)	Distribution of the survey at the end of the conference
Ariail and al., 2013	Writing center	Assess students’ satisfaction with the writing center	Survey	Students	2514	904 (36)	Email
Bouzaher et al., 2020	Service learning project	Assess students’ perception of the service-learning project	Survey	Students	238	168 (70.6)	Email
Cameron et al., 2019	Life coaching program	Assess students’ satisfaction, feasibility and acceptability of the program	Surveys	Students	40	37 (92.5)	Announcements; information sessions; Flyers; E-mail
Campbell et al., 2020	Social Determinants of health program	To ascertain students’ views on the program	Survey; semi-structured interviews	Students	38	19 (50)	During the course
Cole et al., 2024	Antietam staff walk course	To determine if the objectives were met and to investigate possible approaches to improve this experience	Qualitative analysis of student reflexive papers	Student	1	NA	NS
				PhD curriculum researcher with expertise in program evaluation	1
				Military-medical historians	2
Curry, 2018	Clinical anesthesia	To determine what is taught in clinical anesthesia to medical students in the 133 hospitals in the U.S.	Survey	PD	125	85 (68)	E-mail; phone calls
Deora et al., 2020	Neurosurgery residency	Determine what are the different neurosurgical residency programs in middle-income countries and how they are certified, what resources are available and what are the most effective, what resources are needed to improve access to neurosurgical education, as well as to study the feasibility of improving neurosurgical residency programs in these countries assess the feasibility of improving neurosurgical residency programs in these countries	Surveys	Students	1000	195 (19.5)	E-mail; social network groups
Dubois et al., 2023	Undergraduate nutrition program	Determine students’ perception of nutrition training and performance	Surveys	Students	192	61 (31.8)	E-mail
Edwards et al., 2021	Oncology inpatient narrative medicine workshop	Assess the well-being of interns. Assess the views of teachers and PDs	Surveys semi-structured interviews	Students	36	NS	NS
				Workshop facilitator	NS	NS	E-mail
				PD	NS	2
				Other: Researcher, author	1	1 (100)
Egle & Mittal, 2014	General surgery residency	Identify whether changes in accreditation council for graduate medical e ducation (ACGME) requirements have induced changes in residents’ exposure to transplantation or individual program requirements. Determine PDs' views on these new requirements	Survey	PD	207	103 (49.8)	E-mail
Elmore et al., 2020	Radiation therapy residency	To evaluate the knowledge, attitudes and practices of radiation therapy PDs in global health	Surveys	PD	111 of 91 different programs	64 (57.6) of 55 different programs	E-mail
Eno and al., 2023	A longitudinal, within-program integration of humanities and ethics into the pre-clekship medical education curriculum	To evaluate the effectiveness of the program	Multisource: Interviews, essays, surveys	Students	168	Interviews: 27 (16,1) Essays: 168 (100) End-of-year survey: 168 (100)	Interviews: Surveys and school-based email accounts
Fernando et al., 2015	Community health Determinants course	Evaluate student satisfaction, career plans and skills acquisition	Surveys	Students	144	58 (40.3)	E-mail
Finnie et al., 2021	University of Vermont larner college of medicine pre-clinical program in social medicine (UVM larner)	Evaluate an expanded and innovative preclinical social medicine curriculum at the university of Vermont’s larner college of medicine (UVM larner) to propose a framework for the design of the social medicine curriculum	Surveys	Students	118	71 (60.2)	Newsletter information by e-mail
Finnie et al., 2021			Surveys	Faculty members	178	40 (22.5)	Email
Fisher et al., 2015	Undergraduate delirium curriculum	Describe the content of undergraduate delirium education in the UK and how it is delivered	Surveys	Deans	NS	31	Mail and e-mail to deans who select respondents with the most comprehensive view
Fisher et al., 2015	Undergraduate delirium curriculum		Surveys	Teachers	NS	24	Selected by deans or members of the british geriatrics society education and training committee in the absence of a response from the dean
Foster et al., 2018	Food supplement program	Assess the perceived usefulness of the teaching	Survey	Students	65	65 (100)	E-mail to PDs and then to students during the course period; on-site distribution of paper surveys
Gazibara et al., 2015	Epidemiology e-learning course	Evaluate the reasons for participation in e-learning and expectations and then assess students’ satisfaction and impressions	Survey	Students	516	60 (11.6) pre-courses 36 (7) post-courses	Systematic proposal during first practical teaching; email
Ge and al., 2022	Emergency medicine program	Take stock of the current state of emergency medicine teaching	Survey	Students	NS	571	WeChat social network transferred by teachers
Gonzalez & Donnelly, 2016	Anesthesia mentoring programs	Describe existing programs and outline areas residency directors feel should receive special attention	Surveys	PD	131	45 (34.4)	E-mail; phone calls
Hermann et al., 2021	Disaster medicine and humanitarian assistance courses	Assess satisfaction	Survey	Students	102	82 (80.4)	Distribution at the end of the course
Jaafari et al., 2019	Geriatrics program	Evaluate the quality of a geriatrics program in terms of the satisfaction of graduates, faculty members and students, and assess the concerns of educational staff regarding general medicine trainers	Semi-structured interviews; focus groups; field observation	Students	NS	6	Individual selection, then written invitation
				Faculty members		15
				Patients		6
				Other: Nurses		5
Jiang et al., 2020	Doctor-patient communication program	Evaluate the satisfaction, relevance, benefits and outcomes of a peer mentoring program; assess the acceptability and usefulness of the program	Surveys	Students	727	675 (92.8)	Directly during the course
Jujo et al., 2021	Cardiac ultrasound program	Evaluate program feasibility. Identify possible program improvements	Surveys	Students	NS	6	E-mail; public display; snowball word-of-mouth by student-investigator
Kamel et al., 2018	Urology residency	Assess the current use of simulation, acceptability of integrating a simulation program, and barriers to its implementation in urology residencies	Survey	PD	122	43 (35.2)	E-mail
Lapp et al., 2018	Mentoring program	Evaluate content, organization and topics covered	Surveys	Students	NS	Ten mentee groups of 5 to 20 students (100)	Paper document given out at the end of each session and collected before leaving; introductory events; word of mouth
Lapp et al., 2018	Mentoring program	Evaluate content, organization and topics covered	Surveys	Teachers	NS	NS	During regular meetings
Lebastchi & Yuh, 2014	Cardiothoracic surgery residency	Evaluate the motivations, satisfaction and ambitions of residents	Survey	Students	83	57 (68.7)	Electronic platform
Lepard et al., 2020	Neurosurgery residency	Determine teaching paradigms for neurosurgery residencies	Survey	PD	110	102 (92.7)	Automatic e-mail
Li et al., 2020	Entrepreneurship education program	Assess the quality of entrepreneurship education and student-teacher collaboration	Survey	Students	NS	16,660	E-mail with QR code
London et al., 2017	Residency in emergency medicine	Determine the state of patient experience education and assessment in emergency medicine residency programs in the U.S.	Survey	PDs and faculty members	194	139 (71.6) (including 15 faculty members)	E-mail
London et al., 2017	Residency in emergency medicine		Survey	Teachers	NS	153 (168 individuals answers)	E-mail
London et al., 2018	Neurology residency	Establish current PD demographics, training structure, assessment methods, program support and response to specific ACGME mandates	Surveys	PD	150	106 (70.7)	E-mail
Lotfipour et al., 2017	Cataract surgery	Evaluate factors enabling the early practice of phacoemulsification by residents	Survey	PD	116	51 (44)	E-mail
Mann et al., 2014	Quality improvement programs in pediatric residency programs	Define the structure, content, evaluation methods and objectives of quality improvement programs in pediatric residency programs in the U.S.	Surveys	PD	197	104 (52.8)	E-mail
Muszkat et al., 2014	Poetry-based integrated educational intervention	Evaluate whether the effects of an integrated, mandatory poetry-based intervention in a heterogeneous student population are related to the diversity of students’ characteristics and the timing of the program during medical school	Surveys	Students	144	112 (77.8)	On-site distribution of a paper at the end of the program
Naifeh et al., 2023	Pediatric residency	Evaluate changes related to the COVID-19 pandemic in residents’ educational experience, the effects of its disruptions on residents' readiness for more prominent roles	Surveys	PD	199	110 (55)	E-mail
Nayar et al., 2015	Plastic surgery residency programs	Evaluate exposure to global surgery in plastic surgery residency programs	Survey	PD	81	64 (79)	E-mail
Nevin et al., 2014	Gamification program	To evaluate the acceptability and use of a new gamification-based medical knowledge software designed to complement traditional medical training for internal medicine residents and to determine the retention of information on retesting	Focus groups	Students	152	17 (11.2)	E-mail; announcements at program conferences
Ryll et al., 2023	Otolaryngology in medical schools	To provide an up-to-date assessment of the state of OTO teaching at U.S. medical schools, to evaluate medical school factors contributing to the variability of subspecialty education, and to provide recommendations on how medical schools can improve their teaching in OTO	Survey	PD, program coordinators, clerkship directors, medical education coordinators, deans for medical education and registrars	155	68 (43.9)	Email, and, on occasion, phone calls to school departments to obtain new contact information
Shankar et al., 2022	Dr pete Dehnel public health advocacy Fellowship	To inform the development of program components, to respond effectively to changing needs	Online surveys (fellows and mentors), semi-structured interviews (between fellows and staff)	Students	51	22 (43.1) surveys	NA
Shufflebarger et al., 2023	Emergency social medicine	Evaluate the effects and impacts of the program on residents	Surveys	Residents	32	23 (71.9) pre-test; 14 (43.7) post-test; 7 (21.9) later	E-mail
				Students	1	1 (100) pre-test
				Faculty members	11	11 (100) pre-test; 4 (36.3) post-test; 3 (27.3) later
Siddiqui et al., 2016	Musculoskeletal ultrasound in physical medicine and rehabilitation residency programs	Identify the most commonly used training and assessment modalities and examine their effectiveness. Examine the effectiveness of these programs in training experienced and competent musculoskeletal sonographers	Survey	PD	78	36 (46.2)	E-mail
Smeds et al., 2021	Vascular surgery residency	Evaluate current vascular surgery training programs in the U.S. and compare them to traditional training programs	Survey	PD	112	80 (71.4)	E-mail
Soliman et al., 2017	Educational environment	Determine the perception of the second graduating class of king saud university of their educational environment	Survey	Students	245	62 (25.3)	E-mail
Sorensen et al., 2019	Teaching cultural competence in medical studies in europe	Evaluate the teaching of cultural competence in medical studies in europe	Survey	PD	12	12 (100)	E-mail sent to european partners communicating evaluation to DPs
Stewart-Brown et al., 2018	Wellness and mindfulness program	Determine the factors influencing students’ learning regarding their understanding and valuing of wellness activities and mindfulness	Survey	Students	NS	95	Verbal invitation in class; invitation on class social networks
Stewart-Brown et al., 2018	Wellness and mindfulness program		Semi-structured interviews	Students	NS	13	Individual selection for interviews
Sun et al., 2021		Assess whether residents are satisfied with their anesthesia training, characterize their difficulties and determine whether residents feel they are sufficiently prepared for the competencies required by the 6 ACGME core competencies for autonomous practice	Survey	Students	12,929	4707 (36.4)	On entry to the first clinical year, E-mail
Thompson Rodriguez et al., 2021	Pre-med pair up, a distance mentoring program between medical students and undergraduate students	Evaluate the “pre-med pair up” mentoring program	Survey	Students	26 mentors + 43 mentees = 69	11 mentors + 13 mentees = 24 (34.8)	E-mail
Veazey et al., 2015	Optional teaching of family planning	Assess students’ motivation to participate in the course, their perception of the course, their goals for future practice and their overall satisfaction after the course	Phone interview	Students	NS	29	Letter from teaching directors themselves recruited by e-mail
Wanstreet et al., 2020	Teaching of anatomy by ENT residents	Determine whether ENT residents teach anatomy at the medical school, whether the program director perceives the benefits, and how the program manages the logistics of resident involvement	Survey	PD	121	45 (37.2)	E-mail
Ward et al., 2018	Student wellness program	Determine whether interns perceive wellness training as useful, relevant and exciting to consider the integration of an ongoing program into mandatory training. Determine what interns found relevant and exciting, what improvements could be made to the future program, and perceptions of job satisfaction and work-life balance. Explore workplace stressors that may affect interns’ wellness	Survey	Students	NS	35	E-mail
Ward et al., 2018	Student wellness program		In-person or phone interview	Students	NS	6	E-mail
Weis et al., 2019	Team simulation program	Assess quality via students’ satisfaction with the program and faculty	Survey post-intervention	Students	53	52 (98.1)	E-mail
Zumberg et al., 2015	Oncohaematology programs	Assessment of oncohaematology teaching; evaluating satisfaction	Survey	PD	130	68 (52.3)	E-mail

Note: NS = not specified; NA = not applicable.

RQ1

Stakeholders Role Identified in MEd Program Evaluations

Our review identified seven distinct groups of stakeholders involved in MEd program evaluations. Students (n = 33, 62%) and program directors (n = 20, 38%) were the most frequently included stakeholders. Other stakeholder groups identified included teachers (n = 4, 8%), faculty members (n = 5, 9%), patients (n = 1, 2%), and nurses (n = 1, 2%). A few studies included a broader category of stakeholders encompassing deans, program evaluation experts, military medical historians, and researchers (n = 4, 8%). Notably, nine studies (17%) intentionally incorporated multiple perspectives by gathering input from various combinations of students, teachers, program directors, researchers, faculty members, patients, and nursing paramedics. These results are presented in Table 2.

Table 2.

Stakeholders Identified in MEd Program Evaluations

Stakeholder group	Number of studies (N = 53)	Percentage (%)
Students	33	62
Program directors	20	38
Teachers	4	8
Faculty members	5	9
Patients	1	2
Nurses	1	2
Other	4	8
Multiple points of view	9	17

Note. The “Other” category includes deans, program evaluation experts, military medical historians, and researchers. Nine studies (17%) included multiple stakeholder groups.

RQ2

Stakeholder Engagement, and Facilitators and Barriers in MEd Program Evaluations

Stakeholder Engagement

Our review found that stakeholders were predominantly engaged as participants providing feedback rather than actively shaping the evaluation process itself. Most studies utilized email (n = 43, 81%) to recruit participants, while other methods such as phone calls (n = 3, 6%), direct recruitment during class (n = 11, 21%), and individual recruitment or social media (n = 3, 6%) were less common. The specific methods used to engage stakeholders varied across groups. Students (n = 33) primarily participated through surveys (n = 29, 87.9%), with fewer instances of interviews (n = 9, 27.3%), focus groups (n = 2, 6.1%), or activity analysis (essays or clinical practice) (n = 3, 9%). Program directors, teachers, and faculty members were consistently engaged through surveys in all relevant studies. The response rate varied widely from one study to another, especially for those using questionnaires: 11.2% to 100%. Eighteen (34%) authors who used questionnaires achieved a minimum response rate of 60%. Among the authors’ objectives, fifteen (28%) are concerned with student satisfaction. One study also interviewed program directors, while another utilized semi-structured interviews, focus groups, and field observations with faculty members. Patients and paramedics were involved in only one study, participating through semi-structured interviews and focus groups. Two studies employed unique approaches to stakeholder engagement. Cole et al. (2024) utilized a diverse evaluation team and student reflexive essays, while Shankar et al. (2022) commissioned an independent evaluation organization. Additionally, six studies combined various methods, such as surveys, interviews, focus groups, and field observations, to better understand stakeholder perspectives.

Facilitators and Barriers to Participation in MEd Program Evaluations

Appendix 3 provides an overview of the facilitators and barriers identified in the reviewed studies regarding participation in MEd program evaluations. It is important to note that these facilitators and barriers primarily pertain to participants in the evaluation process, rather than active stakeholders, as most studies engaged them as such.

Facilitators to Participation

Most studies (64%, n = 34) reported multiple facilitators to participation in MEd evaluations. Anonymity was highlighted as a significant factor favouring participation in many studies (45%, n = 24). Additionally, the voluntary (11%, n = 6), optional (2%, n = 1), or confidential (6%, n = 3) nature of participation, as well as the right to refuse to answer any question (2%, n = 1), were also considered encouraging. To maintain anonymity and confidentiality, some studies chose not to use official channels or avoided evaluating the presence of teachers, preferring independent researchers. Another key facilitator was the use of reminders, which were sent via email (42%, n = 22), phone (4%, n = 2), orally (2%, n = 1), or through unspecified methods (2%, n = 1).

Some studies aimed to systematize the recruitment process to enhance response rates. In 19% of the studies (n = 10), researchers utilized a national database of program directors’ email addresses. Additional dissemination channels included newsletters and the country’s most popular social network. Some studies also sent requests to more people than initially anticipated or involved third parties. Others took advantage of students' presence in class to distribute questionnaires (9%, n = 5). Limiting the number of questions or ensuring a short duration was also beneficial.

Some studies adopted a targeted approach. One team used personalized emails, either directly or through departmental channels. Another study specified that the person most involved in the program should complete the questionnaire. In another case, a student investigator was employed to recruit other students for the evaluation.

In some instances, program evaluation was integrated into a regular, protocolized process, such as allocating time away from teaching for evaluation, habitual program evaluation by students, or making it part of the teachers’ mission. In terms of incentives, some studies mentioned that access to further education was conditional on completing the evaluation. In contrast, others (15%, n = 8) offered material incentives like gifts, prizes, and compensation, all targeting students.

Barriers to Participation

The barriers to participation in MEd program evaluations were less frequently specified in studies (72%, n = 38). Some studies faced recruitment difficulties due to invalid or unavailable email addresses (6%, n = 3). Using social network groups as dissemination channels did not reach all potential reviewers. Similarly, relying on program directors to disseminate evaluations reduced information transfer and student involvement. Collecting responses was challenging due to evaluators’ lack of time, as they were preoccupied with faculty and personal obligations, or because the evaluation took place far from the program being evaluated. Repeated requests for responses led to student fatigue. One study reported a loss of responses due to a questionnaire not suited for residency programs with different organizational structures. While most studies viewed voluntary participation as beneficial, one study found that it did not motivate participants to engage in program evaluation. Finally, three studies (6%) mentioned the lack of incentives, especially material ones, as a barrier.

Discussion

This scoping review examined the roles and participation levels of stakeholders in MEd program evaluations, including the facilitators and barriers to their participation. We found 53 studies that addressed these questions, and our review yielded several interesting findings.

First, we found a growing number of publications in MEd program evaluation during the last decade. Indeed, there has been a notable increase in publications on MEd program evaluation, with 45% of these works published between 2020 and 2024 and 30% between 2016 and 2019. This trend reflects a growing interest in the field of MEd program evaluation. This reflects the need to develop such evaluation to respond to social accountability and the accreditation process of medical schools (Santen et al., 2019; WFME & WHO, 2005). It is also found in a number of publications guiding to develop this evaluation (Balmer et al., 2020, 2023; Cook, 2010; S. R. Shrivastava & P. S. Shrivastava, 2023).

Secondly, we identified seven primary stakeholder groups involved in MEd program evaluation: students, program directors, teachers, faculty members, patients, nurses, and other healthcare professionals. While students (n = 33, 62%) and program directors (n = 20, 38%) were the most frequently included, the near absence of community members and patients underscores a critical gap in current evaluation practices. This underrepresentation is particularly concerning given their unique perspective as the primary beneficiaries of medical training and their central role in social accountability (Boelen et al., 2016). Engaging these stakeholders more meaningfully could offer invaluable insights into program effectiveness and relevance to community needs (Moreau & Eady, 2023).

Third, we found that stakeholders were predominantly engaged as participants providing feedback rather than actively shaping the evaluation process. This represents a missed opportunity to empower them to contribute to the design and interpretation of evaluations. Strategies to actively engage these groups should be prioritized. One of the first strategies would be reflections on the evaluation approach as a first step in developing a program evaluation by considering, for example, design thinking (Fish et al., 2022; Sandars & Goh, 2020), utilization-focused approach (Patton, 2012), or empowerment evaluation (Fetterman, 1996; Fetterman et al., 2010). Another would be diversifying data collection methods and incorporating a wider range of perspectives that could lead to more comprehensive and meaningful evaluations.

Finally, several factors were identified that influence participation in MEd program evaluations. Facilitators included anonymity, reminders, convenience, targeted communication, and incentives. Conversely, recruitment difficulties, challenges in collecting responses, the inadequacy of voluntary participation alone, and lack of incentives were barriers to participation. It is crucial to note that these factors primarily relate to participants in the evaluation process, not active stakeholders, as most studies engaged them as such. Therefore, future studies should prioritize investigating the perspectives of active stakeholders, such as program administrators, faculty, and employers, to gain a more comprehensive understanding of the factors that influence their engagement in MEd program evaluations.

Strengths and Limitations of the Study

This study is the first to review the participation of stakeholders in medical education program evaluation. The research carried out in this work is targeted and structured. Several databases were searched, with a reading by two independent practitioners supported by a program evaluation expert. The large number of articles included lends power to the literature review, but it could have benefited from two more independent reviewers for more validity to the results. Due to limited publication of medical program evaluations, future scoping reviews should include grey literature, such as unpublished reports and non-peer-reviewed sources, to gain valuable insights into stakeholder engagement in practice settings. The method does not allow us to understand the individual reasons behind stakeholder participation in evaluation.

Conclusion

Evaluating medical programs and teaching is essential to the medical school accreditation process. Many MEd publications reflect a growing interest in program evaluation, which aligns with the importance of medical schools’ social responsibility. The evaluation depends on the participation of its stakeholders: students, program directors, teachers, and faculty members. It should systematically involve patients and paramedics. This participation could be considered an involvement in the evaluation process instead of mere feedback. To encourage stakeholder participation in program evaluation processes, consideration could be given to the choice of evaluative approaches before the implementation of evaluations. This work could be completed to understand better and perceive stakeholders' motivations for participating in the evaluation of medical programs and teaching.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors thank the financial support from the Région Auvergne Rhône-Alpes, France, with the Pack Ambition Internationale for the publication of this article (22PACK-CHANELIERE-3303). The authors reveiced no financial support for the research and authorship.

ORCID iDs

Juliette Macabrey

David Buetti

Appendix

References

Alerhand

Situ-Lacasse

Ramdin

Gottlieb

(2021). National survey of point-of-care ultrasound scholarly tracks in emergency medicine residency programs. Western Journal of Emergency Medicine: Integrating Emergency Care with Population Health, 22(5), 1095–1101. https://doi.org/10.5811/westjem.2021.5.52118

Alt-Epping

Lohse

Viebahn

Steinbüchel

N. V.

Benze

Nauck

(2014). On death and dying – an exploratory and evaluative study of a reflective, interdisciplinary course element in undergraduate anatomy teaching. BMC Medical Education, 14(1), 15. https://doi.org/10.1186/1472-6920-14-15

Ariail

Thomas

Smith

Kerr

Richards-Slaughter

Shaw

(2013). The value of a writing center at a medical university. Teaching & Learning in Medicine, 25(2), 129–133. https://doi.org/10.1080/10401334.2013.770739

Balmer

D. F.

Anderson

West

D. C.

(2023). Program evaluation in health professions education: An innovative approach guided by principles. Academic Medicine, 98(2), 204. https://doi.org/10.1097/ACM.0000000000005009

Balmer

D. F.

Riddle

J. M.

Simpson

(2020). Program evaluation: Getting started and standards. Journal of Graduate Medical Education, 12(3), 345–346. https://doi.org/10.4300/JGME-D-20-00265.1

Berk

R. A.

(2009). Using the 360° multisource feedback model to evaluate teaching and professionalism. Medical Teacher, 31(12), 1073–1080. https://doi.org/10.3109/01421590802572775

Boelen

Heck

J. E.

WHO . (1995). Defining and measuring the social accountability of medical schools (WHO/HRH/95.7. Unpublished). World Health Organization. https://apps.who.int/iris/handle/10665/59441

Boelen

Pearson

Kaufman

Rourke

Woollard

Marsh

D. C.

Gibbs

(2016). Producing a socially accountable medical school: AMEE Guide No. 109. Med Teach, 38(11), 1078–1091. https://doi.org/10.1080/0142159X.2016.121902927608933.

Bouzaher

M. H.

Trinkle

D. B.

Mutcheson

R. B.

(2020). Interprofessional service learning in medical education: A year-over-year assessment of student feedback. Medical Science Educator, 30(2), 775–781. https://doi.org/10.1007/s40670-020-00948-7

10.

Cameron

Dromerick

L. J.

Ahn

Dromerick

A. W.

(2019). Executive/life coaching for first year medical students: A prospective study. BMC Medical Education, 19(1), 163. https://doi.org/10.1186/s12909-019-1564-4

11.

Campbell

Didwania

(2020). Health equity in graduate medical education: Exploring internal medicine residents’ perspectives on a social determinants of health curriculum. Journal of Health Care for the Poor & Underserved, 31, 154–162.

12.

Cole

Van Shufflin

M. W.

Smith

D. C.

Woodard

S. C.

(2024). The antietam staff walk at the uniformed services university: A program evaluation. Military Medicine, 189(1–2), e298–e305. https://doi.org/10.1093/milmed/usad317

13.

Committee on Accredidation of Canadian Medical Schools . (2024). Guide to the medical school self-study.

14.

Cook

D. A.

(2010). Twelve tips for evaluating educational programs. Medical Teacher, 32(4), 296–301. https://doi.org/10.3109/01421590903480121

15.

Curry

(2018). Teaching medical students clinical anesthesia. Anesthesia and Analgesia, 126(5), 1687–1694. https://doi.org/10.1213/ANE.0000000000002802

16.

Deora

Garg

Tripathi

Mishra

Chaurasia

(2020). Residency perception survey among neurosurgery residents in lower-middle-income countries: Grassroots evaluation of neurosurgery education. Neurosurgical Focus, 48(3), Article E11. https://doi.org/10.3171/2019.12.FOCUS19852

17.

Dubois

Ducharme

Lalonde

Rysdale

(2023). Undergraduate medical students’ perceptions of nutrition education at the northern ontario school of medicine. Canadian Journal of Dietetic Practice and Research, 84(1), 28–32. https://doi.org/10.3148/cjdpr-2022-025

18.

Edwards

Kim Y.

Stevenson

Johnson

Sharp

Reisman

Srinivasan

(2021). When it’s needed most: A blueprint for resident creative writing workshops during inpatient rotations. BMC Medical Education, 21(1), 535. https://doi.org/10.1186/s12909-021-02935-x

19.

Egle

J. P.

Mittal

V. K.

(2014). Program directors’ perspective of transplant training during general surgery residency. The American Surgeon, 80(8), 796–800. https://doi.org/10.1177/000313481408000831

20.

Elmore

S. N. C.

Royce

T. J.

Oladeru

O. T.

Freese

C. M.

Wakefield

D. V.

Vapiwala

Zietman

A. L.

(2020). Global health perspectives among radiation oncology residency program directors: A knowledge, attitudes, and practices survey. International Journal of Radiation Oncology*Biology*Physics, 107(3), 419–425. https://doi.org/10.1016/j.ijrobp.2020.02.467

21.

Eno

Piemonte

Michalec

Adams

C. A.

Budesheim

Felix

Hack

Jensen

Leavelle

Smith

(2023). Forming physicians: Evaluating the opportunities and benefits of structured integration of humanities and ethics into medical education. Journal of Medical Humanities, 44(4), 503–531. https://doi.org/10.1007/s10912-023-09812-2

22.

Fernando

Jusko-Friedman

Catton

Nyhof-Young

(2015). Celebrating 10 Years of undergraduate medical education: A student-centered evaluation of the princess margaret cancer centre—determinants of community health year 2 program. Journal of Cancer Education, 30(2), 225–230. https://doi.org/10.1007/s13187-014-0674-2

23.

Fetterman

D. M.

(1996). Empowerment evaluation: An introduction to theory and practice. In Empowerment evaluation: Knowledge and tools for self-assessment & accountability (pp. 3–46). Sage Publications, Inc. https://doi.org/10.4135/9781452243573

24.

Fetterman

D. M.

Deitz

Gesundheit

(2010). Empowerment evaluation: A collaborative approach to evaluating and transforming a medical school curriculum. Academic Medicine, 85(5), 813–820. https://doi.org/10.1097/ACM.0b013e3181d74269

25.

Finnie

Brach

Dawson

Epstein

Goyal

Lounsbury

Eldakar-Hein

Lahey

(2021). A new roadmap for social medicine curriculum design based on mixed methods student and faculty evaluations of the preclinical curriculum. BMC Medical Education, 21(1), 2021. https://doi.org/10.1186/s12909-021-02885-4

26.

Fish

A. M.

Fields

J. M.

Ziring

McCoy

Ostroff

Hayden

(2022). Curriculum development by design thinking: Analyzing a program for social determinants of health screening by pre-clerkship medical students. Journal of Medical Education and Curricular Development, 9, 238212052210807. https://doi.org/10.1177/23821205221080701

27.

Fisher

J. M.

Gordon

A. L.

MacLullich

A. M. J.

Tullo

Davis

D. H. J.

Blundell

Field

R. H.

Teodorczuk

(2015). Towards an understanding of why undergraduate teaching about delirium does not guarantee gold-standard practice—results from a UK national survey. Age & Ageing, 44(1), 166–170.

28.

Foster

Corbin

Kwan

LeClair

(2018). Family medicine resident perspectives on curricula in herbal medicine. Family Medicine, 50(8), 589–596. https://doi.org/10.22454/FamMed.2018.217704

29.

Gazibara

Marusic

Maric

Zaric

Vujcic

Kisic-Tepavcevic

Maksimovic

Denic

Grujicic

Pekmezovic

Grgurevic

(2015). Introducing E-learning in epidemiology course for undergraduate medical students at the faculty of medicine, university of belgrade: A pilot study. Journal of Medical Systems, 39(10), 201. https://doi.org/10.1007/s10916-015-0302-7

30.

Feng

Liu

Guo

Gao

Zhao

Shi

Liu

Zhu

(2022). Current situation and expectation of emergency medicine education for 8-year Doctor of Medicine degree program in China. Hong Kong Journal of Emergency Medicine, 29(6), 338–348. https://doi.org/10.1177/10249079211040971

31.

Global Consensus for Social Accountability of Medical Schools . (2010). Global consensus for social accountability of medical schools. https://healthsocialaccountability.sites.olt.ubc.ca/files/2011/06/11-06-07-GCSA-English-pdf-style.pdf

32.

Gonzalez

L. S.

Donnelly

M. J.

(2016). A survey of residency program directors in anesthesiology regarding mentorship of residents. Journal of Clinical Anesthesia, 33, 254–265. https://doi.org/10.1016/j.jclinane.2016.03.004

33.

Hermann

Gerstner

Weiss

Aichele

Stricker

Gorgati

Rosenberger

Wunderlich

(2021). Presentation and evaluation of a modern course in disaster medicine and humanitarian assistance for medical students. BMC Medical Education, 21(1), 2021. https://doi.org/10.1186/s12909-021-03043-6

34.

Jaafari

Delavari

Bazrafkan

(2019). Evaluation of the geriatric curriculum implemented at shiraz university of medical sciences, Iran, since 2017: A qualitative study. F1000Research, 8, 417. https://doi.org/10.12688/f1000research.16040.1

35.

Jiang

Shi

Cao

Zhu

Sha

Ning

Hong

Dai

Wei

(2020). Effectiveness of clinical scenario dramas to teach doctor-patient relationship and communication skills. BMC Medical Education, 20(1), 473. https://doi.org/10.1186/s12909-020-02387-9

36.

Jujo

Lee-Jayaram

Sakka

Nakahira

Kataoka

Izumo

Kusunose

Athinartrattanapong

Oikawa

Berg

(2021). Pre-clinical medical student cardiac point-of-care ultrasound curriculum based on the American society of echocardiography recommendations: A pilot and feasibility study. Pilot and Feasibility Studies, 7(1), 2021. https://doi.org/10.1186/s40814-021-00910-3

37.

Kamel

Eltahawy

E. A.

Warford

Thrush

C. R.

Noureldin

Y. A.

(2018). Simulation-based training in urology residency programmes in the USA: Results of a nationwide survey. Arab Journal of Urology, 16(4), 446–452. https://doi.org/10.1016/j.aju.2018.06.003

38.

Lapp

Makowka

Recker

(2018). Peer-mentoring program during the preclinical years of medical school at bonn university: A project description. GMS Journal for Medical Education, 35(1), 2018. https://doi.org/10.3205/zma001154

39.

Lebastchi

A. H.

Yuh

D. D.

(2014). Nationwide survey of US integrated 6-year cardiothoracic surgical residents. The Journal of Thoracic and Cardiovascular Surgery, 148(2), 401–407. https://doi.org/10.1016/j.jtcvs.2014.03.040

40.

Lepard

J. R.

Shank

C. D.

Agee

B. S.

Hadley

M. N.

Walters

B. C.

(2020). Neurosurgical resident research education: A survey of United States residency program directors. Journal of Neurosurgery, 133(5), 1527–1536. https://doi.org/10.3171/2019.7.JNS19632

41.

Shen

(2020). Quality evaluation of entrepreneurship education in Chinese medical colleges–from the perspective of student cognition. Frontiers in Psychology, 11, 1093. https://doi.org/10.3389/fpsyg.2020.01093

42.

London

Druck

Silver

Finefrock

(2017). Teaching the emergency department patient experience: Needs assessment from the CORD-EM task force. Western Journal of Emergency Medicine, 18(1), 56–59. https://doi.org/10.5811/westjem.2016.9.30667

43.

London

Z. N.

Khan

Cahill

Schuyler

Wold

Southerland

A. M.

(2018). 2017 Program Director Survey: Feedback from your adult neurology residency leadership. Neurology, 91(15), 2018. https://doi.org/10.1212/WNL.0000000000006315

44.

Lotfipour

Rolius

Lehman

E. B.

Pantanelli

S. M.

Scott

I. U.

(2017). Trends in cataract surgery training curricula. Journal of Cataract and Refractive Surgery, 43(1), 49–53. https://doi.org/10.1016/j.jcrs.2016.10.020

45.

Mann

K. J.

Craig

M. S.

Moses

J. M.

(2014). Quality improvement educational practices in pediatric residency programs: Survey of pediatric program directors. Academic Pediatrics, 14(1), 23–28. https://doi.org/10.1016/j.acap.2012.11.003

46.

Moreau

K. A.

(2017). Twelve tips for planning and conducting a participatory evaluation. Medical Teacher, 39(4), 334–340. https://doi.org/10.1080/0142159X.2017.1286310

47.

Moreau

K. A.

Eady

(2023). Program evaluation use in graduate medical education. Journal of Graduate Medical Education, 15(1), 15–18. https://doi.org/10.4300/JGME-D-22-00397.1

48.

Muszkat

Barak

Lalazar

Mazal

Schneider

Levi

I. M.-Y.

Cohen

M. J.

Canetti

Yehuda

A. B.

Naparstek

(2014). The effect of medical students’ gender, ethnicity and attitude towards poetry-reading on the evaluation of a required, clinically-integrated poetry- based educational intervention. BMC Medical Education, 14(1), 188. https://doi.org/10.1186/1472-6920-14-188

49.

Naifeh

M. M.

Stevenson

M. D.

Abramson

E. L.

Aston

C. E.

Combs

R. M.

Decker

H. R.

S.-T. T.

(2023). The early effects of the coronavirus disease-19 pandemic on pediatric resident education: A national assessment. Academic Pediatrics, 23(5), 993–1000. https://doi.org/10.1016/j.acap.2022.11.003

50.

Nayar

H. S.

Salyapongse

A. N.

Mount

D. L.

Bentz

M. L.

(2015). The current state of global surgery training in plastic surgery residency. Plastic and Reconstructive Surgery, 136(6). 830e–837e. https://doi.org/10.1097/PRS.0000000000001817

51.

Nevin

C. R.

Westfall

A. O.

Rodriguez

J. M.

Dempsey

D. M.

Cherrington

Roy

Patel

Willig

J. H.

(2014). Gamification as a tool for enhancing graduate medical education. Postgraduate Medical Journal, 90(1070), 685–693. https://doi.org/10.1136/postgradmedj-2013-132486

52.

Patton

M. Q.

(1997). Utilization-focused evaluation: The new century text (3rd ed.). Sage Publications.

53.

Patton

M. Q.

(2012). A utilization-focused approach to contribution analysis. Evaluation, 18(3), 364–377. https://doi.org/10.1177/1356389012449523

54.

Patton

M. Q.

Asibey

Dean-Coffey

Kelley

Miranda

Parker

Newman

G. F.

(2014). What is evaluation? American Evaluation Association. https://www.eval.org/About/What-is-Evaluation

55.

Peters

M. D. J.

Godfrey

C. M.

Khalil

McInerney

Parker

Soares

C. B.

(2015). Guidance for conducting systematic scoping reviews. JBI Evidence Implementation, 13(3), 141. https://doi.org/10.1097/XEB.0000000000000050

56.

Ryll

Pellegrini

Richards

Zhou

Newlands

Levi

(2023). A cross-sectional survey study evaluating United States medical school curricula in otolaryngology. Annals of Otology Rhinology and Laryngology, 132(11), 1412–1417. https://doi.org/10.1177/00034894231164220

57.

Sandars

Goh

P.-S.

(2020). Design thinking in medical education: The key features and practical application. Journal of Medical Education and Curricular Development, 7, 238212052092651. https://doi.org/10.1177/2382120520926518

58.

Santen

Feldman

Weir

Blondino

Rawls

DiGiovanni

(2019). Developing comprehensive strategies to evaluate medical school curricula. Medical Science Educator, 29(1), 291–298. https://doi.org/10.1007/s40670-018-00640-x

59.

Shankar

Phinney

Krapek

Surapaneni

V. L.

(2022). Building a community-centered public health advocacy training program for medical students. Journal of Public Health Management & Practice, 28(4), E692–E701. https://doi.org/10.1097/PHH.0000000000001486

60.

Shrivastava

S. R.

Shrivastava

P. S.

(2023). Program evaluation in the field of medical education: Need and approaches. Current Medical Issues, 21(3), 181. https://doi.org/10.4103/cmi.cmi_31_23

61.

Shufflebarger

E. F.

Willett

Sontheimer

S. Y.

Hicks

Khoury

C. A.

Walter

L. A.

(2023). Feasibility of a multifaceted social emergency medicine curriculum for emergency medicine residents. Western Journal of Emergency Medicine, 24(3), 2023. https://doi.org/10.5811/WESTJEM.59009

62.

Siddiqui

I. J.

Luz

Borg‐Stein

O’Connor

Bockbrader

Rainey

Way

Colachis

Bahner

D. P.

Kohler

M. J.

(2016). The current state of musculoskeletal ultrasound education in physical medicine and rehabilitation residency programs. PM&R, 8(7), 660–666. https://doi.org/10.1016/j.pmrj.2015.11.010

63.

Smeds

Sheahan M. G.

Shames

M. L.

Abularrage

C. J.

(2021). A modern appraisal of current vascular surgery education. Journal of Vascular Surgery, 73(4), 1430–1435. https://doi.org/10.1016/j.jvs.2020.10.028

64.

Soliman

Sattar

Alnassar

Alsaif

Alswat

Alghonaim

Alhaizan

Al-furaih

(2017). Medical students’ perception of the learning environment at king saud university medical college, Saudi arabia, using DREEM inventory. Advances in Health Sciences Education and Pratice, 8. https://doi.org/10.2147/AMEP.S127318

65.

Sorensen

Norredam

Suurmond

Carter-Pokras

Garcia-Ramirez

Krasnik

(2019). Need for ensuring cultural competence in medical programmes of European universities. BMC Medical Education, 19. https://doi.org/10.1186/s12909-018-1449-y

66.

Stewart-Brown

Cader

Walker

Janjua

Hanson

Chilton

A.-M.

(2018). Experiences with a universal mindfulness and well-being programme at a UK medical school. Health Education, 118(4), 304–319. https://doi.org/10.1108/HE-10-2017-0053

67.

Sun

Chen D.

Warner

D. O.

Zhou

Nemergut

E. C.

Macario

Keegan

M. T.

(2021). Anesthesiology residents’ experiences and perspectives of residency training. Anesthesia and Analgesia, 132(4), 1120–1128. https://doi.org/10.1213/ANE.0000000000005316

68.

Thompson

Ponkratz

Gallagher

Treat

St Clair

N. E.

Chou

Lauck

S. M.

(2021). Establishment and retrospective analysis of a pilot peer mentorship program. Wisconsin Medical Journal, 120(1), 17–22.

69.

Tricco

A. C.

Lillie

Zarin

O’Brien

K. K.

Colquhoun

Levac

Moher

Peters

M. D. J.

Horsley

Weeks

Hempel

Akl

E. A.

Chang

McGowan

Stewart

Hartling

Aldcroft

Wilson

M. G.

Garritty

Straus

S. E.

(2018). PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Annals of Internal Medicine, 169(7), 467–473. https://doi.org/10.7326/M18-0850

70.

Veazey

Nieuwoudt

Gavito

Tocce

(2015). Student perceptions of reproductive health education in us medical schools: A qualitative analysis of students taking family planning electives. Medical Education Online, 20(1), 28973. https://doi.org/10.3402/meo.v20.28973

71.

Wanstreet

Callaham

O’Brien

Carr

(2020). Program directors’ opinions about otolaryngology resident teaching medical school anatomy. Cureus Journal of Medical Science, 12(10), 2020. https://doi.org/10.7759/cureus.10999

72.

Ward

Outram

Heslop

(2018). Perceived utility and relevance of intern well‐being sessions. Internal Medicine Journal, 48(6), 645–650. https://doi.org/10.1111/imj.13769

73.

Weis

J. J.

Croft

C. L.

Bhoja

Lee

J. H.

Scott

D. J.

(2019). Multidisciplinary simulation activity effectively prepares residents for participation in patient safety activities. Journal of Surgical Education, 76(6), e232–e237. https://doi.org/10.1016/j.jsurg.2019.07.015

74.

WFME . (n.d.). Accreditation. The World Federation for Medical Education. https://wfme.org/accreditation/

75.

WFME . (2020). Basic medical education WFME global standards 2020. https://wfme.org/standards/bme/

76.

WFME WHO . (2005). WHO-WFME guidelines for accreditation of basic medical education.

77.

Zumberg

Broudy

Bengtson

Gitlin

(2015). Preclinical medical student hematology/oncology education environment. Journal of Cancer Education, 30(4), 711–718. https://doi.org/10.1007/s13187-014-0778-8

Who’s at the Table? A Scoping Review of Stakeholder Engagement in Medical Education Program Evaluation

Abstract

Keywords

Role of Stakeholder Engagement in Evaluation

Research Gaps and Questions

Methods

Search Strategy

Study Selection

Data Extraction

Data Analysis and Presentation

Results

Studies Characteristics

Stakeholder Engagement

Facilitators and Barriers to Participation in MEd Program Evaluations

Facilitators to Participation

Barriers to Participation

Discussion

Strengths and Limitations of the Study

Conclusion

Footnotes

Declaration of Conflicting Interests

Funding

ORCID iDs

Appendix

References