Sage Journals: Discover world-class research

Abstract

Introduction

The Accreditation Council of Graduate Medical Education requires all Graduate Medical Education programs to show evidence of scholarly activity. Fulfilling this obligation remains a challenge for residency programs and remains a common citation by the specialty Residency Review Committees, especially in community hospital settings with limited academic resources. This study evaluated the impact of implementing a bundled intervention on resident scholarly activity within a family medicine residency program in a community hospital.

Methods

This study employed a quasiexperimental pre- and postdesign to assess the impact of a multifaceted bundled intervention. The scholarly output of residents graduating between 2016 and 2018 (preintervention) was compared to those graduating from 2019 to 2021 (postintervention). Statistical Process Control charts with JMP Pro 17.0 were used to display data and perform phase analysis. Fisher's exact test and the chi-square test were used to compare the demographics, while the Mann-Whitney U-test and one-way analysis of variance were also used to detect the difference in scholarly output between the 2 groups.

Results

Thirty-one family medicine residents graduated from the program in the study period. A significant increase in the average number of scholarly activities, including all manuscripts and presentations, was noted. The average number of all-scholarly activities increased 4-fold postintervention (P < .001). The increase in all-presentations and all-manuscripts was also statistically significant postintervention (P < .001 and .0038, respectively).

Conclusions

Implementing a multifaceted bundled intervention containing a 13-step structured roadmap with a quality improvement approach is associated with increasing residents’ scholarly output in residency programs at a community hospital.

Keywords

scholarship scholarly activity research capacity curriculum family medicine residency graduate medical education accreditation

Introduction

Although the definition and categorization of scholarship or scholarly activity have been debated by scholars like Boyer,¹ Fincher et al,² Glassick et al,³ and Irby et al,⁴ it is existential to medical education. While Glassick's criteria are generally well accepted as what constitutes scholarship, Boyer's classification¹ also makes sense for scholarly activity in medical education. Boyer classified scholarly activity into 4 major types: discovery of new knowledge, integration into the existing body of knowledge, implementation and application of existing knowledge, teaching, and dissemination. Residency Review Committees (RRCs) have historically lacked standardized criteria for evaluating scholarly activity, with only 6 of the 27 RRCs implementing rubrics to evaluate residents’ contributions to scholarly work. To address this, Grady et al⁵ proposed a comprehensive rubric based on Boyer's 4 components of scholarship—discovery, integration, application, and teaching. The rubric was built based on faculty promotion criteria from 30 institutions across the United States, and it provides a structured and objective method for assessing scholarly contributions in residency programs. The framework supports objective and consistent accreditation reviews across institutions while allowing RRCs flexibility to emphasize focus areas based on specific institutional needs. This standardization reduces the subjectivity in evaluating scholarly activity, promoting a fairer and more transparent assessment process.⁵

The culture and scholarship in family medicine residency programs influence how the next generation of the primary workforce will practice.⁶ For these reasons, the Accreditation Council for Graduate Medical Education (ACGME), which sets standards for Graduate Medical Education (GME) in America, requires a culture of scholarly activity in all GME programs. The Family Medicine Review Committee at the ACGME requires that every graduating resident complete at least 2 scholarly activities, one being quality improvement.⁷ Despite the need for a culture of scholarship, many programs continue to struggle to fulfill these requirements.⁸ In a national survey conducted in 2011 involving 450 US family medicine program directors, factors associated with higher resident scholarly productivity were examined.⁸

Numerous strategies and educational plans proposed to enhance scholarly output have been reported with varying success.⁹ A systematic review found that establishing a research track and appointing a research director increased the number of resident publications. Most studies indicate that providing protected research time, implementing a research curriculum, and assigning dedicated research days have increased scholarly output. Mentorship within the specific departmental specialty as a component of Resident Scholarly Activity Programs (RSAP), along with funding for resident scholarly endeavors, correlates with program adherence to ACGME guidelines and increases research and scholarly productivity.¹⁰ Additionally, annual local research days, formal recognition of scholarship activities, and program director and faculty involvement promote research work among residents.⁸

Waheed et al¹¹ implemented a structured roadmap comprising of 13-step to complete a scholarly activity spanning the PGY1 to PGY3 years. This initiative yielded higher publication rates in a family medicine residency program at a large academic center. Our study aimed to determine if a similar framework can be successfully implemented at a community hospital with more limited academic resources.

Methods

Study Design and Study Setting

Utilizing a retrospective review, this study employed a quasiexperimental pre- and postdesign deploying quality improvement strategies. It aimed to assess the effects of implementing a structured scholarly activity roadmap within a single family medicine residency program. This program is the only accredited graduate medical education program in that community hospital. This site has been the seat of family medicine residency training for more than 30 years. This current program was established in 2014 after the previous program had moved to another location by the previous GME sponsor. The primary outcome was the total number of scholarly activities by each resident. The study included 2 cohorts:

The preintervention group consisted of residents who concluded training in 2016, 2017, and 2018.

The postintervention group, comprising those who graduated in 2019, 2020, and 2021, followed the implementation of the 13-step structured roadmap.¹¹

This study followed the SQUIRE 2.0 Guidelines for quality assessment and reported results (Supplemental Material).¹² The WellSpan Health Institutional Review Board (IRB) determined it to be “Not Human Subjects Research” via study notification 1887432-2.

The Intervention

It was a multifaceted bundled intervention built on a previously created and studied curriculum and related concise handbook outlining “Guidelines for Scholarly Activity” for easy reference, utilizing the work previously done by Waheed et al.^11,13,14 The 13-step structured roadmap developed and reported by Waheed et al¹¹ is summarized in Table 1. It outlines a structured approach for postgraduate medical residents to undertake a research project or another scholarly activity over the 3 years of their training. In addition to employing the 13-step structured roadmap as in the original study,¹¹ we implemented a multifaceted bundled intervention strategy. This multifaceted bundled intervention encompassed several components based on Waheed et al.s’ expanded work,^13,14 also shown in Figure 1. The implementation of this bundle required a dedicated 0.2 FTE as previously reported by Waheed et al.^11,13,14 In this study, this was initially undertaken by the residency program director. The responsibility was then transitioned to another core faculty while the program director continued to mentor the faculty responsible with keen interest.

Figure 1.

Simultaneous Interventions are Implemented in the Residency Program.

Table 1.

Goals of the 13-Step Roadmap During Residency Training Years.

Goal of Postgraduate Year 1 (Steps 1 Through 7)	Goals of Postgraduate Year 2 (Steps 8 Through 10)	Goals of Postgraduate Year 3 (Steps 11 Through 13)
Prepare a project for Institutional Review Board (IRB) submission	Obtain IRB approval	Interpret data
	Collect and analyze data	Present scholarly activity
	Collect and analyze data	Publish research work

Data Collection and Data Analysis

Comprehensive data pertaining to scholarly output, demographic characteristics, and residency records were collected and analyzed. Statistical analysis was conducted using JMP Pro 17.0 for SAS. Statistical Process Control (SPC) charts display and monitor data over time, followed by a phase analysis. We utilized Fisher's exact and chi-square tests to compare the demographics and characteristics of the 2 groups before and after implementing the 13-step roadmap. The Mann-Whitney U-test was used for nonparametric comparisons. Finally, one-way ANOVA was used to compare the mean scholarly output between the 2 groups pre- and postintervention. An alpha of < 0.05 was used for statistical significance.

Results

Between July 2016 and June 2021, 31 Family Medicine residents graduated from the residency program. Before the implementation of the roadmap, the initial cohort comprised 14 residents, while the subsequent cohort consisted of 16 residents. Table 2 provides an overview of the baseline demographic characteristics of the 2 cohort groups. Most participants were Asian, practiced in a nonacademic setting, and did not possess an additional degree. The P-values calculated using Fisher's exact test indicated no significant differences between the 2 groups using a priori cut-off at 0.05. This indicates the comparability of the group's pre- and postimplementation.

Table 2.

Baseline Characteristics of the 2 Cohort Groups.

		Before the Implementation of the Roadmap	After the Implementation of the Roadmap	P Value
Gender	Male	6 (42.9%)	9 (56.3%)	.715
Gender	Female	8 (57.1%)	7 (43.7)	.715
Race	White	3 (21%)	5 (32%)	.49
	Asian	9 (64%)	8 (50%)
	Black	1 (7%)	3 (18%)
	Other	1 (7%)	0
Current practice	Academic	2 (14%)	0	.209
	Nonacademic	12 (86%)	16 (100%)	.209
Additional degree	Yes	4 (29%)	2 (12.5%)	.378
Additional degree	No	10 (71%)	14 (87.5%)	.378
Publication record before starting residency	One or no prior publications	10 (71.4%)	10 (62.5%)	.604
Publication record before starting residency	Two or more prior publications	4 (28.6%)	6 (37.5%)	.604

For all residents who graduated in the academic years 2016 to 2018 (preintervention) and 2019 to 2021, the number of all scholarly activities was plotted on the SPC chart. The individual-moving range (X-mR) control chart of all types of scholarly activities, regional, national, or international presentations, and publications showed a shift in the process indicated by the special cause test. The control chart with phase analysis showed a change in the mean from preintervention to the postintervention phase and special cause variation with Nelson rule number 1 (astronomical data) in the direction of improvement. The average number of all scholarly activities per resident at graduation was 0.9 in the preintervention phase. This increased to an average of 4 activities per resident after implementing a bundled intervention (Figure 2).

Figure 2.

Impact of a Structured Road Map on Scholarly Output: SPC (Statistical Process Control) Phase Analysis Control Chart Tracking the Scholarly Activities Completed by Individual Residents, Divided Into Preintervention (0) and Postintervention (1) Phases Separated by the (Blue) Intervention Line. μ₀ Represents the Mean as the Center Green Line. UCL: 3-Sigma Upper Control Limit. LCL: 3-Sigma Lower Control Limit. Values in UCL and LCL Represent 3 Standard Deviations Above and Below the Mean, Respectively. Special Cause Variation Is Highlighted in Red With the Nelson Rule Number 1 and a Change in the Mean From the Preintervention to the Postintervention Phase.

The total number of scholarly activities by all residents graduating by their graduation years is shown in Figure 3. It illustrates the trajectory of the average number of scholarly activities per resident across the graduation years preintervention (2016-2018) and postintervention (2019-2021). There is an increasing trend in the number of scholarly activities produced by residents as time progressed from the graduation year 2016 through 2021; however, a notable surge is observed in the postintervention phase, where the average number of scholarly activities quadrupled from 0.9 per resident to 4 per resident. This difference was statistically significant with a P-value < .05 with the Mann-Whitney U-test and ANOVA. Further subset analysis also showed statistically significant differences in all scholarly activities, including the number of presentations (regional and national level) and number of all manuscripts (P < .05) as shown in Table 3. The trend continued at the end of the study.

Figure 3.

Trend of Scholarly Productivity Before and After Intervention.

Table 3.

Number of all Scholarly Activities and All Manuscripts by Pre- and Postgroups.

	Pregroup2016-2018N (Per Resident)	Postgroup2019-2021N (Per Resident)	P-ValueMann-Whitney- Wilcoxon
All scholarly activities	13 (0.9)	66 (4.12)	<.0001
All presentations	13 (0.9)	53 (4.06)	<.0001
Local level	7 (0.35)	15 (0.93)	.001
Regional level	6 (0.5)	35 (2.18)	<.0001
National international	0 (0)	3 (0.19)	<.0001
All manuscripts	0 (0)	13 (0.81)	<.0001
Original articles/QI	0 (0)	5 (0.31)	<.0001
Case report and case series	0 (0)	6 (0.38)	<.0001
Nonpeer reviewed/other publications	0 (0)	2 (0.12)	<.0001

Discussion

This study demonstrates the impact of a bundled intervention on the scholarly output of residents in a community program, as evidenced by an increase following the intervention. The findings are similar to what Waheed et al¹¹ reported from a family medicine residency program in a large academic medical center using the same 13-step structured roadmap with bundled intervention. Waheed et al¹¹ study reported doubling of the published manuscripts in their first study while 3.3-fold increase in overall scholarly activity in the second phase of the study.¹² It is consistent with the 4-fold increase in overall scholarly activity in the current study.

This study is particularly relevant as it addresses a common concern raised by the Residency Review Committee (RRC) regarding the lack of scholarly activity within family medicine residency programs. According to one metanalysis, the most common strategies included having a well-defined research curriculum, faculty mentorship, and protected research time.⁹ In our study, utilizing these 3 common strategies led to increased scholarly output. These interventions foster growth, improve residents’ research skills, encourage critical thinking, and support evidence-based practice.

Overall, many interventions have been implemented in the past, including introducing a point system to encourage a culture of scholarly activity in family medicine residents, with findings comparable to our study.^9,15 A more recent national survey of family medicine residency program directors by Ringwald et al¹⁶ indicates that many family medicine residency programs continue to struggle with building a culture of scholarship. They reported that only 17% of the programs had 25% or more of their residents publish their work. The proportion of programs with 25% or more of their residents presenting at local, regional, and/or national conferences was higher at 50% of the programs. Although it is higher than the proportion of programs where residents publish, it is concerning that residents from every other program are not able to present at the local, regional, or national level during their training.¹⁶ They also showed that the small proportion of programs that exceeded the scholarly activity requirements of ACGME had a more robust infrastructure of mentorship, curricula, IRB, and access to support staff, including librarians and/or statisticians. The multifaceted bundled intervention presented in our study, including the previously reported 13-step structured roadmap for scholarly activity, presents solutions similar to those reported by the successful programs in the survey by Ringwald et al.¹⁶ In addition, the interventions presented in our study do not bring a significant additional cost to the program associated with hiring more experts. The National Research Network (NRN) by the American Academy of Family Physicians (AAFP) provides infrastructure for IRB and other resources that might be a challenge for smaller community hospitals with a single residency program.¹⁷

Conducting literature reviews expands knowledge, while collaboration in multidisciplinary teams enhances workflow. Although increased publications may make candidates more desirable for fellowship positions, there is not enough data to support this. Our study primarily consisted of residents who did not pursue academic careers; however, their scholarly activity increased with our interventions. Residents who choose not to pursue fellowships also benefit from the skill to analyze medical information critically.

There are many reported barriers to creating a culture of scholarship in residency. Ravi and colleagues highlighted challenges, including a lack of residents’ time due to busy schedules, a shortage of faculty mentors, and variability in residents’ commitment to research.¹⁸ Community programs may face additional challenges and barriers to conducting research compared to academic institutions. These programs often have fewer faculty members, limited access to research networks, and fewer opportunities for collaboration or mentorship.

The 13-step roadmap to scholarly activity provides a clear, well-defined timeline to navigate the research process throughout postgraduate training. Having a structured, organized roadmap is especially useful during residency, where demanding schedules and time constraints are common challenges residents face when trying to engage in scholarly activity. The diversity of the residents in the program allows the results to be generalizable; therefore, similar bundled interventions may be implemented and studied in other programs.

There are several limitations of the study. The study focuses on a single-family medicine residency program in a community hospital, which may affect its generalizability. There was a change in the residency program director, along with hiring 6 additional core faculty members in the program at the time of implementation of the intervention bundle. The program director had previous experience in creating and implementing similar curricula elsewhere. The engagement and interest of the key stakeholders is a key feature of all studies reporting quality improvement and implementation of science approaches. One should caution against the attribution of causality of the outcome to the intervention bundle. Additional factors may contribute to a resident's scholarly output, such as personal interest in research and the natural academic and personal growth that occurs throughout training. This study did not differentiate whether the same study was presented multiple times. Each presentation was considered a separate scholarly activity. Lastly, this study did not examine the research experience and qualifications of the faculty and research mentors in the program before and after implementing the intervention bundle. Future work should focus on core faculty mentorship within the family medicine residency programs.

Conclusion

Implementing a comprehensive curriculum with the 13-step structured roadmap to scholarly activity in a multifaceted bundled intervention with a quality improvement approach resulted in a notable increase in residents’ scholarly output, contributing to establishing a culture of scholarship within the residency program at a community hospital with limited academic resources. This comprehensive and well-defined roadmap holds significant potential for widespread adoption among programs aiming to fulfill ACGME's standards for scholarly activity.

Supplemental Material

sj-docx-1-mde-10.1177_23821205251358091 - Supplemental material for Improving Scholarship in a Community Hospital Residency Program With a Curriculum Featuring a Structured Roadmap, Individual Accountability, and Measurement of Outcomes

Supplemental material, sj-docx-1-mde-10.1177_23821205251358091 for Improving Scholarship in a Community Hospital Residency Program With a Curriculum Featuring a Structured Roadmap, Individual Accountability, and Measurement of Outcomes by Abdul Waheed, Shehar Bano Awais, Faiza Butt, Hamna Salimi, Zain Elabideen, Muhammad Ali Chaudhary, Mudasir Umer and Erum Azhar in Journal of Medical Education and Curricular Development

Footnotes

Acknowledgements

The authors acknowledge contributions by Dr Paul V. Atiken Jr, MD, MPH, FAAFP, Program Director of Penn State Family & Community Medicine Residency (2005-2012), in the development of “Scholarly Activity Guidelines,” which became the basic framework for the curriculum and for developing the intervention in this study.

ORCID iDs

Abdul Waheed

Shehar Bano Awais

Hamna Salimi

Mudasir Umer

Ethical Considerations

The WellSpan Health Institutional Review Board (IRB) determined it to be “Not Human Subjects Research” via notification 1887432-2.

Author Contributions

The authors confirm their contribution to the paper as follows: AW, EA, and FB contributed to the study conception. AW, FB, MAC, and EA designed the study protocol and submitted it to the Institutional Review Board. AW, FB, HS, ZEA, MU, and MAC participated in data collection and entry. Data analysis was performed by EA, SBA, MU, and AW. All authors were involved in the interpretation of the results. The manuscript was written by SBA, FB, and AW. All authors contributed intellectually and edited the manuscript. All authors read and approved the final manuscript. AW is the principal investigator (PI), corresponding author, and is responsible for the integrity of the work.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Patient Consent

Patient/Research Subject consent was not applicable since it is not a Human-Subjects-Research.

Presentations

(1) Pennsylvania Academy of Family Physicians (PAPF) Annual Spring Conference, March 2023, Camel Back Resort, PA, (2) WellSpan Discovery Day, May 18, 2023, York, PA.

Supplemental Material

Supplemental material for this article is available online.

References

Boyer

. 1990. Scholarship Reconsidered: Priorities of the Professoriate.

Fincher

Simpson

Mennin

, et al. Scholarship in teaching: an imperative for the 21st century. Acad Med. Sep 2000;75(9):887-894. doi:https://doi.org/10.1097/00001888-200009000-00009

Glassick

CEHM

Maeroff

Boyer

. Scholarship assessed : evaluation of the professoriate. In: Carnegie Foundation Adv Teaching. 1st ed. 2024: 22-26.

Irby

Cooke

Lowenstein

Richards

. The academy movement: a structural approach to reinvigorating the educational mission. Acad Med. 2004;79(8):729-736. doi:https://doi.org/10.1097/00001888-200408000-00003

Grady

Roise

Barr

, et al. Defining scholarly activity in graduate medical education. J Grad Med Educ. Dec 2012;4(4):558-561. doi:https://doi.org/10.4300/jgme-d-12-00266.1

Anandarajah

Gupta

Jain

El Rayess

Goldman

. Scholarly development for primary care residents. Clin Teach. Dec 2016;13(6):415-421. doi:https://doi.org/10.1111/tct.12463

ACGME. Accreditation council for graduate medical education. ACGME-I Program Requirements for Graduate Medical Education in Family Medicine. 2011:1-63.

Crawford

Seehusen

. Scholarly activity in family medicine residency programs: a national survey. Fam Med. May 2011;43(5):311-317.

Wood

McCollum

Kukreja

, et al. Graduate medical education scholarly activities initiatives: a systematic review and meta-analysis. BMC Med Educ. 2018;18(1):318. doi:https://doi.org/10.1186/s12909-018-1407-8

10.

Stevenson

Smigielski

Naifeh

Abramson

Todd

. Increasing scholarly activity productivity during residency: a systematic review. Acad Med. 2017;92(2):250-266. doi:https://doi.org/10.1097/acm.0000000000001169

11.

Waheed

Nasir

Azhar

. Development of a culture of scholarship: the impact of a structured roadmap for scholarly activity in family medicine residency program. Cureus. 2020;12(3):e7153. doi:https://doi.org/10.7759/cureus.7153

12.

Ogrinc

Davies

Goodman

Batalden

Davidoff

Stevens

. SQUIRE 2.0 (standards for quality improvement reporting excellence): revised publication guidelines from a detailed consensus process. BMJ Qual Saf. Dec 2016;25(12):986-992. doi:https://doi.org/10.1136/bmjqs-2015-004411

13.

Waheed

. Enrichment of Culture of Scholarship in a Family Medicine Residency Program: effect of Structured Roadmap, Individual Accountability, and Measurement of Outcomes . (Master's Thesis). Public Health Sciences; 2021. Accessed July 2, 2025. https://etda.libraries.psu.edu/catalog/23900axw24.

14.

Waheed

Azhar

Aziz

Nasir

Chaudhary

Wang

. Impact of a structured roadmap, individual accountability and support, and outcome measurement on the culture of scholarship in a residency program. J Med Educ Curric Dev. 2024;11:23821205241287449. doi:https://doi.org/10.1177/23821205241287449

15.

Seehusen

Asplund

Friedman

. A point system for resident scholarly activity. Fam Med. 2009;41(7):467-469.

16.

Ringwald

Taylor

Seehusen

Middleton

. Family medicine resident scholarly activity infrastructure, output, and dissemination: a CERA survey. Ann Fam Med. 2024;22(5):400-409. doi:https://doi.org/10.1370/afm.3160

17.

AAFP. National research network: supporting research that advances high quality patient care. AAFP. Accessed July 2, 2025. https://www.aafp.org/family-physician/patientcare/nrn.html

18.

Ravi

Cruz

Ali

Burge

Wood

Rodriguez

. Outcomes of a scholarly activity curriculum for family medicine residents. Fam Med. 2021;53(4):285-288. doi:https://doi.org/10.22454/FamMed.2021.812680

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB

Improving Scholarship in a Community Hospital Residency Program With a Curriculum Featuring a Structured Roadmap,Individual Accountability,and Measurement of Outcomes

Abstract

Introduction

Methods

Results

Conclusions

Keywords

Introduction

Methods

Study Design and Study Setting

The Intervention

Data Collection and Data Analysis

Results

Discussion

Conclusion

Supplemental Material

sj-docx-1-mde-10.1177_23821205251358091 - Supplemental material for Improving Scholarship in a Community Hospital Residency Program With a Curriculum Featuring a Structured Roadmap, Individual Accountability, and Measurement of Outcomes

Footnotes

Acknowledgements

ORCID iDs

Ethical Considerations

Author Contributions

Funding

Declaration of Conflicting Interests

Patient Consent

Presentations

Supplemental Material

References

Supplementary Material