Resident Research Bootcamp: Preparatory Course for Required ACGME Research in O-HNS

As Otolaryngology-Head and Neck Surgery (O-HNS) training has evolved in the midst of exponential advances in scientific research, we have been struck by the lack of uniform preparation for O-HNS trainees for their Accreditation Council for Graduate Medical Education (ACGME) mandated “…at least 3 months of a structured research experience for residents” with a tangible outcome being “a completed manuscript suitable for publication in a peer-reviewed journal.” ¹ In an effort to address this deficit, we developed a preparatory course (also known as resident research bootcamp) consisting of a 2-week curriculum presented at the beginning of postgraduate year (PGY) 3 before the research block. It was felt that group learning could efficiently overcome the following knowledge gaps: (1) education on topics useful to all regardless of research thrust (eg, statistical methods) and/or necessary from a regulatory standpoint (eg, Institutional Review Board (IRB) training), (2) topics not formally covered during residency training (eg, professional development), and (3) teambuilding. Presented herein is the design, implementation, and results of this program at the 5-year mark.

Description of this educational process and early results are considered exempt by our IRB (45CFR46.104(d)(1)). A customized curriculum was conceived of to fit within the first 2-weeks of July (9 business days) (Table 1). As early as possible in the schedule, an overview of institutional resources is covered. For the subsequent 8 days, modules are taught in increments of 2-hour modules totaling 70 hours of instruction.

Table 1.

Schedule of educational activities covered during resident research bootcamp.

DAY 1 Instituional Resources and Clinical Trials Overview (8 hr) – Laws, regulations, and guiding principles of research – Life cycle of a clinical trial including study documentation, monitoring, and responsibilities for specific team members (eg, PI, co-I, and study coordinator) – Informed Consent (IC) including electronic IC – Protocol implementation – Tutorial on clinicaltrials.gov – Recruitment and retention of study participants – Adverse event reporting	DAY 2 Critical Manuscript Review (6 hr) – Preparation: read How to review journal manuscripts – Active exercise: review note-worthy articles (eg, Danish sham study) given during module with internet access restricted) with number of articles equal to number of participants and each participant being lead reviewer for one article Critical Writing (2 hr) – Active exercise: forced writing on topic of participants choice (eg, introduction to paper) with reading aloud to group soliciting feedback	DAY 3 Grant Primer (2 hr) – Overview of National Institute of Health (NIH) funding mechanisms (eg, K and R awards) – Discussion of review process (R1, R2, R3; score and impact factor; discussed versus not-discussed; amended applications) Grant Mock Review (2 hr) – NIH grant from faculty library distributed and group reviews – Actual NIH review shared Research Contracts (2 hr) – Research and grants managers cover budgetary issues (eg, NIH salary cap, indirect rate, carry-over) Curriculum Vitae (2 hr) (NIH biosketch, ORCID)	July 4th Holiday	DAY 4 Data Base Design and Management self-directed REDCap MOOC (12 hr) Research Data Collection Strategy (6 videos over 69 minutes, one reading, 2 practice exercises Electronic Data Capture Fundamentals (9 videos over 130 minutes, one practice exercises) Planning a Data Strategy for a Prospective Study (9 videos over 92 minutes, one practice exercise) Practicing What We’ve Learned: Implementation (9 videos over 101 minutes, one practice exercise)
DAY 5 Professional Development (2 hr) – Demystifying professional societies – Academic rank progression Publishing/Journals (2 hr) – Reuters Web of Science (impact factor, h-factor) – International Committee of Medical Journal Editors (ICMJE) authorship criteria – Levels of evidence – Open access (NIH compliance versus pay to publish Social Media (2 hr) – Doximity, LinkedIn, and Twitter tutorials stressing professionalism Ethics (2 hr) – Understanding of IC, overlapping surgical cases, and conflict of interest	DAY 6 Statistical Methods(6 hr) – General concepts (1 hr) – Parametric testing (1.5 hr) – Nonparametric testing (1.5 hr) – Linear and nonlinear regression modeling (1 hr) – Individual project consultations (2 hr) Graphical Presentation of Data (2 hr)	DAY 7 Individual Projects (8 hr) – Each participant is given 1 hour to present their primary research project and receive constructive criticism via group discussion focusing on methodology and scope. – Each participant identifies regulatory issues which will need to be addressed during the regulatory affairs module the subsequent day.	DAY 8 Regulatory Affairs (8 hr) – Institutional Animal Care and Use Committee(IACUC) primer (1 hr) – Institutional Review Board(IRB) primer (2 hr) – IRB Individual project consultations with intent of submitting application (2 hr) – Food and Drug Administration (FDA) Investigation New Drug (IND)/Investigational Device Exemption (IDE) Overview (1 hr) – IRB or IACUC Meeting (2 hr) Note: IRB and/or IACUC meetings typically occur as a working lunch. Participants are paired for this activity due to space considerations of the meetings.	DAY 9 Well-Being/Burn-Out Prevention (2 hr) – Modeled after Google’s Search Inside Yourself Team Building Exercise (6 hr) – Escape Room or Rope Climbing course Breakdown Session (2 hr)

Abbreviations: IC = informed consent, NIH = National Institutes of Health, IACUC = Institutional Animal Care and Use Committee, IRB = Institutional Review Board, ORCID = Open Researcher and Contributor Identification, REDCap = Research Electronic Data Capture, MOOC = Massive Open Online Course, PI = principle investigator, I = investigator

Prior to Day 2, participants are asked to read How to review journal manuscripts ² following which they actively review noteworthy papers from O-HNS literature selected by the chiefs of each service line (eg, Danish Sham Study ³ ) Titles are not provided in advance and internet access is prohibited during reviews. One resident is selected as lead reviewer with subsequent group discussion moderated by the faculty mentor. Day 2 concludes with a spontaneous writing exercise followed by reading aloud to solicit constructive feedback. Day 3 covers grants being taught by an experienced NIH-funded faculty member who has served on study section followed by mock review of faculty member’s submitted grant on NIH template review criteria forms. After group discussion, the actual NIH review is shared. Day 3 also includes budgetary issues associated with grants and curriculum vitae exercises for NIH (www.ncbi.nlm.gov) and ORCID (www.orcid.org) format. Due to difficulty scheduling in proximity to the July 4th holiday, Day 4 consists of self-directed learning via Coursera’s massive open online course (www.coursera.org/learn/clinical-data-management) on Research Electronic Data Capture (REDCap). ^4,5 Day 5 focuses on professional development and metrics especially publishing (impact factor, h-index) via online exploration of Reuters Web of Science (www.webofknowledge.com). Criteria for authorship, ⁶ levels of evidence, ⁷ and open access including both NIH-compliance and pay-to-publish journals ^8,9 are presented. Social media and ethics are presented by the same faculty member. Day 6 covers statistical methods using the open-source freeware “R” (www.r-project.org) and graphical presentation of data. On Day 7, participants present their intended research projects soliciting feedback and identifying regulatory hurdles to be addressed on Day 8. The final day begins with a presentation on resiliency skills to add in the prevention of burnout including a copy of Google’s Search Inside Yourself ¹⁰ followed by a teambuilding exercise (eg, escape room, rope climbing courses).

For the 5-year time period since inception, 3 classes graduated without participating and 2 classes graduated after participating. Peer-reviewed articles by each of these classes are presented in Figure 1 noting that both mean and upper limits were higher for those classes which participated in bootcamp although not statistically significant. Additionally, for the 3 classes prior to bootcamp, 50% of each class each year went in academics while for the 2 classes participating in bootcamp, 80% accepted academic positions.

Figure 1.

Number of publications per resident per graduating class from 2016 until 2020. While the research preparatory course was started in 2016, the first graduating class that participated as PGY3’s graduated in 2019. A nonsignificant increase in average ± standard deviation (5.1 ± 5.0 to 6.9 ± 5.8) as well as peak publications (15 in 2017 for R1 compared to 20 in 2020 for R1 and R2) was seen comparing pre research bootcamp (2016-2018) to post research bootcamp (2019, 2020). (Note that each graduating class from 2016 through 2018 had 5 residents but 2019 had 4 and 2020 had 6 residents because a resident had to repeat a year of training because of personal reasons. In 2019 and 2020, a resident in each graduating class had no publications.). PGY indicates postgraduate year.

Ten years ago, we began a formal process for selection of research projects beginning with an exposure night during early PGY2 at which faculty and collaborators give brief overviews of projects. By Thanksgiving, a mentor and primary project are chosen in time for submission of an American Academy of Otolaryngology-Head and Neck Surgery Centralized Otolaryngology Research Efforts grant in January. While this initial step codified identification of research projects, it failed to address skills necessary to carry out research. To address this, the curriculum presented herein was developed. The most frequently cited need for improvement by the participants is expanded statistical training. While we cannot definitively state that the bootcamp has directly lead to increased publication and academic positions, we feel confident that it has refocused the importance of research for both faculty—who often sacrifice normal resident coverage during the bootcamp—and trainees.