Online Infectious Diseases Subspecialty Supplementary Curriculum for Medical Students and Residents: Moving Beyond “You Get What You Get”

Abstract

OBJECTIVES

Students and residents rotating through infectious diseases (ID) electives are instructed primarily by participation in rounds and clinics, with teaching focused on diseases encountered. This “you get what you get” approach allows learners to apply knowledge directly to patient care, however, may miss topics encountered in standardized testing. This multisite study investigates the use of asynchronous web-based learning modules and its impact on student and resident knowledge.

METHODS

Students and residents rotating through an ID elective were assigned to their standard elective (old) or asked to complete asynchronous web-based learning modules in addition to the standard curriculum (new). Learners submitted pre- and post-tests and scores were tabulated. In the following academic year, learners at the host site were provided the learning modules and a post-elective survey.

RESULTS

Nine learners (100%) completed the pre-test and 5 (55.6%) completed the post-test in the standard (old) curriculum group, while 15 (100%) completed the pre-test and post-test in the new curriculum group. The mean percentage change in accuracy was 9% and 5.3% in the old and new curricula, respectively. Most (94%) survey respondents recommended continued use of the curriculum and expansion to other subspecialty electives.

CONCLUSIONS

Subspecialty electives have multiple purposes including assisting learners in exploring careers, providing a well-rounded medical experience, or preparing learners for content in standardized tests. Consistent curricula are not always provided with electives to supplement the clinical experience. Our web curriculum was well-received with perceived knowledge gain, though with very small pre-post-test groups a score improvement could not be determined. An asynchronous online curriculum for learners in ID was feasible and well-received among faculty, and learners felt their knowledge was enhanced. Content areas supplemented those encountered during the ID elective. While an improvement in post-test scores was not demonstrated, learners and faculty felt modules were beneficial.

Keywords

infectious diseases subspecialty elective asynchronous curriculum

Introduction

Medical students and internal medicine (IM) residents have the opportunity to complete elective rotations in infectious diseases (ID) and other IM subspecialties.¹ Goals for these electives are multifaceted; in some instances, they help learners investigate career options and in others, they are directed to help prepare learners for associated content in standardized exams.² While curriculums differ by institution, students and residents rotating through subspecialty clinical electives often learn by participation in clinic and hospital rounds, with teaching focused on the disease processes encountered.^3,4 This “you get what you get” approach is different from the curriculum in the first three years of a traditional four-year medical school curriculum which includes well-defined learning objectives and curricula focused on preparation for standardized exams including United States Medical Licensing Examination (USMLE) and National Board of Medical Examiner (NBME) Shelf Examinations. Learning objectives and curricula for subspecialty electives are often ill-defined.³ At the medical school level, there is little guidance on what learning objectives or topics a subspecialty elective experience should cover. These are often left entirely to the discretion of the institutional division hosting the elective. At the graduate medical education (residency) level, the Accreditation Council for Graduate Medical Education (ACGME) provides limited guidance on this matter and recommends IM program directors identify board-certified faculty to serve as “subspecialty education coordinators” who will be accountable for the coordination of the residents’ subspecialty educational experiences in that subject area.² Within a subspecialty division, the approach to teaching senior medical students and IM resident trainees also varies from the teaching directed to the division's subspecialty fellow trainees. Fellows have a dedicated, all-encompassing curricula to prepare them for independent subspecialty practice and subspecialty board exams. Fellow-level conferences are presented at a more advanced level and over a longer period of time (1–3 years), limiting the utility of these conferences in meeting the educational needs of student and resident learners (rotating days to 1 month). The ACGME has program requirements for the subspecialties themselves; however, these offer little to no instruction regarding optimal learning goals for medical students or IM resident learners, suggesting only that these learners do not interfere with fellow-level education.⁵

There is variable data on the impact of subspecialty electives on standardized test scores. Some studies have failed to identify a statistically significant relationship between elective exposure and subspecialty American Board of Internal Medicine (ABIM) score reports in an academic IM residency program; although electives varied as to inclusion of pre- and post-testing, required reading, and subspecialty conference exposure. In this same study, there was also not a statistically significant relationship between repeated elective exposures in a single specialty and ABIM performance suggesting quantity of time per elective may not promote additional gains in medical knowledge.² Small studies of subspecialty electives have demonstrated that rotations with well-defined learning objectives can lead to an improvement in standardized test scores^3,4 and in medical knowledge related to the rotation objectives among residents.^5,6 In-training exam (ITE) scores can be improved by multiple-choice testing curricula, IM core conference attendance, and time spent in independent reading on medical topics.^7–11 Previous studies have also indicated that performance on the IM ITE predicts performance on the ABIM examination,¹² therefore, initiatives that improve IM ITE scores may also lead to improved ABIM scores. While performance on standardized testing such as the USMLE and the ABIM examinations are not the only measure of competency for medical school or residency, successful completion of these exams with a passing score is imperative to the success of both medical students and residents as well as their respective institutions.

The prevalence of the “you get what you get” curriculum is not entirely clear, as it is unknown what percentage of IM Subspecialty Electives already have dedicated curricula for rotating students and residents. A 2018 SurveyMonkey® inquiry of IM Residency and ID fellowship program directors across the United States recruited through the Association of Program Directors in Internal Medicine (APDIM) discussion forum and via email to ID fellowship program directors was conducted to assess needs related to ID elective didactic curricula. In total, 56 surveys were received; 39 were from IM residency program directors and 17 were from ID fellowship program directors. Response rate could not be accurately calculated due to the fluid membership and use of the forum. Only 31 (55%) of respondents reported dedicated didactic curricula for residents, and only 17 (30%) reported dedicated didactic curricula for fourth-year students (unpublished data).

The primary aim of this study was to investigate whether a dedicated online curriculum for fourth-year students and IM resident subspecialty learners in ID would (1) increase medical knowledge and (2) provide a feasible and useful supplemental curriculum for ID elective learners and faculty.

Methods

Using the Wix.com platform, a website, www.jayhawkid.com was created to house 11 ID modules: human immunodeficiency virus (HIV), sexually transmitted diseases (STDs), hospital-acquired infections, upper respiratory tract infections, community-acquired pneumonia, central nervous system infections, endocarditis, osteomyelitis, antibiotics, tuberculosis, and immunocompromised infections. Nine of the 11 topics selected were chosen given their inclusion in the American College of Physicians Internal Medicine Essentials for Students text, and 2 additional topics (Antibiotics and Immunocompromised infections) were added based on comments from students rotating through the host ID elective. Each topic included in the web curriculum had a downloadable, editable PowerPoint file as well as an accompanying voiceover PowerPoint video with associated clinical practice guideline links as applicable. Each module was designed to take less than 45 min to complete. An asynchronous web platform was chosen given the variable and overlapping schedules of learners and time constraints of a repeated synchronous curriculum. Cases and an Individual Readiness Assurance Test (IRAT), a short formative question set ensuring learners attained concepts from the material, were included to make the resources as interactive as possible.¹³ The PI's email was included on the site for questions and comments as well.

Participants were informed of the study via email and asked to move forward with voluntary survey completion if they consented for participation. This study was approved on October 15, 2020, by the University of Kansas Medical Center (KUMC) Human Research Protection Program under the KUMC Policy for Flexible IRB Review, IRB# STUDY00146427. Subjects included fourth-year medical students and IM residents rotating through a clinical ID elective block in three different academic medical centers in the United States over a 12-month period (2019–2020). Learners were excluded from data collection if their rotation time was less than five days as this was thought to be insufficient time to complete the online curriculum when combined with the standard, concurrent clinical elective commitments; rotations varied from 5 days to 1 month. Prior to the start of the clinical ID elective, learners were provided with an email notification of the study and a web link for the online curriculum which could be accessed upon completion of a 20-multiple choice question (MCQ) pre-test (Supplementary 1) related to material in the curriculum. This test was piloted for clarity to ten students at the host institution. The online curriculum consisted of 11 modules, which cover a range of ID subjects. In this prospective cohort study, learners in alternating blocks were assigned to the supplemental curriculum (standard clinical rotation plus curriculum) and the comparison group completed the standard clinical rotation only. Alternating blocks were chosen to account for expected medical knowledge acquisition expected during an academic year. Students exposed to the curriculum were asked to complete the modules at their own pace before the conclusion of their elective block. On the last day of their rotation, all learners were asked to complete an identical MCQ post-test. For students in the comparison group, access was provided to the web curriculum at the conclusion of their elective for future reference or to study from independently, as desired.

Statistical analysis

Individual pre-post-test changes, pre-test scores and post-test scores as well as pre-post-test absolute and relative improvement in scores were summarized by group (intervention, control) using means and 95% confidence intervals. Any unanswered questions on a partially completed test were assumed incorrect. Any tests not started were assumed missing (Table 1). Scores were also plotted for comparison. Data review was completed by the University of Kansas Department of Biostatistics and Data Science.

Table 1.

Summary statistics of medical knowledge scores.

DOMAIN		OLD CURRICULUM, MEAN (95% CI)		NEW CURRICULUM, MEAN (95% CI)	ABSOLUTE DIFFERENCE, MEAN (95% CI)
Number of questions correct out of 20
Pre-test	n = 9	11.4 (10.2, 12.7)	n = 15	10.7 (9.3, 12.1)	−0.8 (−2.5, 1)
Post-test	n = 5	14 (11.8, 16.2)	n = 15	11.7 (10.4, 13.1)	−2.3 (−4.5, 0)
Change	n = 5	1.8 (−0.2, 3.8)	n = 15	1.1 (−0.4, 2.5)	−0.7 (−2.9, 1.5)
Precent of questions correct out of 20
Pre-test	n = 9	57.2 (51.1, 63.3)	n = 15	53.3 (46.3, 60.4)	−3.9 (−12.7, 4.9)
Post-test	n = 5	70 (59.2, 80.8)	n = 15	58.7 (51.9, 65.4)	−11.3 (−22.5, −0.2)
Change	n = 5	9 (−1.2, 19.2)	n = 15	5.3 (−2, 12.6)	−3.7 (−14.6, 7.3)

To evaluate the second aim, in the subsequent academic year (2020–2021), learners in the clinical ID elective at the host academic institution were enrolled in the standard plus online curriculum group and invited to complete a brief survey using a 5-point Likert scale to evaluate their satisfaction with the online curriculum. Responses were obtained via a SurveyMonkey® link on the curriculum website. The survey assessed the learners’ general satisfaction with the online curriculum, topics, platform, and self-assessed learning using the modules. At the conclusion of the study period, the ID elective faculty (faculty and subspecialty fellows) were asked to complete a satisfaction survey (Supplementary 2) addressing general feasibility of and satisfaction with the ID online curriculum for teaching purposes within the elective. For both parts of the study use of the curriculum was voluntary with use self-reported via the satisfaction survey. Participants were informed of the study via email and asked to move forward with voluntary survey completion if they consented for participation. The study was reviewed and approved by the University of Kansas Medical Center (KUMC) Human Research Protection Program under the KUMC Policy for Flexible IRB Review. The Authors declare that there was no conflict of interest.

Results

Medical knowledge scores

Nine learners in the standard (old) curriculum group (100%) completed the pre-test and five (55.6%) completed the post-test. The mean number of correct responses in the pre-test was 11.4 out of 20 questions. The mean number of correct responses in the post-test was 14 out of 20 questions. The mean number of correct responses change from the pre- to post-test was 1.8. The mean percentage of questions correct in the pre-test was 57.2% and the mean percentage of questions correct in the post-test was 70%, with a mean percent change of 9%.

Fifteen learners in the standard plus online (new) curriculum group (100%) completed the pre-test and 15 (100%) completed the post-test. The mean number of correct responses in the pre-test was 10.7 out of 20 questions. The mean number of correct responses in the post-test was 11.7 out of 20 questions. The mean number of correct responses change from the pre- to post-test was 1.1. The mean percentage of questions correct in the pre-test was 53.3% and the mean percentage of questions correct in the post-test was 58.7% with a mean percent change of 5.3%. Individual pre-post-test changes and pre-post absolute and relative improvement in scores were also plotted (Figure 1).

Figure 1.

Individual pre-post-test changes.

Survey results

Of the learners who participated in the standard plus online (new) curriculum during the 2020–2021 academic year, 38/48 (79%) completed the satisfaction survey (indicating completion of at least one module) which included questions regarding ease of use and perceived knowledge gains (Table 2). All survey respondents felt their knowledge was enhanced by the curriculum. All participants agreed that the online ID curriculum was easy to use. The top three topics that participants reported having the least experience with while on their ID rotations in (one) institution included tuberculosis (74%), HIV (65%), and STDs (62%). Most participants agreed that the videos were a helpful addition to the PowerPoint modules. Most (94%) of the respondents recommended continued use of the curriculum and (94%) of the respondents also recommended expansion to other IM subspecialty electives.

Table 2.

Satisfaction survey.

Satisfaction survey questions

The online ID curriculum was easy to use.

The podcasts were a helpful addition to the PowerPoint modules.

My knowledge in the ID specialty area was enhanced by the online ID curriculum.

I would recommend continuation of the online ID curriculum.

I would recommend online curriculums for other subspecialty electives.

Additional comments (examples)

“I really liked the curriculum and think other specialties should do it too! The antibiotic lecture was super helpful to do before starting the rotation because day one you will be exposed to so many different antibiotics its helpful to have that video as a starting point and reference”

“The online curriculum was helpful in guiding conversations during my ID rotation. The fellow I worked with would have me watch a video each day so we could have a discussion about it. It helped reinforce the knowledge I was gaining on the wards”

“It was very helpful to have this curriculum while on the ID rotation. It was useful when needing to refresh on topics and to know a bit more about management. It gave me a good starting point in my learning while on this rotation. I also watched a majority of the videos the weekend prior to starting my rotation. This helped immensely to prepare for the rotation”

Nineteen of 20 (95%) of ID elective faculty completed the faculty survey. All respondents agreed that the online curriculum was of value for learners. Most 16/19 (84%) agreed that the online curriculum made the teaching experience more efficient, and 84% agreed that the curriculum helped reinforce their teaching regarding topics commonly encountered on the ID rotation. Most (89%) disagreed that the online ID curriculum negatively impacted their teaching experience for learners. Survey comments were uniformly positive and included, “Students and residents both were impressed with the website and topics and material quality. Pharm ID has provided to their pharm students who also reported it to be quite helpful…”; “The lectures were great for learners to use as a framework to then delve deeper into the topics as they applied them to the patients they encountered…the curriculum is well done”; “An excellent tool for students to use as an initial reference and for us to refer them to”; “Feedback from resident/student learners very positive. I have pulled site up during discussion time and reviewed with the learner”; “The online curriculum was very helpful to ensure learners had a good resource for commonly encountered ID topics. It also provides me with a good framework to ensure I discussed a broad range of topics with my students”; “Great resource for all students, gives them a different way to learn other than on wards. Helps them focus on ‘big pictures’”; “Online curriculum is well structured and covers good basic ID topics…I need to remember to use it as a supplemental resource more frequently.”

Discussion

While the ACGME requires clinical experiences in each of the IM subspecialties, it does not dictate the length or content of these experiences.¹ The Liaison Committee on Medical Education (LCME) Data Collection Instrument, the main report documentation metrics of the LCME for Medical School Institutions, does not require course-level details for fourth-year electives. The LCME requires only that the curriculum offers electives to supplement core required experiences for exposure to non-core specialties and pursue personal academic interests (standard 6.5).¹⁴ This online curriculum was developed to fill a gap in didactic curricula for medical student and resident learners on ID elective rotations identified through our national needs assessment survey (unpublished data). The online modules offer materials that can be used to supplement topics encountered during experiential learning encountered on the rotation or during other conferences. They cover core ID topics and provide a framework around which students can organize information. The modules can be used for self-directed study or as structured presentation materials for faculty to use during in-person teaching sessions and included an optional pre-post-test and IRAT. Small enrollment numbers in this study given the limiting factors of response rates for pre and post-tests as well as the effect of the COVID-19 pandemic on elective rotation meant that the study was not powered as originally designed to conduct planned statistical analyses. Feedback, however, was favorable. In part 2 of the project, survey data suggested the supplemental online ID curriculum was well-received and easy to access. This result is similar to a prior online ID curriculum that was well-received among learners.³ Our curriculum adds to the existing resources in that our curriculum includes both presentation slides as well as videos and a pre-post-test; the addition of videos was primarily related to feedback from medical students who favor video as the modality of choice for pre-learning at the host institution. Our curriculum covers topics commonly encountered on standardized exams and adds five additional content areas not currently covered through other free, online ID modules geared toward the student or resident learner. We focused on keeping the modules short (<45 min), and in revised editions shortened the videos into portions <20 min each so that they could be more easily watched within a consult day. Notably, our curriculum also covers three topics identified by learners as areas where they reported little or no clinical experience during their ID elective (tuberculosis, HIV and STDs). These commonly tested, but perhaps less often encountered topics or “you get what you get” from the elective host institution, may be especially beneficial to include in online supplemental curricula. These topics would likely vary across hospital systems, geographic locations, various patient populations, and inpatient versus outpatient focus. Institutional needs assessments could identify topics from which learners may benefit. Additionally, these online modules can serve a model for further expansion to other elective rotations in the future to augment the learners’ and faculty experience.

Learners reported that the online curriculum enhanced their knowledge and experience, though a large limitation of our study was that it was not powered to conduct statistical analyses to make comparisons to assess for measurable knowledge gains. Additionally, demographic data aside from student versus resident level of learner was not collected. Furthermore, rotation changes related to the COVID-19 pandemic likely impacted participation and response rates during a portion of the first year of the study during the 2019–2020 academic year. Additionally, participation in the host site was most robust and this may have been attributed in part to direct involvement of the primary investigator as teaching faculty during the elective.

Conclusions

Future directions for subspecialty elective asynchronous curricula include the development and incorporation of an institutional needs assessment to determine which topic areas should be emphasized based on typical clinical experiences during the rotation. This assessment could be hosted on the online platform for elective course directors to use and subsequently assign or suggest modules for students or residents to supplement their electives. Conducting a similar study in which more enrollees complete the pre and post-test assessments may result in the ability to conduct accurate statistical analyses. Further, formative questions could be embedded into the module to provide learners additional feedback on understanding the material and for standardized examination practice as well as addition of links to primary literature for review on each topic area.

Supplemental Material

sj-docx-1-mde-10.1177_23821205231171206 - Supplemental material for Online Infectious Diseases Subspecialty Supplementary Curriculum for Medical Students and Residents: Moving Beyond “You Get What You Get”

Supplemental material, sj-docx-1-mde-10.1177_23821205231171206 for Online Infectious Diseases Subspecialty Supplementary Curriculum for Medical Students and Residents: Moving Beyond “You Get What You Get” by Nikki Miller, Jo Wick, Vera P Luther and Jessica R Newman in Journal of Medical Education and Curricular Development

Supplemental Material

sj-docx-2-mde-10.1177_23821205231171206 - Supplemental material for Online Infectious Diseases Subspecialty Supplementary Curriculum for Medical Students and Residents: Moving Beyond “You Get What You Get”

Supplemental material, sj-docx-2-mde-10.1177_23821205231171206 for Online Infectious Diseases Subspecialty Supplementary Curriculum for Medical Students and Residents: Moving Beyond “You Get What You Get” by Nikki Miller, Jo Wick, Vera P Luther and Jessica R Newman in Journal of Medical Education and Curricular Development

Footnotes

Acknowledgments

The authors would like to thank Dr David Lindholm and Lynne Goulding for their assistance in recruiting and communicating with learners. Also, students and residents from: University of Kansas Medical Center, Wake Forest University School of Medicine, and Uniformed Services University of the Health Sciences.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors received financial support for the Wix subscription, web domain, a Logitech C270 Widescreen HD Webcam, and Dr Wick's statistical analysis via a University of Kansas School of Medicine Academy of Medical Educators-Medical Education Mentored Grant totaling $4715.88

Ethical approval

This study was reviewed and approved October 15, 2020, by the University of Kansas Medical Center (KUMC) Human Research Protection Program under the KUMC Policy for Flexible IRB Review, IRB# STUDY00146427.

Informed consent

Participants were informed of the study via email and asked to move forward with voluntary survey completion if they consented for participation.

Trial registration

Not applicable because this article does not contain any clinical trials.

ORCID iD

Jessica R Newman

Supplemental material

Supplemental material for this article is available online.

References

ACGME Program Requirements for Graduate Medical Education in Internal Medicine. Accreditation Council for Graduate Medical Education. 2022 July [cited 7 Dec 2022]; Available from: https://www.acgme.org/globalassets/pfassets/programrequirements/140_internalmedicine_2022v4.pdf.

Shanmugam

Tsagaris

Schilling

, et al. Impact of subspecialty elective exposures on outcomes on the American board of internal medicine certification examination. BMC Med Educ. 2012;12:94.

Hale

. What are they here to learn? Meeting the needs of medical students and residents on an infectious disease service. Open Forum Infect Dis. 2017;4(Suppl 1):S450.

Miller-Chang

Gauer

Butler

, et al.

Does an endocrinology subspecialty residency rotation enhance resident endocrine clinical knowledge?

BMC Med Educ. 2022;22(1):49.

ACGME Program Requirements for Graduate Medical Education in Infectious Diseases 2022. July [cited 6 Dec 2022]; Available from: https://www.acgme.org/globalassets/pfassets/programrequirements/146_infectiousdisease_2022_tcc.pdf.

O’Dell

Klassen

Moore

. The use of outcome measures to evaluate clincial rheumatology curriculum changes. J Rheumatol. 1993;20(6):1033‐1036.

Haponik

Bowton

Chin

, et al. Pulmonary section development influences general medical house officer interests and ABIM.

Hellmann

Flynn

. Development and evaluation of a coordinated, ambulatory rheumatology experience for internal medicine residents. Arthritis Care Res. 1999;12(5):325‐330.

Fox

Kirk

. Subspecialty training in the ambulatory clinic: a preliminary investigation of an endocrinology curriculum. Acad Med. 2003;78(11):1170‐1174.

10.

Mathis

Warm

Schauer

Holmboe

Rouan

. A multiple choice testing program coupled with a year-long elective experience is associated with improved performance on the internal medicine in training examination. J Gen Intern Med. 2011;26(11):1253‐1257.

11.

McDonald

Zeger

Kolars

. Associations of conference attendance with internal medicine in-training examination scores. Mayo Clin Proc. 2008;83(4):449‐453.

12.

Babbott

Beasley

Hinchey

Blotzer

Holmboe

. The predictive validity of the internal medicine in-training examination. Am J Med. 2007;120(8):735‐740.

13.

Parmelee

Michaelsen

Cook

Hudes

. Team-based learning: a practical guide: AMEE guide No. 65. Med Teach. 2012;34(5):e275‐e287.

14.

Liaison Committee on Medical Education. Standard 6.5: elective opportunities. Functions and Structure of a Medical School: Standards for Accreditation of Medical Education Programs Leading to the M.D. Degree. 2021. Accessed January 2, 2023. https://lcme.org/publications/.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB

0.02 MB