A Novel Educational Tool to Improve Medical Student Knowledge Acquisition During Surgical Rotations

Abstract

Background

The addition of a novel education tool helps students improve understanding of general surgery topics. However, the effect of the new tool on objective exam performance is unknown.

Materials

A 10-item card of high-yield general surgery topics was implemented in the third-year surgery clerkship. Students reviewed these topics with general surgery residents. Scores from the National Board of Medical Examiners (NBME) surgery subject exam and Step 2 Clinical Knowledge (CK) board exam were compared among students who completed the Ask-a-Resident Topic card to a control group.

Results

Students who participated in the curriculum demonstrated significantly better scores on the NBME Surgery exam, t (236) = −2.56, P = .006. There was not a significant effect of the curriculum on Step 2 CK scores, although students who participated in the curriculum (M = 250.7, SD = 13.4) achieved higher scores than the control group (M = 247.8, SD = 14.2).

Discussion

The novel curriculum may improve objective student performance on standardized surgery exams.

Introduction

Medical student education during clinical years, and particularly during surgical rotations, is formative for both students and the field of surgery itself. Student experiences and the learning environment can dramatically shape their perception of surgery and motivate their career decisions.^1,2 Unfortunately, the rigor of surgical rotations and traditional curricula can lead to burnout in medical students. ³ Burnout, in turn, is associated with reported mistreatment and neglect, which occurs disproportionately more frequently during surgical rotations. ⁴ A feedback loop exists—students who enter surgical rotations fearful of adverse treatment by residents and attendings and have a negative educational experience may then confirm their preconceived assumptions about surgery and perceive mistreatment.^5,6

In addition, residents play a key role in medical student education and can foster effective role-modeling and reinforce knowledge. Residents are primary educators and the students’ primary point of contact during the rotation.^7,8 However, evidence suggests the quality of surgical resident teaching and interactions with medical students have worsened over time. ⁹ This change in the student-resident dynamic can have implications on student exam performance, with better exam scores associated with higher quality teaching from residents.^10,11

Our group recently published the effect of a novel, resident-driven curriculum for teaching medical students during their surgical rotation. This curriculum involved the use of an educational tool, called Ask-a-Resident Topic (ART) cards, which was designed to create informal opportunities for residents to teach medical students. We found that when residents engaged students and taught them basic surgical knowledge, students felt more prepared for exams and completed their rotation with a greater interest in pursuing surgery. ¹²

We now seek to follow-up these promising results by evaluating the addition of the ART cards on objective student performance on the surgery clerkship. The primary goal of this study, therefore, is to compare exam scores of students who participated in the new curriculum to those who did not. We hypothesized that the introduction of the ART cards would improve student exam scores.

Methods

Study Design

This study was conducted at the University of North Carolina (UNC) School of Medicine. The study was approved by the Institutional Review Board at the University of North Carolina with a waiver of informed consent.

Details of the methodology are described in full in our prior publication. ¹² Briefly, an ART card was implemented as part of the third-year surgery clerkship curriculum at UNC at the start of 2018-2019 academic year. Ten high-yield core surgical topics were selected including right upper quadrant pain, postoperative fever, hernia, breast mass, and trauma evaluation, among others. Students were asked to review a topic and prepare for an informal, 5-minute conversation about that topic with at least one of the general surgery residents. Key points to discuss for each topic were included on the assignment sheet. Students were required to complete at least 6 of 10 topics by the end of the rotation for credit.

Raw National Board of Medical Examiner (NBME) surgery subject exam scores, as well as raw scores from the United States Medical Licensing Examination Step 2 Clinical Knowledge (CK) board exam from third-year medical students who completed their surgery rotation at UNC during the 2018-2019 academic year (n = 120) were compared to medical students from the 2017 to 2018 academic year (n = 118). The 2017-2018 student cohorts were chosen as the control group because they completed the surgery clerkship prior to implementation of the ART card; however, all other aspects of the curriculum were identical.

Statistical Analysis

Qualitative data were previously collected and reported in our prior publication. ¹² The primary objective of this study was to evaluate the effect of the ART curriculum on objective outcomes using the NMBE surgery subject exam and Step 2 CK board raw exam scores. The distribution of scores did not exceed the a priori threshold for skewness; therefore, parametric tests were used for analysis. Independent one-tailed, two-sample t-tests were calculated using Stata 16 (StataCorp LLC, College Station, TX). ¹³ P < .05 was considered significant.

Results

NBME Scores

120 students who participated in the ART curriculum and 118 students in the control group took the NBME surgery subject exam. Students who participated in the ART curriculum (M = 76.3, SD = 7.3) demonstrated significantly better scores compared to students who did not (M = 73.9, SD = 6.7), t (236) = −2.56, P = .006. Complete range of scores are depicted in Figure 1.

Figure 1.

Range of NBME surgery subject exam scores by academic year. Abbreviations: AY, academic year; NBME, National Board of Medical Examiners.

Step 2: CK Scores

105 students who participated in the ART curriculum and 108 students in the control group took the Step 2 CK board exam at the time of analysis. There was not a significant difference in Step 2 CK scores between groups, although students who participated in the curriculum achieved higher scores than the control group (M = 250.7, SD = 13.4 and M = 247.8, SD = 14.2, respectively), t (211) = −1.53, P = .06. Complete range of scores are depicted in Figure 2.

Figure 2.

Range of Step 2 CK board exam scores by academic year. Abbreviations: AY, academic year; CK, Clinical Knowledge.

Discussion

Medical student education shifts from lectures to hands-on learning, and this is especially prominent during surgery rotations. This abrupt change in learning environment during surgical rotations has been associated with medical student burnout and negative attitudes toward surgery.^3,5,6 Our group recently published a study that introduced a novel educational tool—the ART card—to the third-year surgical clerkship curriculum. Our results previously demonstrated that the ART card is mutually beneficial for both medical students and residents, is easy to implement, not burdensome to students, and can foster a more positive learning environment.

Our study extends these findings and corroborates the subjective data previously collected. Students who participated in the new curriculum earned significantly better scores on the NBME surgery subject exam than those who did not participate in the ART curriculum. This suggests that new learning strategies and content delivery during clinical training supplements previous traditional didactics. Importantly, although calculation of overall student grades can vary among different medical schools, the NBME surgery subject exam is nationally standardized, and therefore, can be used as a generalizable metric for curriculum evaluation.

Our data are also in line with a similar interactive curriculum, in which medical students briefly presented patients during daily morning surgical rounds. ¹¹ Mean NBME scores were significantly higher in the group of students who participated in this curriculum than those who did not. Taken together, evidence supports that medical students receive a large proportion of their surgical education from residents, and an interactive curriculum may enhance knowledge acquisition.^11,12,14

Students who completed the ART cards did not score significantly higher on the Step 2: CK exam than students who did not have this curriculum. However, mean scores in the intervention group were greater than those in the control group. This likely represents the nature of the Step 2: CK exam vs. the NBME subject exams. Namely, Step 2: CK tests knowledge across all medical specialties and not just surgical-related topics. ¹⁵ It is possible that students in the intervention group performed better on specific surgery questions, but these data are not available. Further, it is possible that our analysis of Step 2: CK scores is underpowered. Notably, at the time of our study, fewer students completed their board exams than their NBME surgery clerkship. This discrepancy is because students at this institution have the opportunity to take a planned absence between the third and fourth school year. It is feasible to reexamine these data once the entire original cohort finishes Step 2; however, this may take up to several years.

This study has several limitations. First, our study is based on a single institution experience on a surgical clerkship, and curriculums can vary somewhat among different medical schools. However, our results are likely generalizable to other institutions as certain components of the surgery clerkship curriculum are required in order to receive accreditation, and the content within ART cards is considered basic knowledge. In addition, the use of a national standardized examination as a metric of improved performance may enhance reliability and reproducibility. Second, this is a cohort study rather than a randomized trial. As a result, the findings are observational and not causal in nature. However, implementation of the ART cards was the only substantial curriculum modification between the control and intervention groups, and therefore, our results are likely reflective of the effect of this change.

The ART curriculum currently addresses commonly tested and important general surgery topics, although our institution has recently expanded use of the ART curriculum to surgical subspecialties. It would be important to evaluate the impact of the specialty specific topics to student performance in the future. In addition, implementation of the ART curriculum at other institutions would allow for standardization of the curriculum and reliability testing.

Conclusions

A positive learning environment is necessary for effective medical student education and for reducing the risk of burnout on rigorous clinical rotations, such as surgery. Innovative tools, such as the ART cards, have been shown to enhance traditional learning modalities and are mutually beneficial for residents and students. ¹² Importantly, this benefit extends to objective student performance on the surgery clerkship. ART cards may enhance student knowledge of surgical topics, improve exam scores, and may be a reliable addition to the surgery curriculum.