Abstract
Objectives
To evaluate how medical students perceive COVID-19 has impacted various otolaryngology educational experiences and appreciate useful adaptations/resources that can be implemented moving forward.
Methods
We circulated a web-based questionnaire among pre-clinical/clinical medical students. Participants were asked to rate their experiences in the realms of research, clinical exposure, faculty interaction, interest group (IG) activities, advising, and USMLE preparation considering the COVID-19 pandemic. Qualitative responses were mapped into thematic domains.
Results
Completed responses from 201 medical students demonstrated an effect across all educational activities. Pre-clinical students experienced a negative impact on their IG involvement (77%; 86/111) and clinical students reported a diminished clinical experience (60%; 40/67), with 54% (36/67) citing below-expectation surgical exposure. Overall, 51% (90/178) of students reported a perceived decrease in beneficial otolaryngology career advising, decreased research involvement (57%, 101/178), and reduced faculty interaction (79%, 141/178). Faculty mentorship programs, curated video resources, and online question banks were selected as valuable resources during COVID-19 (21%, 20%, and 13% of selections, respectively) that may offer ongoing benefit. Free-response feedback also indicated that COVID-19 negatively impacted students’ mental well-being and inhibited opportunities to complete away sub-internships. As for supplemental resources, students appreciated virtual grand rounds/teaching, interest group events, online resources, and meetings with attendings/residents.
Conclusions
According to students, COVID-19 negatively impacted a broad array of otolaryngology educational experiences including research, faculty interaction, and interest group involvement. Students also indicated several well-received adaptations, such as virtual mentorship programs, curated video materials, remote research opportunities, and online question banks. This feedback helps contextualize selection committee decisions and, importantly, assists clerkship directors and undergraduate medical educators to optimize their curriculums.
Introduction
Like many other facets of healthcare, undergraduate medical education across the United States continues to experience changes due to the COVID-19 pandemic. To continue effectively educating future physicians, while also prioritizing safety, medical schools introduced a variety of alterations to their educational curricula. As a result, students needed to adjust to the new realities of virtual classroom learning, research experiences, faculty interactions, USMLE preparation, and clinical exposure.
Matching into otolaryngology residency is a rigorous and competitive process—requiring strong faculty letters of recommendation, excellent clinical performance on sub-internships, and high grades and USMLE scores. 1 COVID-19 precautions introduced an added layer of challenges and stress as students attempt to assess their interest in the field and compile a competitive application. With most programs limiting or suspending away rotations the past two cycles, many residency directors have also had to grapple with evaluating candidates without the prior rapport often established through visiting sub-internships.2,3
Early in the pandemic, many of these challenges resulted in innovative adaptations. Key challenges to overcome included truncated clinical experience for 3rd and 4th year students, minimal in-person patient interactions, and suspension of in-person residency interviews. Proposed innovative solutions to COVID-imposed limitations ranged from hosting virtual “meet and greet” sessions with residency applicants to using novel interactive online resources to advance surgical knowledge.3-5
Few studies have provided direct feedback from medical students about their educational experiences during COVID-19. In one of these studies on general medical education, Coffey et al surveyed colleagues at UC San Diego and found that while many students appreciated the flexibility of hybrid learning and expanded access to online resources, a significant number of students also felt increased anxiety, isolation, and difficulty concentrating on studies while home amid the pandemic. 5
In this study, we sought to investigate perceptions among medical students regarding the impact of COVID-19 on their otolaryngology educational experiences. We conducted a nationwide survey of students attending medical schools selected to provide broad geographic representation and address fundamental aspects of medical education, including pre-clinical/clinical learning, USMLE preparation, faculty interactions, and research experience.
Patients and Methods
Design
This study was approved by the Columbia University Institutional Review Board (IRB AAAT0653). Data were obtained via a survey distribution. All co-authors collaborated to develop our final questionnaire. The survey was created via Qualtrics (Provo, Utah). 6 Utilizing “select-all that-apply,” “multiple choice,” and “free response,” questions, both quantitative and qualitative feedback were obtained. Thirty questions were included taking an average of 12 minutes to complete. Survey flow branched primarily based on respondents’ medical school progression: pre-clinical (1st, 2nd year), clerkship year (3rd year), or post-clerkship year (4th year). Questions were tailored for each phase of medical education.
The questionnaire was designed to evaluate two major themes: (1) factors influencing respondents’ decision to pursue a residency in otolaryngology (independent of COVID-19) and, (2) the impact of pandemic-imposed social restriction on student education and experiences in otolaryngology. This manuscript focuses on the results of the latter set of questions. Respondents were asked how COVID-19 impacted a variety of experiences including research, clinical exposure, faculty interaction, interest group activities, advising, USMLE preparation, clerkship rotation, and their anatomy course. The survey can be viewed in Supplementary Appendix 1.
Population and distribution
A Qualtrics-generated survey link was distributed to otolaryngology interest group (IG) leaders at 37 allopathic medical schools which were targeted to provide comprehensive geographic representation of medical training in the United States. IG leaders were instructed that all members were eligible to participate in the survey, regardless of their intention to pursue an otolaryngology residency. To improve response rate, participants who provided their email addresses were entered in a pool to receive $50 gift cards. The survey was anonymous and optional with all participants providing written informed consent prior to participation. It was initially distributed via email on November 17, 2020, to the primary IG leadership contacts. Follow-up emails were sent to non-respondents in two-week intervals for an additional 6 weeks. Additional IG contacts were also utilized when available.
Statistical analysis
Only complete responses were included in the analysis. Counts and frequencies were used to generate descriptive statistics. Qualtrics software was utilized for data compilation and crosstab analyses before being exported into Microsoft Excel for final statistical analysis. When describing our findings, the terms “majority” or “most” were utilized to indicate that at least 51% of respondents held the specified position.
Two free-response questions yielded a total of 321 narrative comments, which were analyzed and thematically mapped by two independent graders (AH and VY). Themes were then reviewed by authors to finalize thematic domains. Representative comments were reported.
Results
Response and demographics
Respondents’ general characteristics.
aSchool rank according to US News & World Report Research Rankings, 2021.
Pre-clinical directed questions
Anatomy course, ENT exposure, and interest group impact
Three questions regarding students’ anatomy course, ENT exposure, and IG activities were answered solely by pre-clinical students. A majority of pre-clinical students (67%; 74/111) indicated that COVID-19 had a “somewhat” or “very negative” impact on their anatomy course, with 16% (18/111) believing it had a neutral effect. Most pre-clinical students (62%; 69/111) also believed COVID-19 negatively impacted their ability to gain ENT exposure. Finally, 77% (86/111) of students stated that COVID-19 somewhat or very negatively impacted their ability to become involved with the ENT IG. Figure 1 further illustrates these pre-clinical results. Pre-clinical questions; A. COVID-19 impact on anatomy course/lab; B. COVID-19 impact on ENT exposure; C. COVID-19 impact on interest group activities.
Clinical directed questions
Clinical/surgical exposure, assessing interest
Most clinical students (60%; 40/67) indicated that COVID-19 had a “somewhat” or “very negative” impact on their ENT clinical experience. Similarly, 54% (36/67) believed COVID-19 negatively impacted their ability to gain ENT surgical exposure. Moreover, 46% (31/67) of clinical students felt COVID-19 had a negative impact on their ability to assess their interest in ENT. Figure 2 depicts these findings. Clinical questions; A. COVID-19 impact on clinical experience; B.COVID-19 impact on ENT surgical exposure; C.COVID-19 impact on assessing interest in ENT.
Faculty/department support and app competitiveness
At 54% (36 of 67), most clinical students felt that COVID-19 had a “somewhat” or “very negative” impact on their application competitiveness, with 30% (20/67) reporting a neutral impact. As for faculty/department support, 49% (33/67) of clinical students believed COVID-19 had a neutral effect, followed by 24% (16/67) who felt it had a negative effect.
Additional helpful resources
In a “select all that-apply” question shown in Figure 3, clinical students most often selected a formal faculty mentorship program as the most helpful resource to integrate into the otolaryngology curriculum moving forward during COVID-19 (21%; 43 of 209 selections), followed by curated video material (20%; 41 of 209 selections), and online question banks (13%; 28 of 209 selections). Helpful resources to integrate during COVID-19. *Select-all-that-apply question format; therefore, total will not equal number of clinical students in study.
Broadly directed questions
Advising
Overall, 51% (90/178) of students indicated that COVID-19 had a somewhat/very negative impact on ENT advising. However, when stratified by educational progression, 65% (72/111) of pre-clinical students felt that advising was negatively impacted by COVID-19 compared to 27% (18/67) of clinical students (Figure 4a). Broadly directed questions; A. COVID-19 impact on advising—comparison by educational progression; B. COVID-19 impact on USMLE STEP exam preparation; C. COVID-19 impact on research; D. COVID-19 impact on faculty interaction.
USMLE exam, research, and faculty interaction
Among all students, 33% (59/178) felt that COVID-19 had a neutral impact on USMLE exam preparation, followed by 26% (47/178) who were unsure, and 25% (44/178) who indicated it had a negative impact. A majority of students (57%; 101/178) stated that COVID-19 had a negative impact on their ability to obtain ENT research experience. Moreover, when stratified, 64% of pre-clinical students agreed that COVID-19 had a negative impact on research involvement. As for faculty interaction, 79% (141/178) of students overall stated that COVID-19 had a negative impact. Comparison for these factors by educational progression can be viewed in Figure 4b-d.
Qualitative results
Qualitative response domains.
Ten thematic domains were also developed from the free-response question, “What supplemental resources are available to you during COVID to enhance your learning and exposure to otolaryngology?” These domains include comments about virtual interest group meetings, online resources/textbooks, and Zoom meetings with mentors (Table 2b).
Discussion
This national survey study of 201 medical students sought to better understand how COVID-19 impacted their otolaryngology education and key experiences. We also elucidated medical students’ opinions on which novel resources were most valuable. Qualitative, free-response comments expounded on (and often supported) many of the multiple choice, quantitative findings. We found that most students believe COVID-19 negatively impacted a broad array of pre-clinical and clinical otolaryngology experiences.
Interest group involvement
The adverse impact felt by students begins in the pre-clinical stages. In our general otolaryngology survey, most pre-clinical students agreed that the IG provided their “most significant exposure” to otolaryngology. 7 In this COVID-19 oriented survey, pre-clinical students report that their involvement in IG activities was in fact negatively affected. This finding is substantial for several reasons. Even prior to the pandemic, students’ otolaryngology exposure was already limited due to a variety of factors, including the optional nature of many otolaryngology clerkship rotations, limited time allocated for head and neck pre-clinical education, and the relatively small size of the field.8,9 Otolaryngology interest groups provide a unique avenue for students (especially pre-clinical) to gain valuable, multifaceted exposure to the field via research, faculty/resident mentorship, and shadowing opportunities. 10 These early experiences are crucial, as exposure to the field plays a major role in students’ ultimate decision about whether to pursue the specialty.10-12
While most students indicated that their IG was hindered by COVID-19 (e.g., suspending all events), some stated that their IGs adjusted operations and served to keep them engaged remotely. When asked about valuable supplemental resources during COVID-19, these students replied, “interest group [remote] talks/meetings” and “the student IG is still active and has hosted events over Zoom.” Accordingly, we propose that virtual IG activities, such as speaker series, resident Q&As, and faculty lectures, can serve as invaluable connections to otolaryngology for medical students, particularly if in-person activities are curtailed once again.
Research opportunities
A majority of pre-clinical students (64%) reported an adverse impact on research opportunities, possibly due to limited in-person interactions with faculty and patients. This sentiment was captured by numerous comments; when asked how COVID-19 impacted their plans, students responded, “[I] planned on completing a prospective study over the summer, but it was canceled,” “summer research programs [were terminated]” and “it has made obtaining research more difficult.” This limitation may have a significant adverse effect on applicants since research experience provides students with valuable exposure to otolaryngology and opportunities for mentorship. 1 The impact of research hindrance is even more substantial today, as program directors place an increasing emphasis on research when evaluating applicants around the era of COVID-19. 3
Nevertheless, a subset of students continued, or even increased, their engagement in research projects during the pandemic. Students stated that supplemental resources provided to them included, “doing research remotely,” and engaging in “…chart reviews for retrospective studies.” Shahrvini et al similarly found that several students utilized the increased flexibility during COVID-19 to maximize research engagement. 13 By prioritizing research opportunities which can be completed remotely, such as literature reviews, meta-analyses, and retrospective chart reviews, otolaryngology departments can continue engaging students and developing their research portfolios, regardless of in-person guidelines.
Faculty interactions and mentorship
Seventy-nine percent of students in our study reported that COVID-19 negatively impacted their interactions with otolaryngology faculty members. This impact on mentorship was well-captured by several comments. One respondent stated, “it is much harder to interact with faculty and begin the process of mentorship,” and another said, “…[I] could not participate in the structured mentorship program over the summer as it was canceled.” The importance of developing and fostering faculty relationships in a field as intimate as otolaryngology cannot be overstated. Through faculty mentorships, students gain clinical/lifestyle insights into the field, productive research opportunities, and letters of recommendation for residency applications. In a survey of program directors, Kasle et al found that 82.8% expressed that a strong faculty letter of recommendation was the most important factor in their evaluation of applicants in the era of COVID-19, confirming the importance of valuable mentor relationships in the otolaryngology application process. 3
Although COVID-19 limited in-person faculty interactions for most, virtual relationship-building (through zoom, phone calls, etc.) with faculty and residents remained a useful means of mentorship for many. Several students commented that valuable resources, included “increased online mentorship opportunities” and “resident mentorship programs…developed during COVID for MS3s…planning to apply to otolaryngology.” Confirming these qualitative findings, “faculty mentorship program” was most often selected by students as the most valuable additional resource to be integrated during COVID-19. Otolaryngology departments and medical school administrations would be well served to facilitate these types of virtual mentorship opportunities for students should in-person interactions be once again limited. Furthermore, even after the era of COVID-19, the use of more virtual mentorship approaches can serve as an effective supplement to in-person mentorship, as this form of communication is both valuable to students and offers increased flexibility for faculty whose time is often greatly limited by clinical and administrative responsibilities.
Remote learning resources
Following faculty mentorship programs, students indicated that the next three most selected valuable resources to be integrated during COVID-19 included curated video materials, question banks, and case modules. Similar to our findings, Shahrvini et al found that 84% of pre-clinical students at UC San Diego agreed that online question banks were valuable learning resources during COVID-19, followed by 83% and 64% saying the same about recorded didactic lectures and Complete Anatomy, respectively. 13 These additional online learning resources likely increased in importance given COVID-19’s perceived negative impact on anatomy courses, according to most pre-clinical respondents (67%), as well as on clinical and surgical experiences per clinical students (60% and 54%, respectively).
Students in our survey specified several didactic resources they found helpful during COVID-19. Among these were included Headmirror.com (a website providing general otolaryngology information and statistics/advice regarding matching into the field), Pasha’s “Otolaryngology-Head and Neck Surgery,” OnlineMedEd (providing lecture-style videos), and Aquifer (a case-based online learning tool with supplemental questions).14-17 Hussain et al similarly recommended several resources that could be helpful for students during the pandemic as well: LearnENT to provide case-based content, Mayo Clinic’s Otolaryngology—Head and Neck Surgery Video Atlas to help bolster students’ surgical knowledge, and Great Lakes Otolaryngology Consortium for lecture-based content.4,18-20 To augment student learning, medical school administrators and clerkship directors could integrate some of these online resources into their curriculums, regardless of in-person teaching limitations.
USMLE STEP exam and mental health
We also sought to elucidate how students’ USMLE STEP exams were impacted by COVID-19. Several students commented that their STEP 1 or 2 exams were postponed or canceled. Regarding exam preparation, most students (59%) responded that they were either neutrally impacted or unsure, with one-fourth of all respondents feeling negatively impacted. On the other hand, Shahrvini et al found that most medical students (56.7%) felt that COVID-19 negatively impacted their STEP 1 exam preparation. 13
Personality and studying preferences may influence how each individual perceives COVID-19's impact on their exam preparation. For instance, a more introverted and self-motivated individual might prefer the extended seclusion and the study timing flexibility that the pandemic provides. Alternatively, an extravert who prefers learning among peers and the structure of a social daily routine or library study spaces might struggle with COVID-era USMLE preparation. Mental health and wellness play an important role in this regard as well. For many students, remaining home during COVID-19, away from colleagues and an adequate social support system, left them feeling isolated, lonely, and struggling to concentrate. 5
Educational restrictions implemented during the COVID-19 pandemic impacted numerous factors that are important pillars of a successful otolaryngology application, including clinical/surgical exposure, research involvement, conference networking, faculty mentorship, letters of recommendation, and USMLE preparation/performance. Moreover, our qualitative findings suggest that many COVID-era clinical students are being evaluated on a pass/fail basis due to shortened rotations, which may increase well-being but limit a student’s ability to stand out among their peers.21-24 These diverse effects of COVID-19 necessitate a holistic review approach when considering applicants over the next few cycles—for instance, residency directors may reconsider primarily using quantitative measures (e.g., Step 1 score and number of Honor grades) to screen applicants while placing increased emphasis on the interview and valuable extracurricular involvements in their review process. 25
Limitations & strengths
Our study has limitations. First, due to our survey distribution method (via IG leaders at various medical schools to IG members), we were unable to calculate an overall response rate. Instead, we can report an interest group response rate (of the 37 ENT IGs contacted, 30 (81%) forwarded the survey on to their members and provided at least 1 response). Second, while our study elicits respondents’ general attitudes about how COVID-19 impacted various aspects of their otolaryngology education (positively or negatively), we could not always capture the more nuanced details of why respondents answered as they did. To address this limitation, we designed two open-ended questions to capture qualitative, free-response answers from students, as well as a free-response text box for any additional comments.
Lastly, although we report several important ways COVID-19 has impacted students’ otolaryngology-related experiences, we cannot comment on outcomes—for example, number of applicants matching to otolaryngology, quality of applicants, or preparedness for residency. For instance, although we show that pre-clinical students believe their IG involvement has been adversely affected, we cannot conclude that the number of applicants will decline as a result. This limitation is related to our study population, which includes medical students who have yet to apply into otolaryngology, as well as those of varying interest in the field. However, given that the majority of our study participants indicated they were probably/definitely applying into otolaryngology (51% of students overall; 63% of clinical students), we believe our findings can be reasonably applied to the applicant pool. Nevertheless, the longterm effects of COVID-19 on otolaryngology applicants should be studied in the future once these classes have gone on to apply and match to residency programs.
There are several strengths of this study. The currently published literature presents valuable otolaryngology resources developed during the pandemic, individual author’s opinions about the impact of COVID-19 on medical education, or survey data on general COVID-era medical education.5,13,25-31 However, our study is the first to specifically evaluate how students felt their otolaryngology education/experiences were impacted by COVID-19. Moreover, our study is among the largest published (on the topic) in the literature to-date, with over 200 students. We were also able to capture a diverse population of students by educational progression and geographic representation. Finally, our study is unique in that it includes both quantitative and qualitative narrative results which often support and expound upon each other.
Conclusion
We surveyed over 200 medical students from across the country to better understand how COVID-19 impacted their otolaryngology educational experiences. Students expressed that the pandemic had a broad negative influence including on their anatomy lab, otolaryngology IG involvement, surgical/clinical exposure, faculty interactions, advising, and research. Respondents valued integration of virtual faculty mentorship programs, curated video materials, and question banks into the curriculum, representing opportunities for continued inclusion of these resources and support moving forward.
These findings may assist residency directors as they consider and train candidates in the upcoming residency classes. Our study can also provide medical school administrators and otolaryngology clerkship directors with valuable student feedback regarding how the pandemic has impacted them and which helpful adaptive resources/changes could be carried forward beyond the era of COVID-19.
Supplemental material
Supplemental material - Medical students’ perspectives on how COVID-19 has impacted their otolaryngology educational experience: A survey study
Supplemental material for Medical students’ perspectives on how COVID-19 has impacted their otolaryngology educational experience: A survey study by Andy M. Habib, MPH, Victoria Yu, MD, Michelle Yu, MD, Jessica R Levi, MD, David A. Gudis, MD, and Jonathan Overdevest MD, PhD in Ear, Nose & Throat Journal
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Ethical approval
Ethical approval for this study was obtained from the Columbia University Institutional Review Board (IRB AAAT0653).
Statement of human and animal rights
All procedures in this study were conducted in accordance with the Columbia University Institutional Review Board (IRB AAAT0653).
Statement of informed consent
Written informed consent was obtained from the participant(s) for their anonymized information to be published in this article.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
