Sage Journals: Discover world-class research

Abstract

Objective

To provide an updated evaluation of otolaryngology residency program websites during a socially distanced application cycle. Criteria were selected to evaluate for level of comprehensiveness, new content accommodating a virtual interview cycle, and stated commitment to diversity and inclusion.

Study design

Review of otolaryngology residency program websites.

Setting

Online websites between December 2020 and January 2021.

Methods

Otolaryngology residency program websites were evaluated for selected criteria. Programs were categorized by geographic location, size, and Doximity rankings. Differences between groups were analyzed using Student’s t-test and analysis of variance with a significance level of P < .05.

Results

118 otolaryngology residency program websites contained a mean (SD) of 16.3 (4.7) of 31 criteria (52.6%). Fewer than one third included virtual/video tours of facilities or the surrounding area. Only 33% had a stated commitment to diversity and inclusion. There was no difference in number of criteria included when categorized by geographic location (P = .22). Larger programs (17.9 vs 14.5, P < .001) and programs ranked in the top 50 by Doximity (17.7 vs 15.2, P = .003) included a significantly greater number of criteria.

Conclusion

While the comprehensiveness of otolaryngology residency program websites has improved, considerable room for improvement remains. Programs should strive to maintain updated websites that highlight training opportunities, program culture, and commitment to diversity and inclusion. Programs also should consider prioritizing the development of new online resources that may be especially useful to applicants during a virtual interview cycle.

Keywords

program websites otolaryngology residency residency recruitment social media diversity and inclusion

Introduction

During 2020 and 2021, the COVID-19 pandemic changed our world dramatically. The otolaryngology community has responded with admirable adaptability and creativity. As social distancing measures aimed at curtailing the spread of the virus have limited traditional office visits, otolaryngologists have turned to telemedicine.¹ Ongoing research is examining how such services can be delivered in an equitable manner.^2,3 The socially distanced world also has altered medical education. Some institutions have incorporated medical students into telemedicine clinics,⁴ while others have hosted virtual electives.^5–7 A wealth of otolaryngology-specific online learning materials that can be accessed at no cost by medical students also has been developed.^8,9

The otolaryngology residency recruitment process has undergone significant restructuring. Taking into consideration the risks associated with travel during the COVID-19 pandemic, the Coalition for Physician Accountability recommended that away rotations during the 2020–2021 academic year be limited to learners without an otolaryngology residency program at their own institution.¹⁰ Furthermore, they recommended that all programs commit to conducting virtual interviews during the current application cycle; this statement was quickly supported by the Society of University Otolaryngologists.^10,11 Otolaryngologists involved in the residency recruitment process have considered the implications of such a virtual application cycle, as well as proposed strategies for applicant recruitment and assessment.¹²

In this context, it is logical that without the usual opportunities to engage with programs face-to-face, students may be more dependent on online materials. Although many previously published studies have examined the content of websites maintained by residency programs in other surgical specialties,^13–16 to the authors’ knowledge, only one previous paper, Svider et al. published in 2014, has explored otolaryngology residency program websites.¹⁷ Thus, the purpose of the current study was to provide an updated analysis of otolaryngology residency program websites given their significance during a virtual application cycle, anticipating that the comprehensiveness of content provided on these websites has improved since 2014.

In addition to the criteria used previously to assess residency websites, this study also sought to evaluate an expanded number of components, some of which may be especially useful during a virtual application cycle, including video tours and photos of resident activities. Furthermore, as many residency programs have increased their social media presence in an effort to connect with applicants, we evaluated websites for the presence of links to social media accounts.^18,19 Many of these expanded criteria may allow applicants to appreciate a program’s culture, an intangible quality which may be more difficult to convey on traditional text-dominated websites. Finally, as the importance of diversity within the field of otolaryngology has become increasingly recognized, we evaluated websites for stated commitment to diversity and inclusion.²⁰ Previous studies have suggested that program websites which mention diversity and inclusion initiatives may help attract a diverse applicant pool.^21,22 Another survey study demonstrated that women and nonwhite applicants prioritize program diversity more than their white male counterparts do.²³

Methods

Program website identification

Since all information obtained and used in this study is publicly available, this study was not human subjects research and did not require approval by the Drexel University College of Medicine Institutional Review Board. The American Medical Association’s Fellowship and Residency Electronic Interactive Database (FREIDA) was accessed for a listing of otolaryngology residency programs on December 23, 2020. The web address provided for each program website was accessed and categorized as functional, a broken link, a general institutional or graduate medical education website not specific to otolaryngology, or absent if no web address was provided. Programs websites associated with non-functional links were identified via google search.

Website evaluation

The otolaryngology residency program websites were evaluated between December 23, 2020 and January 25, 2021, during the interview season for the 2020–2021 academic year and thus corresponding to a time during which websites might be accessed heavily by applicants. All included websites were searched by the authors (J.Y.T. and A.H.) for the presence or absence of criteria listed in Table 1. These criteria were selected based on Svider et al.,¹⁷ in order to allow for relative comparisons between the two time points, and expanded criteria felt by the authors to be especially beneficial during the virtual application cycle of the 2020–2021 academic year. Information was considered to be present on a program website only if available directly on the otolaryngology website itself; links to outside pages such as institutional graduate medical education websites were not evaluated.

Table 1.

Presence of criteria used to evaluate otolaryngology residency program websites.

Information Evaluated	% of Websites (N = 118)	Previously Reported % (N = 99)
Application process
Link to ERAS (electronic residency application service)	66	55
Contact email	92	77
Selection criteria	38	30
Incentives
Salary	32	23
Benefits	43	22
Parking	18	9
Meal allowance	23	7
Information about surrounding area	54	34
Instruction
Specific extra courses (e.g., temporal bone, anatomy)	78	61
Description of didactics	75	80
Research
Research requirements	42	69
Active/past research projects in department	78	37
Clinical training
Comprehensive faculty listings (name, training, and subspecialty)	93	88
Current resident listings	97	74
Facility descriptions	64	70
Intern year schedule (PGY1)	86	70
Rotation schedule (PGY2-5)	77	64
Surgical case/responsibility progression	45	47
Call schedule	41	40
Career placement of past residents	64	28
Surgical statistics (general overall numbers or specific cases)	34	18
Other
Message from the chairperson	57	23
Message from the program director	50	33
Expanded criteria
2020–2021 interview dates	30	−
Video highlighting resident life	25	−
Photos of resident activities	53	−
Link(s) to social media account(s)	41	−
Testimonials from current and/or past residents	51	−
Virtual/video facility tour	19	−
Virtual/video tour of surrounding area	28	−
Stated commitment to diversity and inclusion	33	−

Program categorization

In order to examine potential variations in program websites, programs were categorized by geographic locations based on the Midwest, Northeast, South, and West regions designated by the U.S. Census Bureau. Puerto Rico was placed in the South as it was not included in the U.S. Census Bureau mapping. In addition, programs were categorized as “large” or “small” in size by comparing the number of currently enrolled residents to the national median program size of 15; programs of 15 residents or more were categorized as “large.” Those with fewer than 15 residents were categorized as “small.” Finally, Doximity rankings of otolaryngology residency programs by reputation were examined. Programs were categorized based on whether or not the program or a residency clinical training site was ranked among the top 50 by either source.

Statistical Analysis

Data analysis was largely descriptive, using counts and percentages for all criteria evaluated. Based upon the methods of Svider et al,¹⁷ student’s t-test and analysis of variance (ANOVA) were used to compare the mean number of criteria included by programs for different categories. A value of P < .05 was considered statistically significant. SPSS version 27 software (IBM Corporation) was used for statistical analysis.

Results

The FREIDA search yielded 124 programs. 6 military-based programs were excluded, and the remaining 118 programs were included in this study. Of these, the FREIDA database provided functional web addresses for 72 programs, 2 broken links, and 29 general websites not specific to otolaryngology. There was no web address provided for 7 programs.

The 118 otolaryngology program websites included in this study contained a mean (SD) of 16.3 (4.7) of the 31 factors evaluated (52.6%). Only 7 program websites contained at least 3 quarters of the criteria evaluated, while 70 program websites contained at least half. Most program websites included comprehensive faculty listings, current resident listings, and a contact email address for interested applicants. Compared to the results published in 2014, a greater percentage of program websites mentioned most of the criteria evaluated previously. The percentage of program websites including a description of didactics, research requirements, facility descriptions, and surgical case/responsibility decreased over this time period. With regard to our expanded criteria, fewer than one third of program websites contained interview dates, videos highlighting resident life, virtual/video facility tours, virtual/video tours of the surrounding area, and statements of commitment to diversity and inclusion. These results are summarized in Table 1, alongside the results reported by Svider et al.¹⁷

When categorizing programs by geographic location, there was no difference in the number of criteria contained on program websites (ANOVA, P = .22). When categorizing programs by size, websites maintained by “large” programs included more criteria (17.9 [3.9]) than those maintained by “small” programs (14.5 [4.9]; t-test, P < .001). When categorizing programs based on Doximity rankings, programs ranked among the top 50 otolaryngology residency programs by reputation included more criteria (17.7 [4.0]) than those who were not (15.2 [4.9]; t-test, P = .003). These results are summarized in Table 2.

Table 2.

Comprehensiveness of otolaryngology residency program websites by region, size, and ranking.

Variable	Number of 31 criteria included mean (SD)
Overall	16.3 (4.7)
Geographic location
Midwest	14.9 (5.0)
Northeast	16.9 (3.8)
South	16.4 (4.8)
West	17.5 (5.1)
Size*
Large	17.9 (3.9)
Small	14.5 (4.9)
Doximity ranking*
Top 50	17.7 (4.0)
Not top 50	15.2 (4.9)

*Statistically significant (P < .05) difference between groups.

Discussion

This study provides an updated evaluation of otolaryngology residency program websites during an application cycle that has eliminated traditional face-to-face interviews and placed an unprecedented level of importance on the online information delivered through program websites. A recent commentary published by Chou et al recommended that residency programs consider using the cost savings incurred by the virtual interview cycle to create and add new content to program websites and social media. In an effort to help applicants better appreciate a program’s culture and “see” the facilities and surrounding areas, they suggested the addition of videos highlighting resident life, photos of residency events, testimonials from current residents and graduates, as well as video tours of facilities and the surrounding area.¹²

Given their relevance to the current application cycle, these were included in the criteria we used to evaluate each program website. Some programs appeared to have uploaded new videos specifically for this virtual application cycle, including videos highlighting resident life, testimonials from current residents and program alumni, and virtual tours. This finding is encouraging in that it provides evidence that some programs have acknowledged the previously published recommendations. However, fewer than one third of program websites included virtual/video facility or surrounding area tours, suggesting that there is substantial room for improvement. Such tours represent a standard component of traditional interviews that are not available in virtual interviews at many programs. Video tours recorded by the residents and faculty of a program may also provide a sense of the program culture through the personalities of those giving the tour. Otolaryngology programs may consider collaborating with other residency programs at their institution to pool the resources and skills needed to record a tour of shared facilities or the surrounding area that could then be featured on the websites of all residency programs affiliated with that training site. In addition to these newly created videos, a few program websites featured recordings of virtual electives or “open house” events that they had hosted. Not only did these recordings provide an abundance of program-specific information, but many also revealed a sense of the program culture by displaying candid interactions between current residents and faculty. As interviews continue to be conducted virtually, more programs should consider prioritizing efforts to create and include similar video content on their websites.

Many programs have leveraged their social media presence to connect with applicants during the socially distanced cycle, with one study reporting that 62.7% of programs participate on Instagram.¹⁸ These social media accounts, often run by current residents, help to provide a sense of program culture that is difficult to derive exclusively from brief conversations during a virtual interview. In addition, they may provide insight into the day-to-day life of current residents in the program. However, this study found that only 41% of program websites included links to social media accounts. Programs should be advised to include their social media accounts on their websites so that they are identified more easily by applicants.

The current study also sought to evaluate the commitment to diversity and inclusion displayed on program websites. The importance of creating an otolaryngology workforce that accurately mirrors the society it serves has become recognized widely. Yet, women and minority racial and ethnic groups are underrepresented in the pool of applicants to otolaryngology residency programs.^20,24 Among medical specialties, otolaryngology consistently ranks low in gender and ethnic diversity. A recent study found that general surgery programs have a higher mean proportion of female residents than do otolaryngology programs.²² Our study found that only 33% of program websites contained a stated commitment to diversity and inclusion. A previous study suggested that general surgery residency websites that display a program’s commitment to diversity and inclusion may be important in attracting a diverse applicant pool.²¹ Another study found that otolaryngology programs with websites that either mentioned the importance of diversity and inclusion, listed diversity and inclusion initiatives, or included specific and measurable diversity goals had a significantly higher proportion of female residents.²² Although further research is needed to clarify strategies through which a diverse applicant pool can be recruited, programs should be encouraged to work with their departments to develop sustainable diversity and inclusion initiatives that are visible on their websites.²¹

The same criteria used by Svider et al in 2014 were evaluated so that comparison of previous results could be made to those of the current study.¹⁷ Increases in the percentage of program websites containing the majority of the criteria evaluated by Svider et al, provided encouraging evidence of an increasing level of comprehensiveness of program websites. However, there remains substantial room for improvement. The percentage of program websites including a description of didactics, research requirements, facility descriptions, and surgical case/responsibility decreased compared to previous results. Programs should be reminded not to neglect these components of a comprehensive website.

Consistent with previously published results, the current study found no differences in the number of criteria mentioned on program websites when categorized by geographic location. Likewise, “large” programs included a greater number of criteria on their websites than did “small” programs (17.9 vs 14.5; P < .001). Our study also found that websites of programs ranked by Doximity contained more criteria than programs that were not. However, it is unclear whether these differences of 2-3 criteria substantially improve potential applicants’ understanding or perspective of a program. These differences may suggest that larger or more highly ranked programs have increased resources available for creating and updating the content displayed on their websites.

Limitations of this study include the subjective nature by which the criteria evaluated were selected. However, the authors felt it appropriate to include the same criteria evaluated previously by Svider et al.¹⁷ These were felt to encompass a variety of domains relevant to clinical training and quality of life, which represented likely components of an applicant’s meaningful consideration of otolaryngology residency programs. The authors also aimed to include a number of expanded criteria that were felt to be especially relevant to applicants during a virtual interview cycle. Despite these efforts to include a broad range of pertinent criteria, other aspects of otolaryngology residency programs important to applicants may not be included in this study. A future survey of otolaryngology applicants may help confirm the suitability of the presently included criteria, as well as offer suggestions for additional factors that should be evaluated. Given our focus on otolaryngology residency program websites, the present study did not evaluate general institutional or graduate medical education websites; these websites may contain additional information that would meet the evaluated criteria. Similarly, the present study did not evaluate content solely released on social media accounts or other platforms.

Conclusion

Otolaryngology residency program websites appear to have improved in level of comprehensiveness since 2014, with an increased percentage of program websites including many of the criteria evaluated. Many program websites also feature new videos aimed at accommodating the current virtual interview cycle. Only a minority of program websites include a statement of commitment to diversity and inclusion; this should be improved as part of a concerted effort to recruit a diverse applicant pool and ultimately to promote a diverse otolaryngology workforce. There remains substantial room for improvement. Programs should be encouraged to maintain up-to-date websites with a comprehensive overview of the opportunities and training culture offered, allowing applicants to make informed decisions throughout the resident recruitment process.

Footnotes

Author Contributions

Concept and design: Tong, Sataloff

Acquisition, analysis, and interpretation of data: Tong, HartkornDrafting of the manuscript: Tong

Critical revision of the manuscript for important intellectual content: Tong, Hartkorn, Sataloff

Accountability for all aspects of the work: Tong, Hartkorn, Sataloff

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Alyssa R Hartkorn

Robert Sataloff

References

Lechien

Radulesco

Distinguin

, et al. Patient and otolaryngologist perceptions of telemedicine during COVID-19 pandemic. Eur Arch Oto Rhino Laryngol. 2021;278:4101-4105. doi:10.1007/s00405-021-06624-9.

Kidane

Kim

Sharon

. Moving toward equitable telemedicine in otolaryngology-head and neck surgery. JAMA Otolaryngol Head Neck Surg. 2021;147:219. doi:10.1001/jamaoto.2020.4329.

Kolb

Born

Banker

Barth

Aaronson

. Improving attendance and patient experiences during the expansion of a telehealth-based pediatric otolaryngology practice. Otolaryngol Head Neck Surg. 2021;164(5):952-958.

Pellegrini

Danis

3rd Levi

. Medical student participation in otolaryngology telemedicine clinic during COVID-19: A hidden opportunity. Otolaryngol Head Neck Surg. 2021;164(6):1131-1133.

Steehler

Pettitt-Schieber

Studer

Mahendran

Pettitt

Henriquez

. Implementation and evaluation of a virtual elective in otolaryngology in the time of COVID-19. Otolaryngol Head Neck Surg. 2021;164(3):556-561.

Farlow

Marchiano

Fischer

Moyer

Thorne

Bohm

. Addressing the impact of COVID-19 on the residency application process through a virtual subinternship. Otolaryngol Head Neck Surg. 2020;163(5):926-928.

Chao

Frost

Brody

, et al. Creation of an interactive virtual surgical rotation for undergraduate medical education during the COVID-19 pandemic. J Surg Educ. 2021;78(1):346-350.

Grose

Best

Liao

, et al. LearnENT: The development of a free open access medical education app in otolaryngology-head and neck surgery. J Surg Educ. 2021;78:1046-1051. doi:10.1016/j.jsurg.2020.10.014.

American Academy of Otolaryngology–Head and Neck Surgery . AcademyU. OTO Logic; 2021. http://academyu.entnet.org/. Accessed February 11, 2021.

10.

Accreditation Council for Graduate Medical Education . Coalition for physician accountability publishes recommendations for 2020-2021. ACGME; 2021. https://www.acgme.org/Newsroom/Newsroom-Details/articleId/10252. Accessed February 11, 2021.

11.

Society of University Otolaryngologists . COVID-19 resources. Society of University Otolaryngologists; 2021. https://suo-aado.org/page/covid-19_resources. Accessed February 11, 2021.

12.

Chou

Pletcher

Bruss

, et al. Otolaryngology residency interviews in a socially distanced world: Strategies to recruit and assess applicants. Otolaryngol Head Neck Surg. 2021;164(5):903-908.

13.

Chen

Hoang

Garner

. Do websites provide what applicants need? Plastic surgery residency program websites versus applicant self-reported needs. Plast Reconstr Surg Glob Open. 2018;6(10):e1900.

14.

Hashmi

Policherla

Campbell

Khan

Schumaier

Al-Mufarrej

. How informative are the plastic surgery residency websites to prospective applicants? J Surg Educ. 2017;74(1):74-78.

15.

Lee

Eisig

Koch

. Oral and maxillofacial surgery program websites under-report content related to resident recruitment and education. J Oral Maxillofac Surg. 2018;76(9):1841-e7.

16.

Stoeger

Freeman

Bitter

Helmer

Reyes

Vincent

. Evaluation of general surgery residency program websites. Am J Surg. 2019;217(4):794-799.

17.

Svider

Gupta

Johnson

, et al. Evaluation of otolaryngology residency program websites. JAMA Otolaryngol Head Neck Surg. 2014;140(10):956-960.

18.

DeAtkine

Grayson

Singh

Nocera

Rais-Bahrami

Greene

. #ENT: Otolaryngology residency programs create social media platforms to connect with applicants during COVID-19 pandemic. Ear Nose Throat J. 2020:145561320983205. Published online.

19.

Mecham

Menapace

Bowe

Carlson

. Recruitment and networking with social media for the otolaryngology match in the COVID-19 pandemic. Otolaryngol Head Neck Surg. 2020:194599820957952. Published online.

20.

Lopez

Farzal

Ebert

Jr Shah

Buckmire

Zanation

. Recent trends in female and racial/ethnic minority groups in US otolaryngology residency programs. Laryngoscope. 2021;131(2):277-281.

21.

Driesen

AMDS

Romero Arenas

Arora

, et al.

Do general surgery residency program websites feature diversity?

J Surg Educ. 2020;77(6):e110-e115.

22.

Smith

Chiu

Sykes

Eck

Hierl

Villwock

. Diversity in academic otolaryngology: An update and recommendations for moving from words to action. Ear Nose Throat J. 2021;100(10):702-709.

23.

Fairmont

Farrell

Johnson

Cabrera-Muffly

. Influence of gender and racial diversity on the otolaryngology residency match. Otolaryngol Head Neck Surg. 2020;162(3):290-295.

24.

Truesdale

Baugh

Brenner

, et al. Prioritizing diversity in otolaryngology-head and neck surgery: Starting a conversation. Otolaryngol Head Neck Surg. 2021;164(2):229-233.

Otolaryngology residency program websites in a socially distanced application cycle

Abstract

Objective

Study design

Setting

Methods

Results

Conclusion

Keywords

Introduction

Methods

Program website identification

Website evaluation

Program categorization

Statistical Analysis

Results

Discussion

Conclusion

Footnotes

Author Contributions

Declaration of conflicting interests

Funding

ORCID iDs

References