Sage Journals: Discover world-class research

Abstract

Background:

Otorhinolaryngology diseases are well suited for artificial intelligence (AI)-based interpretation. The use of AI, particularly AI based on deep learning (DL), in the treatment of human diseases is becoming more and more popular. However, there are few bibliometric analyses that have systematically studied this field.

Objective:

The objective of this study was to visualize the research hot spots and trends of AI and DL in ENT diseases through bibliometric analysis to help researchers understand the future development of basic and clinical research.

Methods:

In all, 232 articles and reviews were retrieved from The Web of Science Core Collection. Using CiteSpace and VOSviewer software, countries, institutions, authors, references, and keywords in the field were visualized and examined.

Results:

The majority of these papers came from 44 nations and 498 institutions, with China and the United States leading the way. Common diseases used by AI in ENT include otosclerosis, otitis media, nasal polyps, sinusitis, and so on. In the early years, research focused on the analysis of hearing and articulation disorders, and in recent years mainly on the diagnosis, localization, and grading of diseases.

Conclusions:

The analysis shows the periodical hot spots and development direction of AI and DL application in ENT diseases from the time dimension. The diagnosis and prognosis of otolaryngology diseases and the analysis of otolaryngology endoscopic images have been the focus of current research and the development trend of future.

Keywords

artificial intelligence otorhinolaryngology CiteSpace VOSviewer

Introduction

Artificial intelligence (AI), which was proposed in the 1950s, is a branch of computer science in which machine-based methods are used to make predictions. AI is currently a hot and controversial topic, and all industries can use AI, especially the deep learning (DL) subtype. Machine learning (ML) has become central to modern data science because its algorithms can automatically analyze interesting and useful patterns in large datasets.¹ It began to have an impact on medicine on 3 different levels: for patients, potentially through the ability to process their data for health; for clinicians, primarily through quick and accurate image interpretation and for health systems, potentially through improved workflow and a decrease in medical errors.²

DL techniques are beginning to dominate due to their simplicity in not requiring manual manipulation, their efficiency in dedicated massively parallel hardware, and their sophistication in performing a large number of tasks.³ AI is more likely to be applied in diagnostic disciplines based on image analysis, such as pathology, ultrasound, and radiology, than in other applications in disease treatment.^4,5

In otolaryngology, AI and DL are widely used. In otology, studies related to AI combined with tympanic images for diagnosis and classification of otitis media and cholesteatoma^6,7 and assessment of hearing loss⁸ are relatively well established. In addition, research on AI to quantify auditory hair cells,⁹ screen for vestibular dysfunction,¹⁰ combine with audiometric data to diagnose vestibular nerve sheath tumors,¹¹ combine with electrophysiological measurements to predict cochlear nerve (CN) function status in individual cochlear implant (CI) users,¹² and combine with microRNA (miRNA) analysis to diagnose Ménière’s disease¹³ is in continuous development. In rhinology, studies such as AI interpretation of sinus computed tomography and diagnosis of sinusitis^14,15 are currently hot topics. In laryngology, AI can be used to classify and predict surgical complications after laryngectomy,¹⁶ to detect real-time laryngeal cancer in combination with laryngoscopy,¹⁷ and to assess the vocal cord condition.¹⁸ With advances in endoscopic techniques, the trend of combining AI and DL with the diagnosis and treatment of ENT diseases is becoming more evident.

First proposed by American bibliographers in 1969, bibliometrics is a discipline that applies mathematical and statistical methods to the study of books and other communication medias.¹⁹ Bibliometrics also allows qualitative and quantitative evaluation of trends in bibliographic research, which not only helps scholars to quickly grasp the research hot spots and trends in specific research fields, but also assesses the distribution of countries/regions, authors, and journals in the research field, laying the foundation for future research directions and developments as well as promoting further regional cooperation.²⁰

Aim of This Study

Although the number of studies discussing the use of DL and AI in treating otolaryngology diseases has grown dramatically year by year, there are few bibliometric analyses that have systematically studied this field. This study aims to explore the hot spots and trends in the application of AI and DL in ENT diseases in the past 20 years and to map them using CiteSpace and VOSviewer software (Leiden University) to provide new ideas for basic and clinical research.

Methods

Data Collection and Search Strategy

Data were extracted from Web of Science Core Collection and downloaded within 1 day on November 4, 2022. The search formula was set to Topic (TS) = [(artificial intelligence) OR (machine learning) OR (deep learning)] AND TS = [(otology) OR (rhinology) OR (laryngology) OR (otorhinolaryngology) OR (otolaryngology) OR (ear) OR (nose) OR (larynx)] and the search dates were from January 1, 2003 to November 3, 2022.

Screening Strategy

A total of 1671 articles were yielded and 1439 unrelated articles were retrieved, including conference abstracts, editorial materials, corrections, letters, retractions, proceeding papers, and studies unrelated to the topic (they did not focus on otorhinolaryngology or AI). The screening process was done independently by 2 coauthors (QW and XZ), and controversial articles were discussed in team meetings before a decision was made.

Finally, a total of 232 articles were exported as all records and references, saved as plain text files, and stored in download_txt format. They were the ones included in this bibliometric analysis (Figure 1).

Figure 1.

Flowchart of literature selection.

Data Analysis

Overview

The main aim of knowledge domain visualization is to detect and monitor the development of knowledge, which allows presenting the organization, rules and distribution of scientific knowledge through visualization.²¹ Knowledge mapping is a new field driven by information technology that can visualize research hot spots and evolution in each domain in the knowledge system and predict the development trend of each domain. It is an effective method and tool for analyzing large-scale data.²²

The Software We Used

We used Microsoft Excel 2019, VOSviewer, and CiteSpace for our visual analysis. Microsoft Office Excel 2019 was used to analyze the trends in the number of published articles for the year while CiteSpace and VOSviewer analyzing the country/region and institution distribution, author contributions, core journals, keywords, and timelines.

VOSviewer is a program for building and viewing bibliometric maps. It can be used to build author or journal maps based on collaborative data, or keyword maps based on co-occurrence data. The software comes with a viewer that enables thorough and in-depth analysis of bibliometric maps. The primary objective of creating VOSviewer was to study bibliometric networks and create visual network maps to gain a thorough grasp of the dynamics and structure of scientific research.²³ CiteSpace is a citation visualization and analysis software focused on analyzing the potential knowledge contained in the scientific literature and developed progressively in the context of scientometrics and data visualization.²²

Results

The Trend of Publication Outputs

The number of articles published in each period reflects the trend of research in the field. As shown in Figure 2, the number of articles on AI and DL applied to ENT diseases has increased gradually. From 2003 to 2014, the publication output in this period was extremely low and research remained stagnant. From 2015 to 2018, the number of publications steadily increased, indicating that the field of AI and DL applied to otolaryngology diseases is starting to gain attention. From 2019 to 2022, the number of published articles exploded, reaching 72 publications in 2022.

Figure 2.

Trends in the number of publications on AI and DL applied to otolaryngology diseases over the past 20 years.

Distribution of Countries/Regions and Institutions

A total of 232 articles from 44 different countries and 498 institutions were published. As seen in Table 1, the largest number of publications came from the United States (63, 27.16%) and China (37, 15.95%), while the United Kingdom (0.80), Australia (0.52), and the United States (0.32) showing a high degree of centrality. The research institutions with most published papers were Fudan University (9, 3.88%) and the University of Toronto (8, 3.45%). According to Figures 3 and 4, only a few countries such as the United Kingdom, China, Australia, and the United States, as well as a few institutions, such as the University of Sydney, Harvard Medical School, and the University of Toronto, have shown active collaboration. However, most countries and research institutions are scattered and lack stable and intensive relationships of collaborative and communicative.

Table 1.

Top 10 Countries and Institutions Related to AI and DL Applied to ENT Diseases.

Rank	Country	Year	Centrality	Count (%)	Institution	Year	Centrality	Count (%)
1	USA	2003	0.32	63 (27.16)	Fudan Univ.	2019	0.00	9 (3.88)
2	People’s Republic of China	2014	0.19	37 (15.95)	Univ. Toronto	2014	0.01	8 (3.45)
3	Germany	2006	0.18	23 (9.91)	Univ. Sydney	2020	0.00	6 (2.59)
4	South Korea	2016	0.00	21 (9.05)	Harvard Med. Sch.	2020	0.01	5 (2.16)
5	Canada	2014	0.00	17 (7.33)	Stanford Univ.	2019	0.00	5 (2.16)
6	England	2015	0.80	14 (6.03)	Ludwig Maximilians Univ Munchen	2017	0.00	5 (2.16)
7	Australia	2018	0.52	12 (5.17)	Westmead Hosp.	2015	0.00	5 (2.16)
8	Italy	2018	0.08	11 (4.74)	Univ. Munich	2006	0.00	4 (1.72)
9	Japan	2019	0.11	9 (3.88)	Fu Jen Catholic Univ.	2022	0.00	3 (1.29)
10	Taiwan	2018	0.08	8 (3.45)	Emory Univ.	2022	0.00	3 (1.29)

Authors and co-cited authors.

Abbreviations: AI, artificial intelligence; DL, deep learning.

Figure 3.

Distribution of publications from different countries/regions. Each circle represents a country, and the size of the circle indicates the publication output of that country. The line between the circles indicates the cooperation between countries, the wider the line, the closer the cooperation.

Figure 4.

Distribution of publications of different institutions. Each circle represents a country, and the size of the circle indicates the publication output of that country. The line between the circles indicates the cooperation between countries, the wider the line, the closer the cooperation.

A total of 1288 authors were involved in writing literatures on AI and DL applied to ENT diseases, and top 10 authors are shown in Table 2. As shown in Figure 2, Habib, Singh, and Wong published the highest number of papers (6, 2.59%). Figure 5 shows that there are certain collaborative networks among different authors such as Singh, Habib, Perry, Kumar and Sacks, Zhang, Hu, Zhong and Wang, Cheah, Holdsworth, Moore and Ell. Co-cited authors are 2 or more authors who are simultaneously cited by one or more other papers, and these 2 or more authors constitute a co-cited relationship. Of the 6688 co-cited authors, 7 were cited more than 20 times (Figure 2). Cha (25), Pichichero (25), and Crowson (25) were the most frequently cited authors, followed by Myburgh (24). Among the top 10 authors, we can see a high centrality of Szegedy (0.15) and Breiman (0.14), which indicates that these 2 authors have a strong influence on each other’s works as well as on the work of other groups.

Table 2.

Top 10 Authors and Co-Cited Authors Related to AI and DL Applied to ENT Diseases.

Author	Count (%)	Co-cited author	Citation	Centrality
Habib	6 (2.59)	Cha	25	0.02
Singh	6 (2.59)	Pichichero	25	0.06
Wong	6 (2.59)	Crowson	25	0.03
Crowson	5 (2.16)	Myburgh	24	0.08
Sacks	3 (1.29)	Esteva	24	0.01
Kumar	3 (1.29)	Lecun	23	0.00
Wang	3 (1.29)	Kingma	21	0.00
Hu	3 (1.29)	Breiman	20	0.14
Swanepoel	3 (1.29)	Simonyan	20	0.01
Ahmadi	2 (0.86)	Szegedy	20	0.15

Journals and co-cited academic journals.

Abbreviations: AI, artificial intelligence; DL, deep learning.

Figure 5.

Author’s CiteSpace visualization map related to the application of AI and DL applied to the field of otolaryngology diseases. Each circle represents an author, the lines between circles indicate the connections between authors, and the different colored networks of connections indicate clusters of collaboration between different authors.

We performed a visual analysis of the published journals using VOSviewer software. We found 232 articles related to AI and DL applied to ENT diseases published in 84 academic journals. The journals Laryngoscope (8, 3.44%) and Otolaryngology Head and Neck Surgery (8, 3.44%) had the highest number of outputs, followed by PLOS One (7, 3.02%). Among the top 10 academic journals, the highest impact factor (IF) was Computers in biology and medicine (6.698). The impact of journals depends on the number of times they are co-cited, which reflects whether the journal has a significant impact in a given area of research. Among the top 10 co-cited academic journals, 9 journals were cited more than 100 times. As shown in Table 3, the most cited journal is Laryngoscope (226), followed by Journal of the Acoustical Society of America (223).

Table 3.

Top 10 Journals and Co-Cited Journals Related to AI and DL Applied to the Field of Otorhinolaryngology Diseases.

Journal	Count (%)	IF (2021)	JCR	Co-cited journal	Citation	IF (2021)	JCR
Laryngoscope	8 (3.44)	2.970	Q3	Laryngoscope	226	2.970	Q3
Otolaryngology Head and Neck Surgery	8 (3.44)	5.591	Q3	Journal of the Acoustical Society of America	223	2.482	Q3
PLOS One	7 (3.02)	3.752	Q3	Otolaryngology Head and Neck Surgery	148	5.591	Q3
Computers in Biology and Medicine	6 (2.59)	6.698	Q3	Otology & Neurotology	144	2.619	Q3
Scientific Reports	6 (2.59)	4.996	Q3	Medical Physics	141	4.506	Q3
Applied Sciences-Basel	5 (2.16)	2.838	Q3	PLOS One	130	3.752	Q3
Ear and Hearing	5 (2.16)	3.562	Q3	Scientific Reports	118	4.996	Q3
IEEE Access	5 (2.16)	3.476	Q3	Ear and Hearing	113	3.562	Q3
Journal of Laryngology and Otology	5 (2.16)	2.187	Q4	International Journal of Pediatric Otorhinolaryngology	107	1.626	Q4
Annals of Otology Rhinology and Laryngology	4 (1.72)	1.973	Q4	European Archives of Oto-Rhino-Laryngology	95	3.236	Q3

Abbreviations: AI, artificial intelligence; DL, deep learning; IF, impact factor; JCR, journal citation reports.

The dual-map overlay of journals shows the distribution of relationships between journals. The green path in Figure 6 shows that publications in medicine/clinical/ophthalmology journals are frequently cited by molecular/genetics/dermatology journals. The red path shows that literatures published in mathematics/systems journals are often cited by systems/computer journals.

Figure 6.

The dual-map overlay of journals related to AI and DL applied to the field of otorhinolaryngology diseases. Cited journals are on the left and co-cited journals are on the right. The colored paths between them indicate the cited relationships.

Co-Cited References and References Burst

As a research method to measure the degree of relationship between articles, co-citation is defined as 2 or more articles cited by one or more papers simultaneously, and these 2 articles are considered to be in a co-citation relationship. Figure 7 shows that the first co-cited reference began in 2015. Most commonly cited references have been frequently cited over the past 10 years, meaning that research related to the application of AI and DL in the field of ENT diseases may continue to explode in the future.

Figure 7.

CiteSpace visualization map of top 18 references with the strongest citation bursts related to AI and DL applied to the field of otorhinolaryngology diseases.

The Analysis of Hot spots and Frontiers

Keywords are the core of a paper. By analyzing the keywords, we can summarize the research themes in a specific field and explore the research hot spots and directions. According to Table 4, besides classification (39), the keywords that appeared more frequently in this study were convolutional neural network (CNN) (23), diagnosis (21), and otitis media (17). Among these keywords, classification, CNN, and diagnosis appeared more than 20 times, indicating that these areas are hot spots for research related to AI and DL applied in otolaryngology diseases.

Table 4.

Top 10 Keywords Related to AI and DL Applied to the Field of Otolaryngology Diseases.

Rank	Keywords	Count	Centrality	Rank	Keywords	Count	Centrality
1	Classification	39	0.00	11	System	10	0.13
2	Convolutional neural network	23	0.03	12	Model	10	0.16
3	Diagnosis	21	0.09	13	Image	9	0.00
4	Otitis media	17	0.00	14	Inner ear	8	0.30
5	Neural network	15	0.20	15	Hearing loss	8	0.10
6	Children	15	0.33	16	Algorithm	8	0.40
7	Tympanic membrane	15	0.02	17	Head	8	0.02
8	Management	14	0.24	18	Segmentation	7	0.09
9	Cancer	12	0.16	19	Ménière’s disease	7	0.48
10	Accuracy	11	0.11	20	Risk	7	0.00

Abbreviations: AI, artificial intelligence; DL, deep learning.

Based on keywords co-occurrence analysis, the web maps were clustered to reflect the underlying knowledge structure of the relevant research areas. We used VOSviewer software to cluster the keywords in the literature. The circles and labels form a cell, and the different colors are composed of different clusters. As shown in Figure 8, we can see the red, green, blue, purple, and yellow clusters, which represent 5 different research directions.

Figure 8.

VOSviewer visualization map of keyword clustering analysis related to AI and DL applied to the field of otolaryngology diseases. Green cluster: transfer learning, images, algorithm, and radiotherapy. Red cluster: squamous-cell carcinoma, classification, prediction, survival, and identification. Yellow cluster: children, recognition, perception, hearing, and epidemiology. Purple cluster: inner ear, Ménière’s disease and endolymphatic. Blue cluster: middle ear, otitis media, otoscopy, tympanic membrane, and diagnosis.

The timeline viewer helps explore evolutionary trajectories and stage characteristics in research areas based on the interactions and changes between keywords within a given domain. Figure 9 is a timeline viewer of keywords related to AI and DL applied to ENT diseases drawn based on CiteSpace software in this study, which visually shows the stage hot spots and development direction of AI and DL applied to ENT disease research from the time dimension. From 2003 to 2018, the research mainly focused on the analysis of hearing disorders and articulation. From 2019 to 2020, the research is mainly based on algorithmic systems for the diagnosis, localization, and classification of ENT diseases, with the main keywords. From 2020 to 2022, the research is based on the analysis of endoscopic images for diagnosis and the exploration of AI and DL for radiation therapy, with the following keywords radiation therapy, images, laryngoscopy, inner ear imaging, MRI, and diagnostic-accuracy. Notably, over the past 20 years, research on AI and DL applied to the field of otolaryngological diseases has been closely related to various diseases (Figures 8 and 9), such as otosclerosis, otitis media, nasal polyps, cancer, sinusitis, and Ménière’s disease.

Figure 9.

CiteSpace timeline visualization map of keywords related to AI and DL applied to the field of otolaryngology diseases.

Discussion

General Information

The number and trend of papers published each year can reflect the speed and progress of researches, and can also indicate the concentration of research in the field. As can be seen in Figure 2, the overall number of publications is on the rise. Specifically, from 2003 to 2014, the relatively small amount of literatures indicates that researches on AI and DL applied in ENT diseases are still in its infancy. From 2015 to 2018, the literature showed a steady increase. From 2019 to 2022, there is an explosion of researches related to AI and DL applied in otolaryngological diseases, with the number of published papers reaching 72 in 2022. It can be seen that the application of AI and DL in the researches of ENT diseases is a popular researching direction in recent years, and a good development trend in the future.

Based on the distribution of countries/regions and institutions in Table 1, we can see that the country with the highest number of publications is the United States (63, 27.16%), followed by China (37, 15.95%), which together account for 43.11% of the total. It indicates that China and the United States are the leading countries in AI and DL applied in ENT diseases researches. Centrality is a measure of the importance of nodes in a network. Usually, nodes greater than 0.1 are considered relatively important. Among the top 10 countries in Table 1, Australia has the highest centrality (0.52), which means it plays a key role as a bridge in the global collaborative network of countries. Among the top 10 research institutions in terms of number of publications, 3 were from the United States, 1 from China, and 2 from Australia, with Harvard Medical School (0.01) and the University of Toronto (0.01) having a greater impact. However, from Figures 3 and 4, the distribution of individual countries and institutions is not considered concentrated, and the main research subjects, such as China and the United States, Fudan University, and the University of Toronto, do not form a network, which indicates a lack of academic exchange between countries and research institutions. This situation hinders the development of this research area. Therefore, it is strongly recommended that research institutions in the United States and China, as well as other countries, remove academic barriers and develop collaboration and communication to facilitate progress in the application of AI and DL to the diagnosis and treatment of otolaryngological diseases.

In terms of authors and co-cited authors, most of the top 10 high publication authors and top 10 highly cited authors are from developed countries, suggesting that scholars from developed countries are in a dominant position in the research of AI and DL, and scholars from developing countries should consider increasing cooperative exchanges with scholars from developed countries to improve their research capacity.

According to the journals and co-cited journals in Figure 3, the journals with the most articles about AI and DL applied in ENT diseases were Laryngoscope (8, 3.44%) and Otolaryngology Head and Neck Surgery (8, 3.44%). As for the co-cited journals, Laryngoscope (226) was the most cited journal, followed by Journal of the Acoustical Society of America (223), Otolaryngology Head and Neck Surgery (148), and Otology & Neurotology (144). The analysis of the literature sources helped to find the core journals that published researches related to AI and DL applied in ENT diseases. As shown in Table 3, most of the cited literature came from important journals in the field of otolaryngology, and the highest IF of the cited journals reached 5.591, which indicates that research on AI and DL applied to ENT diseases is highly valued in the academic field.

The Hot Spots and Frontiers

Based on keyword co-occurrence analysis, we identified the research hot spots and development frontiers of AI and DL applied in ENT diseases.

Tympanic membrane images can show changes in the morphology of the tympanic membrane (TM) that is frequently associated with inflammatory diseases of the middle ear, such as otitis media and cholesteatoma. Access to ENT in communities, especially in rural and distant locations where there is a shortage of ENT experts, is also hampered by the continued reliance on skilled physicians to assess pictures and diagnose disease. The proper diagnosis is also affected by the clinical experience gap between senior and junior doctors. Since AI has advanced in recent years, it is possible to train CNNs or other AI algorithms with a set number of TM photos. Once trained, the AI can diagnose TM images quickly and more accurately. For instance, Zebin Wu et al trained and evaluated MobileNet-V2 and Xception, 2 of the most popular CNN designs, and found that they had high overall accuracies of 95.72% and 97.45%, respectively, for endoscopic image recognition of the ear. Tympanic pictures captured using a smartphone and WI-FI otoscope had great overall accuracy for both models, with 90.66% (95% CI 90.21%-90.98%) and 88.56% (95% CI 87.86%-90.05%), respectively.²⁴ To diagnose cholesteatoma stromal lesions with 34.6% and 42.3% sensitivity and 81.3% and 87.5% specificity, respectively, on SPECTRA A and SPECTRA B, Miwa et al⁷ trained a CNN using a collection of photos chosen by the otolaryngologist. By uploading 1366 endoscopic ear pictures to the Google Cloud Vision AutoML (Google Inc.) platform and changing consensus labels for each image, Devon Livingstone et al trained the system. In subsequent tests, the system correctly detected based on the photos 88.7% of the time. The typical accuracy of physician diagnosis was 58.9%.²⁵ The diagnostic sensitivity and specificity of ML using TM images for middle ear disease classification were 93% (95% CI, 90%-95%) and 85% (95% CI, 82%-88%), respectively, according to a meta-analysis by Zuwei Cao et al that included 16 studies with 20,254 TM images.⁶ A significant issue in otology is hearing loss. Poor patient collaboration is a frequent clinical event, despite the high degree of patient cooperation needed to get precise and reliable hearing thresholds. Analysis of video inflation otoscopy (VPO) images may aid in diagnosing conductive hearing loss (CHL). Hayoung Byun et al developed a DL algorithm for analyzing VPO images that successfully detected the presence of CHL brought on by middle ear effusion, auditory bone fixation, otosclerosis, and adhesive otitis media. This is because VPO results show the presence of middle ear effusion and the dynamic motion of the TM and part of the auditory tuberosity. The interpretation of VPO by this algorithm also demonstrates potential as a diagnostic tool for distinguishing between conductive and sensorineural hearing loss, which is especially helpful for patients who are uncooperative.²⁶ Zeng et al created the DL model (to predict a mean air–bone gap greater than 10 dB) and a logistic regression model based on otoscopic features to predict CHL using otoscopic pictures (particularly when CHL cannot be evaluated). When predicting CHL based on otoscopic pictures, the DL model performed better than the logistic regression model and 3 otologists while taking much less time.⁸ Taewoong Uhm and colleagues created a new prognostic prediction model for idiopathic sudden sensorineural hearing loss (ISSHL) based on ML, which includes prediction models for 5 ML techniques: least absolute shrinkage and selection operator, decision tree, random forest (RF), support vector machine, and boosting. According to the test results, the ISSHL prognostic prediction model demonstrated greater predictive power than conventional logistic regression techniques, with the RF approach achieving the highest predictive power and the other techniques having acceptable predictive power.²⁷ In addition, Hajime Koyama et al²⁸ discovered that RF model and other ML techniques have the potential to accurately predict postoperative air–bone conduction differences. In addition, Lenatti et al²⁹ examined the performance of such machine learning models and found that multivariate ML models applied to noisy speech hearing screening tests can help accurately detect hearing loss.

The treatment of inner ear problems has also benefited from the application of AI and DL. Spontaneous nystagmus, head-shaking nystagmus, and vHIT were cited by Du et al¹⁰ as the 3 main factors to examine while screening for vestibular disorders. Carey et al¹¹ discovered that when diagnosing individuals with vestibular nerve sheath tumors solely based on hearing data, ML methods performed comparably to rule-based assessments. Cortada et al developed a DL model and validated it in untreated cisplatin and gentamicin-treated Corti explant organs to measure hair cell survival in the murine organ of Corti explants. The availability of this DL model provides the opportunity to eliminate inter-assessor reliability disparities and the time-consuming manual counting process.⁹ Ménière’s illness still presents a significant difficulty in diagnosis and therapy. Matthew Shew et al introduced a novel diagnostic concept, namely diagnosis by detecting biomarkers, without having a thorough understanding of the pathophysiology. The researchers trained DL to predict Ménière’s disease using the perilymph they had collected since miRNA perilymph analysis is a secure technique comparable to ‘liquid biopsy’. The top model tested had a 66% accuracy rate. Although ML models are less effective at forecasting Ménière’s Sensorineural hearing loss (SNHL), they offer fresh perspectives for future study, including the discovery and identification of new biomarkers for testing to support diagnosis and therapy.¹³ A machine learning-based analytical model created by Jeffrey Skidmore et al predicts the functional status of the CN in individual CI users, serving as a crucial clinical care reference.¹² Cochlear implants are frequently used to treat severe to total deafness.

DL and AI are being developed for rhinology. To increase the effectiveness of rhinological disease diagnosis, Parsel et al³⁰ combined clinical data with AI, including peripheral eosinophil fraction, past medical history, patient-reported quality of life instrument ratings, and objective findings from nasal endoscopy. Rhinology can benefit from the usage of CNNs, a class of AI or DL that is mainly utilized for image analysis and classification. With a diagnostic accuracy of 81%, Parmar et al trained CNN algorithm to collect high-resolution computed tomography scans of the sinuses to identify pneumatization of the middle turbinates. This work paves the way for future AI-based research into other clinically significant anatomical sites in ENT.¹⁴

The focus of AI and DL in laryngology research is the analysis of laryngoscopic images. Wang et al analyzed laryngoscopic images with Automated Glottic Action Tracking by artificial Intelligence (AGATI), a vocal fold action tracking software driven by AI, and found a difference in the value of the maximum anterior glottis angle (AGA) calculated between patients with vocal cord paralysis and control patients. Using this indicator, AGATI predicted Unilateral vocal fold paralysis (UVFP) with a sensitivity of 77% and a specificity of 92%.³¹ Parker et al³² applied CNN to laryngoscopy images to analyze the granuloma and ulceration of patients after extubation in intensive care unit, proving that ML is feasible to increase the objectivity of laryngoscopy analysis. Dunham et al developed 2 classifiers based on AI: benign classifier and binary classifier. The benign classifier divided benign lesions into 5 categories: normals, nodules, papilloma, polyps, and webs, with an overall accuracy of 80.8%. The binary classifier correctly identified 92.0% of malignant–premalignant lesions, with an overall accuracy of 93.0%.³³ In a retrospective experimental study, Muhammad Adeel Azam et al identified a suitable CNN model (ensemble algorithm YOLOv5s and YOLOv5m-TTA) for laryngeal squamous cell carcinoma (LSCC) detection in white light and narrow-band imaging video laryngoscopes. The detection performance is very promising, which is very suitable for real-time detection of LSCC.¹⁷

Otolaryngology telemedicine is becoming popular as a potential method to improve access to rural populations, shorten wait times for specialists, and lower total healthcare system expenditures. Researchers have created several otoscopes that allow patients, their parents, or primary care physicians to photograph the eardrum and middle ear and send the data to an otolaryngologist for review.³⁴ AI and ML play a crucial part in this. To maximize the effectiveness of the operating room, clinicians and patients can benefit from AI and ML. Miller et al³⁵ assessed various ML algorithms based on their performance in forecasting operating room case duration in otolaryngology and discovered that they might increase prediction accuracy and produce economic benefits.

In conclusion, ML and AI already have many benefits and are becoming more and more crucial in ENT clinical applications. However, there are a few crucial difficulties that demand our attention. First and foremost, a uniform protocol for ENT image acquisition and annotation should be created.⁶ Take DL of ear image training as an example. In the process of manually editing labels for each image, each research group will adopt subjective criteria to classify and label, which leads to differences in the reliability of research between different groups. Different groups find it challenging to communicate effectively and properly due to differences in labeling and classification standards. Second, there are not many accessible code sources, which means that the raw clinical data can only be used with currently available tools by rebuilding the model from scratch.³⁶ Therefore, a great deal of interpretation must be done before the analysis process. In addition, when conducting clinical research, researchers need to further study larger samples and use more complex ML algorithms and interpretability techniques, so as to more fully study the influence of input characteristics on the disease under study and improve the accuracy of model judgment.²⁹

Limitation

There are several drawbacks with VOSviewer and CiteSpace software that need to be solved since they are not a perfect replacement for systematic searching. First, the period covered by the literature we found ranges from 2003 to 2022, although due to the Web of Science Core Collection (WoSCC) literature’s ongoing updating, there are some discrepancies between the search results for this study and the actual number of included works. Second, the variable quality of the literature used in the study’s articles and reviews could undermine the validity of the visual analysis. Last but not least, certain important keywords from the article were only partially analyzed because of insufficient keyword extraction. But there is no doubt that the visual analysis based on the literature provides the groundwork for academics to easily comprehend the research issue.

Conclusion

With a wide range of potential applications, the study of clinical applications of AI and deep (machine) learning in otolaryngology offers significant research value. Visual analysis research employing CiteSpace and VOSviewer software have grown significantly in recent years. China and the United States are the top 2 nations in the world according to this study. Greater cooperation and communication between various nations and institutions are required. The majority of the works on deep (machine) learning and AI in clinical settings in ENT are cited from internationally renowned journals in this subject, which shows the broad interest in this area. At present, the research in this field mainly focuses on the analysis of endoscopic images to diagnose, predict diseases, and judge prognosis, which will also be the focus of future research. Building a common set of image capture and annotation techniques as well as a number of DL-related computing issues are the present challenges. With the development of computer technology, AI and DL will play an increasingly important role in the diagnosis and treatment of ENT diseases.

Footnotes

Author Contribution Statement

Tianhong Zhang is the corresponding author, and she contributes to the conception of the study. Tianyu Ma contributes to design. Qilong Wu, Li Jiang, and Xiaoyun Zeng contribute to collection and assembly of data. Yuyao Wang, Yi Yuan, and Bingxuan Wang contribute to data analysis and interpretation. All authors contributes to the writing of the manuscript.

Data Availability Statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The research was funded by the Heilongjiang Provincial Nature Foundation.

Ethical Statement

Because this work was a retrospective bibliometric review of previously published studies, ethics committee approval was not necessary.

ORCID iD

Tianyu Ma

References

Jun

Jung

Lee

. Medical data science in rhinology: background and implications for clinicians. Am J Otolaryngol. 2020;41(6):102627. doi:10.1016/j.amjoto.2020.102627

Topol

. High-performance medicine: the convergence of human and artificial intelligence. Nat Med 2019;25(1):44–56. doi:10.1038/s41591-018-0300-7

Roberts

Datta

, et al. Deep learning in clinical natural language processing: a methodical review. J Am Med Inform Assoc. 2020;27(3):457–470. doi:10.1093/jamia/ocz200

. Application of artificial intelligence in ophthalmology. Int J Ophthalmol. 2018;11(9):1555–1561. Published online Sep 18, 2018. doi:10.18240/ijo.2018.09.21

Esteva

Kuprel

Novoa

, et al. Dermatologist-level classification of skin cancer with deep neural networks [published correction appears in Nature. 2017 Jun 28;546(7660):686]. Nature. 2017;542(7639):115–118. doi:10.1038/nature21056

Cao

Chen

Grais

, et al. Machine learning in diagnosing middle ear disorders using tympanic membrane images: a meta-analysis. Laryngoscope. 2023;133(4):732–741. doi:10.1002/lary.30291

Miwa

Minoda

Yamaguchi

, et al. Application of artificial intelligence using a convolutional neural network for detecting cholesteatoma in endoscopic enhanced images. Auris Nasus Larynx. 2022;49(1):11–17. doi:10.1016/j.anl.2021.03.018

Zeng

Kang

Chen

, et al. A deep learning approach to predict conductive hearing loss in patients with otitis media with effusion using otoscopic images. JAMA Otolaryngol Head Neck Surg. 2022;148(7):612–620. doi:10.1001/jamaoto.2022.0900

Cortada

Sauteur

Lanz

Levano

Bodmer

. A deep learning approach to quantify auditory hair cells. Hear Res. 2021;409:108317. doi:10.1016/j.heares.2021.108317

10.

Ren

Liu

. Machine learning method intervention: determine proper screening tests for vestibular disorders. Auris Nasus Larynx. 2022;49(4):564–570. doi:10.1016/j.anl.2021.10.003

11.

Carey

Jacobson

Warburton

, et al. Machine learning for vestibular schwannoma diagnosis using audiometrie data alone. Otol Neurotol. 2022;43(5):e530–e534. doi:10.1097/MAO.0000000000003539

12.

Skidmore

Chao

, et al. Prediction of the functional status of the cochlear nerve in individual cochlear implant users using machine learning and electrophysiological measures. Ear Hear. 2021;42(1):180–192. doi:10.1097/AUD.0000000000000916

13.

Shew

Wichova

Bur

, et al. MicroRNA profiling as a methodology to diagnose Ménière’s disease: potential application of machine learning. Otolaryngol Head Neck Surg. 2021;164(2):399–406. doi:10.1177/0194599820940649

14.

Parmar

Habib

Mendis

, et al. An artificial intelligence algorithm that identifies middle turbinate pneumatisation (concha bullosa) on sinus computed tomography scans. J Laryngol Otol. 2020;134(4):328–331. doi:10.1017/S0022215120000444

15.

Walker

Surda

. Unsupervised learning techniques for the investigation of chronic rhinosinusitis. Ann Otol Rhinol Laryngol. 2019;128(12):1170–1176. doi:10.1177/0003489419863822

16.

Koenen

Arens

Olze

Dommerich

. Classifying and predicting surgical complications after laryngectomy: a novel approach to diagnosing and treating patients. Ear Nose Throat J. Jul 30, 2021. doi:10.1177/01455613211029749

17.

Azam

Sampieri

Ioppi

, et al. Deep learning applied to white light and narrow band imaging videolaryngoscopy: toward real-time laryngeal cancer detection. Laryngoscope. 2022;132(9):1798–1806. doi:10.1002/lary.29960

18.

DeVore

Adamian

Jowett

, et al. Predictive outcomes of deep learning measurement of the anterior glottic angle in bilateral vocal fold immobility. Laryngoscope. Published online Nov 3, 2022. doi:10.1002/lary.30473.

19.

Smith

. Bibliometrics, dermatology and contact dermatitis. Contact Dermatitis. 2008;59(3):133–136. doi:10.1111/j.1600-0536.2008.01405.x

20.

. Global research trends on prostate diseases and erectile dysfunction: a bibliometric and visualized study. Front Oncol. 2021;10:627891. doi:10.3389/fonc.2020.627891

21.

Chen

. Searching for intellectual turning points: progressive knowledge domain visualization. Proc Natl Acad Sci USA. 2004;101(Suppl 1):5303–5310. doi:10.1073/pnas.0307513100

22.

Yang

Guan

, et al. A bibliometric analysis of pyroptosis from 2001 to 2021. Front Immunol. 2021;12:731933. doi:10.3389/fimmu.2021.731933

23.

van Eck

Waltman

. Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics. 2010;84(2):523–538. doi:10.1007/s11192-009-0146-3

24.

Lin

, et al. Deep learning for classification of pediatric otitis media. Laryngoscope. 2021;131(7):E2344–E2351. doi:10.1002/lary.29302

25.

Livingstone

Chau

. Otoscopic diagnosis using computer vision: an automated machine learning approach. Laryngoscope. 2020;130(6):1408–1413. doi:10.1002/lary.28292

26.

Byun

Park

Chung

Cho

. Automatic prediction of conductive hearing loss using video pneumatic otoscopy and deep learning algorithm. Ear Hear. 2022;43(5):1563–1573. doi:10.1097/AUD.0000000000001217

27.

Uhm

Lee

, et al. Predicting hearing recovery following treatment of idiopathic sudden sensorineural hearing loss with machine learning models. Am J Otolaryngol. 2021;42(2):102858. doi:10.1016/j.amjoto.2020.102858

28.

Koyama

Kashio

Uranaka

Matsumoto

Yamasoba

. Application of machine learning to predict hearing outcomes of tympanoplasty [published online ahead of print, 2022 Oct 26]. Laryngoscope. Published online October 26 2022. doi:10.1002/lary.30457

29.

Lenatti

Moreno-Sánchez

Polo

Mollura

Barbieri

Paglialonga

. Evaluation of machine learning algorithms and explainability techniques to detect hearing loss from a speech-in-noise screening test. Am J Audiol. 2022;31(3S):961–979. doi:10.1044/2022_AJA-21-00194

30.

Parsel

Riley

Todd

Thomas

McCoul

. Differentiation of clinical patterns associated with rhinologic disease. Am J Rhinol Allergy. 2021;35(2):179–186. doi:10.1177/1945892420941706

31.

Wang

Adamian

Song

, et al. Application of a computer vision tool for automated glottic tracking to vocal fold paralysis patients. Otolaryngol Head Neck Surg. 2021;165(4):556–562. doi:10.1177/0194599821989608

32.

Parker

Brodsky

Akst

Ali

. Machine learning in laryngoscopy analysis: a proof of concept observational study for the identification of post-extubation ulcerations and granulomas. Ann Otol Rhinol Laryngol. 2021;130(3):286–291. doi:10.1177/0003489420950364

33.

Dunham

Kong

McWhorter

Adkins

. Optical biopsy: automated classification of airway endoscopic findings using a convolutional neural network. Laryngoscope. 2022;132(Suppl 4):S1–S8. doi:10.1002/lary.28708

34.

Ezzibdeh

Munjal

Ahmad

Valdez

. Artificial intelligence and tele-otoscopy: a window into the future of pediatric otology. Int J Pediatr Otorhinolaryngol. 2022;160:111229. doi:10.1016/j.ijporl.2022.111229

35.

Miller

Goedicke

Crowson

Rathi

Naunheim

Agarwala

. Using machine learning to predict operating room case duration: a case study in otolaryngology. Otolaryngol Head Neck Surg. 2023;168(2):241–247. doi:10.1177/01945998221076480

36.

Bulfamante

Ferella

Miller

, et al. Artificial intelligence, machine learning, and deep learning in rhinology: a systematic review. Eur Arch Otorhinolaryngol. 2023;280(2):529–542. doi:10.1007/s00405-022-07701-3

Artificial Intelligence and Machine (Deep) Learning in Otorhinolaryngology: A Bibliometric Analysis Based on VOSviewer and CiteSpace

Abstract

Background:

Objective:

Methods:

Results:

Conclusions:

Keywords

Introduction

Aim of This Study

Methods

Data Collection and Search Strategy

Screening Strategy

Data Analysis

Overview

The Software We Used

Results

The Trend of Publication Outputs

Distribution of Countries/Regions and Institutions

Co-Cited References and References Burst

The Analysis of Hot spots and Frontiers

Discussion

General Information

The Hot Spots and Frontiers

Limitation

Conclusion

Footnotes

Author Contribution Statement

Data Availability Statement

Declaration of Conflicting Interests

Funding

Ethical Statement

ORCID iD

References