Sage Journals: Discover world-class research

Abstract

Background

A considerable volume of possible applications of artificial intelligence (AI) in the field of rhinology exists, and research in the area is rapidly evolving.

Objective

This scoping review aims to provide a brief overview of all current literature on AI in the field of rhinology. Further, it aims to highlight gaps in the literature for future rhinology researchers.

Methods

OVID MEDLINE (1946-2022) and EMBASE (1974-2022) were searched from January 1, 2017 until May 14, 2022 to identify all relevant articles. The Preferred Reporting Items for Systematic Reviews and Meta-analyses Extension for Scoping Reviews checklist was used to guide the review.

Results

A total of 2420 results were identified of which 62 met the eligibility criteria. A further 17 articles were included through bibliography searching, for a total of 79 articles on AI in rhinology. Each year resulted in an increase in the number of publications, from 3 articles published in 2017 to 31 articles published in 2021. Articles were produced by authors from 22 countries with a relative majority coming from the USA (19%), China (19%), and South Korea (13%). Articles were placed into 1 of 5 categories: phenotyping/endotyping (n = 12), radiological diagnostics (n = 42), prognostication (n = 10), non-radiological diagnostics (n = 7), surgical assessment/planning (n = 8). Diagnostic or prognostic utility of the AI algorithms were rated as excellent (n = 29), very good (n = 25), good (n = 7), sufficient (n = 1), bad (n = 2), or was not reported/not applicable (n = 15).

Conclusions

AI is experiencing an increasingly significant role in rhinology research. Articles are showing high rates of diagnostic accuracy and are being published at an almost exponential rate around the world. Utilizing AI in radiological diagnosis was the most published topic of research, however, AI in rhinology is still in its infancy and there are several topics yet to be thoroughly explored.

Keywords

artificial intelligence artificial neural network convolutional neural network machine learning rhinology phenotyping endotyping diagnostics prognostics sinus

Introduction

Artificial intelligence (AI) is an increasingly exciting area of research in medicine. Amongst other benefits, AI has the potential to automatically perform complex tasks with great speed¹ and precision. Various applications of AI in medicine have already evolved from theoretical or proof-of-concept to being used in clinical practice, such as the automatic detection of atrial fibrillation via a smartphone or smartwatch-based ECG monitors^2,3 or continuous glucose monitoring to prevent hypoglycaemia.^3,4

In this field there are several definitions that should be considered (Figure 1). Machine learning (ML) is a subset of AI that uses prior data to make improved decisions about future data.⁵ ML algorithms can be split into 3 main categories: supervised learning, unsupervised learning, and reinforcement learning.⁶ Supervised learning requires labelled datapoints for a ML algorithm to learn from, to later make predictions on unlabeled data.⁵ Unsupervised learning algorithms find patterns (eg, in cluster analysis) in unlabeled datasets.⁵ Reinforcement learning is the training of a ML model to make a sequence of decisions to solve a task through a process of trial and error.⁷ An artificial neural network (ANN) utilizes components of supervised and reinforcement learning to solve problems. ANNs use layers of processing to make sense of input information. The output of a layer becomes the input for the next layer, until it has been transformed into an output that can be used by the network.⁸ Deep learning is a subset of ANNs where at least 3 layers are used in the network.⁹ A convolutional neural network (CNN) is a type of ANN that processes data with grid-like structures (such as images).¹⁰ Natural language processing (NLP) is an entirely different subset of AI that allows computers to understand humans by converting language into data, that are processable by a computer.¹¹

Figure 1.

Summary of the types of artificial intelligence commonly used in medicine.

A considerable volume of research about possible applications of AI in the field of rhinology is available, yet no application has a widely used clinical application to date. This scoping review aimed to provide a brief overview of all current literature on AI in the field of rhinology. Further, it aimed to highlight implications for clinical practice and future research for rhinology researchers.

Materials and Methods

Following a preliminary search of the literature on the usages of AI in rhinology, a scoping review was conducted. The Preferred Reporting Items for Systematic Reviews and Meta-analyses Extension for Scoping Reviews¹² (PRISMA-ScR) checklist was used to guide the review. Registration was not applied as PROSPERO does not accept scoping reviews, literature reviews, or mapping reviews.

Eligibility Criteria

Eligible studies described an application of AI to solve a clinical question in the field of rhinology. Articles needed to be published since 2017 to capture the most up-to-date literature in this rapidly expanding field of research.¹³ Unpublished literature is often reported in scoping reviews,¹² as one of its purposes is to map a body of knowledge to identify gaps in research.¹⁴ Hence conference and poster abstracts were eligible for inclusion provided they had been presented since 2020. Grey literature from prior to 2020 were excluded. Articles were excluded if AI was only used in the statistical analysis of a paper, or if the use of AI was only to calculate radiation dosages in oncological settings. Articles that described the automatic segmentation of nasopharyngeal cancers in radiological scans were excluded if they were utilized for the purpose of determining radiation therapy/treatment. Works without original data, animal studies, and studies unavailable in English were also excluded.

Information Sources

A systematic electronic search was performed for relevant studies using the Ovid MEDLINE (1946-2021) and EMBASE (1974-2021) databases from January 1, 2017 until the May 14, 2022 using a defined search strategy (Table 1). AI terms including “DEEP LEARNING” and “articifial adj2 intelligen*,” rhinology terms including “NOSE” and “Rhin*,” and investigation terms including “Endosco*” and “MRI*” were combined using the Boolean operators (AND and OR) to broaden and limit the search where appropriate. A manual bibliographic screen from the included and other relevant studies was performed to search for additional relevant articles undetected by the original search. An attempt was made to contact authors of studies where the full-text was not found.

Table 1.

Search Strategy. Table of Terms Used in MEDLINE (2017-2022) and EMBASE (2017-2022) Databases. Search Performed on May 14, 2022.

Artificial intelligence terms	Rhinology terms	Investigation terms
1. AI 2. artificial adj2 intelligen* 3. Automat* 4. Convolutional 5. DEEP LEARNING 6. Deep learn* 7. MACHINE LEARNING 8. Machine adj2 learn* 9. MobileNet 10. Neural network 11. Pseudo-label* 12. ResNet 13. Self-train* 14. Self-taught* 15. Semi-supervised 16. SENet 17. Supervised 18. Unsupervised	19. Adenoid* 20. Ethmoid* 21. Frontal* adj2 sinus* 22. Inverted adj2 pap* 23. Maxil* 24. Meatal* 25. Meatus* 26. Nasal* 27. Naso* 28. NOSE 29. Nose* 30. Olfact* 31. Rhin* 32. Septal* 33. Septum* 34. Sinonas* 35. Sinus* 36. Smell* 37. Spheno* 38. AERD 39. AFRS 40. Allerg* 41. Atop* 42. CCAD 43. Central compartment 44. CRS* 45. eCRS* 46. fung*	47. CT 48. Comput* 49. Endonasal* 50. Endosco* 51. Fib??optic* 52. Intranasal* 53. Magnet* 54. MRI* 55. Exam* adj3 (nasal* OR nose) 56. Nasendo 57. Nasopharyngolaryngosc* 58. Nasopharyngosc* 59. Rhinoscop* 60. PET 61. Positron* 62. Scop* 63. Tomograph*
64. 1-18 OR 65. 19-46 OR 66. 47-63 OR 67. 64 AND 65 AND 66 68. 67 Limit to English and human 69. 68 Limit 2017-current

Study Selection

Duplicate studies were automatically removed using the OVID duplicate removal function, and later, manually. The remaining studies were exported to Rayyan (Qatar Computing Research Institute, Qatar),¹⁵ an online review tool, for screening against the eligibility criteria outlined above. Study selection was performed by 2 authors (GO and RK); uncertainties were resolved by consensus. Studies were screened in 3 phases: first by title, then by abstract, and finally by full-text. Articles that met the eligibility criteria were included for data collection.

Data Collection

Data extracted from individual studies were recorded in Microsoft Excel for Mac (Version 16.63.1).¹⁶ Data fields collected included names of first and second authors, year of publication, study design, number of participants in total, as well as training, validation, and testing sets, demographic data of participants, country of publication, journal of publication, field of research, anatomical structures defined (if any), pathologies identified (if any), type of AI used, specific AI programs, algorithms, and software used, and diagnostic or prognostic accuracy (if applicable). There were several possible metrics of diagnostic accuracy. In order of preference, receiver operator characteristics area under the curve (ROC-AUC), Dice coefficient, and accuracy were used, however other metrics were satisfactory. Diagnostic or prognostic accuracy was rated as excellent if the ROC-AUC (or next best metric) was more than 0.9, very good if 0.8-0.9, good if 0.7-0.8, sufficient if 0.6-0.7, or bad if 0.5-0.6.

Critical Appraisal of Individual Sources of Evidence

A critical appraisal or risk of bias assessment of individual studies is not typically performed in scoping reviews as their purpose is to identify the available evidence on a topic regardless of the methodological quality.¹²

Results

Study Selection

The search strategy yielded a total of n = 2420 studies. After the removal of 680 duplicates, n = 1740 titles were screened. Abstract screening was performed on 205 articles. Full-text analysis was performed on 129 articles (2 full-texts were unable to be located), resulting in 62 articles being included from the search. An additional 35 records were identified and screened from manual citation searching, of which 17 were included in the final analysis (7 of which were from journals not indexed by MEDLINE or EMBASE), leading to a combined total of 79 articles (Figure 2).

Figure 2.

Preferred reporting items for systematic reviews and meta-analyses extension for scoping reviews flow chart highlighting the study selection process.

Characteristics

Of the 79 articles identified, 3 were published in 2017,^17–19 5 were published in 2018,^20–24 9 were published in 2019,^25–33 15 were published in 2020,^34–48 31 were published in 2021,^49–79 and 16 were published in 2022^80–95 until May 14th (Figure 3). Study designs of the included articles were either cross-sectional (n = 61)^{19,21,22,24,25,29,31–35,37–48,50–52,54–56,58,59,62–65,67,69,71–82,84–95} or cohort studies (n = 18).^{17,18,20,23,26–28,30,36,49,53,57,60,61,66,68,70,83} Of the cross-sectional studies, 7 were presented at scientific conferences as abstracts,^{20,32,40,47,56,78,95} and 1 was presented at a conference as a poster.⁷⁷

Figure 3.

Number of articles on artificial intelligence in the field of rhinology since 2017.

Articles were published in journals of otolaryngology (n = 19),^{17,18,23,26,28,30,36,38,39,42,46,52,57,65,72,77,86,87,94} radiology (n = 21),^{19,22,25,29,31,33,34,45,48,51,54,55,58,67,73,74,81,83–85,95} medical sciences (n = 19),^{32,41,44,47,53,61,63,66,68–70,76,78–80,89–91,93} or other areas of medicine (n = 20).^{20,21,24,27,35,37,40,43,49,50,56,59,60,62,64,71,75,82,88,92} Countries of publication included United States of America (n = 15),^{18,19,23,26,30,33,35,36,39,46,65,68,72,86,93} China (n = 15),^{24,37,40,45,49,67,69,74–76,80,84,87,92,94} South Korea (n = 10),^{28,29,52,54,55,57,64,82,85,89} Germany (n = 8),^{25,34,50,51,60,61,77,91} Japan (n = 5),^{27,31,58,62,63} Hong Kong (n = 3),^48,73,81 Indonesia (n = 3),^32,47,79 Taiwan (n = 3),^41,53,83 Australia (n = 2),^38,42 Italy (n = 2),^22,59 Netherlands (n = 2),^66,78 and Brazil,²¹ Finland,⁸⁸ France,²⁰ Morocco,¹⁷ Poland,⁴³ Russia,⁴⁴ Singapore,⁷⁰ Switzerland,⁷¹ Turkey,⁹⁵ United Kingdom,⁵⁶ and Vietnam⁹⁰ (each n = 1).

Applications of AI in rhinology were arranged into 5 major categories (Figure 4):

Phenotyping or endotyping

Figure 4.

Major applications of artificial intelligence in rhinology flow chart.

There were 12 articles that used AI for phenotyping or endotyping pathology.^{17,18,20,23,28,32,47,65,72,76,79,87} The articles used either cluster analysis (n = 7)^{17,18,20,23,28,65,79} or ML (n = 5)^{32,47,72,76,87} in pathologies such as acute sinusitis (n = 3),^32,47,79 allergic rhinitis (AR) (n = 1),²⁰ and chronic rhinosinusitis (CRS) (n = 8).^{17,18,23,28,65,72,76,87} Diagnostic utility was rated as excellent (n = 5)^{32,47,72,76,79} or very good (n = 1),⁸⁷ or was not applicable/not reported (n = 6)^{17,18,20,23,28,65} (Appendix 1).

2. Radiology Diagnostics

There were 42 articles that used an application of AI for radiology/imaging diagnostics in rhinology.^{19,21,22,25,26,29,31,38–43,45,46,48–51,54,55,58,59,62–64,67,69,75,78,80–86,89,91–93,95} Imaging tools used include CT (n = 25),^{21,22,25,26,38–43,45,46,55,58,59,63,69,75,81–83,89,91,92,95} MRI (n = 10),^{19,48–51,78,80,84,86,93} and X-ray (n = 6),^{29,31,54,62,64,85} with 1 article using both CT and MRI.⁶⁷ ML techniques including CNNs and deep learning were utilized by all 42 studies.

Of the 42 articles total, 15 aimed to identify or characterize anatomical structures^{21,22,25,38,41,43,45,50,51,59,78,91–93,95} including the location of the anterior ethmoid artery (n = 1),³⁸ the maxillary sinus (n = 4),^21,41,43,45 the inferior turbinate (n = 1),⁴¹ the nasal cavity (n = 4),^25,59,91,93 the olfactory bulb (n = 3),^50,51,78 the depth of the olfactory fossa (n = 1),⁹⁵ the sphenoid sinus (n = 2),^22,92 and the entire upper airway (n = 1).⁹³ Diagnostic accuracy was rated as excellent (n = 5),^{21,45,59,91,92} very good (n = 5),^{38,41,51,93,95} good (n = 1),⁵⁰ or was not applicable/not reported (n = 4)^22,25,43,78 (see Appendix 2).

The remaining 27 articles aimed to identify or differentiate between various pathologies^{19,26,29,31,39,40,42,46,48,49,54,55,58,62–64,67,69,75,80–86,89} including concha bullosa (n = 1),⁴² CRS (n = 3),^39,46,83 fungal ball (n = 1),⁸² inverted papilloma (n = 3),^19,69,86 nasal cancers (n = 4),^19,49,63,86 maxillary defects (n = 1),⁷⁵ maxillary sinus lesions (n = 3),^55,58,81 maxillary sinusitis (n = 6),^{29,31,54,62,64,85} nasal fractures (n = 1),⁸⁹ nasopharyngeal cancers (n = 4)^40,48,67,80 or other pathologies (n = 1),⁸⁴ and osteomeatal complex inflammation (n = 1).²⁶ Diagnostic accuracy was rated as excellent (n = 12),^{39,40,42,46,48,49,67,69,80,82,84,89} very good (n = 12),^{19,26,29,31,54,58,62–64,81,83,86} or good (n = 3)^55,75,85 (see Appendix 3).

3. Prognostication

There were 10 articles that used AI to predict prognosis or outcomes in rhinological issues.^{27,30,36,53,57,60,61,66,68,88} Aims of studies included prediction of outcomes in CRS (n = 3),^57,60,88 AR (n = 1),⁵³ sinonasal SCC (n = 1),²⁷ and acute rhinosinusitis (n = 1),⁶⁶ olfactory loss in rhinitis (n = 1)⁶¹ and CRS (n = 2),^30,68 and predicting SNOT scores after functional endoscopic sinus surgery (n = 1).³⁶ The articles used AI with ML (n = 8)^{27,36,53,57,61,66,68,88} or cluster analysis (n = 2).^30,60 Prognostic accuracy was rated as excellent (n = 1),²⁷ very good (n = 3),^53,57,60 good (n = 2),^68,88 sufficient (n = 1),⁶¹ bad (n = 1),⁶⁶ or was not applicable/not reported (n = 2)^30,36 (Appendix 4).

4. Non-radiological diagnostics

There were 7 articles that discussed AI applications for non-radiological diagnostic tools^{24,33,37,44,52,70,94} such as endoscopy (n = 5),^{24,33,52,70,94} histopathology (n = 1),³⁷ and optic coherence tomography (OCT) (n = 1).⁴⁴ All 7 papers utilized CNNs. The aims of the papers ranged from detecting nasopharyngeal carcinomas (n = 4),^24,37,70,94 differentiating nasal polyps from inverted papillomas (n = 1),⁵² automatically identifying anatomical structures (n = 1)³³ to differentiating chronic rhinitis (n = 1).⁴⁴ ML techniques were utilized in all 7 studies. Diagnostic accuracy was rated as excellent (n = 3),^37,44,94 very good (n = 2),^52,70 good (n = 1),²⁴ or was not applicable/not reported (n = 1)³³ (see Appendix 5).

5. Surgical planning and assessment

The use of AI in surgical planning and assessment was explored by 8 studies.^{34,35,56,71,73,74,77,90} Each study was unique in their objective, including prediction of nasal bone morphology based on photographs of the nose (n = 1),⁹⁰ automated analysis of surgical steps and identification of structures during pituitary surgery (n = 2),^56,71 real-time 3-dimentional anatomical annotation during transnasal surgery (n = 1),⁷³ automating the generation of operation reports in sinus surgery (n = 1),⁷⁷ prediction of navigational positions during endoscopy (n = 1),³⁴ automated prediction of whether patients had had rhinoplasty based on mobile photographs (n = 1),³⁵ and automated clipping of nasal endoscopic videos (n = 1).⁷⁴ ML was utilized in 7 of the studies^{34,35,56,71,73,74,90}; NLP was used in 2 of the studies.^34,77 Accuracy was rated as excellent (n = 3),^56,74,90 very good (n = 2),^35,73 bad (n = 1),³⁴ or was not applicable/not reported (n = 2)^71,77 (see Appendix 6).

Discussion

The majority of publications on the use of AI in rhinology were in diagnostics (n = 49); most of these articles (n = 42) discussed radiological diagnostics in the form of CT, MRI, or X-ray imaging to automate identifying anatomical and pathological findings with CNNs, while the remaining (n = 7) detailed other diagnostic tools such as nasal endoscopy, OCT, or histopathology. Phenotyping/Endotyping was the next most common category of AI usage in rhinology (n = 12). The majority of these articles utilized cluster analysis to generate novel sub-type groups in pathologies such as CRS, acute sinusitis, and AR. In the prognostication group (n = 10), ML techniques (unsupervised or semi-supervised) and cluster analysis were used to predict outcomes or responses to treatments most commonly in CRS and AR. Surgical planning and assessment in rhinology was the least common area of AI research (n = 8), with newly evolving CNNs to aid pre-surgical assessment of nasal morphology and endoscopic navigational positions, intra-operative assessment with live annotation, analysis of surgical steps and video clipping, and post-operative tools such as surgical report generation and determination of prior rhinoplasty. The higher volume of AI rhinology research in radiological diagnostics compared to topics such as surgical planning and assessment is perhaps explained by a larger foundation of AI research in radiology in medicine as a whole, which is probably magnified by the investments in the field by tech giants such as Google.^96,97 This would allow researchers to broadly apply previously published AI techniques to their own rhinological niche, whereas developing an AI for surgical planning and assessment requires a more novel technological approach. Throughout the articles, various AI programs were used; notably U-Net, a CNN designed for biomedical image segmentation and Res-Net, an ANN deigned for image recognition were most common.

Articles frequently reported high diagnostic accuracy of their AI algorithms, with 36.7% having an ROC-AUC (or other similar metric) rated as excellent, 31.6% rated as very good, 8.9% as good, 1.3% as sufficient, 2.5% as bad, and 19.0% not reporting.

Implications for Clinical Practice

There are several promising areas in the field where AI has the potential to augment a rhinologist's practice. Although the risk of being entirely replaced by AI and robots is slim,⁹⁸ there are potential ethical and legal issues⁹⁹ with the implementation of AI in medicine which need to be addressed before AI can be considered for adoption into mainstream clinical practice.

In supervised learning models, data need to be labelled to train the ML algorithms. If the data is labelled incorrectly, the algorithm learns incorrectly, amplifying bias. This idea of ‘garbage in, garbage out’¹⁰⁰ has been prevalent in computing since prior to the development of AI,¹⁰¹ and in the field of AI is commonly referred to as algorithm bias.¹⁰² A meta-analysis comparing the decisions of a ML software, called Watson For Oncology, to a multidisciplinary team of experts, found discordance in the decision making for treatment choices in lung cancers.¹⁰³ This was partially explained by the use of synthetic data to train the algorithm¹⁰⁴ as well as patient demographic differences.¹⁰³ Very large amount of precisely labelled real data is essential for the development of supervised ML algorithms.^5,105 It is also important that the data used to train ML algorithms is reflective of the diversity of the population. A landmark study in 2018 demonstrated that facial recognition AI developed by companies including IBM and Microsoft performed significantly worse at recognizing darker-skinned females than lighter-skinned males, with error rates up to 34.7% compared to 0.8%, respectively.¹⁰⁶ The algorithmic bias was attributed to the databases used to train the AI being disproportionately comprised of lighter-skinned subjects.^106,107

Reinforcement learning models such as ANNs, and unsupervised leaning models such as cluster analyses, often use ‘black box’ algorithms, which are AI systems in which the methodology for obtaining an output is hidden to the operator.¹⁰⁸ An argument can be made in support of block box algorithms, as theoretically it shouldn’t matter how an algorithm gets to an answer if that the answer is always correct.¹⁰⁹ The counterargument is that these algorithms don’t always produce a correct answer and that transparency is essential in bias detection.¹¹⁰ Furthermore, clinicians are also more likely to utilize AI if they are able to make sense of how the AI makes its decisions.¹¹¹

Another challenge for implementing AI in medicine is navigating liability regimens. At present, clinicians would be liable if they acted against best practice due to the advice of an AI algorithm.^99,112 In the future, if (or when) AI becomes more routine, we may have the corresponding case where clinicians could be liable if they act against the advice of the AI.

Implications for Future Research

The obvious gaps in the research identified from this review are the comparative lack of publications about AI in phenotyping or endotyping pathologies, prognostication, non-radiological diagnostics, and surgical planning and assessment in comparison to radiological diagnostics.

However, the most important gap is the shortage of large datasets which are needed to develop rigorous algorithms. As previously discussed, large datasets of precisely labelled data are needed for supervised learning models to prevent algorithm bias. Before widespread adoption of AI into rhinology clinical practice, there needs to be a collaborative effort to collect appropriate datasets that reflect the demographics of the population. National registries have been used in orthopaedics¹¹³ and cardiology¹¹⁴ for this purpose. However, with the increased volume of patients in national registries may come a decrease in the quality of the data.¹¹⁵

Conclusions

AI research is increasingly common, with publications per year from around the globe increasing at an almost exponential rate. Utilizing AI in radiological diagnosis was the most published topic of research, however, AI in rhinology is still in its infancy and there are several topics in prognostication and surgical planning yet to be thoroughly explored.

Supplemental Material

sj-docx-1-ajr-10.1177_19458924231162437 - Supplemental material for A Scoping Review of Artificial Intelligence Research in Rhinology

Supplemental material, sj-docx-1-ajr-10.1177_19458924231162437 for A Scoping Review of Artificial Intelligence Research in Rhinology by Gabriel Osie, Rhea Darbari Kaul, Raquel Alvarado, Gregory Katsoulotos, Janet Rimmer, Larry Kalish, Raewyn G. Campbell, Raymond Sacks and Richard J. Harvey in American Journal of Rhinology & Allergy

Supplemental Material

sj-docx-2-ajr-10.1177_19458924231162437 - Supplemental material for A Scoping Review of Artificial Intelligence Research in Rhinology

Supplemental material, sj-docx-2-ajr-10.1177_19458924231162437 for A Scoping Review of Artificial Intelligence Research in Rhinology by Gabriel Osie, Rhea Darbari Kaul, Raquel Alvarado, Gregory Katsoulotos, Janet Rimmer, Larry Kalish, Raewyn G. Campbell, Raymond Sacks and Richard J. Harvey in American Journal of Rhinology & Allergy

Supplemental Material

sj-docx-3-ajr-10.1177_19458924231162437 - Supplemental material for A Scoping Review of Artificial Intelligence Research in Rhinology

Supplemental material, sj-docx-3-ajr-10.1177_19458924231162437 for A Scoping Review of Artificial Intelligence Research in Rhinology by Gabriel Osie, Rhea Darbari Kaul, Raquel Alvarado, Gregory Katsoulotos, Janet Rimmer, Larry Kalish, Raewyn G. Campbell, Raymond Sacks and Richard J. Harvey in American Journal of Rhinology & Allergy

Supplemental Material

sj-docx-4-ajr-10.1177_19458924231162437 - Supplemental material for A Scoping Review of Artificial Intelligence Research in Rhinology

Supplemental material, sj-docx-4-ajr-10.1177_19458924231162437 for A Scoping Review of Artificial Intelligence Research in Rhinology by Gabriel Osie, Rhea Darbari Kaul, Raquel Alvarado, Gregory Katsoulotos, Janet Rimmer, Larry Kalish, Raewyn G. Campbell, Raymond Sacks and Richard J. Harvey in American Journal of Rhinology & Allergy

Supplemental Material

sj-docx-5-ajr-10.1177_19458924231162437 - Supplemental material for A Scoping Review of Artificial Intelligence Research in Rhinology

Supplemental material, sj-docx-5-ajr-10.1177_19458924231162437 for A Scoping Review of Artificial Intelligence Research in Rhinology by Gabriel Osie, Rhea Darbari Kaul, Raquel Alvarado, Gregory Katsoulotos, Janet Rimmer, Larry Kalish, Raewyn G. Campbell, Raymond Sacks and Richard J. Harvey in American Journal of Rhinology & Allergy

Supplemental Material

sj-docx-6-ajr-10.1177_19458924231162437 - Supplemental material for A Scoping Review of Artificial Intelligence Research in Rhinology

Supplemental material, sj-docx-6-ajr-10.1177_19458924231162437 for A Scoping Review of Artificial Intelligence Research in Rhinology by Gabriel Osie, Rhea Darbari Kaul, Raquel Alvarado, Gregory Katsoulotos, Janet Rimmer, Larry Kalish, Raewyn G. Campbell, Raymond Sacks and Richard J. Harvey in American Journal of Rhinology & Allergy

Footnotes

Author Contributions

GO: formulation of research question, development of search strategy, study selection, data extraction, writing of manuscript, editing, accepting final version of manuscript. RDK: development of search strategy, study selection, data extraction, editing, accepting final version of manuscript. RA, GK, RGC, LK, JR, RS: development of search strategy, data extraction, editing, accepting final version of manuscript. RJH: formulation of research question, development of search strategy, data extraction, editing, accepting final version of manuscript.

Declaration of Conflicting Interests

The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Richard J Harvey is consultant/advisory board with Medtronic, Novartis, GSK and Meda pharmaceuticals. Research grant funding received from Glaxo-Smith-Kline. He has been on the speakers’ bureau for Glaxo-Smith-Kline, Astra-zeneca, Meda Pharmaceuticals and Seqirus. Janet Rimmer has honoraria with Sanofi Aventis, Novartis, Mundipharma, BioCSL, Stallergenes. Larry Kalish is on the speakers’ bureau for Care Pharmaceuticals, Mylan and Seqirus Pharmaceuticals. Raymond Sacks is a consultant for Medtronic and is in the speaker bureau for Meda Pharmaceuticals. All other authors have no financial disclosures or conflicts of interest.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Gabriel Osie

Supplemental Material

Supplemental material for this article is available online.

References

Titano

Badgeley

Schefflein

, et al. Automated deep-neural-network surveillance of cranial images for acute neurologic events. Nat Med. 2018;24(9):1337–1341.

Siontis

Noseworthy

Attia

Friedman

. Artificial intelligence-enhanced electrocardiography in cardiovascular disease management. Nat Rev Cardiol. 2021;18(7):465–478.

Briganti

Le Moine

. Artificial intelligence in medicine: today and tomorrow. Front Med 2020;7(27).

Vettoretti

Cappon

Facchinetti

Sparacino

. Advanced diabetes management using artificial intelligence and continuous glucose monitoring sensors. Sensors (Basel). 2020;20(14):3870.

Taulli

. Artificial Intelligence Basics: A Non-Technical Introduction. Apress; 2019.

Coursera. 3 types of machine learning you should know: Coursera; 2022. https://www.coursera.org/articles/types-of-machine-learning.

Osinski

Budek

. What is reinforcement learning? The complete guide: Deepsense.ai; 2018. https://deepsense.ai/what-is-reinforcement-learning-the-complete-guide/.

Marr

. What are artificial neural networks - a simple explanation for absolutely anyone: Forbes; 2018. https://www.forbes.com/sites/bernardmarr/2018/09/24/what-are-artificial-neural-networks-a-simple-explanation-for-absolutely-anyone/?sh=368f5c191245.

Kavlakoglu

. AI vs. machine learning vs. deep learning vs. neural networks: what’s the difference?: IBM; 2020. https://www.ibm.com/cloud/blog/ai-vs-machine-learning-vs-deep-learning-vs-neural-networks.

10.

Valueva

Nagornov

Lyakhov

Valuev

Chervyakov

. Application of the residue number system to reduce hardware costs of the convolutional neural network implementation. Math Comput Simul. 2020;177:232–243.

11.

IBM Cloud Education. Natural Language Processing (NLP): IBM; 2020. https://www.ibm.com/cloud/learn/natural-language-processing.

12.

Tricco

Lillie

Zarin

, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–473.

13.

Tang

Ding

Song

. The pace of artificial intelligence innovations: speed, talent, and trial-and-error. J Informetr. 2020;14(4):101094.

14.

Peters

MDJ

Godfrey

Khalil

McInerney

Parker

Soares

. Guidance for conducting systematic scoping reviews. JBI Evid. Implement.. 2015;13(3):141–146.

15.

Ouzzani

Hammady

Fedorowicz

Elmagarmid

. Rayyan—a web and mobile app for systematic reviews. Syst Rev. 2016;5(1):210.

16.

Microsoft Excel. Microsoft Corporation; 2022.

17.

Adnane

Adouly

Khallouk

, et al. Using preoperative unsupervised cluster analysis of chronic rhinosinusitis to inform patient decision and endoscopic sinus surgery outcome. Eur Arch Otorhinolaryngol. 2017;274(2):879–885.

18.

Divekar

Rank

Squillace

Kita

Lal

. Unsupervised network mapping of commercially available immunoassay yields three distinct chronic rhinosinusitis endotypes. Int Forum Allergy Rhinol. 2017;7(4):373–379.

19.

Ramkumar

Ranjbar

Ning

, et al. MRI-based texture analysis to differentiate sinonasal squamous cell carcinoma from inverted papilloma. AJNR Am J Neuroradiol. 2017;38(5):1019–1025.

20.

Caimmi

Baiz

Sanyal

Banerjee

Demoly

Annesi-Maesano

. Discriminating severe seasonal allergic rhinitis. Results from a large nation-wide database. PLoS One. 2018;13(11):e0207290.

21.

Giacomini

Pavan

ALM

Altemani

JMC

, et al. Computed tomography-based volumetric tool for standardized measurement of the maxillary sinus. PLoS One. 2018;13(1):e0190770.

22.

Gibelli

Cellina

Gibelli

, et al. Volumetric assessment of sphenoid sinuses through segmentation on CT scan. Surg Radiol Anat. 2018;40(2):193–198.

23.

Lal

Hopkins

Divekar

. SNOT-22-based clusters in chronic rhinosinusitis without nasal polyposis exhibit distinct endotypic and prognostic differences. Int Forum Allergy Rhinol. 2018;8(7):797–805.

24.

Jing

, et al. Development and validation of an endoscopic images-based deep learning model for detection with nasopharyngeal malignancies 08 Information and Computing Sciences 0801 Artificial Intelligence and Image Processing. Cancer Commun. 2018;38(1):59.

25.

Laura

Hofmann

Drechsler

Wesarg

Automatic detection of the nasal cavities and paranasal sinuses using deep neural networks. 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019); 2019:1154–1157.

26.

Chowdhury

Smith

Chandra

Turner

. Automated classification of osteomeatal complex inflammation on computed tomography using convolutional neural networks. Int Forum Allergy Rhinol. 2019;9(1):46–52.

27.

Fujima

Shimizu

Yoshida

, et al. Machine-learning-based prediction of treatment outcomes using MR imaging-derived quantitative tumor information in patients with sinonasal squamous cell carcinomas: a preliminary study. Cancers 2019;11(6):800.

28.

Kim

Huh

Rhee

, et al. Unsupervised cluster analysis of chronic rhinosinusitis with nasal polyp using routinely available clinical markers and its implication in treatment outcomes. Int Forum Allergy Rhinol. 2019;9(1):79–86.

29.

Kim

Lee

Sunwoo

, et al. Deep learning in diagnosis of maxillary sinusitis using conventional radiography. Invest Radiol. 2019;54(1):7–15.

30.

Morse

Shilts

Ely

, et al. Patterns of olfactory dysfunction in chronic rhinosinusitis identified by hierarchical cluster analysis and machine learning algorithms. Int Forum Allergy Rhinol. 2019;9(3):255–264.

31.

Murata

Ariji

Ohashi

, et al. Deep-learning classification using convolutional neural network for evaluation of maxillary sinusitis on panoramic radiography. Oral Radiol. 2019;35(3):301–307.

32.

Arfiani

Rustam Z

Pandelaki

Siahaan

Kernel spherical K-means and support vector machine for acute sinusitis classification. IOP Conf Ser: Mater Sci Eng. 2019;546(5):052011.

33.

Liu

Sinha

Ishii

, et al. Dense depth estimation in monocular endoscopy with self-supervised learning methods. IEEE Trans Med Imaging. 2020;39(5):1438–1447.

34.

Bieck

Heuermann

Pirlich

Neumann

Neumuth

. Language-based translation and prediction of surgical navigation steps for endoscopic wayfinding assistance in minimally invasive surgery. Int J Comput Assist Radiol Surg. 2020;15(12):2089–2100.

35.

Borsting

DeSimone

Ascha

. Applied deep learning in plastic surgery: classifying rhinoplasty with a mobile app. J Craniofac Surg. 2020;31(1):102–106.

36.

Chowdhury

Chandra

Turner

. Baseline mucus cytokines predict 22-item Sino-Nasal Outcome Test results after endoscopic sinus surgery. Int Forum Allergy Rhinol. 2020;10(1):15–22.

37.

Diao

Hou

, et al. Computer-aided pathologic diagnosis of nasopharyngeal carcinoma based on deep learning. Am J Pathol. 2020;190(8):1691–1700.

38.

Huang

Habib

Mendis

, et al. An artificial intelligence algorithm that differentiates anterior ethmoidal artery location on sinus computed tomography scans. J Laryngol Otol. 2020;134(1):52–55.

39.

Humphries

Centeno

Notary

, et al. Volumetric assessment of paranasal sinus opacification on computed tomography can be automated using a convolutional neural network. Int Forum Allergy Rhinol. 2020;10(11):1218–1225.

40.

Deng

Xia

, et al. Development of a self-constrained 3D DenseNet model in automatic detection and segmentation of nasopharyngeal carcinoma using magnetic resonance images. Oral Oncol. 2020;110:104862.

41.

Kuo

CFJ

Leu

Huang

Siao

Leon

KBP

. Application of intelligent automatic segmentation and 3D reconstruction of inferior turbinate and maxillary sinus from computed tomography and analyze the relationship between volume and nasal lesion. Biomed Signal Process Control. 2020;57:101660.

42.

Parmar

Habib

Mendis

, et al. An artificial intelligence algorithm that identifies middle turbinate pneumatisation (concha bullosa) on sinus computed tomography scans. J Laryngol Otol. 2020;134(4):328–331.

43.

Przystanska

Rewekant

Sroka

, et al. Sexual dimorphism of maxillary sinuses in children and adolescents - a retrospective CT study. Ann Anat. 2020;229:151437.

44.

Soloviev

Khilov

Shakhova

, et al. Machine learning aided automated differential diagnostics of chronic rhinitis based on optical coherence tomography. Laser Phys Lett. 2020;17(11):115608.

45.

Wang

Zhou

Liu

Jiang

Chen

. Automatic CT image segmentation of maxillary sinus based on VGG network and improved V-Net. Int J Comput Assist Radiol Surg. 2020;15(9):1457–1465.

46.

Massey

Beswick

Ramakrishnan

Humphries

. Clinical validation of an automated deep- learning-based algorithm for quantitative sinus computed tomography analysis. Otolaryngol–Head Neck Surg. 2020;163(1_suppl):135.

47.

Rustam

Hartini

Pandelaki

. Kernel perceptron algorithm for sinusitis classification. J Phys Conf Ser. 2020;1490(1):012025.

48.

Wong

King

QYH

, et al. Convolutional neural network for discriminating nasopharyngeal carcinoma and benign hyperplasia on MRI. Eur Radiol. 2021;31(6):3856–3863.

49.

Chen

Qin

Cheng

Gao

Zhou

. Texture analysis of fat-suppressed T2-weighted magnetic resonance imaging and use of machine learning to discriminate nasal and paranasal sinus small round malignant cell tumors. Front Oncol. 2021;11:701289.

50.

Desser

Assuncao

Yan

Alves

Fernandes

Hummel

. Automatic segmentation of the olfactory bulb. Brain Sci. 2021;11(9):1141.

51.

Estrada

Diers

, et al. Automated olfactory bulb segmentation on high resolutional T2-weighted MRI. Neuroimage. 2021;242:118464.

52.

Girdler

Moon

Bae

Ryu

Bae

. Feasibility of a deep learning-based algorithm for automated detection and classification of nasal polyps and inverted papillomas on nasal endoscopic images. Int Forum Allergy Rhinol. 2021;11(12):1637–1646.

53.

Huang

Wen

Guo

, et al. Prenatal exposure to air pollutants and childhood atopic dermatitis and allergic rhinitis adopting machine learning approaches: 14-year follow-up birth cohort study. Sci Total Environ. 2021;777:145982.

54.

Jeon

Lee

Sunwoo

, et al. Deep learning for diagnosis of paranasal sinusitis using multi-view radiographs. Diagnostics (Basel). 2021;11(2):250.

55.

Jung

Lim

Lee

Cho

Song

. Deep active learning for automatic segmentation of maxillary sinus lesions using a convolutional neural network. Diagnostics. 2021;11(4):688.

56.

Khan

Luengo

Barbarisi

, et al. Automated operative workflow analysis of endoscopic pituitary surgery using machine learning: development and preclinical evaluation (IDEAL stage 0). Brain and Spine. 2021;1(Suppl 2):100580.

57.

Kim

Lim

Eun

, et al. Subepithelial neutrophil infiltration as a predictor of the surgical outcome of chronic rhinosinusitis with nasal polyps. Rhinology. 2021;59(2):173–180.

58.

Kuwana

Ariji

Fukuda

, et al. Performance of deep learning object detection technology in the detection and diagnosis of maxillary sinus lesions on panoramic radiographs. Dentomaxillofac Radiol. 2021;50(1):20200171.

59.

Leonardi

Lo Giudice

Farronato

, et al. Fully automatic segmentation of sinonasal cavity and pharyngeal airway based on convolutional neural networks. Am J Orthod Dentofacial Orthop. 2021;159(6):824–835.e1.

60.

Lotsch

Hintschich

Petridis

Pade

Hummel

. Machine-learning points at endoscopic, quality of life, and olfactory parameters as outcome criteria for endoscopic paranasal sinus surgery in chronic rhinosinusitis. J Clin Med. 2021;10(18):4245.

61.

Lotsch

Hummel

. Data science-based analysis of patient subgroup structures suggest effects of rhinitis on all chemosensory perceptions in the upper airways. Chem Senses. 2021;46:1–13.

62.

Mori

Ariji

Katsumata

, et al. A deep transfer learning approach for the detection and diagnosis of maxillary sinusitis on panoramic radiographs. Odontology. 2021;109(4):941–948.

63.

Ogawa

Osaga

Shiraki

, et al. Utility of CT texture analysis to differentiate olfactory neuroblastoma from sinonasal squamous cell carcinoma. Sci Rep. 2021;11(1):4679.

64.

Kim

Lee

, et al. Effective end-to-end deep learning process in medical imaging using independent task learning: application for diagnosis of maxillary sinusitis. Yonsei Med J. 2021;62(12):1125–1135.

65.

Parsel

Riley

Todd

Thomas

McCoul

. Differentiation of clinical patterns associated with rhinologic disease. Am J Rhinol Allergy. 2021;35(2):179–186.

66.

Abu-Hanna

van Esch

TEM

, et al. Explaining heterogeneity of individual treatment causal effects by subgroup discovery: an observational case study in antibiotics treatment of acute rhino-sinusitis. Artif Intell Med. 2021;116:102080.

67.

Chen

, et al. Computer-aided diagnosis and regional segmentation of nasopharyngeal carcinoma based on multi-modality medical images. Int J Comput Assist Radiol Surg. 2021;16(6):871–882.

68.

Ramakrishnan

Arbet

Mace

, et al. Predicting olfactory loss in chronic rhinosinusitis using machine learning. Chem Senses. 2021;46:1–9.

69.

Ren

Tian

. Deep learning framework for preoperative recognition of inverted papilloma and nasal polyp. IEEE Access. 2021;9:120502–11.

70.

Shu

Yan

Zheng

, et al. Deep learning-guided fiberoptic Raman spectroscopy enables real-time in vivo diagnosis and assessment of nasopharyngeal carcinoma and post-treatment efficacy during endoscopy. Anal Chem. 2021;93(31):10898–10906.

71.

Staartjes

Volokitin

Regli

Konukoglu

Serra

. Machine vision for real-time intraoperative anatomic guidance: a proof-of-concept study in endoscopic pituitary surgery. Oper Neurosurg (Hagerstown). 2021;21(4):242–247.

72.

Thorwarth

Scott

Lal

Marino

. Machine learning of biomarkers and clinical observation to predict eosinophilic chronic rhinosinusitis: a pilot study. Int Forum Allergy Rhinol. 2021;11(1):8–15.

73.

Tong

Liu

, et al. Real-to-virtual domain transfer-based depth estimation for real-time 3D annotation in transnasal surgery: a study of annotation accuracy and stability. Int J Comput Assist Radiol Surg. 2021;16(5):731–739.

74.

Wang

Pan

Zhao

Gao

Liu

. Hard frame detection for the automated clipping of surgical nasal endoscopic video. Int J Comput Assist Radiol Surg. 2021;16(2):231–240.

75.

Wang

Pastewait

, et al. 3D Morphometric quantification of maxillae and defects for patients with unilateral cleft palate via deep learning-based CBCT image auto-segmentation. Orthod Craniofac Res. 2021;24(Suppl 2):108–116.

76.

Chen

Ren

, et al. Artificial intelligence for cellular phenotyping diagnosis of nasal polyps by whole-slide imaging. EBioMedicine. 2021;66:103336.

77.

Wildfeuer

Kunz

Bieck

Sorge

Pirlich

. Clinical evaluation of keyword-based, computer-generated reports of sinus operations. Laryngorhinootologie. 2021;100(0):S32.

78.

Postma

Noothout

JMH

Boek

, et al. Applying olfactory bulb volume in the clinic: relating clinical outcome measures to olfactory bulb volume using convolutional neural networks. Chem Senses. 2021;46:39.

79.

Rustam

Shandri

Siswantining

Pandelaki

. Kernel entropy based fuzzy C-means (KEFCM) for acute sinusitis. J Phys Conf Ser. 2021;1752:012040.

80.

Deng

, et al. The contrast-enhanced MRI can be substituted by unenhanced MRI in identifying and automatically segmenting primary nasopharyngeal carcinoma with the aid of deep learning models: an exploratory study in large-scale population of endemic area. Comput Methods Programs Biomed. 2022;217:106702.

81.

Hung

QYH

King

Bornstein

Wong

Leung

. Automatic detection and segmentation of morphological changes of the maxillary sinus mucosa on cone-beam computed tomography images using a three-dimensional convolutional neural network. Clin Oral Investig. 2022;26(5):3987–3998.

82.

Kim

Chung

Cho

Jung

. Detection of maxillary sinus fungal ball via 3-D CNN-based artificial intelligence: Fully automated system and clinical validation. PLoS One. 2022;17(2):e0263125.

83.

Kuo

Liao

Barman

Liu

. Semi-supervised deep learning semantic segmentation for 3D volumetric computed tomographic scoring of chronic rhinosinusitis: clinical correlations and comparison with Lund-Mackay scoring. Tomography. 2022;8(2):718–729.

84.

Hua

, et al. Anatomical partition-based deep learning: an automatic nasopharyngeal MRI recognition scheme. J Magn Reson Imaging. 2022;56(4):1220–1229.

85.

Lim

Kim

, et al. Aux-MVNet: auxiliary classifier-based multi-view convolutional neural network for maxillary sinusitis diagnosis on paranasal sinuses view. Diagnostics (Basel). 2022;12(3):736.

86.

Liu

Yang

Kim

, et al. Deep learning classification of inverted papilloma malignant transformation using 3D convolutional neural networks and magnetic resonance imaging. Int Forum Allergy Rhinol. 2022;12(8):1025–1033.

87.

Liu

Zhang

, et al. Microvessel quantification by fully convolutional neural networks associated with type 2 inflammation in chronic rhinosinusitis. Ann Allergy Asthma Immunol. 2022;128(6):697–704.e1.

88.

Nuutinen

Haukka

Virkkula

Torkki

Toppila-Salmi

. Using machine learning for the personalised prediction of revision endoscopic sinus surgery. PLoS One. 2022;17(4):e0267146.

89.

Seol

Kim

Cheon

Kim

. A study on 3D deep learning-based automatic diagnosis of nasal fractures. Sensors (Basel). 2022;22(2):506.

90.

Tuan

HNA

Hai

NDX

Thinh

. Shape prediction of nasal bones by digital 2D-photogrammetry of the nose based on convolution and back-propagation neural network. Comput Math Methods Med. 2022;2022:5938493.

91.

Waldmann

Grosch

Witzler

, et al. An effective simulation- and measurement-based workflow for enhanced diagnostics in rhinology. Med Biol Eng Comput. 2022;60(2):365–391.

92.

Wen

Fan

, et al. Human identification performed with skull's sphenoid sinus based on deep learning. Int J Legal Med. 2022;136(4):1067–1074.

93.

Xie

Udupa

Tong

, et al. Automatic upper airway segmentation in static and dynamic MRI via anatomy-guided convolutional neural networks. Med Phys. 2022;49(1):324–342.

94.

Wang

Bian

, et al. Deep learning for nasopharyngeal carcinoma identification using both white light and narrow-band imaging endoscopy. Laryngoscope. 2022;132(5):999–1007.

95.

Dursun

Yildiz

Ozturk

Senel

. Success of the Keros classification in paranasal sinus CT with deep learning method. Neuroradiology. 2022;64(2):416.

96.

Hamilton

. Google's DeepMind created an AI for spotting breast cancer that can outperform human radiologists: Business Insider Africa; 2020. https://africa.businessinsider.com/googles-deepmind-created-an-ai-for-spotting-breast-cancer-that-can-outperform-human/w9n29x6.

97.

The Medical Futurist. Top artificial intelligence companies in healthcare to keep an eye on 2023. https://medicalfuturist.com/top-artificial-intelligence-companies-in-healthcare/.

98.

Basu

Sinha

Ong

Basu

. Artificial intelligence: how is it changing medical sciences and its future? Indian J Dermatol. 2020;65(5):365–370.

99.

Gerke

Minssen

Cohen

Ethical and legal challenges of artificial intelligence-driven healthcare. Artif Intell Healthcare. 2020:295–336. Epub 2020 Jun 26. doi:10.1016/B978-0-12-818438-7.00012-5.

100.

Geiger

Cope

, et al.

“Garbage in, garbage out” revisited: what do machine learning application papers report about human-labeled training data?

Quant Sci Stud. 2021;2(3):795–827.

101.

Mellin

. Work with new electronic ‘brains’ opens field for army math experts. Hammond Times. 1957;10:66.

102.

Igoe

. Algorithmic bias in health care exacerbates social inequities — how to prevent it: Harvard; 2021. https://www.hsph.harvard.edu/ecpe/how-to-prevent-algorithmic-bias-in-health-care/.

103.

Jie

Zhiying

. A meta-analysis of Watson for oncology in clinical application. Sci Rep. 2021;11(1):5792.

104.

Brown

IBM Watson reportedly recommended cancer treatments that were ‘unsafe and incorrect’. Gizmodo. 2018.

105.

Datagen. The danger of hand labeling your data: Datagen; 2021. https://datagen.tech/blog/the-danger-of-hand-labeling-your-data/.

106.

Buolamwini

Gebru

, eds. Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. FAT; 2018.

107.

Perkowitz

The bias in the machine: facial recognition technology and racial disparities. MIT Case Studies in Social and Ethical Responsibilities of Computing. 2021;Winter 2021.

108.

Wigmore

. Black box AI: TechTarget; 2019. https://www.techtarget.com/whatis/definition/black-box-AI.

109.

London

. Artificial intelligence and black-box medical decisions: accuracy versus explainability. Hastings Cent Rep. 2019;49(1):15–21.

110.

Rudin

Radin

. Why are we using black box models in AI when we don’t need to? A lesson from an explainable AI competition. Harvard Data Sci Rev. 2019;1(2).

111.

Poon

AIF

Sung

JJY

. Opening the black box of AI-medicine. J Gastroenterol Hepatol. 2021;36(3):581–584.

112.

Maliha

Gerke

Cohen

Parikh

. Artificial intelligence and liability in medicine: balancing safety and innovation. Milbank Q. 2021;99(3):629–647.

113.

Kita

Uemura

Takao

, et al. Use of artificial intelligence to identify data elements for The Japanese Orthopaedic Association National Registry from operative records. J Orthop Sci. 2022.

114.

Choi

Kim

Sung

, et al. On behalf of the connected network for EMS comprehensive technical support using artificial intelligence investigators. Development of prediction models for acute myocardial infarction at prehospital stage with machine learning based on a nationwide database. J Cardiovasc Dev Dis. 2022;9(12):430.

115.

Khera

Haimovich

Hurley

, et al. Use of machine learning models to predict death after acute myocardial infarction. JAMA Cardiol. 2021;6(6):633–641.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB

0.01 MB