Sage Journals: Discover world-class research

Abstract

Background

Radiomics is a quantitative approach to medical imaging, aimed to extract features into large datasets. By using artificial intelligence (AI) methodologies, large radiomic data can be analysed and translated into meaningful clinical applications. In rhinology, there is heavy reliance on computed tomography (CT) imaging of the paranasal sinus for diagnostics and assessment of treatment outcomes. Currently, there is an emergence of literature detailing radiomics use in rhinology.

Objective

This systematic review aims to assess the current techniques used to analyze radiomic data from paranasal sinus CT imaging.

Methods

A systematic search was performed using Ovid MEDLINE and EMBASE databases from January 1, 2019 until March 16, 2024 using the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) checklist and Cochrane Library Systematic Reviews for Diagnostic and Prognostic Studies. The QUADAS-2 and PROBAST tools were utilized to assess risk of bias.

Results

Our search generated 1456 articles with 10 articles meeting eligibility criteria. Articles were divided into 2 categories, diagnostic (n = 7) and prognostic studies (n = 3). The number of radiomic features extracted ranged 4 to 1409, with analysis including non-AI-based statistical analyses (n = 3) or machine learning algorithms (n = 7). The diagnostic or prognostic utility of radiomics analyses were rated as excellent (n = 3), very good (n = 2), good (n = 2), or not reported (n = 3) based upon area under the curve receiver operating characteristic (AUC-ROC) or accuracy. The average radiomics quality score was 36.95%.

Conclusion

Radiomics is an evolving field which can augment our understanding of rhinology diseases, however there are currently only minimal quality studies with limited clinical utility.

Keywords

artificial intelligence diagnostics prognostics radiomics rhinology machine learning sinus computed tomography chronic rhinosinusitis nasopharyngeal carcinoma

Introduction

The role of medical imaging is currently also swiftly evolving from a diagnostic tool to assessed by human beings medical experts to a computational and data-analysis by automated computerized methods.^1,2 Techniques such as “radiomics” which allow the extraction of quantitative features from medical imaging are now being used to produce large datasets for clinical problem solving.³ Radiomics can capture tissue and lesion characteristics and features such as contrast, heterogeneity, and shape.⁴ Broadly speaking, radiomic features can be divided into multiple categories based upon types of features.^4–7 First-order features including histogram-based properties, for example, gray-level mean, median, and range for individual voxels.^4–7 Second-order features including textural features, for example, statistical interrelationships between neighboring voxels related to intralesion heterogeneity.^4–7 Higher order features are related to patterns analysis through statistical methods or mathematical transformations.^4–7 Artificial intelligence (AI) techniques offer analysis of this data and has already been established in some areas of medical imaging to potentially augment future radiologists’ practices.² Following radiomics-based data extraction, large datasets can be analyzed through conventional statistical methods and/or AI methodologies (eg, machine learning algorithms).^5,8–10 In medicine, there has been a rapid expansion in the use of radiomics and AI methods in consideration of their ability to identify tumor genotypes and imaging biomarkers with clinical relevance.^5,7–10 In the field of rhinology, which heavily relies on the use of computed tomography (CT) imaging of the paranasal sinus for diagnostics and assessment of treatment outcomes, there are few applications of radiomic analysis to common conditions.¹¹ This systematic review aims to assess the current techniques used to analyze radiomic data from CT images of the paranasal sinuses in rhinology.

Methods

A systematic review of the literature on the assessment of radiomics in CT paranasal sinus imaging was conducted. The Preferred Reporting Items for Systematic Reviews and Meta-analyses 2020 Checklist (PRISMA 2020) and Cochrane Library Systematic Reviews for Diagnostic and Prognostic Studies were referenced for this review.^11–13

Eligibility Criteria

Eligible studies described their approach to radiomics in paranasal sinus CT imaging for feature and pattern recognition in the field of rhinology. Articles included adults 18 years of age and above and were published in the past 5 years (from 2019 to current) to capture up-to-date literature in this rapidly evolving field of research. Gray literature including case reports, conference, and poster abstracts were excluded. Systematic reviews or literature reviews were utilized in bibliographic searching for further relevant articles for inclusion. Articles were excluded if they did not utilize radiomics and only utilized deep learning/machine learning. Articles which utilized other imaging modalities, that is, positron emission tomography (PET), magnetic reference imaging (MRI), X-ray or detailed regions not included in a CT paranasal sinus scan were excluded. Articles which were used for forensics, dental, or radiotherapy treatment were excluded to focus on papers relevant to rhinology.

Information Sources

A systematic electronic search was performed for relevant studies using the Ovid MEDLINE and EMBASE databases from January 1, 2019 until the March 16, 2024 using a defined search strategy.

Search Strategy

Radiomics terms including “RADIOMIC” and “textur* adj2 analys*”, Rhinology terms including “NOSE”, “Rhin*” and investigation terms including “CT” and “Comput* adj2 Tomograph*” were combined using the Boolean operators (AND and OR) for the search strategy (see Table 1). A manual bibliographic screen from the relevant studies and systematic reviews was performed to search for additional relevant articles.

Table 1.

Search Strategy.

Radiomic/imaging terms	Imaging terms	Rhinology terms
1. RADIOMICS	17. CT	22. Rhin*
2. Textur* adj2 analys*	18. CAT adj2 scan	23. Sinonas*
3. Textur* adj2 feature*	19. Comput* adj2 Tomograph*	24. Meatal*
4. Feature adj2 selection	20. COMPUTER ASSISTED TOMOGRAPHY	25. Meatus*
5. Feature adj2 extraction	21. Cone adj2 beam*	26. Nasal*
6. Imaging adj2 analys*		27. Naso*
7. Image adj2 segement*		28. NOSE
8. Automat*		29. Nose*
9. COMPUTER ANALYSIS		30. (Ethmoid* OR Frontal* OR Maxil* OR Spheno) adj2 sinus
10. COMPUTER VISION		31. AERD
11. Computation*		32. ASPIRIN-EXACERBATED RESPIRATORY DISEASE
12. Pixel*		33. AFRS
13. Voxel*		34. AFS
14. Hounsfield*		35. allergic adj2 fung*
15. Gray adj2 level*		36. CCAD
16. Grey adj2 level*		37. Central compartment*
		38. RHINOSINUSITIS
		39. ACUTE RHINOSINUSITIS
		40. CHRONIC RHINOSINUSITIS
		41. SINUSITIS
		42. CRS*
		43. Chronic adj2 rhinosinusitis
		44. eCRS*
		45. fung* adj2 ball
		46. fung* adj2 mycetoma
		47. silent adj2 sinus*
		48. nasal adj2 polyp*
		49. NOSE POLYP
		50. Invert* adj2 papilloma*
	51. 1-16
	52. 17-21
	53. 22-50
	54. 51 AND 52 AND 53
	55. Limit 54 to 2019-currrent

Table of terms used in MEDLINE (2017–2024) and EMBASE (2017–2024) databases. Search performed on March 16, 2024.

Selection Process

OVID was utilized to search the MEDLINE and EMBASE databases. Studies were exported to Rayyan (Qatar Computing Research Institute, Qatar), an online review website.¹⁴ Duplicate studies were automatically removed using the Rayyan duplicate removal function, and also manually reviewed by the author. Study selection was performed by the author (RDK). Any uncertainties were resolved by 2 independent authors (RJH and LK). Studies were screened in 3 phases: first by title, then by abstract, and finally by full text. Articles that met the eligibility criteria were included for data extraction.

Data Extraction

Data extracted from individual studies were recorded by one author (RDK) in Microsoft Excel. Data fields collected included:

- Study details: authors, year of publication, study design, and country of publication;

- Category of research, that is, diagnostic or prognostic studies;

- Pathology of interest;

- Participant details: number of participants, and the distribution with or without disease of interest;

- Radiomics utilized including software used, number of features, feature selection methods;

- Analysis methods utilized including classification models or validation models; and

- Outcome measures including diagnostic or prognostic performance as applicable, that is, area under the curve receiver operating characteristic (AUC-ROC), sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy for the diagnostic studies, and AUC-ROC, C statistic, calibration slope, and calibration intercept for the prognostic studies.

Study risk of bias assessment: The QUADAS-2 tool for diagnostic studies and the PROBAST tool for prognostic studies were used as per the Cochrane systematic reviews for diagnostic studies and prognostic studies handbook.^15,16

Level of evidence: The level of evidence for each study was recorded as per the Oxford Centre for Evidence-Based Medicine 2011 Levels of Evidence table.¹⁷

Radiomics quality assessment: The radiomics quality scores (RQS) were calculated for the included studies, as a published and previously validated score used to assess the methodology, analysis, and reporting of radiomic studies.^1,18 This questionnaire comprises of 6 domains (image protocol, radiomics features extraction, data analysis and statistics, model validation, clinical validity, and open science) with 16 items altogether rated up to a total of 36 points, with 36 points (100%) indicating optimal quality. Currently, there are no available score ranges to characterize a rating of quality in the RQS, however we note that similar radiomics quality scoring tools such as METRICS which are yet to be validated have utilized; 0 ≤ score < 20%, “very low”; 20 ≤ score < 40%, “low”; 40 ≤ score < 60%, “moderate”; 60 ≤ score < 80%, “good”; and 80 ≤ score ≤ 100%, “excellent” quality.¹⁹ One author (RDK) reported the RQS for each study using the online tool.¹⁸

Effect measures: To assess diagnostic and prognostic accuracy, studies reported varying metrics. The most reported metrics for diagnostic studies were the AUC-ROC and accuracy, while the most common for prognostic studies were AUC-ROC and the C statistic. This systematic review used a diagnostic and prognostic utility scoring based on AUC-ROC or accuracy (see Figure 1).²⁰ Due to the heterogeneity of the studies included, a meta-analysis was not performed.

Figure 1.

Diagnostic/prognostic utility score. Scoring used for included studies measured with either area under the curve receiver operating characteristic (AUC-ROC) or accuracy.

Results

Study Selection

The search strategy on MEDLINE and EMBASE resulted in 1450 studies for review. After removal of duplicates, the total was 1029 studies. Following title screening, there were 74 studies and post abstract screening, there were 13 studies identified for full text review. An additional 6 studies were included in the search through bibliographic screening. This resulted in a total of 19 studies for full text review. Of the 19 studies, 5 did not utilize radiomics, 1 was a conference abstract only, and 3 utilized other imaging modailities. These articles were excluded, leaving 10 included studies (see Figure 2).

Figure 2.

Preferred reporting items for systematic reviews and meta-analyses (PRISMA) flow chart. Study selection process for this review.

Study Characteristics

The 10 included studies were categorized into 2 broad areas; diagnostic or prognostic studies (see Table 2). Studies were all recent, with publication in 2019 (n = 1)²¹, 2021 (n = 2)^22,23, 2022 (n = 4)^24–27, and 2023 (n = 3)^28–30. Most included studies were diagnostic cross-sectional studies (n = 6)^{22,24,26,28–30}, along with case series (n = 2)^21,27 and retrospective cohort studies (n = 2)^23,25. The number of radiomic features utilized varied from 4 to 1409, with 2 studies using feature selection methods.^27,29 Machine learning-based analysis was utilized in 7 studies^{21–23,25,27,29,30}, with the methods including Least Absolute Shrinkage and Selection Operator (LASSO) logistic regression (n = 2)^23,30, logistic regression alone (n = 1)²⁵, LASSO with ElasticNet (n = 1)²², unsupervised clustering (n = 1)²¹, and a combination of classification models (n = 2)^27,29. Cross validation (CV) models were used in 5 studies^{22,23,25,29,30}, with 10-fold CV most frequently used in 3 studies^23,29,30.

Table 2.

Table of Characteristics of Included Studies.

Category	Author	Year	Country	Design	Pathology	Subjects	Split of pathologies	Software used	No. of radiomic features	Feature selection methods	Use of AI methods	Classification model	CV model
Diagnostics	Baracnovic et al²²	2022	Serbia	Cross-sectional	Allergic Fungal rhinosinusitis versus non-allergic Fungal rhinosinusitis	37	24:13:00	Image J	4	N/A	No	N/A	N/A
Diagnostics	Costa et al²³	2023	Brazil	Cross-sectional	Odontogenic versus Non-odontogenic maxillary sinusitis	40	20:20	MaZda	11	N/A	No	N/A	N/A
Diagnostics	Guo et al²⁴	2023	China	Cross-sectional	Nasal polyps versus Inverting papilloma	296	144: 152	Python	1288	Boruta, Random Forest, Correlation coefficient	Yes	SVM, Naïve Bayes, XGBoost	10 fold CV
Diagnostics	Ito et al²⁵	2022	Japan	Cross-sectional	Odontogenic versus Non-odontogenic maxillary sinusitis	36	18:18	MATLAB	45	N/A	No	N/A	N/A
Diagnostics	Liang et al²⁷	2019	Singapore	Case series	Nasopharyngeal carcinoma	100	N/A	PyRadiomics, Moddicom	70	N/A	Yes	Unsupervised clustering	Nil
Diagnostics	Ogawa et al²⁸	2021	Japan	Cross-sectional	Olfactory neuroblastoma versus Sinonasal squamous cell carcinoma	43	17:26	SlicerRadiomics	42	N/A	Yes	LASSO, Elastic Net	Leave one out cross validation
Diagnostics	Zhu et al²⁹	2023	China	Cross-sectional	Eosinophilic CRS with nasal polyps versus Non-eosinophilic CRS with nasal polyps	431	N/A	PyRadiomics	10	N/A	Yes	LASSO-LR	10 fold CV
Prognostication	Intarak et al³⁰	2022	Thailand	Retrospective cohort	Nasopharyngeal carcinoma	197	N/A	PyRadiomics	842	N/A	Yes	Logistic regression	5 fold CV
Prognostication	Li et al²⁶	2022	China	Case series	Chronic rhinosinusitis	91	N/A	MaZda	275	Fisher coefficient, classification error probability, combined average correlation coefficient and mutual information	Yes	Original data analysis, principal component analysis, linear classification analysis, nonlinear classification analysis	Nil
Prognostication	Yan et al³¹	2021	China	Retrospective cohort	Nasopharyngeal carcinoma	311	N/A	AccuContour	1409	N/A	Yes	LASSO-LR	10 fold CV

Includes details of all diagnostic and prognostic radiomic studies of computed tomography paranasal sinus imaging and highlights use of AI methods such as classification and CV models.

Abbreviations: AI, artificial intelligence; CV, cross validation.

Risk of Bias

For the diagnostic studies, bias by the QUADAS-2 tool was generally low (Table 3) and very low for the prognostic studies, as reported by PROBAST tool (Table 4).

Table 3.

Risk of Bias for Diagnostic Studies.

Author	Year	Risk of bias				Concerns of applicability
Author	Year	Patient selection	Index test	Reference standard	Flow and timing	Patient selection	Index test	Reference standard
Baracnovic et al	2022	Unclear	Unclear	Low	High	Low	Low	Low
Costa et al	2023	Low	Unclear	Low	High	Low	Low	Low
Guo et al	2023	Unclear	Low	Low	Unclear	High	Low	Low
Ito et al	2022	High	High	Unclear	Unclear	Low	Low	Low
Liang et al	2019	Unclear	High	Low	Low	Low	Low	Low
Ogawa et al	2021	Unclear	Unclear	Low	Unclear	Low	Low	Low
Zhu et al	2023	Unclear	Unclear	Low	Low	Low	Low	Unclear

Risk of bias assessment using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) 2 Tool where red denotes high risk, green low risk, and yellow unclear risk.

Table 4.

Risk of Bias for Prognostic Studies.

Author	Year	Risk of bias				Concerns of applicability			Overall assessment
Author	Year	Participants	Predictors	Outcome	Analysis	Participants	Predictors	Outcome	RoB	Applicability
Intarak et al	2022	Low	Low	Unclear	Low	Low	Low	Unclear	Low	Unclear
Li et al	2022	Low	Low	Unclear	Unclear	Low	Low	High	Unclear	High
Yan et al	2021	Low	Low	Low	Low	Low	Low	Low	Low	Low

Risk of bias (RoB) assessment using Prediction Model Risk of Bias Assessment Tool (PROBAST) where red denotes high risk, green low risk, and yellow unclear risk.

Radiomics Quality Assessment

RQS ranged from 19.44% to 63.89%. The average RQS across all studies was 36.95% which is consider generally poor based on exploratory studies.³¹ The individual RQS for each study are detailed (see Tables 5 and 6).

Table 5.

Characteristics of diagnostic radiomic studies of CT paranasal sinuses imaging.

Category	Author	Year	Index test	Reference test	Pathologies	Level of evidence	Radiomics quality score (%)	AUC-ROC	Sensitivity	Specificity	PPV	NPV	Accuracy	Diagnostic utility rating	Other measure of performance
Diagnostic	Baracnovic et al	2022	CT radiomics	Radiologist interpretation and confirmation with positive fungal molds, serum IgE and allergic mucin	Allergic fungal rhinosinusitis versus Nonallergic fungal rhinosinusitis	2	19.44%	0.73	0.71	0.64	N/A	N/A	N/A	Good	N/A
Diagnostic	Costa et al	2023	CT radiomics	Radiologist interpretation	Odontogenic versus nonodontogenic maxillary sinusitis	2	19.44%	N/A	N/A	N/A	N/A	N/A	N/A	Unclear	Statistically significant difference in 3 radiomic parameters
Diagnostic	Guo et al	2023	CT radiomics	Histopathological diagnosis	Nasal polyps versus inverting papilloma	2	41.67%	0.92	0.85	0.99	0.99	N/A	0.92	Excellent	N/A
Diagnostic	Ito et al	2022	CT radiomics	Radiologist interpretation	Odontogenic versus nonodontogenic maxillary sinusitis	2	27.78%	0.76	0.72	0.72	N/A	N/A	0.72	Good	N/A
Diagnostic	Liang et al	2019	CT radiomics	Correlation to clinical variables of T-, N-categories and gross tumor volume	CT and MRI radiomic features in nasopharyngeal carcinoma	4	27.78%	N/A	N/A	N/A	N/A	N/A	N/A	Unclear	76.1% of CT radiomic features significantly correlated with clinical features
Diagnostic	Ogawa et al	2021	CT radiomics	Radiologist interpretation and Histopathological diagnosis	Olfactory neuroblastoma versus sinonasal squamous cell carcinoma	2	41.67%	0.83	0.71	0.96	0.92	0.83	0.86	Very good	N/A
Diagnostic	Zhu et al	2023	CT radiomics	Serum eosinophils and tissue eosinophils, CT Lund-Mackay scores by radiologist	Eosinophilic CRSwNP versus non-eosinophilic CRSwNP	2	63.89%	0.815	0.731	0.81	0.715	0.817	0.776	Excellent	N/A

Diagnostic utility scoring based on AUC-ROC or accuracy with the ratings based on Figure 1. Other outcomes utilized included sensitivity, specificity, PPV, and NPV.

Abbreviations: AUC-ROC, area under the curve receiver operating characteristic; CT, computed tomography; MRI, magnetic resonance imaging; NPV, negative predictive value; PPV, positive predictive value.

Table 6.

Characteristics of Prognostic Radiomic Studies of CT Paranasal Sinuses Imaging.

Category	Author	Year	Pathology	Prognostic outcomes	Time period	Number of events	Level of evidence	Radiomics quality score (RQS)	C statistic	AUC-ROC	Prognostic utility rating	Calibration slope	Calibration intercept	Other
Prognostication	Intarak et al (a)	2022	Nasopharyngeal carcinoma	Overall survival	3 years	Not recorded	3	41.67%	0.860 ± 0.064	0.886 ± 0.043	Very good	N/A	N/A
Prognostication	Intarak et al (b)	2022	Nasopharyngeal carcinoma	Progression free survival	3 years	Not recorded	3	41.67%	0.757 ± 0.070	0.818 ± 0.052	Very good	N/A	N/A
Prognostication	Intarak et al (c)	2022	Nasopharyngeal carcinoma	Distant metastasis free survival	3 years	Not recorded	3	41.67%	0.819 ± 0.055	0.781 ± 0.047	Very good	N/A	N/A
Prognostication	Yan et al	2021	Nasopharyngeal carcinoma	Progression free survival	7 years	88	3	55.56%	0.873 (95% CI 0.803∼0.943)	0.925	Excellent	0.69	0.053
Prognostication	Li et al	2022	Chronic Rhinosinusitis	Post endoscopic sinus surgery CT findings	Not recorded	Not recorded	4	30.56%	N/A	N/A	Unclear	N/A	N/A	Misclassification rate of 25.27%

Prognostic utility scoring based on AUC-ROC or accuracy with ratings based on Figure 1. Other outcomes utilized included concordance statistic (C statistic), calibration slope, and calibration intercept.

Abbreviations: AUC-ROC, area under the curve receiver operating characteristic; CT, computed tomography.

Summary of Diagnostic Studies

There were 7 diagnostic studies included in this review of CT paranasal sinus radiomics^{21,22,24,26,28–30} All 7 utilized CT radiomics as an index test^{21,22,24,26–30}, with 1 utilizing MRI radiomics in conjunction²¹. The reference tests were either radiologist interpretation alone (n = 3)^26,28,30, histopathological diagnosis alone (n = 1)²⁹, a combination of both (n = 2)^22,24, or clinical features such as TNM categories (n = 1)²¹. The pathologies analyzed included odontogenic maxillary sinusitis (n = 2)^26,28, chronic rhinosinusitis (CRS) with subtypes including allergic fungal rhinosinusitis (AFRS) and eosinophilic CRS with nasal polyps (eCRSwNP) (n = 3)^24,26,30, inverting papilloma (n = 1)²⁹, nasopharyngeal carcinoma (n = 1)²¹, and olfactory neuroblastoma versus sinonasal squamous cell carcinoma (n = 1)²². Five studies reported AUC-ROC in the range of 0.73 to 0.92 with diagnostic utility ratings of excellent (n = 2)^29,30, very good (n = 1)²², and good (n = 2)^24,26. Two studies did not report AUC-ROC or accuracy, with one reporting mean radiomic parameter value with statistically significant differences and one reporting percentage of significant correlation to clinical features^21,28 (see Table 5).

Summary of Prognostic Studies

There were 3 studies which analyzed prognostication in paranasal sinuses CT radiomics.^23,25,27 Two studies used the pathology of nasopharyngeal carcinoma.^23,25 The prognostic outcomes measured were progression free survival (n = 2)^23,25, overall survival (n = 1)²⁵, and distant metastasis survival (n = 1)²⁵. The remaining study utilized CT imaging to predict treatment of CRS post endoscopic sinus surgery.²⁷ Two studies reported C statistic and AUC-ROC, with excellent (n = 1)²³ and very good (n = 1)²⁵ diagnostic utility ratings, while the other study used misclassification of disease rate²⁷ (see Table 6).

Discussion

Radiomics as quantitative field of image analysis was first discussed in 2012 by Lambin et al³² for the purpose studying intratumour features on medical imaging. Since then, there has been a rapid expansion of radiomics to multiple fields of medicine given the routine use of medical imaging for diagnostics.³³ As there is routine use of CT imaging in the field of rhinology, radiomics is becoming an increasingly utilized tool, with 3 previous systematic reviews detailing radiomics use in nasopharyngeal carcinoma.^34–36 This review assesses the use of radiomics in rhinology given the emergence of literature in a variety of rhinological diseases. Radiomics has demonstrated diagnostic accuracy in diseases such as chronic rhinosinusitis including subtypes such as AFRS, odontogenic sinusitis and eosinophilic CRS, nasopharyngeal carcinoma, inverting papillomas, olfactory neuroblastoma and sinonasal squamous cell carcinoma.^{21,22,24,26–30} There are also prognostic benefits with studies utilizing radiomics in nasopharyngeal carcinoma to predict overall survival, progression free survival and distant metastasis free survival due to imaging features of extent or invasiveness that provides prognostics value.^23,25 Initial studies focused on data localized to single functional areas, that is, maxillary sinuses with user defined radiomic features analyzed which used first order or second order features.²⁸ Recent studies exhibit an increasing breadth of complexity and utility of machine learning for radiomic analysis of first-, second-, and higher-order features beyond the capabilities of human alone, for instance, by clustering and identifying eosinophilic chronic rhinosinusitis with nasal polyps and correlating to clinical features.³⁰ Studies using increasing complexities of methods in a variety of rhinology diseases are resulting in more opportunities for clinical translation for medical practitioners. Most studies did not use feature selection methods which are useful for minimizing the impact of overfitting in radiomics and improving interpretability, making it challenging to derive meaningful conclusions about their quality. Classification models varied from more simplistic logistic regression and support vector machine used for binary task classification in smaller datasets, to complex unsupervised clustering to identify multiple new groupings.^37,38 The CV techniques were used to validate the classification models with unseen data. A 10-fold CV was the most common one, as a common method in image analysis, while leave out one CV has value in smaller datasets with comprehensive evaluation although is a time consuming technique.³⁸ Overall, when considering feature selection, classification, and CV models there is no single answer for all scenarios and careful consideration of the intended task helps determine which to use. As a result, interpreting data from differing models is challenging, as one is not truly able to compare like for like unless the exact same task and methodology are used. The results of the identified studies show promising results for clinical translation with diagnostic and prognostic performances in the good and above ranges (>0.7) for 70% of studies (n = 7).^{22–26,29,30} And 30% of studies (n = 3)^23,29,30 even had excellent performances in AUC-ROC with the highest at 0.925.²³

However, the low to average RQS highlight the inadequate methodological quality of current literature. This review found a low RQS of only 36.95%. This may be related to the exploratory and pilot nature of the radiomics involved, that is, identifying radiomic features in odontogenic maxillary sinusitis may have low clinical utility as most ENT surgeons are taught to identify these features of odontogenic sinusitis on CT. The domains of the RQS which most frequently required improvement included public access to protocols, coding and data, lack of standardization in images’ preprocessing, the low number of prospective studies, lack of interscanner phantom assessment, lack of multitemporal imaging and lack of cost-effectiveness analyses. This assessment remains consistent among multiple fields of medicine with a mean RQS of only 26.1% and specific research areas such as cholangiocarcinoma radiomics ranging between 23% and 39% and neuro-oncology radiomics averaging 37.1%.^33,39,40 While there are limitations to the RQS, notably with user interpretation, strict adherence to a single ideal radiomic workflow and minimal weightage to clinical translation, currently this remains the most used metric for quality studies.¹

Limitations of this review included single author study selection as a source of bias, the restriction to the past 5 years for inclusion and the heterogeneous data presented limiting recommendations. This study captured the past 5 years of literature, which was chosen given the dramatic increase in the quantity and quality of radiomic literature.⁷ In the future when more studies are available in this field, subgroup analysis based on pathology would also be beneficial for improved analysis.

A heavily cited issue for radiomics is clinical translation.⁴⁰ While these studies show some promise in prognosticating outcomes in nasopharyngeal carcinoma or aiding radiologist or rhinologist interpretation of CT images, one must ensure that radiomics should be used to help answer clinically relevant questions. Limiting radiomics to identify simple diagnostic features on CT scans which can otherwise be easily interpreted by clinicians, hinders it performing to its full potential. Radiomics is more beneficial when it can identify patterns of disease which require very large datasets and vast experience, or minute and difficult to discern features which are challenging for humans. Diagnostic value may be enhanced with utilizing radiomics for identifying unique and specific features of diseases, and prognostic value with the detection of minute invasion and patterns/characteristics which may proceed the development of disease. Then radiomics can augment current clinical practice, rather than simply replace clinicians as is often questioned in the media and literature with radiomics and AI.

Implications for Future Research

From this systematic review, there are potential avenues for further quality radiomic research to encompass more areas of rhinology beyond the most frequent pathology of nasopharyngeal carcinoma. Radiomic data is likely to form the basis of future AI-based analysis with the potential to investigate radiomic phenotypes in Chronic Rhinosinusitis from CT imaging, with the aim of identifying new clusters for clinical classification and optimizing targeted therapies for improved long-term outcomes (see Figure 3).

Figure 3.

Sample segmentation of the paranasal sinuses for radiomic analysis. Screenshot from 3-dimensional (3D) Slicer (Open Source, http://www.slicer.org) application demonstrating a sample manual segmentation of the paranasal sinuses on computed tomography (CT) imaging, with axial views (A), 3D segmentation of the paranasal sinuses (B), coronal views (C), and sagittal views (D).

There is also a comparative predominance of radiomic research in diagnostics and a lack of prognostication studies, with only 2 studies till date limited to nasopharyngeal carcinoma. Currently the most common pathology studied was nasopharyngeal carcinoma, however there is the potential to expand to other tumor subtypes in the paranasal sinuses. To improve the quality of future radiomic research, there is a need for both standardized processes and more transparent and public imaging data and codes.⁴¹ In 2024, a new tool called METhodological RadiomICs Score “METRICS” was developed via modified Delphi method including an international panel for more flexible methodological assessment scoring to ensure consistency in radiomic studies.¹⁹ Although in its infancy, this offers promise as a method with mandatory and conditional sections depending upon the methodology used, that is, deep learning, machine learning, etc, and giving importance to the key areas of data acquisition, image segmentation, feature extraction and feature analysis. For improving transparency and availability, there are data registries and open source codes available on GitHub in multiple types of oncologic radiomics research such as lung and brain cancers.⁴¹ However there is a dearth in the field of rhinology.⁴¹ Addressing the lack of data and resources, could allow researchers to spend less time on manually intensive data collection and preprocessing, and focus more on delivering solutions for clinical translation in radiomics research.

Conclusion

Considering the reliance of the rhinology profession on CT of the paranasal sinuses, the use of radiomic data is very limited but growing each year in number and complexity. In order to best harness the ability of radiomics, quality methodologies, and presenting clinically relevant questions are vital. This systematic review highlights the beginnings of clinically relevant research with a potential greatly aid the rhinological surgeon in the future.

Supplemental Material

sj-docx-1-ajr-10.1177_19458924241304082 - Supplemental material for Radiomics of the Paranasal Sinuses: A Systematic Review of Computer-Assisted Techniques to Assess Computed Tomography Radiological Data

Supplemental material, sj-docx-1-ajr-10.1177_19458924241304082 for Radiomics of the Paranasal Sinuses: A Systematic Review of Computer-Assisted Techniques to Assess Computed Tomography Radiological Data by Rhea Darbari Kaul, BMed, Peta-Lee Sacks, MBBS, PhD, Cedric Thiel, MD, Janet Rimmer, MD, Larry Kalish, MBBS (Hons I), Raewyn Gay Campbell, BMed (Hons), Raymond Sacks, MBBCH, Antonio Di Ieva, MD, PhD, and Richard John Harvey, MBBS (Hons I), PhD in American Journal of Rhinology & Allergy

Footnotes

Author Contributions

RDK contributed to formulation of research question, development of search strategy, study selection, data extraction, writing of manuscript, editing, and accepting final version of the manuscript. PS, CT, JR, LK, RGC, RS, and ADI were involved in development of search strategy, data extraction, editing, and accepting final version of the manuscript. RJH was involved in formulation of research question, development of search strategy, data extraction, editing, and accepting final version of manuscript.

Data Availability

The authors confirm that the data supporting the findings of this study are available within the article and its Supplemental materials. Any further data that support the findings of this study are available from the corresponding author, RDK, upon reasonable request.

Declaration of Conflicting Interests

The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Richard J Harvey is a consultant/advisory board with Medtronic, Novartis, GSK and Meda pharmaceuticals. Research grant funding received from Glaxo-Smith-Kline. He has been on the speakers’ bureau for Glaxo-Smith-Kline, Astra-Zeneca, Meda Pharmaceuticals and Seqirus. Janet Rimmer has honoraria with Sanofi Aventis, Novartis, Mundipharma, BioCSL, Stallergenes. Larry Kalish is on the speakers’ bureau for Care Pharmaceuticals, Mylan and Seqirus Pharmaceuticals. Raymond Sacks is a consultant for Medtronic. Antonio Di Ieva has received research funding from B Braun, Abbott, Servier, and BrainLab. All other authors have no financial disclosures or conflicts of interest.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Ethics

Nil ethics approval or patient informed consent was not required for this systematic review.

ORCID iD

Rhea Darbari Kaul

Supplemental Material

Supplemental material for this article is available online.

References

Lambin

Leijenaar

RTH

Deist

, et al. Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol. 2017;14(12):749–762. doi:https://doi.org/10.1038/nrclinonc.2017.141

Sanduleanu

Woodruff

de Jong

EEC

, et al. Tracking tumor biology with radiomics: a systematic review utilizing a radiomics quality score. Radiother Oncol. 2018;127(3):349–360. doi:https://doi.org/10.1016/j.radonc.2018.03.033

van Timmeren

Cester

Tanadini-Lang

Alkadhi

Baessler

. Radiomics in medical imaging—“how-to” guide and critical reflection. Insights Imaging. 2020;11(1):91. doi:https://doi.org/10.1186/s13244-020-00887-2

Mayerhoefer

Materka

Langs

, et al. Introduction to radiomics. J Nucl Med. 2020;61(4):488–495. doi:https://doi.org/10.2967/jnumed.118.222893

Jang

Russo

Di Ieva

. Radiomics in gliomas: clinical implications of computational modeling and fractal-based analysis. Neuroradiology. 2020;62(7):771–790. doi:https://doi.org/10.1007/s00234-020-02403-1

Rizzo

Botta

Raimondi

, et al. Radiomics: the facts and the challenges of image analysis. Eur Radiol Exp. 2018;2(1):36. doi:https://doi.org/10.1186/s41747-018-0068-z

Zhang

Guo

Jin

. Radiomics and its feature selection: a review. Symmetry (Basel). 2023;15(10):1834. doi:https://doi.org/10.3390/sym15101834

Jian

Jang

Manuguerra

Liu

Magnussen

Di Ieva

. Machine learning for the prediction of molecular markers in glioma on magnetic resonance imaging: a systematic review and meta-analysis. Neurosurgery. 2021;89(1):31–44. doi:https://doi.org/10.1093/neuros/nyab103

Jian

Liu

Di Ieva

. Artificial intelligence for survival prediction in brain tumors on neuroimaging. Neurosurgery. 2022;91(1):8–26. doi:https://doi.org/10.1227/neu.0000000000001938

10.

Tabassum

Suman

Suero Molina

Pan

Di Ieva

Liu

. Radiomics and machine learning in brain tumors and their habitat: a systematic review. Cancers (Basel). 2023;15(15). doi:https://doi.org/10.3390/cancers15153845

11.

Damen

JAA

Moons

KGM

van Smeden

Hooft

. How to conduct a systematic review and meta-analysis of prognostic model studies. Clin Microbiol Infect. 2023;29(4):434–440. doi:https://doi.org/10.1016/j.cmi.2022.07.019

12.

Deeks

Bossuyt

Leeflang

Takwoingi

. Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy. Chichester, UK: John Wiley & Sons; 2023. Available from: https://training.cochrane.org/handbook-diagnostic-test-accuracy/current .

13.

Page

McKenzie

Bossuyt

, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Br Med J. 2021;372(21):n71. doi:https://doi.org/10.1136/bmj.n71

14.

Ouzzani

Hammady

Fedorowicz

Elmagarmid

. Rayyan—a web and mobile app for systematic reviews. Syst Rev. 2016;5(1):210. doi:https://doi.org/10.1186/s13643-016-0384-4

15.

Whiting

Rutjes

Westwood

, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155(8):529–536. doi:https://doi.org/10.7326/0003-4819-155-8-201110180-00009

16.

Wolff

Moons

KGM

Riley

, et al. PROBAST: a tool to assess the risk of bias and applicability of prediction model studies. Ann Intern Med. 2019;170(1):51–58. doi:https://doi.org/10.7326/M18-1376

17.

Group OLoEW. Oxford Centre for Evidence-Based Medicine 2011 Levels of Evidence 2011 [Available from: http://www.cebm.net/index.aspx?o=5653.

18.

Radiomics Quality Score. RQS: RQS - Radiomics.world 2017.

19.

Kocak

Akinci D’Antonoli

Mercaldo

, et al. METhodological RadiomICs Score (METRICS): a quality scoring tool for radiomics research endorsed by EuSoMII. Insights Imaging. 2024;15(1):8. doi:https://doi.org/10.1186/s13244-023-01572-w

20.

Osie

Darbari Kaul

Alvarado

, et al. A scoping review of artificial intelligence research in rhinology. Am J Rhinol Allergy. 2023;37(4):438–448. doi:https://doi.org/10.1177/19458924231162437

21.

Liang

Tan

Zhang

, et al. Comparison of radiomics tools for image analyses and clinical prediction in nasopharyngeal carcinoma. Br J Radiol. 2019;92(1102):20190271. doi:https://doi.org/10.1259/bjr.20190271

22.

Ogawa

Osaga

Shiraki

, et al. Utility of CT texture analysis to differentiate olfactory neuroblastoma from sinonasal squamous cell carcinoma. Sci Rep. 2021;11(1):4679. doi:https://doi.org/10.1038/s41598-021-84048-5

23.

Yan

Shen

Chen

, et al. CT-based radiomics nomogram for prediction of progression-free survival in locoregionally advanced nasopharyngeal carcinoma. Cancer Manag Res. 2021;2021(13):6911–6923. doi:https://doi.org/10.2147/CMAR.S325373

24.

Bracanovic

Janovic

Antic

, et al. CT and CT image-based texture image analysis in radiological diagnostics of allergic fungal rhinosinusitis. Mycoses. 2022;65(5):551–559. doi:https://doi.org/10.1111/myc.13438

25.

Intarak

Chongpison

Vimolnoch

, et al. Tumor prognostic prediction of nasopharyngeal carcinoma using CT-based radiomics in non-Chinese patients. Front Oncol. 2022;12(775248):1–9. doi:https://doi.org/10.3389/fonc.2022.775248

26.

Ito

Kondo

Andreu-Arasa

, et al. Quantitative assessment of the maxillary sinusitis using computed tomography texture analysis: odontogenic vs non-odontogenic etiology. Oral Radiol. 2022;38(3):315–324. doi:https://doi.org/10.1007/s11282-021-00558-y

27.

Tao

. Radiomics model based on enhanced gradient level set segmentation algorithm to predict the prognosis of endoscopic treatment of sinusitis. Comput Math Methods Med. 2022;2022(9511631):1–7.

28.

Costa

ALF

Fardim

KAC

Ribeiro

, et al. Cone-beam computed tomography texture analysis can help differentiate odontogenic and non-odontogenic maxillary sinusitis. Imaging Sci Dent. 2023;53(1):43–51. doi:https://doi.org/10.5624/isd.20220166

29.

Guo

Zang

, et al. Classification of nasal polyps and inverted papillomas using CT-based radiomics. Insights Imaging. 2023;14(1):188. doi:https://doi.org/10.1186/s13244-023-01536-0

30.

Zhu

, et al. Development and multicenter validation of a novel radiomics-based model for identifying eosinophilic chronic rhinosinusitis with nasal polyps. Rhinology. 2023;61(2):132–143.

31.

Spadarella

Stanzione

Akinci D'Antonoli

, et al. Systematic review of the radiomics quality score applications: an EuSoMII radiomics auditing group initiative. Eur Radiol. 2023;33(3):1884–1894. doi:https://doi.org/10.1007/s00330-022-09187-3

32.

Lambin

Rios-Velazquez

Leijenaar

, et al. Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer. 2012;48(4):441–446. doi:https://doi.org/10.1016/j.ejca.2011.11.036

33.

Pinto Dos Santos

Dietzel

Baessler

. A decade of radiomics research: are images really data or just patterns in the noise? Eur Radiol. 2021;31(1):1–4. doi:https://doi.org/10.1007/s00330-020-07108-w

34.

Duan

Xiong

Tian

Zou

Zhang

. Radiomics in nasopharyngeal carcinoma. Clin Med Insights Oncol. 2022;16:1–10. doi:https://doi.org/10.1177/11795549221079186

35.

Huang

Wang

, et al. Comprehensive integrated analysis of MR and DCE-MR radiomics models for prognostic prediction in nasopharyngeal carcinoma. Vis Comput Ind Biomed Art. 2023;6(1):23. doi:https://doi.org/10.1186/s42492-023-00149-0

36.

Wang

Fang

Zhang

, et al. Radiomics and deep learning in nasopharyngeal carcinoma: a review. IEEE Rev Biomed Eng. 2024;17:118–135. doi:https://doi.org/10.1109/RBME.2023.3269776

37.

Jardim

Valente

Almeida

Mora

. Comparing artificial intelligence classification models to improve an image comparison system with user inputs. SN Computer Sci. 2023;5(1):57. doi:https://doi.org/10.1007/s42979-023-02375-y

38.

Bradshaw

Huemann

Rahmim

. A guide to cross-validation for artificial intelligence in medical imaging. Radiol Artif Intell. 2023;5(4):e220232. doi:https://doi.org/10.1148/ryai.220232

39.

Cannella

Vernuccio

Klontzas

, et al. Systematic review with radiomics quality score of cholangiocarcinoma: an EuSoMII radiomics auditing group initiative. Insights Imaging. 2023;14(1):21. doi:https://doi.org/10.1186/s13244-023-01365-1

40.

Park

Kim

, et al. A systematic review reporting quality of radiomics research in neuro-oncology: toward clinical utility and quality improvement using high-dimensional imaging features. BMC Cancer. 2020;20(1):29. doi:https://doi.org/10.1186/s12885-019-6504-5

41.

Woznicki

Laqua

Al-Haj

Bley

Baeßler

. Addressing challenges in radiomics research: systematic review and repository of open-access cancer imaging datasets. Insights Imaging. 2023;14(1):216.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB