Combining Gabor-local and contextual-global deep features for cholesteatoma classification

Abstract

A key challenge in classifying cholesteatoma images lies in the effective extraction and integration of both local and global features. This paper addresses this challenge by introducing a Gabor-local and contextual-global network. This network enhances local feature extraction through the incorporation of Gabor filtering alongside a ResNet architecture, which captures richer local information at varying scales and orientations. Moreover, the attention mechanism generates an importance score map, enabling the network to dynamically focus on the most relevant regions within the input image by normalizing scores into a weight matrix and supporting Generalized-Mean (GeM) pooling, resulting in a more robust global descriptor. Finally, a novel dynamic feature fusion strategy combines local features, global features, and outputs from the last residual block based on their computed importance scores, further enhancing the model’s overall performance. Experimental results show the network’s superior performance, effectively leveraging both local and global information for enhanced classification outcomes.

1. Introduction

Accurate differentiation between cholesteatoma and a normal middle ear is crucial, as cholesteatoma may cause progressive hearing loss (as shown in Figure 1), tinnitus, and structural damage,^1,2 often necessitating surgical intervention. In contrast, a normal middle ear does not require treatment, highlighting the importance of precise diagnosis. Advanced imaging techniques, in combination with artificial intelligence (AI), play a pivotal role in this process. While imaging can reveal subtle anatomical differences, AI further enhances diagnostic accuracy by analyzing complex data patterns, thereby improving classification performance and supporting personalized treatment planning.

Figure 1.

Image samples of normal middle ear and cholesteatoma.

Early efforts in AI ear image classification employ deep learning on two dimensional Computed Tomography (CT) slices to detect chronic otitis media ³ and apply machine learning to tympanic membrane images for middle ear condition classification. ⁴ Subsequent CT approaches focus on cholesteatoma differentiation and pediatric temporal bone disease diagnosis using architectures such as VGG and hybrid models.^5,6 Structure aware methods also improve classification accuracy. ⁷ In addition to CT, convolutional neural networks (CNNs) apply to intraoperative endoscopic and standard otoscopic images for cholesteatoma detection,^8,9 while lightweight models such as MobileNetV2 support targeted predictive tasks. ¹⁰ Further methodological advances include deep feature fusion with graph isomorphism networks, explainable three dimensional CNNs, and hybrid CNN combined with machine learning strategies.^11–13 These developments include comparative evaluations of architectures such as ResNet, MobileNetV2 and DenseNet against Magnetic Resonance Imaging (MRI),^14,15 as well as systematic reviews of performance. ¹⁶ Bibliometric analyses indicate a growing integration of AI and medical imaging in cholesteatoma research. ¹⁷ Additionally, various works like DenseNet, ¹⁸ SE-ResNet, ¹⁹ Vision Transformer (ViT), ²⁰ Bottleneck Transformer (BoTNet), ²¹ ConvNeXt, ²² iFormer, ²³ TransNeXt, ²⁴ and Visual Attention Network (VAN) ²⁵ effectively utilized network architectures or attention mechanisms to achieve state-of-the-art results in multiple medical image classification tasks.

Previous methods in cholesteatoma image classification have achieved a certain degree of success; however, they still face challenges, particularly in effectively extracting and integrating local and global features. These approaches struggle to capture intricate spatial relationships and may overlook important contextual information, which can lead to less than optimal performance. To address these challenges, this paper introduces several innovative components. First, the proposed Gabor-local feature extraction, combined with a ResNet architecture, enhances local feature capture by obtaining richer information at varying scales. Second, a contextual-global feature extraction aggregation is developed, generating the importance score map that enables the model to dynamically focus on relevant image regions while normalizing scores into a weight matrix and supporting Generalized-Mean (GeM) pooling for a robust global descriptor that incorporates contextual significance. Lastly, a novel dynamic feature fusion strategy combines local features, global features, and outputs from the last residual block based on their computed importance scores. This synergy not only amplifies critical information but also strengthens the model’s capacity for multi-scale analysis, ultimately leading to improved classification performance in challenging cholesteatoma image analysis. Overall, the main contributions of this paper can be summarized as.

- The proposed Gabor-local feature extraction within the network enhances local feature capture, enabling a more nuanced representation of relevant patterns in cholesteatoma images.

- The developed contextual-global aggregation mechanism generates the importance score map, which enables the model to dynamically focus on relevant regions and supports robust global descriptors through GeM pooling.

- A dynamic feature fusion approach combines local and global features together with outputs from the last residual block based on importance scores, which leads to superior classification performance in challenging cholesteatoma image analysis.

- Experimental results demonstrate superior classification performance compared to conventional methods, showcasing the model’s effectiveness in leveraging local and global information.

The remainder of this paper is structured as follows: Section 2 reviews relevant works from various perspectives, establishing the foundation for our study. Section 3 introduces our proposed model, detailing its technical implementation and the underlying theoretical principles. In Section 4, we outline the experimental framework, including the datasets used for evaluation and the performance metrics applied. The paper concludes in Section 5 with a summary of the main findings and proposes possible directions for future research in this area.

2. Related works

2.1. Local feature extraction

CNNs have been frequently utilized for their inherent ability to capture local features across various applications. For example, Tian et al. ²⁶ employed CNNs within a dual network structure to extract both local and global features for image denoising. Concurrently, the effectiveness of convolutional operations for capturing local details was implicitly studied by Utomo et al., ²⁷ who compared models using explicit local feature descriptors against standard CNN performance on medical images. Subsequent research continued to integrate CNNs for their local feature extraction capabilities, often within hybrid models. For instance, Mou et al. ²⁸ designated a CNN channel purely for local feature extraction in a multimodal driver distraction system, while Hu et al. ²⁹ applied a CNN module to summarize local planar features in magnetic resonance images before integrating them. In other imaging areas, Li et al. ³⁰ leveraged the strength of CNNs in computing local dependencies for mineral prospectivity mapping, Wang et al. ³¹ used CNN features to interactively capture local structural information for image super resolution, and Xu et al. ³² proposed a specific dynamic CNN branch to effectively encode local pixel features for hyperspectral image classification. MedViTV2 ³³ incorporated Kolmogorov Arnold Network layers into a transformer, achieving efficient global and local feature perception with improved performance.

2.2. Global feature extraction

Recognizing the limitations of CNNs in capturing long-range dependencies and global context, several approaches have sought to enhance their ability to model global features. He et al. ³⁴ introduced a framework with distinct branches for local and global attention within a CNN architecture to learn comprehensive representations for depression recognition. Wang et al. ³⁵ developed a multitask attention CNN with a global feature shared network designed to extract globally relevant information for tasks like fault diagnosis and working condition identification. Similarly, Li et al. ³⁶ combined global information from the entire image with component details in a multiscale CNN to improve synthetic aperture radar target recognition. Further, Liu et al. ³⁷ integrated a CNN focused on local features with a vision transformer for learning global features to address high-resolution synthetic aperture radar images’ challenges. Xiong et al. ³⁸ designed a CNN model with separate branches for extracting and fusing global and local features from electromyography signals to enhance hand gesture recognition. To tackle the struggle with long distance dependencies, Li et al. ³⁰ integrated a Transformer with a CNN for mineral prospectivity mapping. Khan et al. ³⁹ introduced a model with global and local convolutional components for histopathological slide classification. Lastly, Duan et al. ⁴⁰ combined transformers with CNNs for multi-focus image fusion, enhancing interaction between features using knowledge distillation.

2.3. Feature fusion strategies

Several studies have explored feature fusion techniques within CNN frameworks to enhance performance across various domains. For instance, Khan et al. ⁴¹ integrated low-level handcrafted features with multi-stage deep CNN features for image scene geometry recognition through fusion strategies. Extending this concept, Ding et al. ⁴² developed a multi-feature fusion network combining multi-scale graph convolutional networks and multi-scale CNNs to fuse their outputs for hyperspectral image classification. In a parallel development, Li et al. ⁴³ presented a combined architecture of a CNN for spatial features and an LSTM network for temporal features, fusing these along with intermediate convolutional layer features for motor imagery EEG classification. Similarly, Ou et al. ⁴⁴ designed a CNN framework incorporating feature fusion after applying band selection for hyperspectral image change detection. Xiao et al. ⁴⁵ constructed a multi-scale CNN with an attention mechanism, employing feature fusion strategies to recognize the appearance of spot-welding surfaces. Further, Haq et al. ⁴⁶ employed feature fusion within a deep CNN architecture, combined with ensemble learning, to improve the classification of abnormalities in mammographic images. Zhang et al. ⁴⁷ developed a dual-channel CNN that enhanced feature representation by repeatedly fusing deep and shallow features for wildfire detection.

3. Methodology

In this paper, we present a classification method for cholesteatoma images that leverages a ResNet ⁴⁸ architecture, integrating Gabor filtering for local feature extraction and an attention mechanism for contextual-global feature aggregation. As shown in Figure 2, the input image is first processed through residual blocks to generate feature maps. These feature maps are subsequently refined by Gabor filters, which further enhance local features. Concurrently, the attention mechanism produces an importance score map to identify relevant regions in the feature space. These scores are normalized into a weight matrix and applied using GeM pooling to create a robust global descriptor. Next, the local features, global features, and outputs from the last residual block are fused using the proposed dynamic feature fusion. Finally, this combined representation is passed through a fully connected layer to generate logits, which are then converted into class probabilities via Softmax.

Figure 2.

The input image is processed through residual blocks to create feature maps. Subsequently, Gabor-local feature extraction (GLFE) and contextual-global feature aggregation (CGFA) are applied to generate local and global features. Following this, dynamic feature fusion (DFF) is employed to combine the local and global features with the outputs from the last residual block, based on their significance scores. Finally, the fused features are passed through a fully connected layer to produce logits, which are then converted into class probabilities using the softmax function.

3.1. Gabor-local feature extraction

As illustrated in Figure 3, the Gabor-local feature extraction process begins with an input image $I\in {\mathbb{R}}^{H\times W\times 1}$ , where H and W represent the height and width, respectively. At layer l of the last residual block, the output feature map $X^{[l]}\in {\mathbb{R}}^{h\times w\times c}$ is computed using the formula:

$X^{[l]}=F_{\text{conv}}(X^{[l-1]})+X^{[l-1]},$

(1)

where $F_{\text{conv}}:{\mathbb{R}}^{h\times w\times c}\to {\mathbb{R}}^{h\times w\times c}$ represents the convolution transformation function mapping features from layer l − 1 to layer l, with h, w, and c denoting the height, width, and number of channels of the feature map, respectively. Before applying multi-scale Gabor filtering, an adaptive parameter selection mechanism is employed, in which the Gabor parameters are dynamically adjusted based on feature statistics:

${\lambda }_i={\lambda }_0\cdot {\alpha }_{\lambda }(X^{[l]}),{\sigma }_i={\sigma }_0\cdot {\alpha }_{\sigma }(X^{[l]}),$

(2)

where α _λ and α _σ are adaptive scaling functions that analyze the feature map to determine appropriate parameter values, and λ₀ and σ₀ are base values. The wavelength scaling function α _λ is defined as:

${\alpha }_{\lambda }(X^{[l]})=\text{clip}\left(1+{\beta }_{\lambda }\cdot \text{std}(X^{[l]}),0.5,2.0\right),$

(3)

where $\text{std}(X^{[l]})=\sqrt{1/h\times w\times c{\sum }_{i,j,k}{(X_{i,j,k}^{[l]}-\mu )}^2}$ computes the standard deviation of the feature map, $\mu =1/h\times w\times c{\sum }_{i,j,k}{X}_{i,j,k}^{[l]}$ is the mean of the feature map, and β _λ is a learnable scaling factor. Similarly, the standard deviation scaling function α _σ is defined as:

${\alpha }_{\sigma }(X^{[l]})=\text{clip}\left(1+{\beta }_{\sigma }\cdot \text{mean}(|X^{[l]}|),0.5,2.0\right),$

(4)

where $\text{mean}(|X^{[l]}|)=1/h\times w\times c{\sum }_{i,j,k}|X_{i,j,k}^{[l]}|$ computes the mean absolute value of the feature map, and β _σ is a learnable scaling factor. Once the parameters (λ _i , σ _i ) have been adaptively determined, a multi-scale Gabor filtering approach is applied. A set of Gabor filters ${\{G_i\}}_{i=1}^N$ with varying parameters is defined, where each Gabor filter $G_i\in {\mathbb{R}}^{k\times k}$ is formulated as:

$\begin{array}{c}\hfill G_i(x,y;{\lambda }_i,{\theta }_i,{\psi }_i,{\sigma }_i)=\exp \left(-\frac{x^{\prime 2}+y^{\prime 2}}{2{\sigma }_i^2}\right)\\ \hfill \times \hspace{0.17em}\exp \left(i\left(\frac{2\pi }{{\lambda }_i}{x}^{\prime }+{\psi }_i\right)\right),\end{array}$

(5)

where x′ = x cos θ _i + y sin θ _i and y′ = −x sin θ _i + y cos θ _i are the rotated coordinates, λ _i > 0 is the wavelength, θ _i ∈ [0, π] determines the orientation, ψ _i ∈ [0, 2π] is the phase offset, and σ _i > 0 is the standard deviation of the Gaussian envelope. Moreover, the Gabor filter is decomposed into real and imaginary parts:

$G_{i,\text{Re}}(x,y)=\exp \left(-\frac{x^{\prime 2}+y^{\prime 2}}{2{\sigma }_i^2}\right)\cdot \cos \left(\frac{2\pi }{{\lambda }_i}{x}^{\prime }+{\psi }_i\right),$

(6)

$G_{i,\text{Im}}(x,y)=\exp \left(-\frac{x^{\prime 2}+y^{\prime 2}}{2{\sigma }_i^2}\right)\cdot \sin \left(\frac{2\pi }{{\lambda }_i}{x}^{\prime }+{\psi }_i\right),$

(7)

such that convolution with the feature map yields two separate responses:

$F_{i,\text{Re}}^{[l]}=G_{i,\text{Re}}\ast X^{[l]},F_{i,\text{Im}}^{[l]}=G_{i,\text{Im}}\ast X^{[l]}.$

(8)

Figure 3.

The input feature X^[l−1] is first processed by a residual block to produce the feature map X^[l]. An adaptive parameter selection mechanism then analyzes the statistical characteristics of X^[l] to determine the parameters for the Gabor filters. Using these parameters, multi-scale Gabor filters G_i,Re and G_i,Im are constructed, whose convolution with X^[l] yields the multi-scale feature representation $F_{\text{multi}}^{[l]}$ . This representation is subsequently refined by a spatial attention mechanism, which emphasizes the most informative regions. The resulting attention map is applied to $F_{\text{multi}}^{[l]}$ , producing the enhanced local features $F_{\text{local}}^{[l]}$ for downstream processing.

For each Gabor filter, the concatenated real and imaginary responses are computed as:

$F_{i,\text{concat}}^{[l]}=\text{Concat}\left(F_{i,\text{Re}}^{[l]},F_{i,\text{Im}}^{[l]}\right),$

(9)

where Concat(⋅, ⋅) concatenates along the channel dimension, enabling the preservation of both amplitude and phase information. Collecting responses from N multi-scale Gabor filters yields:

$F_{\text{multi}}^{[l]}=\text{Concat}\left(F_{1,\text{concat}}^{[l]},F_{2,\text{concat}}^{[l]},\dots ,F_{N,\text{concat}}^{[l]}\right),$

(10)

so that $F_{\text{multi}}^{[l]}\in {\mathbb{R}}^{h\times w\times (2N\cdot c^{\prime })}$ , where c′ denotes the number of channels per filter output. To focus on the most relevant spatial locations within the features, a spatial attention mechanism $S:{\mathbb{R}}^{h\times w\times C}\to {\mathbb{R}}^{h\times w\times 1}$ is employed to generate a spatial attention map. The implementation for S involves convolutional layers applied to the input feature map to output a single channel map, typically followed by a sigmoid activation function:

$M^{[l]}=S(F_{\text{multi}}^{[l]})=\text{sigmoid}({\text{Conv}}_{\text{spatial}}(F_{\text{multi}}^{[l]})),$

(11)

where $M^{[l]}\in {\mathbb{R}}^{h\times w\times 1}$ is the spatial attention map. Finally, the enhanced local feature response $F_{\text{local}}^{[l]}$ at layer l is obtained by applying the spatial attention map element-wise:

$F_{\text{local}}^{[l]}=F_{\text{multi}}^{[l]}\odot M^{[l]},$

(12)

where ⊙ denotes element-wise multiplication with broadcasting along channels, followed by batch normalization:

$F_{\text{local}}^{[l]}=\text{BatchNorm}(F_{\text{local}}^{[l]}).$

(13)

3.2. Contextual-global feature aggregation

As shown in Figure 4, contextual-global feature aggregation begins with GeM pooling applied to the feature tensor $X^{[l]}\in {\mathbb{R}}^{H\times W\times C}$ . To ensure numerical stability, the feature map is first rectified as:

${\tilde{X}}^{[l]}=\max (X^{[l]},ϵ),$

(14)

where ϵ is a small positive constant. The GeM pooled global descriptor is then computed channel-wise as:

$F_{\text{preglobal},c}^{[l]}={\left(\frac{1}{HW}\sum _{m=1}^H\sum _{n=1}^W{\left({\tilde{x}}_{c,m,n}^{[l]}\right)}^p\right)}^{\frac{1}{p}},$

(15)

where c = 1, …, C, and p is a learnable pooling parameter controlling pooling sharpness. The preglobal descriptor is then used to generate context-aware attention parameters:

${\theta }_{\text{att}}=g_{\text{cond}}\left(F_{\text{preglobal}}^{[l]}\right),$

(16)

where g_cond(⋅) is the MLP. The attention function f_att is implemented as a 1 × 1 convolution whose parameters are determined by θ_att:

$A_{\text{regional}}=f_{\text{att}}\left(X^{[l]};{\theta }_{\text{att}}\right),$

(17)

where A_regional denotes the raw regional attention score map. The scores are normalized by spatial softmax:

$w{″}_{m,n}=\frac{\exp (a{″}_{m,n})}{{\sum }_{h=1}^H{\sum }_{k=1}^W\exp (a{″}_{h,k})},$

(18)

where $a{\prime \prime }_{m,n}$ denotes the score at position (m, n). The regional feature is obtained by weighted aggregation:

${(F_{\text{regional}}^{[l]})}_c=\sum _{m=1}^H\sum _{n=1}^{W}w{″}_{m,n}\cdot x_{c,m,n}^{[l]},c=1,\dots ,C.$

(19)

Finally, the global descriptor is refined by a residual connection:

$F_{\text{global}}^{[l]}=F_{\text{preglobal}}^{[l]}+F_{\text{regional}}^{[l]}.$

(20)

Figure 4.

The contextual-global feature aggregation first applies GeM pooling to X^[l] to obtain an initial global descriptor $F_{\text{preglobal}}^{[l]}$ , which captures the overall scene context. Conditioned on this descriptor, a dual-MLP network generates attention parameters θ_att, which guide a 1× 1 convolution followed by softmax normalization to produce regional attention weights W_regional. These weights are then applied to X^[l] via weighted summation, yielding a contextually refined regional feature F_{regional_sum}. Finally, a residual addition integrates the initial global descriptor with this refined representation, producing the output $F_{\text{global}}^{[l]}$ .

3.3. Dynamic feature fusion

To further enhance model performance, we develop a dynamic feature fusion approach that integrates local features, global features, and the output from the last residual block. Initially, these features are extracted from the preceding layers of ResNet. Subsequently, they are flattened into three distinct representations: $F_{\text{local}}^{[l]}\in {\mathbb{R}}^{d_1}$ , $F_{\text{global}}^{[l]}\in {\mathbb{R}}^{d_2}$ , and $F^{[last]}\in {\mathbb{R}}^{d_3}$ . Here, d₁, d₂, and d₃ denote the dimensions of the respective feature vectors, each projected to a common dimension via a linear layer before fusion. During the fusion phase, a function $\tilde{S}(\cdot )$ is utilized to compute the importance score for each feature. The function f(X) first applies a linear transformation followed by a non-linear activation, specifically:

$f(X)=\text{ReLU}(W_{f}X+b_f)$

(21)

where W _f is the weight matrix and b _f is the bias term for the transformation. The scoring function $\tilde{S}(X)$ is then defined as:

$\tilde{S}(X)=W_s\cdot f(X)+b_s$

(22)

where W _s is a weight matrix and b _s is a bias term, transforming the processed features into a scalar score. Subsequently, importance scores for each of the three features are calculated:

$w_{\text{local}}=\tilde{S}(F_{\text{local}}^{[l]}),w_{\text{global}}=\tilde{S}(F_{\text{global}}^{[l]}),w_{\text{last}}=\tilde{S}(F^{[last]})$

(23)

These weights are then normalized to ensure their sum equals 1:

$w_{\mathit{total}}=e^{w_{\text{local}}}+e^{w_{\text{global}}}+e^{w_{\text{last}}}$

(24)

and

$w_{\mathit{item}}^{\prime }=\frac{e^{w_{\mathit{item}}}}{w_{\mathit{total}}},item\in \{\text{local},\text{global},\text{last}\}$

(25)

Next, the normalized weights are combined with the three features to yield a composite feature:

$F^{[combined]}=w_{\text{local}}^{\prime }{F}_{\text{local}}^{[l]}+w_{\text{global}}^{\prime }{F}_{\text{global}}^{[l]}+w_{\text{last}}^{\prime }{F}^{[last]}$

(26)

Afterwards, a fully connected layer maps this fused feature into output space. The logits can be expressed as follows, where W _l denotes the linear transformation matrix and b represents the bias term:

$Z=W_l^T{F}^{[combined]}+b$

(27)

Finally, these logits are passed through a Softmax function to obtain class probability distributions:

$P(y=\tilde{c}|X)=\frac{\exp (Z_{\tilde{c}})}{{\sum }_{j^{\prime }=0}^1\exp (Z_{j^{\prime }})},\tilde{c}\in \{\mathrm{0,1}\}$

(28)

During training, the cross-entropy loss function evaluates the discrepancy between predicted results and true labels, where y represents the true label:

$L=-(y\log (P(y=1))+(1-y)\log (P(y=0)))$

(29)

4. Experiment

4.1. Datasets

Our dataset consists of 562 images obtained through CT scanning using a 128 channel multidetector SOMATOM Definition Edge CT scanner (Siemens Inc., Munich, Germany). Each scan is acquired with consistent imaging parameters, including an axial section thickness of 0.625 mm, collimation of 128× 0.6 mm, a field of view measuring 220× 220 mm, a pitch of 0.8, and a matrix size of 224× 224. Imaging is conducted at 120 kV and 240 mAs to maintain uniform image quality throughout the dataset. Clinical labels for each ear derive from definitive diagnoses recorded in patients’ medical records. The dataset maintains a balanced representation of cholesteatoma and normal cases. To safeguard against data leakage and ensure independence between training, validation, and test sets, we apply stratified 5 fold cross-validation at the patient level. This approach preserves the integrity of the evaluation process by preventing patient data overlap across folds. Finally, model selection relies on the average performance across validation folds, promoting robustness and generalizability in the results.

4.2. Implementation details

The proposed classification network is developed and evaluated using PyTorch 1.9.0 on an NVIDIA GeForce RTX 3090. Prior to training, all input CT scans are resampled to a uniform voxel size of 1mm × 1mm × 1mm to ensure consistent spatial resolution across subjects. Subsequently, intensity normalization is performed to achieve zero mean and unit variance throughout the dataset, standardizing the overall distribution of voxel values. To enhance model generalization and mitigate overfitting, various data augmentation strategies are applied, including random rotations, horizontal flips, and brightness adjustments. The network is optimized using the Adam algorithm with an initial learning rate of 0.001 and a weight decay of 1e-5, enabling efficient parameter updates while introducing L2 regularization for improved stability. A learning rate scheduler further refines the training process by reducing the learning rate by a factor of 0.1 every 15 epochs over a total of 100 training epochs, promoting smoother convergence. To prevent unnecessary training and further reduce overfitting, an early stopping strategy is employed, which monitors validation performance and halts training automatically when no improvement is observed over five consecutive epochs, ensuring optimal model performance.

4.3. Evaluation metrics

Accuracy quantifies the frequency with which a model correctly classifies instances. It is defined as the ratio of correctly predicted instances to the total number of instances:

$\text{Accuracy}=\frac{\text{TP}+\text{TN}}{\text{TP}+\text{TN}+\text{FP}+\text{FN}}$

(30)

Here, TP represents true positives, TN indicates true negatives, FP denotes false positives, and FN refers to false negatives.

Sensitivity measures the proportion of actual positive instances correctly identified by the model. It is calculated using the following formula:

$\text{Sensitivity}=\frac{\text{TP}}{\text{TP}+\text{FN}}$

(31)

Specificity assesses how effectively a model identifies negative instances, defined mathematically as:

$\text{Specificity}=\frac{\text{TN}}{\text{TN}+\text{FP}}$

(32)

The Area Under the ROC Curve (AUC) serves as a performance measure for classification models across various threshold settings. It quantifies how well a classifier distinguishes between different classes.

4.4. Validation of Gabor-local feature extraction

The experimental evaluation systematically assesses the effectiveness of Gabor-local feature extraction (GLFE) by comparing different models. Specifically, the analysis contrasts models incorporating GLFE with those relying solely on conventional convolutional layers (Conv) or operating without (w/o) GLFE. As demonstrated in Table 1, the integration of GLFE consistently leads to performance improvements. This enhanced performance stems from GLFE’s unique ability to extract highly discriminative local features, a crucial advantage when processing the complex and nuanced patterns present in cholesteatoma images. Consequently, the results clearly indicate that models enhanced with GLFE exhibit superior class discrimination abilities compared to their conventional counterparts, showcasing the value of localized feature extraction.

Table 1.

Validation of Gabor-local feature extraction.

Method	Accuracy (%)	Sensitivity (%)	Specificity (%)	AUC
Conv	87.36	80.64	89.34	0.8721
w/o GLFE	87.58	85.18	88.10	0.8704
w/GLFE	90.85	89.65	91.12	0.9021

4.5. Effectiveness of contextual-global feature aggregation

This experiment evaluates the specific contribution of Contextual-Global Feature Aggregation (CGFA) by comparing model performance w/and w/o this module. The results, detailed in Table 2, demonstrate a clear performance increase when CGFA is incorporated (accuracy: 90.85%, sensitivity: 89.65%, specificity: 91.12%, AUC: 0.9021). This enhancement is attributed to CGFA’s ability to utilize attention scores, which effectively prioritize relevant features while suppressing irrelevant ones. By dynamically adjusting feature weights based on calculated relevance, CGFA generates more robust global descriptors. Overall, these findings confirm the effectiveness of CGFA in improving the accuracy of this classification task.

Table 2.

Effectiveness of contextual-global feature aggregation.

Method	Accuracy (%)	Sensitivity (%)	Specificity (%)	AUC
w/o CGFA	87.59	81.48	88.89	0.8595
w/CGFA	90.85	89.65	91.12	0.9021

4.6. Impact of dynamic feature fusion

This subsection investigates the effectiveness of Dynamic Feature Fusion (DFF) in integrating local, global, and residual block features within the model architecture. To isolate the specific contribution of DFF, we compare its performance against simpler, static fusion techniques such as concatenation, summation, and averaging, tested across various configurations. The experimental results, presented in Table 3, reveal that DFF’s approach, which dynamically balances the influence of these diverse feature sources, yields superior model performance compared to the static methods. This outcome highlights the importance of adaptive feature fusion strategies like DFF, demonstrating their capability to create more discriminative representations and thereby improve overall classification performance.

Table 3.

Influence of dynamic feature fusion.

Method	Accuracy (%)	Sensitivity (%)	Specificity (%)	AUC
Concatenation	88.78	85.71	88.06	0.8657
Summation	87.65	86.67	87.80	0.8599
Averaging	88.24	83.87	89.34	0.8691
w/o DFF	88.12	81.48	89.88	0.8551
w/DFF	90.85	89.65	91.12	0.9021

4.7. Influence of pooling methods

This experiment evaluates the impact of different pooling strategies on global feature aggregation within the image classification model, specifically comparing Average Pooling, Max Pooling, and GeM Pooling. The empirical results, documented in Table 4, clearly demonstrate that GeM Pooling achieves superior performance compared to both Average and Max Pooling. This suggests that GeM Pooling’s mechanism, which generalizes the pooling operation, is more effective at capturing and retaining discriminative global features essential for classification. Ultimately, the use of GeM Pooling contributes to a more robust feature representation, leading to enhanced model performance.

Table 4.

Influence of pooling methods.

Method	Accuracy (%)	Sensitivity (%)	Specificity (%)	AUC
Average Pooling	89.14	80.64	90.16	0.8562
Max Pooling	88.79	82.14	89.60	0.8543
GeM Pooling	90.85	89.65	91.12	0.9021

4.8. Effect of different residual network depths

This experiment investigates the relationship between the depth of residual blocks in the ResNet architecture and its performance on classifying normal middle ear and cholesteatoma images. Initially, as depicted in Figure 5, increasing network depth leads to improved performance. This gain is attributable to the enhanced feature extraction capabilities of deeper models, which allow them to capture more intricate patterns crucial for distinguishing between the two medical conditions. However, this positive trend reverses beyond a certain threshold (specifically, a depth of 50). At greater depths, the model becomes susceptible to overfitting by learning noise and specifics from the training data rather than generalizable patterns, which consequently diminishes its performance on unseen data.

Figure 5.

Effect of different residual network depths.

4.9. Effectiveness of various input sizes

As shown in Figure 6, the experimental results demonstrate variations in performance across different input image sizes. Through detailed analysis, we observe that larger input dimensions, particularly 112 × 112 and 224 × 224 pixels, generally yield superior classification performance compared to smaller input sizes, albeit at the cost of increased computational overhead. However, our experiments reveal an interesting phenomenon: the network shows diminishing returns in performance improvements beyond 224 × 224, suggesting this resolution represents an optimal balance point between performance and computational efficiency. Based on these findings, we ultimately adopt the size of 224 × 224 as it offers a compelling compromise, maintaining relatively high performance while effectively reducing computational requirements compared to larger input dimensions.

Figure 6.

Effectiveness of various input sizes.

4.10. Visualization of attention maps

As shown in Figure 7, the attention map visualizations demonstrate that the proposed model could concentrate on key pathological regions within cholesteatoma images. This suggests that the importance score mechanism effectively directs the model’s focus toward diagnostically relevant areas. By highlighting these critical regions, the model is able to integrate local and global features more accurately, thereby enhancing its overall representation ability. Consequently, this targeted attention contributes to the observed improvement in classification performance.

Figure 7.

Visualizations of attention maps.

4.11. Comparison with state-of-the-art models

In this section, we conduct a comprehensive comparison of our proposed method with other state-of-the-art approaches to evaluate its performance. Here, the comparison methods including: Vision Transformer, ²⁰ DenseNet, ¹⁸ ConvNeXt, ²² SE-ResNet, ¹⁹ BoTNet, ²¹ iFormer, ²³ VAN ²⁵ and TransNeXt, ²⁴ MobileNetV2, ¹⁴ Inception V3 + SVM, ¹³ MedViTV2s. ³³ As illustrated in Table 5, the experimental results indicate that our method performs competitively against various models, showcasing its effectiveness in achieving superior results across multiple metrics. This competitive performance can be attributed to the use of Gabor-local feature extraction, which enables the model to capture essential local representations effectively. Additionally, the incorporation of contextual-global feature aggregation and dynamic feature fusion further enhances the model’s ability to integrate and leverage broader contextual information, ultimately contributing to its overall performance.

Table 5.

Comparison with state-of-the-art models.

Method	Accuracy (%)	Sensitivity (%)	Specificity (%)	AUC
VIT 20	88.98	81.48	89.77	0.8862
DenseNet 18	89.32	88.89	89.68	0.8589
ConvNeXt 22	89.87	83.87	90.98	0.8906
SE-ResNet 19	90.35	85.18	89.75	0.8810
BoTNet 21	89.75	86.67	88.62	0.8471
iFormer 23	89.67	85.18	88.86	0.8939
VAN 25	88.89	81.48	90.48	0.8645
TransNeXt 24	90.20	88.92	90.48	0.9015
MobileNetV2 14	90.03	89.51	90.56	0.8911
Inception V3 + SVM 13	89.86	88.81	90.91	0.8910
MedViTV2s 33	90.21	89.51	90.91	0.9020
Ours	90.85	89.65	91.12	0.9021

5. Discussion and conclusion

In this paper, our proposed model showcases superior performance in classifying cholesteatoma from normal middle ear images compared to state-of-the-art methods, emphasizing the efficacy of its integrated design. Specifically, the incorporation of Gabor filtering alongside the ResNet architecture effectively improves local feature extraction, enabling the model to capture nuanced textural and structural patterns critical for distinguishing cholesteatoma. Furthermore, the contextual-global feature aggregation enhances the model by generating an importance score map, enabling it to dynamically focus on the most salient image regions and facilitating the creation of a robust, global feature descriptor via GeM pooling. Additionally, the proposed dynamic feature fusion strategy combines Gabor-local features, contextual-global features, and deep semantic features from the final residual block based on their computed importance, supporting a more comprehensive multi-scale analysis. Clinically, this model holds promise by enhancing diagnostic performance, which is crucial for accurately distinguishing between cholesteatoma, requiring timely intervention, and a normal middle ear that does not. This improvement helps reduce misdiagnosis, ensure appropriate and timely patient management, and potentially decrease morbidity, avoid unnecessary surgeries, and optimize the use of healthcare resources in otolaryngology. Besides, this paper is conducted at a single center with a retrospective design, which introduces selection bias and limits the generalizability of the results. The dataset consists exclusively of CT images and does not include MRI data, which provides complementary diagnostic information. To address these limitations, we plan a prospective multicenter study to validate the model in a larger and more diverse patient population, incorporating multimodal imaging data, including MRI, for comprehensive evaluation.