Sage Journals: Discover world-class research

Abstract

Study Design

Retrospective diagnostic study.

Objectives

To develop a fine-grained classification model based on deep learning using X-ray images, to screen for scoliosis, and further to screen for atypical scoliosis patterns associated with Chiari Malformation type I (CMS).

Methods

A total of 508 pairs of coronal and sagittal X-ray images from patients with CMS, adolescent idiopathic scoliosis (AIS), and normal controls (NC) were processed through construction of the ResNet-50 model, including the development of ResNet-50 Coronal, ResNet-50 Sagittal, ResNet-50 Dual, ResNet-50 Concat, and ResNet-50 Bilinear models. Evaluation metrics calculated included accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for both the scoliosis diagnosis system and the CMS diagnosis system, along with the generation of receiver operating characteristic (ROC) curves and heatmaps for CMS diagnosis.

Results

The classification results for the scoliosis diagnosis system showed that the ResNet-50 Coronal model had the best overall performance. For the CMS diagnosis system, the ResNet-50 Coronal and ResNet-50 Dual models demonstrated optimal performance. Specifically, the ResNet-50 Dual model reached the diagnostic level of senior spine surgeons, and the ResNet-50 Coronal model even surpassed senior surgeons in specificity and PPV. The CMS heatmaps revealed that major classification weights were concentrated on features such as atypical curve types, significant lateral shift of scoliotic segments, longer affected segments, and severe trunk tilt.

Conclusions

The fine-grained classification model based on the ResNet-50 network can accurately screen for atypical scoliosis patterns associated with CMS, highlighting the importance of radiographic features such as atypical curve types in model classification.

Keywords

Chiari malformation atypical scoliosis adolescent idiopathic scoliosis deep learning fine-grained classification

Introduction

Chiari malformation (CM), also known as cerebellar tonsillar herniation, is a congenital malformation at the craniovertebral junction characterized primarily by underdevelopment of the posterior cranial fossa and herniation of the cerebellar tonsils through the foramen magnum.^1,2 CM can be divided into 4 types based on anatomical and radiological features, among which Chiari malformation type I (CMI) is the most common subtype.³ Its characteristic feature is a small posterior cranial fossa with cerebellar tonsillar descent below the line of the foramen magnum by more than 5 mm, but normal development of the cerebellum and brainstem.^4,5 Scoliosis is one of the most common complications in children with CMI (scoliosis secondary to CMI, CMS), with an incidence rate of 15-50%. Unlike adolescent idiopathic scoliosis (AIS), CMS patients often have syringomyelia, and even present with neurological symptoms, and the scoliosis is more likely to progress.^5,6 Early posterior fossa decompression (PFD) surgery is necessary to halt disease progression, hence early diagnosis is necessary.

Full-spine X-ray is the widely used method for diagnosis of scoliosis, however, it is difficult to identify CMS from AIS using X-ray images. It has been reported that 2-26% of patients initially diagnosed with idiopathic scoliosis have neurological anomalies, with CMS and syringomyelia being the most common.⁷ Missed or incorrect diagnoses can severely affect treatment outcomes. Magnetic Resonance Imaging (MRI) is the golden standard used for diagnosis of CMI, however, conducting full-spine MRI screenings for all patients with scoliosis does not conform to the principles of medical economics.⁸ Therefore, it is of great importance to quickly and accurately identify CMS from AIS on X-ray images. While previous research has applied machine learning for Chiari malformation screening using MRI data,⁹ to date, no studies have reported the use of X-ray radiographs for screening scoliosis associated with Chiari malformation. Previous research has found that CMS patients exhibit radiological features of neuromuscular scoliosis, with a higher incidence of atypical scoliosis types such as left thoracic, double thoracic, and left thoracic-right lumbar curves.¹⁰ These atypical curve types are more frequently observed in CMS patients compared to AIS. Unlike typical idiopathic scoliosis, where the curve tends to be a right-sided thoracic curve, CMS often presents with unusual scoliosis patterns that may include more complex curve configurations and more severe trunk tilt. As a result, CMS is often categorized as a neuromuscular rather than idiopathic form of scoliosis, due to its distinct radiological features and early progression. However, the recognition of these atypical characteristics is based on rich clinical experiences, and most spine surgeons still face significant challenges in deciding when to perform a full-spine MRI to identify CMS. Early diagnosis is crucial, as timely identification of CMS could lead to early surgical intervention which may prevent further curve progression and alleviate potential neurological deficits. This highlights the clinical need for more efficient, accurate, and non-invasive screening tools that can help differentiate atypical scoliosis related to CMS from typical AIS.

Fine-grained image classification refers to the identification of subcategories within a major image category. The challenge and distinction of fine-grained image analysis tasks from general or generic image tasks lie in the finer granularity of the image categories involved. In recent years, fine-grained classification models based on deep learning technology have rapidly developed, with models such as Inception,¹¹ ResNet,¹¹ DenseNet,¹² and EfficientNet¹³ emerging and achieving remarkable success in image recognition, natural language processing, medical image analysis, and bioinformatics. Our team’s previous research developed the Bilateral CNN fine-grained classification model, which can accurately identify the atrophic characteristics of scoliosis secondary to neurofibromatosis type 1 (NF1-S), thus diagnosing atrophic and non-atrophic NF1-S with an accuracy of 80.36%.¹⁴

The objectives of this study are to: (1) develop a fine-grained classification model based on deep learning technology for accurate diagnosis of scoliosis from X-ray images; (2) apply the fine-grained classification model to achieve accurate identification of X-ray images of atypical scoliosis associated with CMS patients and explore radiological features that influence the model’s classification performance using heatmaps.

Materials and Methods

Subjects

This study protocol was approved by our hospital’s ethics committee (IRB20030115), and informed consent was obtained from all patients and their families. We conducted a retrospective analysis of patients diagnosed with CMS who underwent coronal and sagittal standing full-spine X-ray and cranial MRI examinations at our hospital from June 2016 to November 2023. The inclusion criteria were as follows: (1) diagnosed with CMS with syringomyelia, patients with AIS (case control), and normal adolescents (normal control, NC), all having complete clinical and radiological records; (2) underwent standing full-spine coronal and sagittal X-ray examinations for diagnostic research purposes at first visit. The exclusion criteria were as follows: (1) under 11 years of age or over 18 years old; (2) history of spine surgery or PFD surgery; (3) concurrent spinal trauma or spinal tumors; (4) congenital spinal developmental abnormalities. Finally, a total of 128 CMS cases, along with 200 age- and gender-matched AIS cases and 180 normal adolescents, were included in the study dataset, serving as the CMI group, AIS group, and NC group, respectively.

Datasets and Preprocessing

This study included data from 508 subjects at our hospital, comprising 256 coronal and sagittal X-ray images of 128 CMS patients, 400 coronal and sagittal X-ray images of 200 AIS patients, and 360 coronal and sagittal X-ray images of 180 normal adolescents. 70% of the subjects’ X-ray images were used for training, 20% of the subjects’ X-ray images were used for validation, with the remainder for testing. To ensure the image quality was suitable for deep learning, 100% original data in DICOM format were collected. Considering the higher number of AIS cases in actual classification tasks, the number of AIS patients in the test dataset was higher than that of CMS patients.

The original images in the dataset were uniformly resized to 512*512 pixels and normalized and standardized to facilitate fitting with the pre-trained model. Given the large number of network parameters relative to the smaller training set size and the imbalance among groups in the training set, we used the Augmentor package (https://github.com/mdbloice/Augmentor) for data augmentation. This included random horizontal flips, rotations of 0-10°, translations of 0-15%, and scaling from 75% to 125%. The final dataset grouping is shown in Table 1.

Table 1.

Training, Validation and Testing Dataset Volumes.

	CMS	AIS	NC
Training set (before data augmentation)	90	140	126
Training set (after data augmentation)	270	280	252
Validation set	26	40	36
Testing set	12	20	18

Experimental Environment and Hyperparameters

This study constructed the network architecture using PyTorch (https://pytorch.org/) and MMClassification (https://github.com/open-mmlab/mmpretrain/blob/master/README.md), with the PyTorch version being 1.12.1 and Python version 3.8. We selected a batch size of 16 for the experimental setup, as validation results indicated this hyperparameter choice yielded optimal performance. The complete hyperparameter settings for the ResNet-50 model are presented in Table 2.

Table 2.

Model Hyperparameter Settings.

Parameter Initial Weight Source	Optimizer	Batch Size	Learning Rate^a	Momentum	Weight Decay Rate	Epoch
ImageNet	SGD	16	0.1	0.9	0.0001	100

^aThis study employs a learning rate scheduling strategy where the learning rate is reduced to 10% of its original value at epochs 30, 60, and 90. That is, the initial learning rate is 0.1, which is then reduced to 0.01 after the 30th epoch, further reduced to 0.001 after the 60th epoch, and finally lowered to 0.0001 after the 90th epoch.

Model Development

ResNet-50 Model Architecture

The image classification model constructed in this study is based on the structure of ResNet-50, developed by Microsoft Research in 2015. ResNet-50 is a classic deep convolutional neural network whose core idea involves introducing residual blocks to construct skip connections, effectively alleviating the problem of gradient vanishing in deep networks. ResNet-50 consists of a 50-layer network structure (Figure 1), mainly including the following 3 parts:

(1) Backbone: Serving as the main structure (backbone) of the model, it contains 4 stages, with the output from the last stage used for feature extraction.

(2) Neck: The “neck” structure after the output of the Backbone, where we employ Global Average Pooling (GAP). GAP operation compresses the feature map into a fixed-size vector while preserving its spatial information.

(3) Head: For the classification head, we use a fully connected layer for classification, setting the number of output categories of the model to 3. To compute the loss for imbalanced datasets, we chose Focal Loss.15 When assessing the model’s performance, we utilized Top-1 accuracy.

Figure 1.

This modified ResNet-50 architecture for X-ray image classification consists of 4 stages of convolutional layers for feature extraction, followed by a classification head. Two variants are shown: ResNet-50 Concat, where outputs from 2 streams are concatenated before classification, and ResNet-50 Bilinear, which uses bilinear pooling of the streams’ features for final classification with a SoftMax layer.

X-ray Classification Model

To test the diagnostic performance of coronal and sagittal X-ray images, we constructed 5 ResNet-50 models, including ResNet-50 Coronal, ResNet-50 Sagittal, ResNet-50 Dual, ResNet-50 Concat, and ResNet-50 Bilinear. Among these, the first 3 are single-stream models, and the last 2 are dual-stream models:

(1) ResNet-50 Coronal Model: A single coronal X-ray image is input into the ResNet-50 model to predict the scoliosis classification, where $I_{c o r}$ represents the coronal X-ray image, $Y$ represents the classification result, and $f$ represents the ResNet-50 model.

Y = f (I_{c o r})

(2) ResNet-50 Sagittal Model: A single sagittal X-ray image is input into the ResNet-50 model to predict the scoliosis classification, where $I_{s a g}$ represents the sagittal X-ray image, $Y$ represents the classification result, and $f$ represents the ResNet-50 model.

Y = f (I_{s a g})

(3) ResNet-50 Dual Model: Both coronal and sagittal X-ray images are concatenated along their longest axis and then input into the ResNet-50 model to predict the scoliosis classification, where $C o n c a t (I_{c o r}, I_{s a g})$ represents the input of concatenated coronal and sagittal X-ray images, $Y$ represents the classification result, and $f$ represents the ResNet-50 model.

Y = f (C o n c a t (I_{c o r}, I_{s a g}))

(4) ResNet-50 Concat Model: coronal and sagittal X-ray images are separately input into the ResNet-50 model, and the extracted image features are concatenated along the channel dimension (Figure 1) to predict the scoliosis classification using the merged features. Here, $f (I_{c o r})$ and $f (I_{s a g})$ represent the features extracted from the coronal and sagittal X-ray images through the ResNet-50 model, $C o n c a t$ represents the concatenation of these features, $g$ represents the function using concatenated features for classification, $Y$ represents the classification result, and $f$ represents the ResNet-50 model.

Y = g (C o n c a t (f (I_{c o r}), f (I_{s a g})))

(5) ResNet-50 Bilinear Model: coronal and sagittal X-ray images are separately input into the ResNet-50 model, and the extracted image features are used to compute the tensor outer product (Figure 1) to predict the scoliosis classification using the merged features. Here, $T e n s o r O u t e r P r o d u c t$ represents the tensor outer product calculation of 2 sets of features, $f (I_{c o r})$ and $f (I_{s a g})$ represent the features extracted from the coronal and sagittal X-ray images respectively, $g$ represents the function using the pooled features for classification, $Y$ represents the classification result, and $f$ represents the ResNet-50 model.

Y = g (T e n s o r O u t e r P r o d u c t (f (I_{c o r}), f (I_{s a g})))

Performance Evaluation

Diagnostic Test Indicators

Diagnostic test indicators are calculated based on the number of true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN) as follows:

(1) Accuracy: Accuracy is the ratio of correctly predicted observations to the total observations. It describes the ability of the classifier to correctly predict all predictions.

Accuracy = \frac{T P + T N}{T P + T N + F P + F N}

(2) Sensitivity (also known as Recall): Sensitivity describes the classifier’s ability to correctly identify positive instances. It is the proportion of all true positive instances that were correctly predicted as positive.

Sensitivity = \frac{T P}{T P + F N}

(3) Specificity: Specificity describes the classifier’s ability to correctly identify negative instances. It is the proportion of all true negative instances that were correctly predicted as negative.

Specificity = \frac{T N}{T N + F P}

(4) Positive Predictive Value (PPV, also known as Precision): The positive predictive value describes the proportion of instances predicted as positive that are actually true positive instances.

P P V = \frac{T P}{T P + F P}

(5) Negative Predictive Value (NPV): The negative predictive value describes the proportion of instances predicted as negative that are actually true negative instances.

NPV = \frac{T N}{T N + F N}

X-ray Image Classification Tasks

Task One: Scoliosis Diagnosis System

CMS and AIS patients are considered positive cases, while normal adolescents (NC) are considered negative cases. The classification system’s accuracy, sensitivity, specificity, PPV, NPV, and the confusion matrix are calculated.

Task Two: CMS Diagnosis System

CMS patients are considered positive cases, while AIS patients and NC are considered negative cases. The classification system’s accuracy, sensitivity, specificity, PPV, NPV, and the confusion matrix are calculated. Additionally, 2 spine surgery experts (Expert 1 with 24 years of clinical experience and Expert 2 with 10 years of clinical experience) also assess the images to compare the gap in diagnostic capabilities with the model.

Visualization of Experimental Results

To explore how the ResNet-50 model makes decisions, we use Class Activation Mapping (CAM)¹⁶ to “visualize” the image areas that the model focuses on when making decisions. We generate CAMs using the convolutional feature maps from the model’s last convolutional layer and the gradients of these feature maps for the target class. The generated CAM is a 2D array the same size as the original image, representing the importance of each area of focus for the model. To make it more readable for spinal surgeons, we convert the CAM into a heatmap and overlay it on the original image. In this heatmap, red areas indicate regions considered most important by the model, while blue or darker areas indicate regions considered less important.

Statistical and Data Analysis

Results are presented as single values, with means ± standard errors of the mean (SEM). The comparison between CMS and AIS groups is performed using independent sample t-tests, while comparison among 3 groups is performed using One-way analysis of variance (ANOVA), with the significance level α set at 0.05 for two-tailed tests. We performed statistical analyses with SPSS 22.0 software (SPSS Inc., USA). To evaluate the model’s performance in diagnosing CMS, we use Python (v3.8.16) and various Python packages (including NumPy v1.23.5, scikit-learn v1.3.2, Matplotlib v3.7.1) to plot Receiver Operating Characteristic (ROC) curves and calculate the Area Under the Curve (AUC).

Results

This study included a total of 508 participants, with 356 in the training set, 102 in the validation set and 50 in the testing set. We recorded the age, gender, Risser sign for all the subjects, and evaluated scoliosis-related parameters such as Cobb angle, flexibility, coronal balance and sagittal vertical axis (SVA) for the CMS and AIS groups. Detailed demographic information is presented in Table 3, the 3 groups were age-, gender- and Risser grade-matched. There were no significant differences of scoliosis-related parameters between CMS and AIS group, except for curve flexibility in main curve. CMS patients seemed to have more rigid curvature with lower flexibility than AIS patients.

Table 3.

Demographic Information of the Datasets.

Group	CMS	AIS	Normal	P
N (patients)	128	200	180	-
Age (years)	15.2 ± 2.5	15.1 ± 1.6	15.4 ± 2.0	.337
Gender (male/female)	61/67	92/107	82/98	.935
Risser	2.7 ± 2.1	2.8 ± 1.8	3.0 ± 1.9	.364
Main curve, °	46.4 ± 6.8	45.8 ± 4.9	-	.354
Flexibility, %	43.4 ± 3.4	49.3 ± 5.1	-	<.001
Secondary curve, °	35.4 ± 2.8	34.9 ± 5.3	-	.326
Flexibility, %	78.3 ± 8.3	80.3 ± 10.2	-	.064
Coronal balance, mm	14.7 ± 4.2	15.3 ± 3.4	-	.157
Sagittal vertical axis, mm	−15.2 ± 23.5	−14.7 ± 22.8	-	.848

Task One: Scoliosis Diagnosis System

The classification system for scoliosis (including CMS and AIS) vs normal adolescents is shown in Table 4. Compared to other models, the ResNet-50 Coronal model possesses significantly better classification performance, with accuracy, sensitivity, specificity, PPV, and NPV reaching 94.00%, 90.63%, 100.00%, 100.00%, and 85.71%, respectively. Although the ResNet-50 Bilinear model achieved a sensitivity of 96.88%, as shown in the confusion matrix in Figure 2, the ResNet-50 Bilinear model was nearly unable to recognize NC, leading to an abnormally high sensitivity.

Table 4.

The Quantitative Evaluation of Scoliosis Diagnosis System.

Model	Accuracy	Sensitivity	Specificity	PPV	NPV
ResNet-50 coronal	94.00%	90.63%	100.00%	100.00%	85.71%
ResNet-50 sagittal	64.00%	81.25%	33.33%	68.42%	50.00%
ResNet-50 dual	80.00%	81.25%	77.78%	86.67%	70.00%
ResNet-50 concat	84.00%	81.25%	88.89%	92.86%	72.73%
ResNet-50 bilinear	64.00%	96.88%	5.56%	64.58%	50.00%

^aThe best-performing results highlighted in bold.

Figure 2.

The confusion matrix for the classification system involving CMS, AIS, and NC: The vertical and horizontal axes represent the true labels (from top to bottom: CMS, AIS, and NC) and the predicted labels (from left to right: CMS, AIS, and NC), respectively.

Task Two: CMS Diagnosis System

The classification system distinguishing CMS patients from other subjects (including AIS and NC) is depicted in Table 5. The ResNet-50 Coronal and ResNet-50 Dual models exhibit relatively superior classification performance, exceeding the diagnostic capabilities of mid-career spine surgery experts. Notably, the ResNet-50 Dual model attains the diagnostic level of senior spine surgery experts, with the ResNet-50 Coronal model surpassing senior experts in specificity and PPV. The classification accuracy, sensitivity, specificity, PPV, and NPV for the ResNet-50 Coronal model are 82.00%, 41.67%, 94.74%, 71.43%, and 83.72%, respectively. For the ResNet-50 Dual model, these figures are 84.00%, 58.33%, 92.11%, 70.00%, and 87.50%. Although the ResNet-50 Bilinear model achieves a sensitivity of 66.67%, its specificity and PPV are low, indicating a propensity to diagnose most patients as CMS, leading to numerous misdiagnoses, as shown in the confusion matrix in Figure 2. The AUCs of the ROC curves for diagnosing CMS with the ResNet-50 Coronal and ResNet-50 Dual models are 0.85 (95% Confidence Interval (CI), 0.66-0.98) and 0.78 (95% CI, 0.58-0.93), respectively, with the ROC curves illustrated in Figure 3.

Table 5.

The Quantitative Evaluation of CMS Diagnosis System.

Model	Accuracy	Sensitivity	Specificity	PPV	NPV
ResNet-50 coronal	82.00%	41.67%	94.74%	71.43%	83.72%
ResNet-50 sagittal	64.00%	33.33%	73.68%	28.57%	77.78%
ResNet-50 dual	84.00%	58.33%	92.11%	70.00%	87.50%
ResNet-50 concat	78.00%	41.67%	89.47%	55.56%	82.93%
ResNet-50 bilinear	50.00%	66.67%	44.74%	27.59%	80.95%
Expert 1 (senior)	84.00%	58.33%	92.11%	70.00%	87.50%
Expert 2 (mid-career)	78.00%	50.00%	86.84%	54.55%	84.62%

^aThe best-performing results highlighted in bold.

Figure 3.

The ROC curves for diagnosing CMS with the ResNet-50 Coronal model (left) and the ResNet-50 Dual model (right).

Visualization Heatmaps Based on CAM

Figure 4 presents heatmaps generated by the ResNet-50 Dual model using CAM, showcasing the visualized image features of 2 correctly classified CMS patients. In the image, areas with higher weights are colored red, primarily located on the right side of the thoracic curve and the lumbar curve in the coronal X-ray images, as well as the thoracic kyphosis and the lumbar-sacral region in the sagittal X-ray images. Based on this figure, it can be inferred that atypical curve types, significant lateral shift of scoliotic segments, length of the affected segments, and severe trunk tilt are important factors influencing the model’s classification outcome.

Figure 4.

The heatmaps highlighting critical areas affecting accurate CMS classification by the ResNet-50 Dual model. (A) A 16-year-old male with CMS, characterized by atypical left thoracic curvature and severe vertebral segment lateral shift; (B) A 13-year-old female with CMS, characterized by an atypical long thoracic curve and significant trunk tilt.

Discussion

In recent studies, machine learning approaches have been used to assist in the identification of Chiari I malformation, often relying on MRI for diagnosis and surgical decision-making.^9,17-19 Mesin et al⁹ proposed a machine learning model that leverages morphometric indices extracted from sagittal MRI to predict patients at higher risk of syringomyelia and clinical deterioration following surgery. The study achieved a classification accuracy of 71%, demonstrating the potential of MRI-based machine learning approaches to support neurosurgical management.

Our approach significantly differs from the aforementioned studies, we employ full-spine X-ray images for detection of atypical scoliosis associated with Chiari malformation, making our method more accessible and cost-effective. Full-spine X-rays are routinely performed in clinical practice for scoliosis diagnosis, offering a readily available and non-invasive imaging modality that avoids the need for more expensive and resource-intensive MRI scans.

The pathogenesis of CMS is complex and remains not fully understood. Previous research by our team indicated that 70% of Chiari malformation patients have the primary direction of cerebellar tonsillar herniation aligned with the primary curve direction of their scoliosis.²⁰ Yeom et al found that 83% of patients with syringomyelia had cavities that were biased toward the same direction as the primary curve of their scoliosis.²¹ Attenello et al discovered that for CMS patients with a Cobb angle less than 40°, posterior fossa decompression (PFD) surgery could halt the progression of scoliosis, whereas for those with a Cobb angle greater than 40°, deformity continued to worsen after PFD.²² Based on these clinical observations, it can be inferred that early-stage CM and syringomyelia-induced neurological damage are significant factors contributing to the progression of scoliosis. Subsequently, our team, using multiple invigorated high-definition diffusion imaging methods and fiber disappearance technology, identified microstructural damage in spinal cord nerve fibers in CMS patients with syringomyelia,^23,24 further corroborating clinical observations.

Therefore, compared to idiopathic scoliosis, CMS more frequently exhibits characteristics of neuromuscular scoliosis, including longer affected segments, significant lateral shifts in scoliotic segments, a higher occurrence of atypical curve types, and severe trunk tilt. These atypical features are distinct from those observed in AIS. Additionally, CMS is characterized by earlier onset, faster progression, higher brace treatment failure rate, more clinical symptoms, and a higher proportion of males,^25-27 all of which can serve as clues for achieving precise diagnosis of CMS in clinical settings.

Studies have reported that among patients initially diagnosed with AIS, the prevalence of neuroaxial abnormalities detected by MRI varies considerably, typically ranging between 2% and 12%. Consequently, routine MRI screening is recommended for scoliosis patients to identify underlying neuroaxial anomalies, as these findings significantly impact treatment decisions. As highlighted by Heemskerk et al,²⁸ the presence of neuroaxial anomalies often necessitates neurosurgical interventions prior to scoliosis correction to prevent potential complications during deformity surgery.

Furthermore, missed diagnoses of Chiari malformation can lead to severe clinical consequences. Studies indicate that among patients undergoing spinal fusion, those with Chiari malformation combined with syringomyelia experience intraoperative neuromonitoring abnormalities—indicative of potential neural injury—at rates as high as 28%.²⁹ Additionally, untreated Chiari-associated syringomyelia may progressively enlarge, compressing or damaging spinal neural pathways and neurons, potentially leading to worsening symptoms or even tethered cord syndrome. Importantly, delays in diagnosing and managing these neuroaxial abnormalities can result in irreversible neurological impairment, emphasizing the necessity for timely PFD.

While MRI remains the gold standard for detecting neuroaxial abnormalities, its routine application imposes substantial economic costs and resource burdens. Overuse of MRI in asymptomatic patients or those without clear clinical indications can lead to unnecessary healthcare expenditures, patient anxiety, and potentially unwarranted interventions. Although current literature lacks direct comparative cost-effectiveness analyses between routine and selective MRI screening strategies, existing studies indicate poor predictive ability for MRI screening criteria, resulting in inefficient selection processes. Routine MRI screening thus generates significant economic pressure, strains healthcare resources, and may exacerbate patient and family anxiety due to overdiagnosis and uncertain clinical implications.³⁰

Studies have consistently shown that CMS often presents with left-sided thoracic curves, multiple curve patterns, and severe sagittal imbalances, all of which are challenging to differentiate from other forms of scoliosis. Such atypical curve features, while distinguishable to some extent, are often difficult to identify based solely on visual inspection or traditional radiological methods due to the overlap with other scoliosis subtypes. This is where our model plays a pivotal role. By leveraging advanced image classification techniques, our deep learning-based model is capable of identifying subtle and complex patterns in radiographs that are difficult to distinguish using conventional methods. These atypical scoliosis patterns, such as those commonly found in CMS, represent a crucial area where machine learning can significantly enhance early detection and diagnosis, providing a more accurate and efficient tool for clinicians to differentiate between AIS and neurogenic forms of scoliosis.

Fine-grained image classification has emerged as a promising technique to address the diagnostic challenges mentioned above. The task is challenging due to minimal inter-class differences, substantial intra-class variations, and factors such as perspective, background, and occlusion. The classification strategies can be generally categorized into several approaches. The first is the generic deep convolutional neural network (DCNN) method, represented by our ResNet-50 Corona l, Sagittal, and Dual models, which have demonstrated excellent performance across various image classification tasks. Another approach involves localization-based methods that identify distinctive local regions within images; these include supervised methods requiring detailed annotations and weakly supervised methods using only category labels.³¹ A third approach is the network ensemble strategy, which combines multiple specialized DCNNs to enhance accuracy, exemplified by our ResNet-50 Concat model. Lastly, higher-order feature encoding techniques, such as the bilinear model used in our ResNet-50 Bilinear approach, transform DCNN-extracted features into higher-dimensional descriptors, capturing more complex image patterns.

The backbone network used in this study, ResNet-50, is a widely utilized convolutional neural network structure known for its efficiency and superior performance due to residual connections that effectively mitigate gradient vanishing issues. To enhance the generalization capability and reduce overfitting, we diversified the limited training dataset through common data augmentation methods such as flipping, rotating, translating, and scaling. Additionally, to address dataset imbalance, we employed the Focal loss function, which emphasizes hard-to-classify positive samples, thereby improving performance on minority classes.

The diagnostic test results of this study demonstrate that the use of coronal X-ray images alone can achieve rapid and accurate diagnosis of scoliosis. When sagittal X-ray images are used alone or used in combination with coronal X-ray images, the performance of the model is actually reduced, indicating that compared to providing supplementary information, sagittal X-ray images bring more noise into the task of scoliosis screening. In diagnosing CMS patients, the ResNet-50 Dual and ResNet-50 Coronal models achieved levels comparable to or even exceeding that of senior spine surgery experts. However, although these models surpassed 2 human experts in confirmatory indicators (specificity and PPV), effectively reducing the occurrence of false positives, they only exceeded mid-career spinal surgeons in screening indicators (sensitivity and NPV) and matched the level of senior spinal surgeons, indicating a significant risk of missed diagnoses. Therefore, further optimization of the model’s performance is needed in the future.

Due to sample size limitations, this study did not attempt to use models with a larger number of parameters. Increasing the sample size and trying the application of multiple models might achieve better diagnostic results. The current model currently focuses specifically on detecting atypical scoliosis patterns associated with CMS. While the model demonstrates promising results for CMS, its applicability to other neuroaxial abnormalities remains limited. Future work should aim to enhance the generalization ability of the model, allowing it to accurately recognize atypical scoliosis patterns associated with other neuroaxial abnormalities, such as cerebral palsy, tethered cord syndrome, and spinal cord tumors. Expanding the dataset and incorporating a wider range of neuroaxial conditions will be crucial for improving the model’s performance and ensuring its broader clinical utility. This study is a single-center research, and future work should include external test sets to further test the model’s classification performance.

Conclusion

This is the first study using a fine-grained image classification algorithm based on the ResNet-50 network for identification of atypical scoliosis associated with CMS by X-ray images. Using a generic DCNN approach, a strategy based on network ensemble, and a higher-order encoding strategy of convolutional features, we constructed 5 models for scoliosis screening and CMS diagnosis. Among them, the ResNet-50 Dual and ResNet-50 Coronal models achieved favorable classification results, with heatmaps indicating that image features such as atypical curve types hold significant weight in the classification task. Therefore, spinal surgeons, combining the diagnostic results of the model with patients’ clinical data, can hope to reduce the rates of missed diagnoses of CMS, enabling early intervention to halt the progression of scoliosis. Moreover, by providing a more accurate and efficient screening tool, this model could help reduce the over-reliance on MRI scans, saving healthcare resources and improving clinical decision-making.

Footnotes

Author Note

The manuscript submitted does not contain information about medical device(s)/drug(s). No relevant financial activities outside the submitted work.

Author Contributions

Z.Z.Z., A.Y.L., X.D.Q. initiated the project and the collaboration. Y.C., Z.H., K.G.Y. developed the network architectures, training, and testing setup. Z.L., J.C.C., W.Y.L. designed the clinical setup. E.C.C., Y.Q., X.P.C. created the data set and defined clinical labels. N.L., X.L.L. contributed to the software engineering. Z.H., X.L.L. contributed algorithmic expertise. Y.C., Z.H., Z.Z.Z., A.Y.L. wrote the paper. Y.C., Z.H., K.G.Y have access to and verify the underlying study data.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Key R&D Program of China (2023YFC2507700), the Natural Science Foundation of Jiangsu Province (BK20230147), the AO Spine Asia Pacific National Research Grant (AOSRG2024023), the China Postdoctoral Science Foundation (2022M711581), the Nanjing Medical Science and Technology Development Foundation (YKK22098), and the Jiangsu Provincial Key Research and Development Program (BE2023658).

Ethical Statement

ORCID iDs

Yi Chen

Zhong He

Zhen Liu

Zezhang Zhu

Data Availability Statement

The source code and trained models of ResNet-50 can be requested from the corresponding author upon reasonable request. Images and sensitive patient privacy information will not be disclosed. Additionally, access to this data will require signing a data transfer or access agreement, subject to case-by-case review by Nanjing Drum Tower Hospital.*

References

Loukas

Noordeh

Shoja

Pugh

Oakes

Tubbs

. Hans Chiari (1851-1916). Childs Nerv Syst. 2008;24(3):407-409. doi:10.1007/s00381-007-0535-y

Albert

. Chiari malformation in children. Pediatr Clin North Am. 2021;68(4):783-792. doi:10.1016/j.pcl.2021.04.015

Azahraa Haddad

Qaisi

Joudeh

, et al. The newer classifications of the chiari malformations with clarifications: an anatomical review. Clin Anat. 2018;31(3):314-322. doi:10.1002/ca.23051

Meadows

Kraut

Guarnieri

Haroun

Carson

. Asymptomatic Chiari Type I malformations identified on magnetic resonance imaging. J Neurosurg. 2000;92(6):920-926. doi:10.3171/jns.2000.92.6.0920

Tubbs

Beckman

Naftel

, et al. Institutional experience with 500 cases of surgically treated pediatric Chiari malformation Type I. J Neurosurg Pediatr. 2011;7(3):248-256. doi:10.3171/2010.12.PEDS10379

Dyste

Menezes

. Presentation and management of pediatric Chiari malformations without myelodysplasia. Neurosurgery. 1988;23(5):589-597. doi:10.1227/00006123-198811000-00008

Qiu

Wang

Zhu

. The left thoracic curve pattern: a strong predictor for neural axis abnormalities in patients with “idiopathic” scoliosis. Spine (Phila Pa 1976). 2010;35(2):182-185. doi:10.1097/BRS.0b013e3181ba6623

van Beek

EJR

Kuhl

Anzai

, et al.

Value of MRI in medicine: more than just another test?

J Magn Reson Imaging. 2019;49(7):e14-e25. doi:10.1002/jmri.26211

Mesin

Ponzio

Carlino

, et al. A machine learning approach to support treatment identification for Chiari I malformation. Appl Sci. 2022;12(18):9039.

10.

Kelly

Guillaume

Lenke

. Spinal Deformity Associated with Chiari Malformation. Neurosurg Clin N Am 2015;26(4):579-585. doi:10.1016/j.nec.2015.06.005.

11.

Szegedy, C, Wei L, Yangqing J, Pierre S, Scott R, Dragomir A, Dumitru E, Vincent V, and Andrew R . “Going deeper with convolutions.” In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1-9. 2015.

12.

Huang

Liu

Van Der Maaten

Weinberger

. Densely connected convolutional networks. IEEE Computer Society 2017;4700-4708. doi:10.1109/CVPR.2017.243

13.

Tan

. Efficientnet: rethinking model scaling for convolutional neural networks. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, Long Beach, 9-15 June 2019:6105-6114.

14.

Wang

Qin

, et al. Classification of neurofibromatosis-related dystrophic or nondystrophic scoliosis based on image features using Bilateral CNN. Med Phys. 2021;48(4):1571-1583. doi:10.1002/mp.14719

15.

Lin

T-Y

Goyal

Girshick

Dollár

. Focal loss for dense object detection. IEEE Computer Society 2017;2980-2988. doi:10.1109/ICCV.2017.324

16.

Zhou

Khosla

Lapedriza

Oliva

Torralba

. Learning deep features for discriminative localization. IEEE Computer Society 2016;2921-2929. doi:10.1109/CVPR.2016.319

17.

Urbizu

Martin

Moncho

, et al. Machine learning applied to neuroimaging for diagnosis of adult classic Chiari malformation: role of the basion as a key morphometric indicator. J Neurosurg. 2018;129(3):779-791. doi:10.3171/2017.3.JNS162479

18.

Mesin

Mokabberi

Carlino

. Automated morphological measurements of brain structures and identification of optimal surgical intervention for Chiari I malformation. IEEE J Biomed Health Inform. 2020;24(11):3144-3153. doi:10.1109/JBHI.2020.3016886

19.

Lin

Liu

Dai

, et al. Diagnostic performance evaluation of adult Chiari malformation type I based on convolutional neural networks. Eur J Radiol. 2022;151:110287. doi:10.1016/j.ejrad.2022.110287

20.

Zhu

Yan

Han

, et al. Radiological features of scoliosis in Chiari I malformation without syringomyelia. Spine (Phila Pa 1976). 2016;41(5):E276-E281. doi:10.1097/BRS.0000000000001406

21.

Yeom

Lee

Park

, et al. Scoliosis associated with syringomyelia: analysis of MRI and curve progression. Eur Spine J. 2007;16(10):1629-1635. doi:10.1007/s00586-007-0472-1

22.

Attenello

McGirt

Gathinji

, et al. Outcome of Chiari-associated syringomyelia after hindbrain decompression in children: analysis of 49 consecutive cases. Neurosurgery. 2008;62(6):1307-1313; discussion 1313. doi:10.1227/01.neu.0000333302.72307.3b

23.

Yan

Zhu

Liu

, et al. Diffusion tensor imaging in cervical syringomyelia secondary to Chiari I malformation: preliminary results. Spine (Phila Pa 1976). 2015;40(7):E381-E387. doi:10.1097/BRS.0000000000000781

24.

Yang

, et al. Specific microstructural changes of the cervical spinal cord in syringomyelia estimated by diffusion tensor imaging. Sci Rep. 2021;11(1):5111. doi:10.1038/s41598-021-84164-2

25.

Zhu

Sha

Chu

, et al. Comparison of the scoliosis curve patterns and MRI syrinx cord characteristics of idiopathic syringomyelia versus Chiari I malformation. Eur Spine J. 2016;25(2):517-525. doi:10.1007/s00586-015-4108-6

26.

Sha

Zhu

Lam

, et al. Brace treatment versus observation alone for scoliosis associated with Chiari I malformation following posterior fossa decompression: a cohort study of 54 patients. Eur Spine J. 2014;23(6):1224-1231. doi:10.1007/s00586-014-3265-3

27.

Sha

Zhu

Sun

, et al. Effectiveness of brace treatment of Chiari malformation-associated scoliosis after posterior fossa decompression: a comparison with idiopathic scoliosis. Spine (Phila Pa 1976). 2013;38(5):E299-E305. doi:10.1097/BRS.0b013e318281dba6

28.

Heemskerk

Kruyt

Colo

Castelein

Kempen

DHR

. Prevalence and risk factors for neural axis anomalies in idiopathic scoliosis: a systematic review. Spine J. 2018;18(7):1261-1271. doi:10.1016/j.spinee.2018.02.013

29.

Godzik

Holekamp

Limbrick

, et al. Risks and outcomes of spinal deformity surgery in Chiari malformation, Type 1, with syringomyelia versus adolescent idiopathic scoliosis. Spine J. 2015;15(9):2002-2008. doi:10.1016/j.spinee.2015.04.048

30.

Fruergaard

Ohrt-Nissen

Dahl

Kaltoft

Gehrchen

. Neural axis abnormalities in patients with adolescent idiopathic scoliosis: is routine magnetic resonance imaging indicated irrespective of curve severity? Neurospine. 2019;16(2):339-346. doi:10.14245/ns.1836154.077

31.

Xiao

Yang

Zhang

Peng

Zhang

. The application of two-level attention models in deep convolutional neural network for fine-grained image classification. IEEE Computer Society 2015;842-850. doi:10.1109/CVPR.2015.7298685

Identification of Atypical Scoliosis Patterns Using X-ray Images Based on Fine-Grained Techniques in Deep Learning

Abstract

Study Design

Objectives

Methods

Results

Conclusions

Keywords

Introduction

Materials and Methods

Subjects

Datasets and Preprocessing

Experimental Environment and Hyperparameters

Model Development

ResNet-50 Model Architecture

X-ray Classification Model

Performance Evaluation

Diagnostic Test Indicators

X-ray Image Classification Tasks

Task One: Scoliosis Diagnosis System

Task Two: CMS Diagnosis System

Visualization of Experimental Results

Statistical and Data Analysis

Results

Task One: Scoliosis Diagnosis System

Task Two: CMS Diagnosis System

Visualization Heatmaps Based on CAM

Discussion

Conclusion

Footnotes

Author Note

Author Contributions

Declaration of Conflicting Interests

Funding

Ethical Statement

ORCID iDs

Data Availability Statement

References