Sage Journals: Discover world-class research

Abstract

Purpose

To predict bone marrow metastasis in neuroblastoma using contrast-enhanced computed tomography (CECT) radiomics features and explainable machine learning.

Methods

This cohort study retrospectively included a total of 345 neuroblastoma patients who underwent testing for bone marrow metastatic status. Tumor lesions on CECT images were delineated by two radiologists, and 1409 radiomics features were extracted. Correlation analysis, Least Absolute Shrinkage and Selection Operator regression, and one-way analysis of variance were used to identify radiomics features associated with bone marrow metastasis. A predictive model for bone marrow metastasis was then developed using the support vector machine algorithm based on the selected radiomics features. The performance of the radiomics model was evaluated using the area under the curve (AUC), 95% confidence interval (CI), accuracy, sensitivity, and specificity.

Results

The radiomics model included 16 features, with a predominant focus on texture features (12/16, 75%). In the training set, the model demonstrated an AUC of 0.891 (95% CI: 0.848-0.933), an accuracy of 0.831 (95% CI: 0.829-0.832), a sensitivity of 0.893 (95% CI: 0.840-0.946), and a specificity of 0.757 (95% CI: 0.677-0.837). In the test set, the AUC, accuracy, sensitivity, and specificity were 0.807 (95% CI: 0.720-0.893), 0.767 (95% CI: 0.764-0.770), 0.696 (95% CI: 0.576-0.817), and 0.851 (95% CI: 0.749-0.953), respectively.

Conclusion

Radiomics features extracted from CECT images are associated with the presence of bone marrow metastasis in neuroblastoma, providing potential new imaging biomarkers for predicting bone marrow metastasis in this disease.

Keywords

neuroblastoma radiomics computed tomography bone marrow metastasis explainable machine learning

Introduction

Neuroblastoma is a pediatric solid tumor originating from primitive neuroblasts and is often characterized by distant metastasis, with over 90% of metastatic lesions involving the bone marrow.^1,2 Bone marrow involvement in neuroblastoma is associated with a higher risk of disease progression and poor outcomes.³ Despite aggressive multi-modal combination therapies, the overall survival rate for high-risk patients with metastatic neuroblastoma remains below 50%.⁴ Studies have shown that bone marrow metastasis is a significant risk factor associated with the prognosis of neuroblastoma patients.^3,5 Therefore, timely and accurate detection of bone marrow metastasis is crucial for optimizing treatment strategies and improving patient outcomes.

Bone marrow biopsy and aspiration are commonly used to detect bone marrow metastasis in neuroblastoma. However, due to the typical involvement of multiple bone marrow compartments in neuroblastoma, diagnosis can be challenging, especially when the infiltrating neuroblastoma cells in the bone marrow are less than 30%.⁶ Although commonly used methods such as morphological or immunocytological analysis can detect bone marrow metastasis in neuroblastoma, the sensitivity of these methods varies.^7,8 Some laboratory indicators, such as vanillylmandelic acid (VMA), have been found to be associated with bone marrow metastasis in neuroblastoma.⁹ However, the VMA test is sensitive to many external influences, such as foods and drugs. Additionally, these laboratory indicators can differ significantly across lesions located in different anatomical sites.¹⁰ Furthermore, tumor markers may show limited correlation with minimal residual disease in high-risk neuroblastoma patients.¹¹ Therefore, the investigation for additional supplementary biomarkers associated with bone marrow metastasis in neuroblastoma is crucial. This could aid in the early diagnosis of bone marrow metastasis, particularly for lesions with minimal metastatic involvement.¹²

In medical imaging, positron emission tomography/computed tomography (PET/CT) has the potential to reduce the need for invasive bone marrow biopsy and aspiration in cases where PET/CT results are negative.¹³ However, although total lesion glycolysis and metabolic tumor volume were significantly associated with neuroblastoma bone marrow metastasis in univariate analysis, these conventional PET parameters were not independent risk factors for neuroblastoma bone marrow metastasis in multivariate analysis.⁹ Recently, several studies have suggested that radiomics derived from medical imaging could provide imaging biomarkers for the diagnosis of neuroblastoma.¹⁴ Feng et al.¹⁵ observed a significant association between radiomics features based on PET/CT imaging and bone marrow metastasis in neuroblastoma, with their predictive model demonstrating excellent performance. Radiomics features derived from PET/CT were found to be as important as clinical features and conventional PET parameters.¹⁶ While PET/CT is valuable for detecting distant metastasis and clinical staging of neuroblastoma, preoperative planning for neuroblastoma often relies on contrast-enhanced CT (CECT) or magnetic resonance imaging (MRI).^17,18

Therefore, the aim of this study was to predict bone marrow metastasis in neuroblastoma using CECT radiomics features through an explainable machine learning approach. Additionally, we employed global and local explainable methods to emphasize the model's explainability. These methods also illustrated how the radiomics model could provide personalized predictions for individual cases.

Materials and Methods

Patient Selection

This retrospective study received approval from the Institutional Review Board of Children's Hospital of Chongqing Medical University (approval number: 202235), with a waiver of the requirement for informed consent from patients. All methods were performed in accordance with relevant guidelines and regulations, including the Declaration of Helsinki. The reporting of this study conforms to STROBE guidelines,¹⁹ which is included in Supplemental Materials. Clinical and CECT imaging data were retrospectively and consecutively gathered from neuroblastoma patients who visited our institution between January 2010 and May 2023. All patient details were de-identified. Based on inclusion and exclusion criteria, a total of 345 pediatric patients were included in the study (Figure 1). The data were collected by two operators working together to ensure accuracy and consistency. The inclusion criteria were as follows: (1) pathologically confirmed neuroblastoma; (2) arterial-phase CECT examination; (3) bone marrow metastatic status confirmed by bone marrow aspiration with samples taken from the iliac bones. Exclusion criteria included: (1) prior anti-cancer therapy before the CECT examination; (2) the presence of artifacts or poor image quality. The time interval between the testing of bone marrow metastasis and the acquisition of the CT scan was less than two weeks. Based on the presence of bone marrow metastasis, all cases were categorized into metastasis and non-metastasis groups. The cases were then randomly stratified in a 7:3 ratio to form the training set (n = 242) and the test set (n = 103).

Figure 1.

Flowchart of patient selection.

Image Acquisition, Image Preprocessing, and Tumor Delineation

CECT images were acquired using GE Lightspeed and Philips Brilliance CT scanners. The tube voltage ranged from 80 to 100 kV, with a tube current between 150 and 200 mAs, and a scanning slice thickness of 5.0 mm. A non-ionic iodinated contrast agent (Iodixanol, GE Healthcare) at a concentration of 320 mgI/ml was administered at a dosage of 1.5–2.0 ml/kg of body weight. The contrast agent was delivered through the cubital vein at a flow rate of 0.5 to 3.5 ml/s, and arterial-phase images were captured 20–28 s after the injection. To ensure the generalization of radiomics features across different scanners and scanning protocols, the CECT images were preprocessed before radiomics feature extraction. The preprocessing methods primarily included 1.0 mm × 1.0 mm × 1.0 mm voxel resampling and discretization of images with a bin width of 25.

CECT images for all cases were anonymously retrieved from the Picture Archiving and Communication System. The radiomics flowchart is illustrated in Figure 2. Initially, a pediatric radiologist with more than three years of experience delineated entire primary tumor lesions slice by slice on arterial-phase CECT images using the ITK-SNAP open-source software (version 4.0.0), excluding encased vascular structures (eg aorta, renal artery, veins, etc). Subsequently, another pediatric radiologist with over ten years of experience reviewed all delineated tumor lesions. Both radiologists were blinded to the bone marrow metastatic status of the cases during the delineation process. To assess the reproducibility of radiomics features across different operators, 40 cases were randomly selected from the training set, and the same method was used to delineate tumor lesions again for radiomics feature extraction. The intra-class correlation coefficient (ICC) for radiomics features extracted from twice delineations was calculated using the “psych” R package (https://CRAN.R-project.org/package=psych).

Figure 2.

Radiomics flowchart in this study. ICC, intra-class correlation coefficient; PCC, Pearson correlation coefficient; LASSO, Least Absolute Shrinkage and Selection Operator; GLDM, Gray Level Dependence Matrix; GLCM, Gray Level Co-occurrence Matrix; GLSZM, Gray Level Size Zone Matrix; GLRLM, Gray Level Run Length Matrix; ANOVA, analysis of variance; SVM, support vector machine.

Radiomics Feature Dimension Reduction

A total of 1409 radiomics features were extracted from each tumor lesion using various filters (Table 1). The extraction was performed using the FAE open-source software.²⁰ Since the Pyradiomics package was embedded within the FAE open-source software, the extracted radiomics features complied with the Image Biomarkers Standardization Initiative standards.²¹ All radiomics features were standardized using z-sore standardization to eliminate the effects of different dimensions. In this study, we employed the following statistical methods for selecting radiomics features in the training set. First, radiomics features with an ICC greater than 0.75 were retained to enhance the repeatability of selected features across different operators. ICC values between 0.75 and 0.90 indicate good reliability, and values greater than 0.90 indicate excellent reliability.²² As a result, many radiomics studies used 0.75 as the screening threshold for ICC.^23,24

Table 1.

Details of the Extracted Radiomics Feature Categories

Image filter	Feature type
Original, Wavelet Transform (LLL, HHH, HHL, HLH, HLL, LHH, LHL, LLH), Square, Square Root, Logarithm, Laplacian of Gaussian (sigma: 1.0), Gradient, Exponential	Shape, First-order, Gray Level Co-occurrence Matrix (GLCM), Gray Level Dependence Matrix (GLDM), Gray Level Size Zone Matrix (GLSZM), Gray Level Run Length Matrix (GLRLM), Neighboring Gray Tone Difference Matrix (NGTDM)

To address potential redundancy among radiomics features extracted from a high-dimensional data space, Pearson correlation analysis was applied, and features with a Pearson correlation coefficient (PCC) less than 0.90 were retained. Pearson correlation is a statistical method that measures the linear relationship between two data objects, and it can be used for dimensionality reduction of radiomics features.²⁵ Next, the “glmnet” R package (https://CRAN.R-project.org/package = glmnet) was used to input the remaining features into the Least Absolute Shrinkage and Selection Operator (LASSO) algorithm, which selects radiomics features with non-zero coefficients. The optimal lambda value was determined through five-fold cross-validation. LASSO is a widely used method for high-dimensional data analysis because it performs both regularization and variable selection simultaneously.²⁶ Finally, one-way analysis of variance (ANOVA) was conducted to retain radiomics features with a P-value less than 0.05. ANOVA is a statistical method used to determine if there are significant differences between the means of independent groups. It helps retain the most relevant radiomics features and is effective for improving predictive performance in radiomic studies.²⁷

Machine Learning and Validation

Based on the radiomics features retained after ANOVA analysis, a radiomics model was constructed in the training set using the support vector machine (SVM) algorithm from the “e1071” R package (https://CRAN.R-project.org/package = e1071). The SVM parameters were optimized through grid search to identify the best configuration. The final parameters were set as follows: kernel = “radial”, gamma = “1/feature number”, cost = “1.0”, degree = “3.0”. The predictive performance of the established radiomics model for bone marrow metastasis was then validated in both the training and test sets. Receiver operating characteristic (ROC) curves and precision-recall curves were generated to visualize the predictive efficacy of the radiomics model in both datasets. Additionally, lift charts were created to evaluate the performance of the radiomics model in the training and test sets. To evaluate the robustness of the proposed model, we used five-fold cross-validation within the training set to further validate the model's robustness. This is to assess the consistency of the model's performance across different subsets of the training data, ensuring that the model is not overly dependent on any single data partition. By repeatedly validating the model within the training set, variations in performance can be identified, and the stability and reliability of the model's predictions can be confirmed.

Global and Local Explanations of Machine Learning Model

Despite the outstanding performance of non-linear machine learning algorithms in previous studies, their inherent lack of transparency and explainability has become a significant constraint, limiting their application in the medical field.²⁸ Linear algorithms, such as linear regression, predict outputs based on linear relationships, which makes them relatively explainable. The contribution of each feature can be directly understood through model coefficients, allowing for a clear understanding of each feature's impact on the output. In contrast, non-linear algorithms produce more complex models that capture intricate non-linear relationships between features, but they are difficult to explain with simple mathematical expressions. The importance of features in non-linear models is harder to quantify, often requiring methods like SHapley Additive exPlanations (SHAP) values to interpret the contributions of individual features.^29,30 Therefore, explainable machine learning aims to address key questions, such as identifying the most important features, explaining individual predictions, and understanding the overall behavior of the model.

In this study, we used the “DALEX” R package for both global and local explanations of the radiomics model, which clarified the relationship between input variables and model output (https://CRAN.R-project.org/package = DALEX). To start, we generated feature importance plots and partial dependence plots (PDPs) to provide a global explanation of the features in the radiomics model. The “DALEX” R package employs a permutation-based method for assessing feature importance, determining the significance of each feature by calculating the increase in model prediction error after permuting the feature values.³¹ Additionally, PDPs visualize how different features influence predictions in machine learning models, showing how the model's average predictions change as a single feature varies while other features remain constant.³²

We then provided detailed local explanations for four cases within the test set using Breakdown and SHAP methods. These four cases included: one where bone marrow metastasis occurred with accurate model prediction, one where it occurred with inaccurate model prediction, one where it did not occur with accurate model prediction, and one where it did not occur with inaccurate model prediction. The Breakdown method illustrates which features in the model influence the prediction of a specific case and the extent of their impact, helping to clarify each feature's contribution to the prediction.³³ Additionally, the SHAP method, based on Shapley values, quantifies the contribution of each feature to the model's prediction for a specific case.³⁴ Finally, we used Ceteris-paribus plots to show how the radiomics model's predictions for these four specific cases change with variations in important features.

Statistical Analysis

In this study, we used RStudio (version 4.2.2) and SPSS (version 26.0, IBM Corp., Armonk, NY) for statistical analysis. Age was expressed as mean ± standard deviation, and a Student t-test was conducted to compare age between the two groups. Categorized data were presented as case number (percentage), and a Chi-square test was used to compare these categorized variables between the groups. To evaluate the predictive performance of the radiomics model in both the training and test sets, we calculated the area under the curve (AUC), 95% confidence interval (CI), sensitivity, specificity, accuracy, negative prediction value, and positive prediction value using the “reportROC” R package. Correlation heatmaps of the radiomics features were generated using the “pheatmap” and “corrplot” R packages. A two-sided P-value less than 0.05 was considered statistically significant.

Results

Patient Clinical Information

We enrolled 345 pediatric patients diagnosed with neuroblastoma, consisting of 193 males and 152 females. The mean age was 36 ± 29 months, ranging from 1 month to 14 years. The dataset was divided into a training set (n = 242) and a test set (n = 103). Overall, 54% (187/345) of the cohort exhibited bone marrow metastasis, while 46% (158/345) did not. The clinical information for the metastasis and non-metastasis groups in both the training and test sets is presented in Table 2.

Table 2.

Distribution of Patient Clinical Information in the Training and Test Sets

Information	Training set (n = 242)			Test set (n = 103)
Information	Metastasis (n = 131)	Non-metastasis (n = 111)	P value	Metastasis (n = 56)	Non-metastasis (n = 47)	P value
Age (months)	42 ± 28	29 ± 32	0.001	38 ± 25	36 ± 32	0.768
Sex			0.173			0.208
Male	81 (62%)	59 (53%)		32 (57%)	21 (45%)
Female	50 (38%)	52 (47%)		24 (43%)	26 (55%)
Location			-			-
Neck	0 (0%)	1 (1%)		0 (0%)	1 (2%)
Thorax	5 (4%)	18 (16%)		4 (7%)	4 (9%)
Abdomen	116 (89%)	80 (72%)		51 (91%)	32 (68%)
Pelvis	0 (0%)	7 (6%)		0 (0%)	4 (9%)
Two sites	10 (7%)	5 (5%)		1 (2%)	6 (12%)
Histology			<0.001			<0.001
Favorable	16 (12%)	72 (65%)		9 (16%)	26 (55%)
Unfavorable	89 (68%)	38 (34%)		31 (55%)	21 (45%)
Unclear	26 (20%)	1 (1%)		16 (29%)	0 (0%)
MYCN status			<0.001			0.050
Positive	25 (19%)	7 (6%)		11 (20%)	4 (9%)
Negative	58 (44%)	79 (71%)		25 (45%)	32 (68%)
Unclear	48 (37%)	25 (23%)		20 (35%)	11 (23%)

Reduction of Radiomics Feature Dimensionality

A total of 1409 radiomics features were extracted from each lesion. Forty cases were included in the analysis of radiomics feature reproducibility, comprising 25 boys and 15 girls, with a median age of 14 months (interquartile range: 9 to 35 months). Among these, 23 cases had bone marrow metastasis, while 17 cases did not. The ICC analysis revealed that the average and standard deviation of the ICC for radiomics features extracted from the two delineations were 0.88 and 0.20, respectively. Among all features, 1203 features (1203/1409, 85%) had an ICC greater than 0.75 and were therefore included in the Pearson correlation analysis. PCC analysis indicated that 839 radiomics features (839/1203, 70%) had a PCC greater than 0.90. Consequently, 364 radiomics features (364/1203, 30%) with a PCC less than 0.90 were input into the LASSO algorithm (Figures 3A and 3B). Following LASSO selection, 32 radiomics features (32/364, 9%) with non-zero coefficients were retained (Figures 3C-3E). One-way ANOVA analysis revealed that 16 radiomics features (16/32, 50%) had a P-value less than 0.05. The importance ranking of these features is shown in Figure 3F. In the entire cohort, the distribution differences between the metastasis and non-metastasis groups for the representative radiomics features are depicted in Figure 4.

Figure 3.

Selection process of the radiomics features. Figures A and B display the correlation heatmap of radiomics features before and after Pearson correlation analysis, respectively. Figure C illustrates the determination of the optimal lambda value with the minimum predictive error via a five-fold cross-validation process using the Least Absolute Shrinkage and Selection Operator algorithm. Figure D showcases the selection of radiomics features with non-zero coefficients under the threshold of the optimal lambda value. Figure E presents the correlation heatmap of radiomics features selected by the Least Absolute Shrinkage and Selection Operator. Figure F demonstrates the importance plot of the selected radiomics features with a P-value less than 0.05 through one-way ANOVA analysis.

Figure 4.

Comparison of representative radiomics features between the non-metastasis and metastasis groups in the entire cohort.

Performance of the Radiomics Model

The radiomics model included 16 radiomics features, with a predominant focus on texture features (12/16, 75%). In the training set, the model demonstrated an AUC of 0.891 (95% CI: 0.848-0.933), an accuracy of 0.831 (95% CI: 0.829-0.832), a sensitivity of 0.893 (95% CI: 0.840-0.946), and a specificity of 0.757 (95% CI: 0.677-0.837). In the test set, the AUC, accuracy, sensitivity, and specificity were 0.807 (95% CI: 0.720-0.893), 0.767 (95% CI: 0.764-0.770), 0.696 (95% CI: 0.576-0.817), and 0.851 (95% CI: 0.749-0.953), respectively. The ROC and precision-recall curves for the radiomics model in both the training and test sets are depicted in Figures 5A-5C. The lift charts in the training and test sets indicated that the model's predictive ability for positive samples was superior to random acquisition without the use of this model (Figures 5D and 5E). Table 3 presents a more detailed set of evaluation metrics for the radiomics model.

Figure 5.

Performance of the radiomics model. Figure A displays the receiver operating characteristic curves of the radiomics model in the training and test sets. Figures B and C present the precision-recall curves of the radiomics model in the training and test sets, respectively. Figures D and E showcase the lift curves of the radiomics model in the training and test sets, respectively. The solid curves above the lower dashed lines in Figures D and E indicate that the radiomics model outperforms a random model.

Table 3.

Evaluation Metrics of the Radiomics Model in the Training and Test Sets

	AUC (95% CI)	Accuracy (95% CI)	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)
Training set	0.891 (0.848–0.933)	0.831 (0.829–0.832)	0.893 (0.840–0.946)	0.757 (0.677–0.837)	0.812 (0.749–0.876)	0.857 (0.788–0.926)
5-fold CV in training set	0.700 (0.672, 0.730)	0.707 (0.675, 0.736)	0.716 (0.669, 0.762)	0.724 (0.657, 0.815)	0.769 (0.684, 0.877)	0.631 (0.509, 0.721)
Test set	0.807 (0.720–0.893)	0.767 (0.764–0.770)	0.696 (0.576–0.817)	0.851 (0.749–0.953)	0.848 (0.744–0.952)	0.702 (0.583–0.821)

Notes: AUC, area under the curve; CI, confidence interval; CV, cross-validation; PPV, positive prediction value; NPV, negative prediction value

Explainability of the Radiomics Model

After the permutation of radiomics features, different features exhibited varying degrees of influence on the model performance (Figure 6A). The top three most crucial features were wavelet.LLH.firstorder_Kurtosis, wavelet.LLL_glcm_Imc2, and wavelet.HHH_glrlm_GrayLevelVariance. The PDPs in Figure 6B revealed that different radiomics features had unique correlations with the model's average predictions. Furthermore, in the four cases from the test set, various features in the radiomics model showed differences in importance for predicting these cases. The local explanations of the radiomics model for these four cases are illustrated in Figures 7 –10.

Figure 6.

Global explanation of the radiomics model. Figure A depicts the feature importance of the radiomics features. Figure B illustrates the partial dependence plot of the radiomics features. On the horizontal axis of Figure B, individual feature values are represented, while the vertical axis shows the effects of each feature on the predicted outcomes of the radiomics model. Features 1 to 16 correspond to the radiomics features ranked by ANOVA F-ratio importance in Figure 1F in sequential order.

Figure 7.

Local explanation of the radiomics model for case 1, where bone marrow metastasis actually occurred and the radiomics model correctly predicted it. Figure A presents the axial CECT image of the neuroblastoma lesion in a 4-year-old male patient. For Case 1, the model's prediction was 1.022, surpassing both the average model prediction of 0.656 and that for 92% of all observations (Figure B). The most significant feature was original_glszm_SmallAreaEmphasis, contributing to a prediction increase of 0.081. The second most significant feature was wavelet.LLL_glcm_Imc2, leading to a prediction increase of 0.052 (Figure C). Figure D illustrates how the radiomics model's prediction for Case 1 changes with alterations in these two features, where the blue dots represent the actual predictions for Case 1.

Figure 8.

Local explanation of the radiomics model for Case 2, where bone marrow metastasis actually occurred, but the radiomics model predicted that bone marrow metastasis did not occur. Figure A shows the axial CECT image of the neuroblastoma lesion in a 1-month-old female patient. For Case 2, the model's prediction was −0.004, which was lower than both the average model prediction of 0.656 and that for 96% of all observations (Figure B). The most significant feature was original_glszm_SmallAreaEmphasis, contributing to a prediction decrease of 0.241. The second most significant feature was wavelet.HLL_glszm_SmallAreaEmphasis, leading to a prediction decrease of 0.151 (Figure C). Figure D illustrates how the radiomics model's prediction for Case 2 changes with alterations in these two features, where the blue dots represent the actual predictions for Case 2.

Figure 9.

Local explanation of the radiomics model for Case 3, where bone marrow metastasis did not actually occur and the radiomics model correctly predicted it. Figure A shows the axial CECT image of the neuroblastoma lesion in a 9-month-old male patient. For Case 3, the model's prediction was −0.005, which was lower than both the average model prediction of 0.656 and that for 96% of all observations (Figure B). The most significant feature was wavelet.LLH_firstorder_Kurtosis, contributing to a prediction decrease of 0.267. The second most significant feature was gradient_glrlm_ShortRunLowGrayLevelEmphasis, leading to a prediction decrease of 0.079 (Figure C). Figure D illustrates how the radiomics model's prediction for Case 3 changes with alterations in these two features, where the blue dots represent the actual predictions for Case 3.

Figure 10.

Local explanation of the radiomics model for case 4, where bone marrow metastasis did not actually occur, but the radiomics model predicted the occurrence of bone marrow metastasis. Figure A shows the axial CECT image of the neuroblastoma lesion in a 16-month-old male patient. For Case 4, the model's prediction was 0.769, surpassing both the average model prediction of 0.656 and that for 54% of all observations (Figure B). The most significant feature was square_glcm_Imc1, contributing to a prediction decrease of 0.068. The second most significant feature was gradient_glrlm_ShortRunLowGrayLevelEmphasis, leading to a prediction decrease of 0.066 (Figure C). Figure D illustrates how the radiomics model's prediction for Case 4 changes with alterations in these two features, where the blue dots represent the actual predictions for Case 4.

Discussion

In this study, we employed CECT radiomics analysis to predict bone marrow metastasis in neuroblastoma. The results demonstrated a significant correlation between CECT radiomics features and the risk of bone marrow metastasis. Furthermore, the radiomics model based on these features achieved good predictive performance. These findings suggest that CECT radiomics provides additional imaging biomarkers for predicting bone marrow metastasis in neuroblastoma, potentially enhancing the detection of subtle bone marrow metastases that might be challenging to identify through conventional methods. Additionally, the explainable methods used to explain the radiomics model established by non-linear machine learning help clarify the relationship between input variables and model output.

Different imaging modalities can lead to discrepancies in radiomics features. A recent study comparing radiomics features derived from CECT and MRI for predicting pathological subtypes of neuroblastoma found differences between the features obtained from these modalities, with the CECT radiomics model showing superiority over the MRI radiomics model.³⁵ In two recent studies using radiomics features from CT and MRI to predict neuroblastoma bone marrow metastasis, the optimal CT-based radiomics model outperformed the optimal MRI-based radiomics model in the validation set,^36,37 indicating CT images may better capture the heterogeneity of neuroblastoma. Therefore, investigation focused on CT radiomics relevant to bone marrow metastasis is crucial, as it holds the potential to advance the field of multi-modality imaging for neuroblastoma. In our study, we also identified a significant correlation between CECT radiomics features and the incidence of bone marrow metastasis in neuroblastoma. Among the selected radiomics features, texture features were predominant, suggesting a strong association between the heterogeneity in voxel distribution within CECT images and the occurrence of bone marrow metastasis in neuroblastoma.

It is worth noting that while the final CECT radiomics features selected in our study were primarily texture features, similar to those in another study,³⁷ differences in the specific types of texture features may arise due to variations in CT scanners and scanning protocols. In our preliminary experiments, we found that the radiomics model based on arterial-phase CECT images performed better than one based on venous-phase CECT images in predicting bone marrow metastasis. Consequently, we chose to use arterial-phase images for the formal study to optimize our predictive accuracy. We speculate that arterial-phase CECT images may better reflect the heterogeneity of neuroblastoma because neuroblastoma often encases adjacent vascular structures. Previous studies have also demonstrated that arterial-phase CECT images are effective in differentiating between both histological subgroups of neuroblastoma and high-risk versus non-high-risk subtypes.^38–40

In a previous study, Feng et al¹¹ found that lesions with bone marrow metastasis exhibited significantly higher radiomics scores from PET/CT, further indicating the heterogeneity of image texture in lesions with bone marrow metastases. Prior studies have also highlighted a correlation between texture features observed in medical images and the aggressive biological behavior of neuroblastoma.^41–43 The texture features of CECT images in our study were associated with neuroblastoma bone marrow metastasis, reflecting potential changes in the microscopic structure and biological characteristics of tumor tissues. Texture features are mathematical indicators designed to characterize the distribution of voxels and grayscale variations within an image, with their variations revealing the heterogeneity and complexity of tissues.⁴⁴ The occurrence of metastasis can induce changes in local cell density and extracellular matrix,^45,46 and such alterations may be reflected in the texture features. Moreover, the growth and metastasis of tumors are frequently accompanied by the emergence of new blood vessels and changes in blood flow.⁴⁷ These processes may lead to an uneven grayscale distribution in the images, thereby influencing the texture features.

In prior studies, non-linear machine learning algorithms, including multilayer perceptron and random forest, provided better predictive performance than linear machine learning algorithms for establishing radiomics models to predict neuroblastoma bone marrow metastasis.^36,37 This suggests that non-linear algorithms are better at capturing complex data relationships that linear models might overlook, resulting in more accurate predictions. However, the enhanced performance of non-linear algorithms comes with a trade-off: they tend to lack transparency, making it challenging to understand the contribution of individual features to the model's predictions. This opacity hinders an intuitive grasp of the relationship between input features and model outputs.⁴⁸ In the medical field, explainable artificial intelligence is becoming increasingly important, as it helps clinicians interpret artificial intelligence-generated insights and incorporate them effectively into patient care plans.⁴⁹ Compared to deep learning, traditional machine learning algorithms require fewer computational resources and can operate in smaller computing environments, making them more accessible for use in institutions with limited data and infrastructure.⁵⁰ Many traditional machine learning algorithms perform well on smaller datasets and have fewer hyperparameters, making the tuning process relatively straightforward.

In our established radiomics model, there is a disparity between the feature importance rankings obtained through global explanation and those derived from ANOVA. The SVM algorithm considers the interactions between features, whereas ANOVA typically focuses on the impact of individual features. This suggests that machine learning algorithms are more likely to accurately capture nonlinear relationships when there are interactions between features. In previous radiomics studies, although non-linear machine learning algorithms have shown excellent performance across various prediction tasks, the explainability of these models has been a longstanding concern.⁵¹ Despite their ability to flexibly adapt to complex datasets, the intricate internal workings of these non-linear algorithms pose challenges for explaining the prediction processes of the models.⁵² In this study, we tried to gain a deeper understanding of the local explainability of the radiomics model, particularly focusing on the impact of individual features on the predictions for specific cases. Through local explanation of four cases, we discovered variations in the importance of different radiomics features in specific case predictions. For example, in the case of Small Area Emphasis, high values indicate a concentration of voxel intensity in small regions of the image, while low values suggest a more dispersed distribution of voxel intensity. In Cases 1 and 2, Small Area Emphasis emerges as the most important feature influencing the radiomics model's predictions. This indicates that the distribution of voxel intensity in small regions significantly affects the model's predictions for these cases in CECT images. Therefore, applying local explanations to specific cases helps clarify why the radiomics model generates certain predictive decisions.

This study has some limitations. First, it was a single-center retrospective study, and the imaging data used were derived from the scanners and protocols of our institution. This could limit the application of the radiomics model across different scanners and protocols at other institutions, necessitating further validation through multi-center data to ensure the model's generalizability. Second, our sample size is relatively small, which may impact the statistical power and reliability of the model. Therefore, expanding the sample size in future research is necessary. Third, we retrospectively collected cases continuously in an effort to include as many cases as possible. However, due to the retrospective nature of the study and the availability of cases, a formal sample size calculation was not performed. Then, the extensive time span of this study, covering patients treated over a ten-year period, resulted in some missing or unavailable laboratory markers, which were not incorporated into the combined model construction process. Future studies should aim to establish different combined models for neuroblastoma in various locations, considering that the clinical variables of neuroblastoma in different anatomical sites can differ significantly.⁵³ Finally, in the context of clinical decision-making support systems, deep learning techniques such as Class Activation Mapping are frequently utilized, whereas radiomics-based machine learning technology typically employs radiomics maps. These maps generate heatmaps that display the importance of different regions. However, the primary focus of our study was on explaining the contribution of each feature in a non-linear algorithm model to the model prediction. Therefore, we did not use visual techniques like radiomics maps, as our approach prioritizes quantifying feature importance directly within the context of the non-linear model.

Conclusion

In conclusion, radiomics features derived from CECT images are associated with the presence of bone marrow metastasis in neuroblastoma, offering promising new imaging biomarkers for predicting such metastasis. Moreover, the incorporation of explainable machine learning strategies enhances the explainability of the radiomics model, providing clinicians with more reliable clinical decision support.

Abbreviations

VMA

vanillylmandelic acid

PET/CT

positron emission tomography/computed tomography

CECT

contrast-enhanced CT

MRI

magnetic resonance imaging

ICC

intra-class correlation coefficient

PCC

Pearson correlation coefficient

LASSO

Least Absolute Shrinkage and Selection Operator

ANOVA

analysis of variance

SVM

support vector machine

ROC

Receiver operating characteristic

PDPs

partial dependence plots

SHAP

SHapley Additive exPlanations

AUC

area under the curve

confidence interval

Supplemental Material

sj-docx-1-tct-10.1177_15330338241290386 - Supplemental material for Predicting Bone Marrow Metastasis in Neuroblastoma: An Explainable Machine Learning Approach Using Contrast-Enhanced Computed Tomography Radiomics Features

Supplemental material, sj-docx-1-tct-10.1177_15330338241290386 for Predicting Bone Marrow Metastasis in Neuroblastoma: An Explainable Machine Learning Approach Using Contrast-Enhanced Computed Tomography Radiomics Features by Haoru Wang, Ling He, Xin Chen, Shuang Ding, Mingye Xie and Jinhua Cai in Technology in Cancer Research & Treatment

Footnotes

Acknowledgments

Not applicable.

Authors’ Contributions

Conceptualization: H.W., L.H., X.C., and J.C., Methodology: H.W., Formal analysis: H.W., Investigation: H.W., X.C., SD, and M.X., Writing—original Draft: H.W., Writing—review and editing: all authors, Visualization: H.W., Supervision: J.C. All authors contributed to manuscript revision, read, and approved the submitted version.

Availability of Data and Material

The datasets generated or analyzed during the study are available from the corresponding author on reasonable request.

Consent for Publication

Not applicable.

Competing Interests

Haoru Wang is an Editorial Review Board Member of Technology in Cancer Research & Treatment and was excluded from all editorial decision-making related to the acceptance of this article for publication. The remaining authors declare no conflict of interest.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethics Approval and Consent to Participate

This retrospective study received approval from the Institutional Review Board of Children's Hospital of Chongqing Medical University, and patient informed consent was waived (File No. 202235).

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Key Project of Technology Innovation and Application Development of Chongqing Science and Technology Bureau, Natural Science Foundation of Chongqing Municipality, (grant number No. CSTB2022TIADKPX0151, (CSTB)2023NSCQ-BHX0127).

ORCID iD

Haoru Wang

Supplemental Material

Supplemental material for this article is available online.

References

Fetahu

Esser-Skala

Dnyansagar

, et al. Single-cell transcriptomics and epigenomics unravel the role of monocytes in neuroblastoma bone marrow metastasis. Nat Commun. 2023;14(1):3620. https://doi.org/10.1038/s41467-023-39210-0

Huang

Jiang

Liao

, et al. Bone and bone marrow involvement in neuroblastoma: A case report. Medicine (Baltimore). 2020;99(40):e22505. https://doi.org/10.1097/MD.0000000000022505

Morgenstern

London

Stephens

, et al. Prognostic significance of pattern and burden of metastatic disease in patients with stage 4 neuroblastoma: A study from the International Neuroblastoma Risk Group database. Eur J Cancer. 2016;65:1–10. https://doi.org/10.1016/j.ejca.2016.06.005

Brignole

Pastorino

Perri

, et al. Bone marrow environment in metastatic neuroblastoma. Cancers (Basel). 2021;13(10):2467. https://doi.org/10.3390/cancers13102467

Liu

Yin

Lin

, et al. Metastasis pattern and prognosis in children with neuroblastoma. World J Surg Oncol. 2023;21(1):130. https://doi.org/10.1186/s12957-023-03011-y

Abbasi

Rifatbegovic

Brunner

, et al. Bone marrows from neuroblastoma patients: An excellent source for tumor genome analyses. Mol Oncol. 2015;9(3):545–554. https://doi.org/10.1016/j.molonc.2014.10.010

Beiske

Burchill

Cheung

, et al. Consensus criteria for sensitive detection of minimal neuroblastoma cells in bone marrow, blood and stem cell preparations by immunocytology and QRT-PCR: Recommendations by the international neuroblastoma risk group task force. Br J Cancer. 2009;100(10):1627–1637. https://doi.org/10.1038/sj.bjc.6605029

Schriegel

Taschner-Mandl

Bernkopf

, et al. Comparison of three different methods to detect bone marrow involvement in patients with neuroblastoma. J Cancer Res Clin Oncol. 2022;148(10):2581–2588. https://doi.org/10.1007/s00432-021-03780-7

Feng

Yang

, et al. ¹⁸F-FDG PET/CT-based radiomics nomogram could predict bone marrow involvement in pediatric neuroblastoma. Insights Imaging. 2022;13(1):144. https://doi.org/10.1186/s13244-022-01283-8

10.

Matthay

Neuhaus

, et al. Clinical, biologic, and prognostic differences on the basis of primary tumor site in neuroblastoma: A report from the international neuroblastoma risk group project. J Clin Oncol. 2014;32(28):3169–3176. https://doi.org/10.1200/JCO.2014.56.1621

11.

Uemura

Lin

Mon Thwin

, et al. Limited correlation between tumor markers and minimal residual disease detected by seven neuroblastoma-associated mRNAs in high-risk neuroblastoma patients. Mol Clin Oncol. 2021;15(1):137. https://doi.org/10.3892/mco.2021.2299

12.

Osman

Galli

Hanafy

, et al. Identification of novel biomarkers in neuroblastoma associated with the risk for bone marrow metastasis: A pilot study. Clin Transl Oncol. 2013;15(11):953–958. https://doi.org/10.1007/s12094-013-1030-4

13.

Zhao

Yang

. Predictive value of 18 F-FDG PET/CT versus bone marrow biopsy and aspiration in pediatric neuroblastoma. Clin Exp Metastasis. 2024. Epub ahead of print. https://doi.org/10.1007/s10585-024-10295-1

14.

Wang

Chen

. A narrative review of radiomics and deep learning advances in neuroblastoma: Updates and challenges. Pediatr Radiol. 2023;53(13):2742–2755. https://doi.org/10.1007/s00247-023-05792-6

15.

Feng

Zhou

Liu

, et al. ¹⁸F-FDG PET/CT-based radiomics nomogram for prediction of bone marrow involvement in pediatric neuroblastoma: A two-center study. Acad Radiol. 2024;31(3):1111–1121. https://doi.org/10.1016/j.acra.2023.07.018

16.

Feng

Yang

, et al. Diagnostic value of ¹⁸F-FDG PET/CT-based radiomics nomogram in bone marrow involvement of pediatric neuroblastoma. Acad Radiol. 2023;30(5):940–951. https://doi.org/10.1016/j.acra.2022.08.021

17.

Burnand

Barone

McHugh

, et al. Preoperative computed tomography scanning for abdominal neuroblastomas is superior to magnetic resonance imaging for safe surgical planning. Pediatr Blood Cancer. 2019;66(11):e27955. https://doi.org/10.1002/pbc.27955

18.

Delval

Touitou

Gondry-Jouet

, et al. A non-inferiority study of MRI versus CT for staging and image-defined risk factor assessment in the preoperative work-up of abdominopelvic neuroblastoma. Eur J Radiol. 2024;177:111580. https://doi.org/10.1016/j.ejrad.2024.111580

19.

von Elm

Altman

Egger

, et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: Guidelines for reporting observational studies. Ann Intern Med. 2007;147(8):573–577. https://doi.org/10.7326/0003-4819-147-8-200710160-00010

20.

Song

Zhang

, et al. Feature explorer (FAE): A tool for developing and comparing radiomics models. PLoS One. 2020;15(8):e0237587. https://doi.org/10.1371/journal.pone.0237587

21.

Zwanenburg

Vallières

Abdalah

, et al. The image biomarker standardization initiative: Standardized quantitative radiomics for high-throughput image-based phenotyping. Radiology. 2020;295(2):328–338. https://doi.org/10.1148/radiol.2020191145

22.

Koo

. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155–163. https://doi.org/10.1016/j.jcm.2016.02.012

23.

Schick

Lucia

Dissaux

, et al. MRI-derived radiomics: Methodology and clinical applications in the field of pelvic oncology. Br J Radiol. 2019;92(1104):20190105. https://doi.org/10.1259/bjr.20190105

24.

Zhou

Xie

, et al. Delta radiomics analysis for prediction of intermediary- and high-risk factors for patients with locally advanced cervical cancer receiving neoadjuvant therapy. Sci Rep. 2023;13(1):19409. https://doi.org/10.1038/s41598-023-46621-y

25.

Teng

Zhang

, et al. Improving radiomic model reliability using robust features from perturbations for head-and-neck carcinoma. Front Oncol. 2022;12:974467. https://doi.org/10.3389/fonc.2022.974467

26.

Vasquez

Roe

, et al. Least absolute shrinkage and selection operator type methods for the identification of serum biomarkers of overweight and obesity: Simulation and application. BMC Med Res Methodol. 2016;16(1):154. https://doi.org/10.1186/s12874-016-0254-8

27.

Demircioğlu

. Benchmarking feature selection methods in radiomics. Invest Radiol. 2022;57(7):433–443. https://doi.org/10.1097/RLI.0000000000000855

28.

Ali

Akhlaq

Imran

, et al. The enlightening role of explainable artificial intelligence in medical & healthcare domains: A systematic literature review. Comput Biol Med. 2023;166:107555. https://doi.org/10.1016/j.compbiomed.2023.107555

29.

Swisher

Chung

, et al. On the importance of interpretable machine learning predictions to inform clinical decision making in oncology. Front Oncol. 2023;13:1129380. https://doi.org/10.3389/fonc.2023.1129380

30.

Ghosh

Kandasamy

. Interpretable artificial intelligence: Why and when. AJR Am J Roentgenol. 2020;214(5):1137–1138. https://doi.org/10.2214/AJR.19.22145

31.

Biecek

. DALEX: Explainers for complex predictive models in R. J Mach Learn Res. 2018;19:3245–3249.

32.

Wright

. Interpreting black-box machine learning models using partial dependence and individual conditional expectation plots. Exploring SAS® Enterprise Miner Special Collection. 2018;2018:1950–2018.

33.

Staniak

Biecek

. Explanations of model predictions with live and breakDown packages. R Journal. 2018;10:395–409. https://doi.org/10.32614/RJ-2018-072

34.

Kim

Handayani

Lee

, et al. Feature attribution analysis to quantify the impact of oceanographic and maneuverability factors on vessel shaft power using explainable tree-based model. Sensors (Basel). 2023;23(3):1072. https://doi.org/10.3390/s23031072

35.

Wang

Chen

, et al. A head-to-head comparison of computed tomography- and magnetic resonance imaging-based radiomics in assessing pediatric peripheral neuroblastic tumor cell behavior. Abdom Radiol (NY). 2024;49(8):2942–2952. https://doi.org/10.1007/s00261-024-04411-8

36.

Zhang

, et al. Machine-learning radiomics to predict bone marrow metastasis of neuroblastoma using magnetic resonance imaging. Cancer Innov. 2023;2(5):405–415. https://doi.org/10.1002/cai2.92

37.

Chen

Liu

, et al. Radiomics models to predict bone marrow metastasis of neuroblastoma using CT. Cancer Innov. 2024;3(5):e135. https://doi.org/10.1002/cai2.135

38.

Wang

Xie

Chen

, et al. Development and validation of a CT-based radiomics signature for identifying high-risk neuroblastomas under the revised Children's Oncology Group classification system. Pediatr Blood Cancer. 2023;70(5):e30280. https://doi.org/10.1002/pbc.30280

39.

Zhang

Yang

Ning

, et al. Contrast computed tomography-based radiomics is correlation with COG risk stratification of neuroblastoma. Abdom Radiol (NY). 2023;48(6):2111–2121. https://doi.org/10.1007/s00261-023-03875-4

40.

Wang

Xie

Chen

, et al. Radiomics analysis of contrast-enhanced computed tomography in predicting the international neuroblastoma pathology classification in neuroblastoma. Insights Imaging. 2023;14(1):106. https://doi.org/10.1186/s13244-023-01418-5

41.

Ghosh

Yekeler

Teixeira

, et al. Role of MRI radiomics for the prediction of MYCN amplification in neuroblastomas. Eur Radiol. 2023;33(10):6726–6735. https://doi.org/10.1007/s00330-023-09628-7

42.

Jia

Liang

, et al. Radiomic-based machine learning model for predicting the surgical risk in children with abdominal neuroblastoma. World J Pediatr Surg. 2023;6(3):e000531. https://doi.org/10.1136/wjps-2022-000531

43.

Wang

Ding

, et al. Computed tomography-based radiomics signature for predicting segmental chromosomal aberrations at 1p36 and 11q23 in pediatric neuroblastoma. J Comput Assist Tomogr. 2023;48(3):472–479. https://doi.org/10.1097/RCT.0000000000001564

44.

Castellano

Bonilha

, et al. Texture analysis of medical images. Clin Radiol. 2004;59(12):1061–1069. https://doi.org/10.1016/j.crad.2004.07.008

45.

Wullkopf

West

Leijnse

, et al. Cancer cells’ ability to mechanically adjust to extracellular matrix stiffness correlates with their invasive potential. Mol Biol Cell. 2018;29(20):2378–2385. https://doi.org/10.1091/mbc.E18-05-0319

46.

Elgundi

Papanicolaou

Major

, et al. Cancer metastasis: The role of the extracellular matrix and the heparan sulfate proteoglycan perlecan. Front Oncol. 2020;9:1482. https://doi.org/10.3389/fonc.2019.01482

47.

Lugano

Ramachandran

Dimberg

. Tumor angiogenesis: Causes, consequences, challenges and opportunities. Cell Mol Life Sci. 2020;77(9):1745–1770. https://doi.org/10.1007/s00018-019-03351-7

48.

Hassija

Chamola

Mahapatra

, et al. Interpreting black-box models: A review on explainable artificial intelligence. Cogn Comput. 2020;16:45–74. https://doi.org/10.1007/s12559-023-10179-8

49.

Davenport

Kalakota

. The potential for artificial intelligence in healthcare. Future Healthc J. 2019;6(2):94–98. https://doi.org/10.7861/futurehosp.6-2-94

50.

Sarker

. Deep learning: A comprehensive overview on techniques, taxonomy, applications and research directions. SN Comput Sci. 2021;2(6):420. https://doi.org/10.1007/s42979-021-00815-1

51.

Koçak

. Key concepts, common pitfalls, and best practices in artificial intelligence and machine learning: Focus on radiomics. Diagn Interv Radiol. 2022;28(5):450–462. https://doi.org/10.5152/dir.2022.211297

52.

Zhang

Cheng

, et al. Artificial intelligence-driven radiomics study in cancer: The role of feature engineering and modeling. Mil Med Res. 2023;10(1):22. https://doi.org/10.1186/s40779-023-00458-8

53.

Matthay

Neuhaus

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.04 MB