Radiomics Nomogram Based on Optimal Volume of Interest Derived from High-Resolution CT for Preoperative Prediction of IASLC Grading in Clinical IA Lung Adenocarcinomas: A Multi-Center,Large-Population Study

Abstract

The novel grading system developed by the International Association for the Study of Lung Cancer (IASLC) for clinical stage IA lung adenocarcinomas has demonstrated remarkable prognostic capabilities. Notably, tumors classified as grade 3 have been associated with poor prognostic outcomes, thereby playing a crucial role in the formulation of personalized surgical strategies. The objective of this study is to develop a radiomics nomogram that utilizes the optimal volume of interest (VOI) derived from high-resolution CT (HRCT) scans to accurately predict the presence of grade 3 tumors in patients with clinical IA lung adenocarcinomas.

In this multi-center, large-population study, clinical, pathological, and HRCT imaging data from 1418 patients who were pathologically diagnosed with lung adenocarcinomas were retrospectively collected. The data was obtained from four hospital databases between January 2018 and May 2022. From this patient cohort, 1206 individuals were screened from three databases and randomly divided into training and internal validation datasets in a 7:3 ratio. An additional dataset consisting of 212 individuals was used for external validation dataset. Radiomics features were extracted from HRCT images at various scales, including VOI^−2mm, VOI ^entire, VOI ^+2mm, and VOI ^+4mm. To reduce dimensionality, select relevant features, and build radiomics signatures, the maximal redundancy minimal relevance (mRMR) and least absolute shrinkage and selection operator (LASSO) algorithm were utilized. Univariate and multivariate logistic regression analyses were conducted to identify independent clinic-radiological (Clin-Rad) predictors. Receiver operating characteristic (ROC) curves and corresponding area under the curve (AUC) were used to evaluate the diagnostic efficiency. A nomogram predicting the risk of grade 3 in clinical stage IA lung adenocarcinoma was constructed based on multivariate logistic regression, combining independent predictors and the optimal radiomics signatures.

Multivariate logistic regression revealed that males exhibited a higher prevalence of grade 3 tumors, and solid nodules were frequently observed through radiological assessments. The utilization of radiomics features extracted from the VOI ^entire resulted in significant improvements in predictive performance, as evidenced by AUC values of 0.900 (0.880-0.942), 0.885 (0.824-0.946), and 0.888 (0.782-0.993) for the training, internal validation, and external validation datasets, respectively. Furthermore, the nomogram that combined VOI ^entire -based radiomics signatures and Clin-Rad characteristics, exhibited remarkable predictive performance. This was indicated by AUC values of 0.910(0.873-0.942), 0.891 (0.845-0.937), and 0.905 (0.846-0.964) for the training, internal validation, and external validation datasets, respectively.

The extraction of radiomics features from both the indented and peri-tumoral regions does not offer any additional benefits in predicting grade 3 tumors according to the IASLC system. However, when combining the VOI ^entire-based radiomics model with Clin-Rad characteristics, the resulting integrated nomogram exhibited remarkable predictive performance.

Keywords

lung adenocarcinomas grading system high-resolution CT volume of interest (VOI)radiomics nomogram

Introduction

Lung adenocarcinomas, even at clinical stage I, exhibit heterogeneous histologic features, with approximately 80% to 90% displaying a combination of various patterns.¹ The histological classification of adenocarcinomas established by World Health Organization (WHO) in 2015 characterized these tumors according to their distinct growth patterns, which include lepidic, papillary, acinar, micropapillary, and solid-predominant types. Research has demonstrated that these patterns are associated with patient prognosis.^2–5 In 2020, the Pathology Committee of the International Association for the Study of Lung Cancer (IASLC) proposed a refined three-tier grading system that considers both the predominant histologic subtype and specific high-grade components, such as micropapillary or solid -predominant.⁶ According to the updated grading system criteria, grade 1 now includes a lepidic-predominant component with less than 20% high-grade patterns, grade 2 comprises a papillary or acinar-predominant component with less than 20% high-grade patterns, and grade 3 encompasses any component with 20% or more high-grade patterns.⁶ Subsequently, the fifth edition of the WHO thoracic tumor classification, released in 2021, incorporated this revised grading system.⁷ Further validation studies have confirmed that this comprehensive approach improves prognostic efficacy compared to the WHO 2015 classification, which was based solely on the predominant subtype.^8–11

The IASLC system provides important guidance for determining the most effective treatment regimens for lung adenocarcinomas. In case of grade 3 tumors, which are characterized by their aggressive behavior and poor prognosis, the utilization of segmentectomy is restricted. Instead, lobectomy combined with systemic mediastinal lymph node dissection is recommended.^12,13 Additionally, these patients may benefit from adjuvant chemotherapy.¹⁴ Therefore, preoperative diagnosis of grade 3 tumors could facilitate surgeons in selecting the most suitable surgical interventions.

Currently, puncture biopsy and intraoperative rapid frozen pathology are important approaches in diagnosing lung adenocarcinoma. However, these techniques may lead to inadequate preoperative diagnosis and false negative outcomes, primarily due to limited sample sizes and variations in operator expertise. Additionally, common complications such as pneumothorax, hemorrhage, and pleural effusions should not be overlooked.¹⁵ Studies have indicated that clinic-radiological (Clin-Rad) characteristics, including air bronchogram, consolidation/tumor ratio, and margin assessment, can help predict the IASLC grading system preoperatively.^16,17 Meanwhile, Fujikawa et al¹⁸ identified a higher maximum standardized uptake value (SUVmax) on PET as a significant preoperative predictive factor for grade 3 tumors, achieving a prediction rate exceeding 90%. However, it is important to note that Clin-Rad and PET SUVmax may have a certain level of subjectivity, lack representativeness, and incur substantial costs can be costly.

The radiomics approach, which quantifies the size, shape, margin, and internal density of tumors as imaging biomarkers, has shown potential in various aspects of lung cancer research, including tissue classification, oncogenetic correlations, treatment response, and disease prognosis.^19,20 Yang et al²¹ found that a nomogram integrating CT-based intratumoral radiomics and Clin-Rad resulted in improved predictive performance for the preoperative assessment of the IASLC grading system in lung adenocarcinoma.

Invasion and metastasis represent critical biological characteristics of malignant tumors. Consequently, peritumoral radiomics features have gained increasing attention due to their potential to capture proliferation, migration, invasion, and microenvironmental changes, which are common characteristics of malignancies. Several studies have successfully extracted radiomics features from both intratumoral and peritumoral regions, leading to improved diagnostic efficiency in predicting various aspects of lung adenocarcinoma, such as invasiveness in ground-glass nodules (GGNs),²² preoperative lymphovascular invasion,²³ lymph node metastasis,²⁴ and EGFR mutation assessment.²⁵ However, there is currently a lack of literatures systematically evaluating the radiomics model of multi-scale peritumoral ranges of lung nodules for the preoperative prediction of IASLC grading in clinical stage IA lung adenocarcinomas.

To achieve successful radiomics outcomes, it is crucial to develop predictive models that are reliable, highly accurate, and efficient. In pursuit of this objective, we conducted a multi-center study involving a large population to ensure the robustness and predictive capacity of our models. Our focus was on deriving radiomics features from both intratumoral and peritumoral regions using various volumes of interest (VOIs). To determine the optimal VOI, we performed a comprehensive comparison and selected the most effective combination for constructing a nomogram, thereby enhancing the accuracy of predicting IASLC grading in clinical stage IA lung adenocarcinomas.

Method

Enrollment of Patients

In this multi-center, large-population study, a retrospective analysis was conducted on the clinical, pathological, and HRCT images of patients who underwent surgical resection for lung adenocarcinomas between January 2018 and May 2022 across four hospitals. The inclusion criteria were as follows: (1) Clinical stage IA (cT1N0M0) with a maximum nodule diameter of ≤3 cm; (2) patients who had undergone chest CT within 7 days prior to surgery; (3) availability of HRCT with specified slice thickness; (4) no prior history of chemotherapy or radiotherapy. The exclusion criteria included: (1) the presence of multiple lesions; (2) tissue samples obtained via biopsy rather than surgical resection; and (3) the concurrent presence of other malignant tumors.

Subsequently, after a thorough screening process, a total of 1206 individuals were recruited from three distinct databases: Database 1 (n = 576), Database 2 (n = 439), and Database 3 (n = 191). These individuals were then randomly allocated to form both the training (n = 844) and internal validation (n = 362) datasets, adhering toa ratio of 7:3. Additionally, an independent validation dataset consisting of 212 individuals was established from Database 4 (Figure 1). This study received approval from the local research ethics committee, with the protocol identification numbers: KY2020147, 2021-07-009, 22/244-3446, 2023YJSLL-121, and KY-2024-231. Given the retrospective nature of our study, informed consent was waived.

Figure 1.

Schematic representation of the screening process for enrolled patients.

Histopathology Re-Evaluation

The hematoxylin- and eosin-stained slides from all enrolled patients were re-evaluated and graded according to the IASLC 2020 classification. The assessment was conducted by two experienced pathologists at each participating institution, each with over a decade of expertise in pulmonary pathology, using a multi-head microscope. Consensus on the evaluation was reached through collaborative discussion. In addition to calculating the percentages of the five primary histological subtypes in 5% increments according to the 2015 WHO guidelines, we also examined non-traditional complex glandular patterns.² These patterns included cribriform formations, characterized by nests of neoplastic cells exhibiting sieve-like perforations, and fused gland structures, which are defined as poorly developed fused glands that lack intervening stroma or appear in a ribbon-like arrangement with irregular boundaries, accompanied by single cells infiltrating desmoplastic stroma. Individuals were graded according to the IASLC 2020 classification system⁶ and categorized into two groups based on histopathology evaluation results (Group1: grade1-2; Group2: grade3). Furthermore, other aggressive pathological features such as visceral pleural invasion (VPI), spread through air spaces (STAS), mediastinal lymph node metastasis (LN), and vascular invasion, were also collected.

HRCT Radiological Features Evaluation

A flowchart illustrating the workflow of the study is presented in Figure 2. For details regarding the acquisition of HRCT images, please refer to Supplemental Digital Content 1. Non-enhanced chest HRCT scans in DICOM format were imported into the RadiAnt DICOM Viewer (https://www.radiantviewer.com) to visualize the radiological features of the tumor through multi-planar reconstruction (MPR) and maximum intensity projection (MIP). Two thoracic radiologists, with 10 and 15 years of experience, respectively, assessed and measured the tumors both qualitatively and quantitatively. In case of disagreements regarding qualitative indicators, a consensus was reached through consultation. Quantitative parameters were analyzed based on the average measurements provided by the two radiologists. If any discrepancies arise during the discussion, a consensus will subsequently be achieved. HRCT images were utilized to characterize the lesions in terms of sizes density, shapes boundary, spiculation, lobulation, vascular convergence sign, and vacuole sign. For a comprehensive understanding of the HRCT radiological features, please refer to Figure 3, which summarizes their respective definitions.

Figure 2.

Workflow diagram illustrating the methodological approach employed in the study.

Figure 3.

Compendious overview of High-Resolution Computed Tomography (HRCT) radiological features along with their respective definitions.

CT Images Preprocessing and Tumor Segmentation

The ITK-SNAP software (version 4.0, www.itksnap.org) was utilized for the slice-by-slice delineation of the gross tumor volume along the tumor boundary, with the 3D region defined as the VOI^entire. The initial delineation of the VOI was conducted on CT images under a lung-specific CT window (width: 1500 HU; level: −500 HU). The tumor boundary was identified by a certified cardiothoracic radiologist with 10 years of experience, and the VOI was manually segmented across all two-dimensional sections of the nodule. Subsequently, another radiologist with 15 years of experience in the same field reviewed and adjusted the lesion delineation as necessary.

Previous research has established that an extension of 2 to 4 mm from the apparent border of a tumor is considered as the zone of microscopic disease extension and is recognized as an aggressive region. It has been observed that beyond this distance, there is no significant enhancement in diagnostic performance.^26,27 Therefore, in this study, the multi-scale peritumoral ranges were defined by shrinking or expanding distances of −2 mm, 2 mm, and 4 mm from the tumor boundary, designated as VOI^−2mm, VOI ^+ 2mm, and VOI ^+ 4mm, respectively. These regions were generated using pixel filtering on a Python platform (version 3.11, https://www.python.org). During the process of obtaining these VOIs, vessels, pleura, the chest wall, mediastinum, and bronchus were intentionally excluded based on appropriate pixel thresholding. Artificial exclusion of these structures was conducted for the defined VOIs when necessary.^22–27

Image Preprocessing and Radiomics Feature Extraction

After incorporating CT images from multiple hospitals and utilizing various CT protocols, we implemented a comprehensive image preprocessing pipeline aimed at mitigating the variability of associated with different CT scanners and acquisition parameters. This pipeline included several key steps. First, we performed image resampling using the PyRadiomics package within the Python programming environment to standardize voxel spacing to 1 × 1 × 1 mm³ (x, y, z). The resampling algorithm employed was based on the B-spline curve, with the interpolator set to sitkBSpline (= 3), ensuring smooth and accurate interpolation of the image data.

Next, we adjusted the bin width to partition the pixel intensity range of 0–255 into five intervals. This binning approach effectively discretized the intensity values, thereby reducing the dimensionality of the feature space and facilitating subsequent analysis.

Subsequently, for each VOI, radiomics features were extracted from the original images and those processed using Laplacian of Gaussian (LoG) filtering with sigma values of 2, 3, 4, 5, as well as wavelet filtering applied in various directions. The extracted radiomics features included first-order statistics, shape-based metrics, the gray level dependence matrix, the gray level co-occurrence matrix, the gray level run length matrix, the gray level size zone matrix, and the neighboring gray tone difference matrix. In total, 1239 radiomics features were extracted. The images processed with LoG and wavelet filtering are considered derived images, and the corresponding radiomics features are regarded as derived features. Detailed description of the feature types is presented in Figure 4, as illustrated in the preceding reference.^19,20 Following this, all radiomics features were normalized using the z-score transformation.²⁸

z score = \frac{x - μ}{σ}

Figure 4.

Detailed elucidation of radiomics features, including terminology and interpretative framework.

Where χ represents each radiomics feature, μ represents the mean of all features, σ represents standard deviation of all features.

Radiomics Feature Selection and Model Construction

To mitigate the risk of overfitting, we employed the maximal redundancy minimal relevance (mRMR) algorithm and the least absolute shrinkage and selection operator (LASSO) method to reduce the dimensionality of the radiomics features in the training dataset. Specifically, the mRMR algorithm was utilized with a threshold set at 100, aiming to select features that exhibit minimal redundancy and maximal relevance to the target variable. This process was conducted using the ‘mRMRe’ package in R software, which facilitated the efficient implementation of the algorithm.

Subsequently, we applied the LASSO method to further refine feature selection by penalizing the absolute values of the regression coefficients, thereby promoting a sparse model. This step was performed using the ‘glmnet’ package in R software. To determine the optimal hyperparameter λ that minimizes prediction error, we employed a 10-fold cross-validation process within the LASSO method. This cross-validation approach ensures the robustness and generalizability of our radiomics model by evaluating its performance across multiple subsets of the data.

Features exhibiting non-zero coefficients at the optimal value of λ were selected as the final candidate radiomics signature (Figure 5). Following feature selection, the rad score was established by combining the selected features with their corresponding weights in a linear manner, according to the formula:

rad score = β_{0} + β_{1} F_{1} + β_{2} F_{2} + \dots + β_{n} F_{n}

where β0 represents the intercept, F represents radiomics features, and βn represents the coefficient of Fn.

Figure 5.

Depiction of the feature selection process utilizing Lasso regression and a 10-fold cross-validation technique.

Four distinct types of VOIs were generated to calculate the various rad scores, leading to the development of four radiomics models: VOI ^entire, VOI^−2mm, VOI ^+ 2mm, and VOI ^+ 4mm.The diagnostic efficacy of these models was evaluated using receiver operating characteristic (ROC) curves, and the optimal radiomics model was determined based on the highest area under the curve (AUC).

Statistical Analysis

The Shapiro-Wilk test was used to evaluate the normality of the numeric variables. Numeric variables that exhibited a normal distribution were presented as mean ± standard deviation, and comparisons between the two groups were conducted using two independent samples t-test. For data that did not conform a normal distribution, the median along with 25th and 75th quartiles were utilized, and the Mann-Whitney U test was applied for analysis. For categorical variables, the chi-squared test was applied. Univariate logistic regression analysis was employed to identify predictive factors associated with outcome risk, with a significant threshold set at P < .05. Following this, multivariate logistic regression analysis was conducted to determine independent factors that demonstrated significance in the univariate analysis, also at P < .05. The results were presented as odds ratios (OR) with their 95% confidence intervals (CI) and corresponding P-value. A nomogram was subsequently generated based on the coefficients obtained from the multivariate logistic regression results. To evaluate the predictive performance of the model for the IASLC grading system, ROC curves and their associated AUC values were utilized, in addition to decision curve analysis (DCA) and calibration plots. These assessments were conducted across the training, internal validation, and external validation datasets. All statistical analyses were conducted using the R version 4.3.1, with a two-sided P-value of less than .05 considered statistically significant.

Results

Comparison of Baseline Characteristics among Enrolled Patients

A total of 1418 patients with clinical stage IA lung adenocarcinomas were enrolled in this study. Among them, 505 patients were pathologically diagnosed as grade 1, 804 as grade 2, and 109 was grade 3. The clinical-pathological characteristics corresponding to grades 1 through 3 of the IASLC grading system for clinical stage IA lung adenocarcinomas are presented in Table 1, showing that grade 3 patients are more likely to exhibit aggressive pathological features such as VPI, STAS, and LN compared to grades 1 and 2.

Table 1.

Clinical-Pathological Characteristics Across Grade 1-3 IASLC Grading System in Clinical IA Lung Adenocarcinomas.

Parameter	Total (N = 1418)	Grade1(N = 505)	Grade2(N = 804)	Grade3(N = 109)	P-value
Sex:					< .001
Male	493 (34.8%)	165 (32.7%)	265 (33.0%)	63 (57.8%)
Female	925 (65.2%)	340 (67.3%)	539 (67.0%)	46 (42.2%)
Age(y)	57.6 ± 11.1	54.9 ± 12.4	58.9 ± 10.0	60.7 ± 9.05	< .001
Size(mm)	18.1 ± 6.66	15.6 ± 6.52	19.0 ± 6.28	22.5 ± 5.82	< .001
VPI, n (%):					< .001
Unknow	276 (19.5%)	42 (8.32%)	204 (25.4%)	30 (27.5%)
Absence	997 (70.3%)	441 (87.3%)	504 (62.7%)	52 (47.7%)
Presence	145 (10.2%)	22 (4.36%)	96 (11.9%)	27 (24.8%)
Vascular invasion, n (%):					< .001
Unknow	61 (4.30%)	21 (4.16%)	38 (4.73%)	2 (1.83%)
Absence	1287 (90.8%)	479 (94.9%)	727 (90.4%)	81 (74.3%)
Presence	70 (4.94%)	5 (.99%)	39 (4.85%)	26 (23.9%)
STAS, n (%):					< .001
Unknow	676 (47.7%)	157 (31.1%)	451 (56.1%)	68 (62.4%)
Absence	659 (46.5%)	342 (67.7%)	305 (37.9%)	12 (11.0%)
Presence	83 (5.85%)	6 (1.19%)	48 (5.97%)	29 (26.6%)
LN status, n (%):					< .001
Unknow	440 (31.0%)	93 (18.4%)	308 (38.3%)	39 (35.8%)
Absence	915 (64.5%)	409 (81.0%)	457 (56.8%)	49 (45.0%)
Presence	63 (4.44%)	3 (0.59%)	39 (4.85%)	21 (19.3%)

Abbreviation: VPI, Visceral pleural invasion; STAS, Spread through air spaces; LN, Mediastinal lymph node metastasis; IASLC, international association for the study of lung cancer.

Detailed clinical and radiological characters were recorded and compared among the training, internal validation, and external validation datasets (Table 2), and there were no statistically significant differences (P = .074-.935) were observed among these datasets.

Table 2.

Comparative Analysis of Radiological Features Across Training, Internal Validation, and External Validation Databases.

Variables	Total (n = 1418)	Training database (n = 844)	Internal-validation database (n = 362)	External-validation Database (n = 212)	P-value
Sex, n (%)					.611
Male	493 (34.8)	290 (34.4)	123 (34)	80 (37.7)
Female	925 (65.2)	554 (65.6)	239 (66)	132 (62.3)
Age, Median (Q1, Q3)	58 (51, 66)	58 (52, 66)	58 (51, 66)	58 (51, 67)	.935
Size, Median (Q1, Q3)	17 (12.6, 23.3)	17.1 (12.6, 23.3)	16.8 (12.2, 22.8)	16.9 (12.9, 24.1)	.918
IASLC system, n (%)					.778
Grade 1-2	1308 (92.2)	780 (92.4)	331 (91.4)	197 (92.9)
Grade3	110 (7.8)	64 (7.6)	31 (8.6)	15 (7.1)
Density, n (%)					.767
GGN	618 (43.6)	369 (43.7)	150 (41.4)	99 (46.7)
SN	406 (28.6)	238 (28.2)	111 (30.7)	57 (26.9)
SSN	394 (27.8)	237 (28.1)	101 (27.9)	56 (26.4)
Location, n (%)					.585
RUL	495 (34.9)	286 (33.9)	126 (34.8)	83 (39.2)
RLL	250 (17.6)	149 (17.7)	62 (17.1)	39 (18.4)
RML	99 (7)	64 (7.6)	28 (7.7)	7 (3.3)
LUL	385 (27.2)	231 (27.4)	97 (26.8)	57 (26.9)
LLL	189 (13.3)	114 (13.5)	49 (13.5)	26 (12.3)
Boundary, n (%)					.857
Ill-defined	362 (25.5)	211 (25)	95 (26.2)	56 (26.4)
Well-defined	1056 (74.5)	633 (75)	267 (73.8)	156 (73.6)
Shape, n (%)					.895
Irregular	699 (49.3)	412 (48.8)	180 (49.7)	107 (50.5)
Others	719 (50.7)	432 (51.2)	182 (50.3)	105 (49.5)
Lobulation, n (%)					.496
Absence	560 (39.5)	343 (40.6)	134 (37)	83 (39.2)
Presence	858 (60.5)	501 (59.4)	228 (63)	129 (60.8)
Spiculation, n (%)					.674
Absence	684 (48.2)	410 (48.6)	168 (46.4)	106 (50)
Presence	734 (51.8)	434 (51.4)	194 (53.6)	106 (50)
Vascular convergence sign, n (%)					.074
Absence	338 (23.8)	202 (23.9)	97 (26.8)	39 (18.4)
Presence	1080 (76.2)	642 (76.1)	265 (73.2)	173 (81.6)
Vacuole sign, n (%)					.85
Absence	1149 (81)	681 (80.7)	297 (82)	171 (80.7)
Presence	269 (19)	163 (19.3)	65 (18)	41 (19.3)
Pleural indentation, n (%)					.574
Absence	558 (39.4)	339 (40.2)	134 (37)	85 (40.1)
Presence	860 (60.6)	505 (59.8)	228 (63)	127 (59.9)

Abbreviation: IASLC, International Association for the Study of Lung Cancer; GGN, ground-glass nodule; SN, solid nodule; SSN, subsolid nodule; LUL left upper lobe, LLL left lower lobe, RUL right upper lobe, RML right middle lobe, RLL right lower lobe.

Clin-Rad Model Building

Table 3 presents a comparative analysis of Clin-Rad characteristics between grade 1-2 and grade 3 lung adenocarcinomas within the training datasets. Patients classified as grade 3, compared to grades 1-2, demonstrate t a significantly higher likelihood of exhibiting aggressive radiological features, such as lobulation, spiculation, pleural indentation, and a predominance of solid nodules (SN) density, when compared to those in grades 1-2 (all P < .05). Additionally, a male predominance, advanced age, and larger tumor size were also observed in grade 3 patients (all P < .05).

Table 3.

Comparative Analysis of Radiological Characteristics Between Grade 1-2 and Grade 3 Within the Training Database.

Variables	Total (n = 844)	Grade1-2 (n = 780)	Grade3 (n = 64)	P-value
Sex, n (%)				< .001
Male	290 (34.4)	253 (32.4)	37 (57.8)
Female	554 (65.6)	527 (67.6)	27 (42.2)
Age, Median (Q1, Q3)	58 (52, 66)	58 (52, 66)	62 (53.8, 66.2)	.173
Size, Median (Q1, Q3)	17.1 (12.6, 23.3)	16.6 (12.5, 22.8)	21.5 (16.6, 27.1)	< .001
Density, n (%)				< .001
GGN	369 (43.7)	366 (46.9)	3 (4.7)
SN	238 (28.2)	183 (23.5)	55 (85.9)
SSN	237 (28.1)	231 (29.6)	6 (9.4)
Location, n (%)				.051
RUL	286 (33.9)	268 (34.4)	18 (28.1)
RLL	149 (17.7)	129 (16.5)	20 (31.2)
RML	64 (7.6)	60 (7.7)	4 (6.2)
LUL	231 (27.4)	219 (28.1)	12 (18.8)
LLL	114 (13.5)	104 (13.3)	10 (15.6)
Boundary, n (%)				.652
Ill-defined	211 (25)	197 (25.3)	14 (21.9)
Well-defined	633 (75)	583 (74.7)	50 (78.1)
Shape, n (%)				.171
Irregular	412 (48.8)	375 (48.1)	37 (57.8)
Others	432 (51.2)	405 (51.9)	27 (42.2)
Lobulation, n (%)				.002
Absence	343 (40.6)	329 (42.2)	14 (21.9)
Presence	501 (59.4)	451 (57.8)	50 (78.1)
Spiculation, n (%)				< .001
Absence	684 (48.2)	393 (50.4)	17 (26.6)
Presence	734 (51.8)	387 (49.6)	47 (73.4)
Vascular convergence sign, n (%)				.719
Absence	202 (23.9)	185 (23.7)	17 (26.6)
Presence	642 (76.1)	595 (76.3)	47 (73.4)
Vacuole sign, n (%)				.963
Absence	681 (80.7)	630 (80.8)	51 (79.7)
Presence	163 (19.3)	150 (19.2)	13 (20.3)
Pleural indentation, n (%)				< .001
Absence	339 (40.2)	331 (42.4)	8 (12.5)
Presence	505 (59.8)	449 (57.6)	56 (87.5)

Abbreviation: GGN, ground-glass nodule; SN, solid nodule; SSN, subsolid nodule; LUL left upper lobe, LLL left lower lobe, RUL right upper lobe, RML right middle lobe, RLL right lower lobe.

Multivariate logistic regression analysis identified sex and CT density as independent predictors of grade 3 lung adenocarcinomas, as defined by the IASLC grading system, in patients with clinical stage IA disease (Table 4). Notably, female gender was found to be associated with a decreased risk, yielding an OR of 0.946 (95% CI: 0.913-0.98). In contrast, a higher CT density was significantly correlated with an increased risk of grade 3 lung adenocarcinomas, with an OR of 1.215 (95% CI: 1.16-1.273), using GGN as the reference point. The Clin-Rad model developed by these two independent predictors showed AUCs of 0.855 (95% CI: 0.814-0.896), 0.797 (95% CI: 0.728-0.865), and 0.853 (0.788-0.918) in the training, internal validation, and external validation datasets, respectively.

Table 4.

Univariate and Multivariate Logistic Analysis of Clin-Rad Characteristics for the Prediction of Grade 3 According to the IASLC System in Patients with Clinical Stage IA Disease.

Variables	Univariate		Multivariate
Variables	Odd Ratio (95%CI)	P-value	Odd Ratio (95%CI)	P-value
Sex	0.35 (0.21-0.59)	<.001	0.946(0.913-0.98)	.002
Age	1 (1-1)	.11
Size	1.1 (1-1.1)	<.001	1.001(0.999-1.004)	.343
Density	37 (11-120)	<.001	1.215(1.16-1.273)	<.001
Location	0.99 (0.32-3)	.99
Boundary	1.2 (0.65-2.2)	.55
Shape	0.68 (0.4-1.1)	.14
Lobulation	2.6 (1.4-4.8)	.0021	0.996(0.954-1.04)	.848
Spiculation	2.8 (1.6-5)	<.001	1.007(0.964-1.052)	.742
Vascular convergence sign	0.86 (0.48-1.5)	.61
Vacuole sign	1.1 (0.57-2)	.83
Pleural indentation	5.2 (2.4-11)	<.001	1.029(0.99-1.07)	.143

Performance of Multi-Scale VOI Radiomics Model

The Rad score for each VOI subgroup was constructed using radiomics features selected through LASSO regression. which further facilitated the construction of various VOI^−2mm-based, VOI ^entire-based, VOI ^+ 2mm-based, and VOI ^+ 4mm-based radiomics models. Notably, grade 3 achieved higher Rad scores across all the VOI subgroups when compared to grades 1 and 2, as evidenced in the training, internal validation, and external validation datasets (Figure 6). Overall, the VOI ^entire-based radiomics model led to substantial predictive performance, as indicated by AUCs of 0.900 (95%CI: 0.880-0.942), 0.885 (95%CI: 0.824-0.946), and 0.888 (95%CI: 0.782-0.993) in the training, internal validation, and external validation datasets, respectively (Figure 7, Table 5).

Figure 6.

Comparative analysis of Rad scores across various VOI subgroups within the training, internal validation, and external validation datasets.

Figure 7.

Comparative assessment of Receiver Operating Characteristic (ROC) curves for each VOI subgroup across the training, internal validation, and external validation datasets.

Table 5.

Performance Evaluation of Radiomics Models Utilizing Multi-Scale VOIs for Predicting Grade 3.

Parameter	AUC (95%CI)	ACC	SEN	SPE
VOI _−2mm-based radiomics model
Training database	0.888 (0.849-0.928)	0.819	0.844	0.817
Internal-validation database	0.882 (0.817-0.948)	0.840	0.903	0.834
External-validation database	0.918 (0.869-0.968)	0.840	0.933	0.832
VOI _entire-based radiomics model
Training database	0.900 (0.880-0.942)	0.866	0.812	0.871
Internal-validation database	0.885 (0.824-0.946)	0.820	0.935	0.810
External-validation database	0.888 (0.782-0.993)	0.925	0.800	0.934
VOI _+2mm-based radiomics model
Training database	0.893 (0.857-0.928)	0.783	0.906	0.773
Internal-validation database	0.884 (0.824-0.943)	0.818	0.935	0.807
External-validation database	0.889 (0.809-0.970)	0.807	0.933	0.797
VOI _+4mm-based radiomics model
Training database	0.891 (0.854-0.928)	0.825	0.844	0.823
Internal-validation database	0.883 (0.840-0.927)	0.801	0.903	0.792
External-validation database	0.874 (0.780-0.967)	0.712	0.933	0.695

Abbreviation: AUC, Area under curve; ACC, Accuracy; SEN, Sensitivity; SPE, Specificity; VOI, Volume of internet.

Hybrid Model Construction and Clinical Application of the Nomogram

Given that the VOI ^entire-based radiomics model exhibited superior predictive performance for grade 3 according to the IASLC system in patients with clinical stage IA disease, a hybrid model was subsequently developed, integrating the VOI ^entire-based radiomics with the Clin-Rad model, which incorporates variables such as sex and CT density. The hybrid model demonstrated outstanding predictive performance, as evidenced by AUC values of 0.910 (95%CI: 0.873-0.942), 0.891 (95%CI: 0.845-0.937), and 0.905 (95%CI: 0.846-0.964) for the training, internal validation, and external validation datasets, respectively (Figure 8). Details of the AUCs for the hybrid, radiomics, and Clin-Rad models are presented in Table 6. The DCA indicated that the hybrid model yielded the highest net benefits within a threshold probability range of >8% to <55% in the training dataset, >9% to <38% in the internal validation dataset, and <75% in the external validation dataset. These ranges collectively encompass the broadest spectrum of probabilities (Figure 9).

Figure 8.

Comparative evaluation of ROC curves for the Clin-Rad model, radiomics model, and hybrid model across the training, internal validation, and external validation datasets.

Figure 9.

Comparative evaluation of DCA curves for the Clin-Rad model, radiomics model, and hybrid model across the training, internal validation, and external validation datasets.

Table 6.

Performance Assessment of Radiomics, Radiological, and Hybrid Models in Predicting Grade 3.

Parameter	AUC (95%CI)	ACC	SEN	SPE
Hybrid model
Training database	0.910 (0.873-0.942)	0.795	0.938	0.783
Internal-validation database	0.891 (0.845-0.937)	0.834	0.903	0.828
External-validation database	0.905 (0.846-0.964)	0.892	0.800	0.898
Clin-Rad model
Training database	0.855 (0.814-0.896)	0.773	0.859	0.765
Internal-validation database	0.797 (0.728-0.865)	0.735	0.742	0.734
External-validation database	0.853 (0.788-0.918)	0.689	0.933	0.67
Radiomics model
Training database	0.900 (0.880-0.942)	0.866	0.812	0.871
Internal-validation database	0.885 (0.824-0.946)	0.820	0.935	0.810
External-validation database	0.888 (0.782-0.993)	0.925	0.800	0.934

Abbreviation: AUC, Area under curve; ACC, Accuracy; SEN, Sensitivity; SPE, Specificity.

The hybrid model was transformed into a nomogram that incorporates Rad score, sex, and CT density to quantify the risk of grade 3 in clinical stage IA lung adenocarcinomas (Figure 10). Additionally, the nomogram demonstrated strong calibration capabilities, as shown by the standard curve fitting well with the predicted curve across the training, internal validation, and external validation datasets, respectively (Figure 11).

Figure 10.

Radiomics nomogram for preoperative prediction of IASLC grading in clinical stage IA lung adenocarcinomas.

Figure 11.

Calibration plots for the Clin-Rad model, radiomics model, and hybrid model across the training, internal validation, and external validation datasets.

Discussion

In this multicenter, large-scale study, we conducted a multiscale region-based radiomics analysis to predict the IASLC grading system in clinical stage IA lung adenocarcinomas. Our findings suggest that extraction of radiomic features from both the indented and peritumoral regions does not provide additional benefits in predicting grade 3 according to the IASLC system. The Clin-Rad model, which integrates includes sex and CT density, was identified as an independent predictor of the IASLC grading system in clinical stage IA lung adenocarcinomas. Importantly, the VOI ^entire-based radiomics model, when combined with the Clin-Rad model, resulted in an integrated nomogram that demonstrated remarkable predictive performance, with strong discriminative and calibration capabilities.

In comparison to grades 1 and 2, grade 3 patients are more likely to exhibit aggressive pathological features such as VPI, STAS, LN, and vascular invasion, according to the IASLC grading system for clinical stage IA lung adenocarcinomas. This indicates that grade 3 tumors exhibit more aggressive biological behavior, which aligns with findings from previous studies.^8–11 In the present study, the Clin-Rad model incorporates factors such as male predominance and elevated CT density, indicating that male patients with grade 3 tumors exhibit a slight yet statistically significant increase in risk compared to their female counterparts. Additionally, these tumors primarily characterized as SN in CT image findings, a result that corroborates the outcomes of previous investigations.^8,21

To further enhance diagnostic capabilities, our study explored the impact of multi-scale ranges on the preoperative prediction of IASLC grading by radially adjusting the distances at −2, 2, and 4 mm from the tumor boundary. Our findings suggest that the VOI ^entire-based radiomics model demonstrates robust predictive capability in distinguishing between grade 3 and grades 1-2. Furthermore, the VOI ^entire-based radiomics model outperforms the Clin-Rad model, indicating that the radiomics model provides additional insights that visual assessments alone may not provide. In contrast to previous research,^22–25 the integration of peritumoral radiomics features did not enhance the performance beyond exclusive tumor features. This finding suggests that the diagnostic benefits provided by peritumoral radiomics are adaptive and closely associated with various factors such as nodule type, peritumoral range, and tumor malignancy. Wu et al²⁶ concluded that extending the analysis to a peritumoral region of 5 mm does not provide any additional advantage in distinguishing invasive adenocarcinoma from adenocarcinoma in situ or minimally invasive adenocarcinoma when compared to the 2 mm region. Additionally, another study.²⁹ found that the radiomics features of the surrounding parenchyma of the nodule did not improve the effectiveness of the diagnostic model in distinguishing between benign and malignant solid pulmonary nodules smaller than 2 cm.

The hybrid model, which incorporates the VOI ^entire-based radiomics and Clin-Rad characteristics, exhibits remarkable predictive performance. This approach enables a more comprehensive extraction of information from multiple dimensions. By integrating quantitative imaging features derived from radiomics with Clin-Rad characteristics, the model captures a broader spectrum of disease characteristics, thereby enhancing both accuracy and robustness in diagnostic processes. Additionally, this multimodal fusion helps overcome the limitations of relying solely on either diagnostic model, potentially uncovering concealed patterns or relationships that may not be apparent when utilizing only one type of data. Ultimately, this integration facilitates a more holistic and precise diagnostic assessment.

This study has shown promising results but also has several limitations. Firstly, as a retrospective study, it is susceptible to selection bias, which may have led to a lower-than- anticipated representation of grade 3 patients, potentially underestimating the actual positive rate. To mitigate this concern, future research could improve sampling methods or increase sample sizes. Secondly, although preoperative imaging was utilized to predict grade 3 of the IASLC system, long-term follow-up is essential to elucidate the relationship between Clin-Rad features, radiomics features, outcomes, and survival, thereby improving the understanding of the prognostic significance of radiomics features. Thirdly, there is currently no standardized method for delineating tumoral and peritumoral VOIs for the extraction of radiomics features and the prediction of the IASLC grading system in clinical stage IA lung adenocarcinomas. To address this challenge, further comparative studies with pathological assessments are warranted.

Conclusion

The extraction of radiomic features from both the intratumoral and peritumoral regions does not offer additional benefits in predicting grade 3 according to the IASLC system. Furthermore, when the VOI ^entire-based radiomics model is combined with Clin-Rad characteristics, the resulting nomogram exhibits remarkable predictive performance. The multicenter, large-scale nature of this study further enhances its robustness and accuracy, thereby minimizing redundancy and improving reliability.

Supplemental Material

sj-tif-1-tct-10.1177_15330338241300734 - Supplemental material for Radiomics Nomogram Based on Optimal Volume of Interest Derived from High-Resolution CT for Preoperative Prediction of IASLC Grading in Clinical IA Lung Adenocarcinomas: A Multi-Center, Large-Population Study

Supplemental material, sj-tif-1-tct-10.1177_15330338241300734 for Radiomics Nomogram Based on Optimal Volume of Interest Derived from High-Resolution CT for Preoperative Prediction of IASLC Grading in Clinical IA Lung Adenocarcinomas: A Multi-Center, Large-Population Study by Zhichao Zuo, Guochao Zhang, Shanyue Lin, Qi Xue, Wanyin Qi, Wei Zhang and Xiaohong Fan in Technology in Cancer Research & Treatment

Supplemental Material

sj-doc-2-tct-10.1177_15330338241300734 - Supplemental material for Radiomics Nomogram Based on Optimal Volume of Interest Derived from High-Resolution CT for Preoperative Prediction of IASLC Grading in Clinical IA Lung Adenocarcinomas: A Multi-Center, Large-Population Study

Supplemental material, sj-doc-2-tct-10.1177_15330338241300734 for Radiomics Nomogram Based on Optimal Volume of Interest Derived from High-Resolution CT for Preoperative Prediction of IASLC Grading in Clinical IA Lung Adenocarcinomas: A Multi-Center, Large-Population Study by Zhichao Zuo, Guochao Zhang, Shanyue Lin, Qi Xue, Wanyin Qi, Wei Zhang and Xiaohong Fan in Technology in Cancer Research & Treatment

Footnotes

Author Contributions

Zhichao Zuo, Guochao Zhang, and Shanyue Lin contributed equally to this work. Zhichao Zuo primarily contributed to the writing. Guochao Zhang and Shanyue Lin were responsible for data collection and analysis. Qi Xue and Wanyin Qi participated in the conceptualization and design of the study. Wei Zhang and Xiaohong Fan provided critical revisions and approved the final manuscript. All authors have reviewed and approved the final version of the manuscript.

Compliance with Ethical Requirements

The Ethics Committees of the participating hospitals approved the study (reference numbers: KY2020147, 2021-07-009, 22/244-3446, 2023YJSLL-121, and KY-2024-231) in accordance with the Declaration of Helsinki.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

This work was supported by The National Key R&D Program of China (2022YFC2407404, Qi Xue); Beijing Natural Science Foundation (7232134, Guochao Zhang); National High-Level Hospital Clinical Research Funding (2022-PUMCH-C-043, Qi Xue); Beijing Municipal Science & Technology Commission (Z211100002921058, Qi Xue); Beijing Hope Run Special Fund of Cancer Foundation of China (LC2021L01, Qi Xue).

Informed Consent Statement

Given the retrospective nature of our study, informed consent was waived.

ORCID iD

Zhichao Zuo

Supplemental Material

Supplemental material for this article is available online.

References

Marx

Chan

JKC

Chalabreysse

, et al.

The 2021 WHO classification of tumors of the Thymus and mediastinum: What is new in thymic epithelial, germ cell, and mesenchymal tumors?

J Thorac Oncol. Feb 2022;17(2):200‐213. doi:https://doi.org/10.1016/j.jtho.2021.10.010

Murakami

Ito

Tsubokawa

, et al. Prognostic value of the new IASLC/ATS/ERS classification of clinical stage IA lung adenocarcinoma. Lung Cancer (Amsterdam, Netherlands). Nov 2015;90(2):199‐204. doi:https://doi.org/10.1016/j.lungcan.2015.06.022

Campos-Parra

Avilés

Contreras-Reyes

, et al. Relevance of the novel IASLC/ATS/ERS classification of lung adenocarcinoma in advanced disease. Eur Respir J. May 2014;43(5):1439‐1447. doi:https://doi.org/10.1183/09031936.00138813

Tsuta

Kawago

Inoue

, et al. The utility of the proposed IASLC/ATS/ERS lung adenocarcinoma subtypes for disease prognosis and correlation of driver gene alterations. Lung Cancer (Amsterdam, Netherlands). Sep 2013;81(3):371‐376. doi:https://doi.org/10.1016/j.lungcan.2013.06.012

Yoshizawa

Motoi

Riely

, et al. Impact of proposed IASLC/ATS/ERS classification of lung adenocarcinoma: Prognostic subgroups and implications for further revision of staging based on analysis of 514 stage I cases. Mod Pathol. May 2011;24(5):653‐664. doi:https://doi.org/10.1038/modpathol.2010.232

Moreira

Ocampo

PSS

Xia

, et al. A grading system for invasive pulmonary adenocarcinoma: A proposal from the international association for the study of lung cancer pathology committee. J Thorac Oncol. Oct 2020;15(10):1599‐1610. doi:https://doi.org/10.1016/j.jtho.2020.06.001

International Agency for Research on Cancer. WHO Classification of Tumours. Thoracic Tumours. 5th ed. International Agency for Research on Cancer; 2021.

Deng

Zheng

Zhang

, et al. Validation of the novel international association for the study of lung cancer grading system for invasive pulmonary adenocarcinoma and association with common driver mutations. J Thorac Oncol. Oct 2021;16(10):1684‐1693. doi:https://doi.org/10.1016/j.jtho.2021.07.006

Huang

Zhang

, et al. A prognostic nomogram based on a new classification of combined micropapillary and solid components for stage IA invasive lung adenocarcinoma. J Surg Oncol. Mar 2022;125(4):796‐808. doi:https://doi.org/10.1002/jso.26760

10.

Rokutan-Kurata

Yoshizawa

Ueno

, et al. Validation study of the international association for the study of lung cancer histologic grading system of invasive lung adenocarcinoma. J Thorac Oncol. Oct 2021;16(10):1753‐1758. doi:https://doi.org/10.1016/j.jtho.2021.04.008

11.

Kagimoto

Tsutani

Kambara

, et al. Utility of newly proposed grading system from international association for the study of lung cancer for invasive lung adenocarcinoma. JTO clinical and Research Reports. Feb 2021;2(2):100126. doi:https://doi.org/10.1016/j.jtocrr.2020.100126

12.

Nitadori

Bograd

Kadota

, et al. Impact of micropapillary histologic subtype in selecting limited resection vs lobectomy for lung adenocarcinoma of 2 cm or smaller. J Natl Cancer Inst. Aug 21 2013;105(16):1212‐1220. doi:https://doi.org/10.1093/jnci/djt166

13.

Kamigaichi

Hamada

Tsutani

. Segmentectomy for patients with early-stage pure-solid non-small cell lung cancer. Front Oncol. 2023;13:1287088. doi:https://doi.org/10.3389/fonc.2023.1287088

14.

Jeon

Lee

Shin

, et al. Prognostic impact of micropapillary and solid histological subtype on patients undergoing curative resection for stage I lung adenocarcinoma according to the extent of pulmonary resection and lymph node assessment. Lung Cancer (Amsterdam, Netherlands). Jun 2022;168:21‐29. doi:https://doi.org/10.1016/j.lungcan.2022.04.005

15.

Kim

Buonocore

Petre

, et al. Utility of core biopsy specimen to identify histologic subtype and predict outcome for lung adenocarcinoma. Ann Thorac Surg. Aug 2019;108(2):392‐398. doi:https://doi.org/10.1016/j.athoracsur.2019.03.043

16.

Volmonen

Sederholm

Rönty

Paajanen

Knuuttila

Jartti

. Association of CT findings with invasive subtypes and the new grading system of lung adenocarcinoma. Clin Radiol. Mar 2023;78(3):e251‐e259. doi:https://doi.org/10.1016/j.crad.2022.11.011

17.

Wang

Liu

Ding

, et al. A pretreatment prediction model of grade 3 tumors classed by the IASLC grading system in lung adenocarcinoma. BMC Pulm Med. Oct 7 2023;23(1):377. doi:https://doi.org/10.1186/s12890-023-02690-3

18.

Fujikawa

Muraoka

Kashima

, et al. Clinicopathologic and genotypic features of lung adenocarcinoma characterized by the international association for the study of lung cancer grading system. J Thorac Oncol. May 2022;17(5):700‐707. doi:https://doi.org/10.1016/j.jtho.2022.02.005

19.

Tunali

Gillies

Schabath

. Application of radiomics and artificial intelligence for lung cancer precision medicine. Cold Spring Harbor Perspect Med. Aug 2 2021;11(8). doi:https://doi.org/10.1101/cshperspect.a039537

20.

Chen

Copley

Viola

Aboagye

. Radiomics and artificial intelligence for precision medicine in lung cancer treatment. Semin Cancer Biol. Aug 2023;93:97‐113. doi:https://doi.org/10.1016/j.semcancer.2023.05.004

21.

Yang

Cai

Chen

, et al. A CT-based radiomics nomogram combined with clinic-radiological characteristics for preoperative prediction of the novel IASLC grading of invasive pulmonary adenocarcinoma. Acad Radiol. Sep 2023;30(9):1946‐1961. doi:https://doi.org/10.1016/j.acra.2022.12.006

22.

Gao

Xiang

Zheng

Pang

. CT-Imaging Based analysis of invasive lung adenocarcinoma presenting as ground glass nodules using peri- and intra-nodular radiomic features. Front Oncol. 2020;10:838. doi:https://doi.org/10.3389/fonc.2020.00838

23.

Chen

Shao

Xue

, et al. Intratumoral and peritumoral radiomics nomograms for the preoperative prediction of lymphovascular invasion and overall survival in non-small cell lung cancer. Eur Radiol. Feb 2023;33(2):947‐958. doi:https://doi.org/10.1007/s00330-022-09109-3

24.

Wang

Zhao

, et al.

Can peritumoral radiomics increase the efficiency of the prediction for lymph node metastasis in clinical stage T1 lung adenocarcinoma on CT?

Eur Radiol. Nov 2019;29(11):6049‐6058. doi:https://doi.org/10.1007/s00330-019-06084-0

25.

Shang

Chen

, et al. Computed tomography-derived intratumoral and peritumoral radiomics in predicting EGFR mutation in lung adenocarcinoma. Radiol Med. Dec 2023;128(12):1483‐1496. doi:https://doi.org/10.1007/s11547-023-01722-6

26.

Gao

, et al. The value of various peritumoral radiomic features in differentiating the invasiveness of adenocarcinoma manifesting as ground-glass nodules. Eur Radiol. Dec 2021;31(12):9030‐9037. doi:https://doi.org/10.1007/s00330-021-07948-0

27.

Liu

, et al. Improving the accuracy of prognosis for clinical stage I solid lung adenocarcinoma by radiomics models covering tumor per se and peritumoral changes on CT. Eur Radiol. Feb 2022;32(2):1065‐1077. doi:https://doi.org/10.1007/s00330-021-08194-0

28.

Beaumont

Iannessi

Bertrand

Cucchi

Lucidarme

. Harmonization of radiomic feature distributions: Impact on classification of hepatic tissue in CT imaging. Eur Radiol. Aug 2021;31(8):6059‐6068. doi:https://doi.org/10.1007/s00330-020-07641-8

29.

Zhang

Ren

Linning

. Can peritumoral radiomics improve the prediction of malignancy of solid pulmonary nodule smaller than 2 cm? Acad Radiol. Feb 2022;29(Suppl 2):S47‐s52. doi:https://doi.org/10.1016/j.acra.2020.10.029

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB

1.42 MB