Abstract
Objective:
To evaluate the value of 2-dimensional (2D) and 3-dimensional (3D) computed tomography texture analysis (CTTA) models in distinguishing fat-poor angiomyolipoma (fpAML) from chromophobe renal cell carcinoma (chRCC).
Methods:
We retrospectively enrolled 32 fpAMLs and 24 chRCCs. Texture features were extracted from 2D and 3D regions of interest in triphasic CT images. The 2D and 3D CTTA models were constructed with the least absolute shrinkage and selection operator algorithm and texture scores were calculated. The diagnostic performance of the 2D and 3D CTTA models was evaluated with respect to calibration, discrimination, and clinical usefulness.
Results:
Of the 177 and 183 texture features extracted from 2D and 3D regions of interest, respectively, 5 2D features and 8 3D features were selected to build 2D and 3D CTTA models. The 2D CTTA model (area under the curve [AUC], 0.811; 95% confidence interval [CI], 0.695-0.927) and the 3D CTTA model (AUC, 0.915; 95% CI, 0.838-0.993) showed good discrimination and calibration (P > .05). There was no significant difference in AUC between the 2 models (P = .093). Decision curve analysis showed the 3D model outperformed the 2D model in terms of clinical usefulness.
Conclusions:
The CTTA models based on contrast-enhanced CT images had a high value in differentiating fpAML from chRCC.
Keywords
Introduction
Angiomyolipoma (AML) is the most common benign neoplasm of the kidney. 1 Most typical AMLs can be easily diagnosed on imaging by their mature fat components, which is a dependable method to distinguish AML from renal cell carcinoma (RCC). Approximately 5% of AMLs lack visible fat and mainly consist of blood vessels and smooth muscle cells; these are labeled as “fat-poor AML (fpAML)”. 2 Renal cell carcinoma has been classified into 3 major histologic subtypes: clear cell RCC (65%∼70%), papillary RCC (18.5%), and chromophobe RCC (chRCC; 5%∼7%) by the 2016 World Health Organization. 3 Among the 3 subtypes, chRCC is the rarest and least studied. Malignant chRCC is usually treated with radical nephrectomy, while AML can be monitored without any treatment or can be removed with nephron-sparing surgery. 4 Therefore, it is quite important to accurately distinguish fpAML from chRCC before surgery.
Computed tomography (CT) is widely accepted as the first-line imaging modality for preoperative diagnosis of renal tumors, allowing a noninvasive means of tumor characterization. Fat-poor angiomyolipoma shares overlapping CT features with chRCC. Both fpAML and chRCC appear as a homogeneous renal mass on unenhanced CT with rare necrosis, hemorrhage, or cystic changes. On contrast-enhanced CT, although there is a greater variability in enhancement characteristics of chRCCs, they are most likely to present as a homogeneous mass with an enhancement pattern that is hypovascular relative to clear cell RCC, mostly demonstrating a peak attenuation in the corticomedullary phase (CMP) with washout in the excretory phase (EP). 5 -7 The substantial overlap in the tumor’s enhancement pattern with fpAML makes it difficult to distinguish them relying on basic triphasic CT images. Recently, texture analysis (TA) has been widely used as a technique that can analyze the characteristics and distribution of pixel or voxel gray levels in medical images, 8 providing an evaluation of tumor heterogeneity and revealing details of the tumor microenvironment usually unrecognizable or indistinguishable to the human eye. 9,10 Computed tomography texture analysis (CTTA) can improve experience-dependent and subjective diagnosis of the radiologists by providing a large amount of objective information of the lesion, thereby assisting them in making a more accurate diagnosis. Prior studies suggested that CTTA has great value in distinguishing between fpAML and RCC. 11,12 However, few studies focused on the value of CTTA in preoperative differential diagnosis between fpAML and chRCC. Computed tomography texture analysis can be performed on a single section of the largest cross-sectional diameter of the tumor (2-dimensional, 2D) or on multiple sections or whole tumor volumes (3-dimensional, 3D). 13,14 It is intuitive that 3D texture features may offer more comprehensive information compared with 2D texture features extracted from CT images. However, it is not clear if 3D CTTA, which is time-consuming and labor-intensive, is definitely more valuable than 2D CTTA. 15
The aim of this study was to investigate the value of TA on contrast-enhanced CT images in distinguishing between fpAML and chRCC and to verify whether 3D CTTA has more value than 2D CTTA.
Materials and Methods
Patients
Our institutional review board approved this single-institution retrospective study and waived the demand for informed consent. From June 2009 to January 2018, a pathologic diagnosis of AML or chRCC was selected by searching the pathology database of our hospital. A total of 24 chRCCs (13 males and 11 females; mean age, 52.88 ± 10.86 years; age range, 24-72 years) and 32 fpAMLs (8 males and 24 females; mean age, 50.38 ± 8.66 years; age range, 34-67 years) were enrolled according to the following inclusion criteria: (1) patients with a pathologically confirmed single renal mass, either fpAML or chRCC, after radical or partial nephrectomy; (2) patients had undergone a 3-phase CT scan 2 weeks before receiving any treatment and/or surgery; (3) CT images were of diagnostic quality; (4) there was no visible fat inside the renal masses on CT scan in the fpAML group. The exclusion criteria were: (1) patients who had undergone radiotherapy and/or chemotherapy before surgery; (2) patients suffering from other kidney diseases that might affect image analysis. The diagram for inclusion of patients is shown in Figure 1. Clinical information including age and gender was obtained by searching medical records. The tumor size was defined as the maximum tumor diameter on axial CT images.

Diagram for inclusion of patients into the study. AML indicates angiomyolipoma; chRCC, chromophobe renal cell carcinoma; fpAML, fat-poor AML.
Image Acquisition
All patients underwent contrast-enhanced CT scanning with a 64-slice CT scanner (Siemens Sensation Cardiac 64; Siemens, Forchheim, Germany). The scanning parameters were as follows: tube voltage, 120 kV; tube current, 220-250 mA; slice thickness, 5 mm. Patients held their breath for scanning in the supine position. The scanning area extended from the diaphragm to the lower edge of the kidney. After acquisition of an unenhanced scan, 90 mL nonionic contrast material of iodine (iopromide, Ultravist 370; Bayer Schering Pharma, Berlin, Germany) was administered into an antecubital vein at a rate of 3.0 mL/s using a power injector. Corticomedullary phase, nephrographic phase (NP), and EP CT images were acquired 30, 90, and 300 seconds after contrast injection, respectively.
Computed Tomography Texture Feature Extraction
All the images were exported from the workstation and imported into a radiomics cloud platform V2.1.2 (Huiying Medical Technology Co, Ltd, Beijing, China). The images from 3 phases (CMP, NP, and EP) were stored in DICOM format, and the window width and level were set at 250 and 75 HU, respectively. Two-dimensional regions of interest (ROIs) were delineated to cover the largest potential tumor area, avoiding adjacent large blood vessels and maintaining a consistent maximal cross-sectional area. Three-dimensional ROIs were obtained by integrating 2D ROIs on every section of the entire tumor. An example of 2D and 3D ROIs of fpAML and chRCC is presented in Figure 2. A total of 177 2D texture features and 183 3D texture features were extracted from the ROIs for quantification of internal tumor heterogeneity. There are 3 kinds of texture features: (1) gray level co-occurrence matrix: including contrast, autocorrelation, entropy, and so on 16 ; (2) gray level run length matrix: including long-run emphasis, gray-level nonuniformity, size-zone nonuniformity, and so on 17 ; (3) gray-level size zone matrix: including large area emphasis, gray-level nonuniformity, gray-level variance, and so on. 18,19 The detailed information is provided in the Supplementary Material.

Regions of interest of the tumors. Two-dimensional (A) and 3-dimensional (B) ROIs of fpAML. Two-dimensional (C) and 3-dimensional (D) ROIs of chRCC. chRCC indicates chromophobe renal cell carcinoma; fpAML, fat-poor angiomyolipoma; ROIs, regions of interest.
The inter- and intraclass correlation coefficient (ICC) was computed for evaluation of the inter-reader reliability and intra-reader reproducibility of feature extraction. Twenty cases of CT images (10 fpAMLs and 10 chRCCs) were randomly selected. Two radiologists (reader 1, G.Y and reader 2, Z.W with 8 and 20 years’ experience in abdominal imaging diagnosis, respectively) drew the ROIs of the 20 cases. Reader 1 repeated the segmentations 2 weeks later. An ICC greater than 0.75 suggested good agreement of feature extraction. The ROI segmentation for the remaining 36 cases was completed by reader 1.
Feature Selection and Construction of 2D and 3D CTTA Models
The optimal texture features were selected through the following 3 steps. First, we retained texture features with both inter- and intra-rater ICCs > 0.75 for further analysis to avoid subjective differences in segmenting the ROIs. Second, the significantly different features between fpAML and chRCC were chosen by using 1-way analysis of variance (ANOVA). Finally, to obtain the optimal texture feature, the least absolute shrinkage and selection operator algorithm (LASSO) was performed. The LASSO method is a widely used approach to select the most valuable features from high-dimensional data. The model coefficients were compressed by selecting the optimal harmonic parameter λ in the model by 10-fold cross-validation, and the coefficients of the unrelated variables were reduced to zero while retaining the variables of non-zero coefficients. 20 Finally, the 2D and 3D CTTA models were developed by combining the selected features. A texture score (Tex-score) was calculated for each patient through a linear combination of selected features weighted by their respective LASSO coefficients.
Diagnostic Performance Evaluation of 2D and 3D CTTA Models
The diagnostic performance of the 2D and 3D CTTA models in differentiating between fpAML and chRCC was evaluated by a receiver operating characteristic (ROC) curve. The calibration curve and the Hosmer-Lemeshow test were used to evaluate the goodness-of-fit of the models. The associated area under the ROC curve (AUC), sensitivity, specificity, and accuracy were calculated. The Delong test was used to assess the differences in the AUC between the models. By calculating the net benefits for a range of threshold probabilities, decision curve analysis (DCA) was performed to estimate the clinical utility of the models.
Statistics
All statistical analyses were conducted with a commercial software (SPSS version 24; IBM, Armonk, New York) and an open-source R software (version R × 64 3.5.1, https://www.r-project.org). Univariate analysis was used to compare the differences of age, gender, and Tex-score between fpAML and chRCC patients by using the χ2 test or Fisher exact test for categoric variables and Mann-Whitney U test for continuous variables, where appropriate. One-way ANOVA was used to select the significantly different texture features between the 2D and 3D groups. The LASSO regression analysis was performed by using the “glmnet” package. The ROC analysis was performed using the “ROCR” package. Calibration plots were performed using the “rms” package, and the Hosmer-Lemeshow test was performed using the “generalhoslem” package. Differences in the AUC values between the 2 models were compared using the Delong test. The DCA was performed using the “dca.R.” package. A 2-sided P < .05 was considered statistically significant.
Results
Patients
There was no significant difference in age (U = 323.000, P = .267) and gender (P = .059) between fpAML and chRCC patients. There was significant difference in tumor size (U = 196.500, P = .002) between the 2 groups.
Selection of Texture Features
Of the 177 and 183 texture features extracted from 2D and 3D ROIs, respectively, 154 2D features and 162 3D features were shown having a good inter- and intra-observer agreement, with ICCs ranging from 0.768 to 0.998. A total of 26 2D features and 33 3D features showing significant differences between fpAML and chRCC (P = .000-.050) were entered into the LASSO logistic regression model to select the most valuable features (Figure 3). Finally, 5 2D features and 8 3D features were selected. The contribution of the selected features with their absolute value of coefficients is shown in the histograms in Figure 4.

Texture features selection using the LASSO regularization. Identification of the tuning parameter (λ) selection used 10-fold cross-validation and the minimum criteria. Dotted vertical lines were depicted at the optimal values by using the minimum criteria and the 1 standard error of the minimum criteria. The optimal λ values of 0.1118 and 0.1219, with log (λ) =−2.1914 and −2.1043 were chosen in the 2D (A) and 3D (C) models, respectively. The LASSO coefficient profiles of 26 2D and 33 3D texture features. Vertical line was drawn at the value selected using 10-fold cross-validation in log (λ), where optimal λ resulted in 5 and 8 selected features in the 2D (B) and 3D (D) models, respectively. 2D indicates 2-dimensional; 3-D, 3-dimensional; LASSO, least absolute shrinkage and selection operator.

Two-dimensional (A) and 3D (B) histograms show the contribution of the selected texture features with their absolute value of coefficients to the 2D and 3D radiomics signatures. 2D indicates 2-dimensional; 3-D, 3-dimensional.
Construction of 2D and 3D CTTA Models
The 2D and 3D CTTA models were created by a linear combination of selected texture features and respective LASSO coefficients. The Tex-scores of the 2D and 3D CTTA models were calculated for each patient using the following formulas: Tex-score (2D) = −0.3088 × GrayLevelNonUniformity.glszm.CMP − 0.1307 × RunLengthNonUniformity.glrlm. CMP − 0.0368 × SizeZoneNonUniformity.glszm. CMP − 0.1501 × GrayLevelNonUniformity.glrlm. EP − 0.0890 × ZoneEntropy.glszm.EP. Tex-score (3D) = −0.1553 × GrayLevelNonUniformity.glszm. CMP − 0.1043 × RunLengthNonUniformity.glrlm. CMP − 0.0468 × ZoneEntropy.glszm. CMP − 0.0948 × GrayLevelNonUniformity.glrlm. EP − 0.3029 × ZoneEntropy.glszm. EP − 0.2043 × GrayLevelNonUniformity.glrlm.NP − 0.0068 × GrayLevelNonUniformity.glszm.NP − 0.2210 × ZoneEntropy.glszm.NP.
The Tex-score showed statistically significant differences between fpAML and chRCC (median Rad-score of fpAML: 0.300, range: −0.558 to 0.760; median Rad-score of chRCC: −0.239, range: −1.844 to 0.537; P < .001 by the 2D CTTA model and median Rad-score of fpAML: 0.409, range: −0.349 to 1.237); median Rad-score of chRCC: −0.584, range: −1.225 to 0.489; P < .001 by the 3D CTTA model). The Tex-scores for each patient in the 2D and 3D models are shown in Figure 5.

The texture scores (Tex-scores) for each patient in 2D (A) and 3D (B) CTTA models, respectively. Red bars represent the scores for fpAML patients, while green bars represent the scores for chRCC patients. chRCC indicates chromophobe renal cell carcinoma; CTTA, computed tomography texture analysis; 2D, 2-dimensional; 3-D, 3-dimensional; fpAML, fat-poor angiomyolipoma.
Diagnostic Performance of 2D and 3D CTTA Models
The diagnostic performance for 2D and 3D CTTA models for the diagnosis of fpAML is presented in Table 1. The ROC curves and calibration curves of the 2 models for the diagnosis of fpAML are shown in Figure 6. The Hosmer-Lemeshow test showed good calibration for the 2D model (P = .681) and the 3D model (P = .484). There was no significant difference in AUC values between the 2 models (P = .093). The DCA (Figure 7) showed that the 3D model had a higher overall net benefit in differential diagnosis than the 2D model across the majority of the range of reasonable threshold probabilities.
Performance of the 2D and 3D CTTA Models for Diagnosis of fpAML.
Abbreviations: AUC, area under the curve; CI, confidence interval; CTTA, computed tomography texture analysis; 2D, 2-dimensional; 3-D, 3-dimensional; fpAML, fat-poor angiomyolipoma.
a Numbers in parentheses were used to calculate percentages.

The ROC curves (A, B) and the calibration curves (C, D) of the 2D and 3D CTTA models for diagnosis of fpAML, respectively. The ROC curves show 2D (A) and 3D (B) texture models had favorable predictive value for differentiating fpAML from chRCC with the AUC of 0.811 and 0.915, respectively. Calibration curves indicate the goodness-of-fit of the model. The 45° gray line represents the ideal prediction, and the blue line represents the predictive performance of the models. The closer the blue line approaches the ideal prediction line, the better the predictive efficacy of the nomogram is. The calibration curves of the 2D (C) and 3D (D) CTTA models show the blue lines have closer fit to the gray lines, indicating good predictive accuracy of the 2 models. AUC indicates area under the curve; chRCC, chromophobe renal cell carcinoma; CTTA, computed tomography texture analysis; 2D, 2-dimensional; 3-D, 3-dimensional; fpAML, fat-poor angiomyolipoma; ROC, receiver operating characteristic.

Decision curve analysis for the 2D and 3D CTTA models. The y-axis and x-axis indicate the net benefit and the threshold probability, respectively. The gray line represents the assumption of all fpAML patients, while the horizontal black line represents the assumption of all chRCC patients. The 3D model (red line) provides a higher net benefit than the 2D model (blue line). chRCC indicates chromophobe renal cell carcinoma; CTTA, computed tomography texture analysis; 2D, 2-dimensional; 3-D, 3-dimensional; fpAML, fat-poor angiomyolipoma.
Discussion
Distinction between fpAML and chRCC is especially crucial due to their rather different treatments and prognoses. Fat-poor angiomyolipoma shares overlapping imaging features with chRCC, leading the differential diagnosis a great challenge by using traditional imaging techniques. The present study showed that the enhanced CT-based 2D and 3D texture models had favorable predictive value for differentiating fpAML from chRCC with the AUC of 0.811 and 0.915, respectively. Although no significant difference in AUC was found between the 2 models, the 3D CTTA model outperformed the 2D model in terms of clinical usefulness.
Among the widespread use of up-to-date imaging modalities such as CT, ultrasonography, and magnetic resonance imaging (MRI), CT is the most commonly used imaging method for the diagnosis of renal masses. However, imaging findings of fpAML and chRCC on CT are similar. Both of them appear as a renal mass with a relatively homogeneous density on unenhanced CT with less bleeding and necrosis than clear cell RCC. In addition, there is no specific difference in the enhancement pattern between the 2 entities on contrast-enhanced CT images. Various indexes have been proposed to differentiate fpAML from RCC with conventional CT imaging features. Takahashi et al 21 developed several CT models that combined demographics, unenhanced CT, and enhanced CT features for differentiating fpAML from RCC. Demographic data, size, shape, CT attenuation, and heterogeneity of 24 fpAMLs and 148 RCCs on unenhanced CT and contrast-enhanced CT were analyzed. The model combining various CT and demographic findings achieved high AUC (0.939), high specificity (95%), but low sensitivity (50%) for differential diagnosis. Lim et al 22 recently stated that the diagnosis of fpAML could not be readily established only with CT features, and combining CT and MRI features, including high attenuation without calcification at unenhanced CT, low T2W and/or ADC signal, and avid early enhancement with washout kinetics, was highly accurate for the diagnosis of fpAML. Coy et al 23 found a novel, quantitative CAD algorithm that enabled robust peak HU lesion detection and discrimination of ccRCC from chRCC, papillary RCC, oncocytoma, and fpAML, with AUCs of 0.850, 0.959, 0.792, and 0.825, respectively. Sung et al 6 showed that CT images with non-round shape without capsule and prolonged enhancements may be used to differentiate fpAML from RCC. However, most previous studies were based on qualitative analysis of the imaging features; quantitative analysis might serve as a beneficial method to improve the diagnostic accuracy in differentiating fpAML from RCC.
Recently, TA has been rapidly developed that can reflect the histological and biological characteristics of tumors which are difficult to recognize by the human eye. Previous studies indicate that CTTA shows good prospects in differentiating benign from biologically aggressive or malignant lesions. 24,25 In the study by Bayanati et al, 26 quantitative CTTA showed potential in accurately differentiating malignant from benign mediastinal nodes in lung cancer. Xu et al 27 found that TA could significantly improve the differential diagnosis of bone and soft-tissue lesions on 18F-fluorodeoxyglucose (FDG) positron emission tomography (PET)/CT images. Xu et al 28 reported that CTTA could potentially help differentiate GISTs without the KIT exon 11 mutation from those with the KIT 11 mutation.
Previous studies have shown that TA was helpful in distinguishing fpAML from ccRCC. 29 Hodgdon et al 12 reported that CTTA could be used to differentiate fpAML from RCC on unenhanced images using three 2D ROIs for texture feature extraction. Sixteen fpAMLs and 84 RCCs were analyzed, and the texture model resulted in an AUC of 0.89. Lubner et al 24 investigated whether CT texture features of primary RCCs correlate with pathologic features and oncologic outcomes. The CTTA 2D features of 157 large (>7 cm) RCCs were analyzed. They found CT texture features (in particular, entropy, the mean of positive pixels, and the standard deviation of the pixel distribution histogram) were associated with tumor histologic findings, nuclear grade, and outcome measures. Yan et al 30 investigated the 2D texture features and subjective CT findings of 18 fpAMLs, 18 ccRCCs, and 14 papillary RCCs and found that CTTA might be a reliable quantitative method for the discrimination of fpAML, ccRCC, and papillary renal cell carcinoma (pRCC) with an error rate of less than 9.3% by a nonlinear discriminant analysis. However, few studies focused on CTTA discrimination between fpAML and chRCC. Our results revealed that CTTA could be used as an effective tool for preoperative distinction of fpAML and chRCC with AUCs higher than 0.8.
Being different from previous studies, we focused on 2D and 3D CTTA comparison in distinction of fpAML and chRCC. The delineation of tumors with ROIs took the most time in all procedures of CTTA. Since the attenuation difference between the renal tumor and the surrounding tissue was usually small, the boundary of the tumor contour on CT was blurred. It was difficult to automate segmentation for ROI delineation of renal tumors. Manual segmentation for 3D ROI was far more time-consuming than 2D ROI, especially for a large tumor without a well-defined boundary. Intuitively speaking, 3D TA may provide more abundant and comprehensive image information than 2D TA and may be more helpful for clinical diagnosis. However, whether 3D CTTA is superior to 2D CTTA in the practical application of identification for renal tumors has not been verified. It is not clear that the extra time and labor associated with volumetric assessment are necessary. 24 Ng et al 31 analyzed CTTA features for the largest tumor cross-sectional area and the whole tumor in 55 patients with primary colorectal cancer and evaluated its effect on clinical outcome prediction. They found the whole tumor analysis appeared more representative of tumor heterogeneity. Our study showed a different result: we found that no significant difference in the diagnostic performance existed between 2D and 3D CTTA models, indicating the similar lesion classification efficacy with 2D and 3D ROI segmentation for fpAML and chRCC.
Our study has some limitations. First, as a case–control study, the diagnostic accuracy is usually overestimated; therefore, an independent external validation is needed. Second, the patients were derived from one single institute, and the patient number was relatively small, and thus, a multicenter study with a larger sample is required.
In conclusion, we developed 2D and 3D CTTA models with favorable predictive efficacy in differentiating fpAML from chRCC. As an objective and noninvasive modality, quantitative CTTA model may serve as an effective tool to supplement the traditional imaging techniques for clinical decision-making process.
Supplemental Material
Supplementary_material_(1) - Contrast-Enhanced CT Texture Analysis for Distinguishing Fat-Poor Renal Angiomyolipoma From Chromophobe Renal Cell Carcinoma
Supplementary_material_(1) for Contrast-Enhanced CT Texture Analysis for Distinguishing Fat-Poor Renal Angiomyolipoma From Chromophobe Renal Cell Carcinoma by Guangjie Yang, Aidi Gong, Pei Nie, Lei Yan, Wenjie Miao, Yujun Zhao, Jie Wu, Jingjing Cui, Yan Jia and Zhenguang Wang in Molecular Imaging
Footnotes
Authors’ Note
Aidi Gong and Pei Nie contributed to literature search. Guangjie Yang and Zhenguang Wang contributed to study design. Guangjie Yang, Aidi Gong, Pei Nie, Lei Yan, Wenjie Miao, Yujun Zhao, and Jie Wu contributed to data collection. Guangjie Yang, Aidi Gong, Pei Nie, Jie Wu, Jingjing Cui, and Yan Jia contributed to data analysis. Guangjie Yang, Aidi Gong, and Pei Nie contributed to manuscript writing. Zhenguang Wang contributed to manuscript review.
Author Contribution
Guangjie Yang and Aidi Gong contributed equally to this work
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship and publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by the National Natural Science Foundation of China (81601527 and 81701688); the Natural Science Foundation of Shandong Province (ZR2017MH036 and ZR2017BH096); the Key Research and Development Project of Shandong Province (2018GSF118078); and the Postdoctoral Science Foundation of China (2018M642617). None of these funding sources had any role in study design, the collection, analysis and interpretation of data, the writing of the report, or the decision to submit the paper for publication.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
