Abstract
The combination of immune checkpoint inhibitors (ICIs) and angiogenesis inhibitors (AGIs) is widely used in cancer treatment; however, drug–drug reactions (DDIs) remain unknown. We aimed to identify interaction signals for the concomitant use of ICIs and AGIs. Data were obtained from the US FDA Adverse Event Reporting System (FAERS) from January 1, 2015, to December 31, 2023. Disproportionality analysis was used for data mining by calculating the reporting odds ratio (ROR) and 95% confidence interval (95% CI). Adjusted RORs were analysed using logistic regression analysis, considering age, sex and reporting year. Further confirmation was assessed via additive and multiplicative models. We identified 75,936 reports on ICIs combined with AGIs. Significant interaction signals were observed for hepatobiliary disorders (RORcrude: 5.25, 95% CI: 5.07–5.44, RORadj: 5.01, 95% CI: 4.82–5.22, additive models: 0.2323), investigations (RORcrude: 1.66, 95% CI: 1.62–1.70, RORadj: 1.63, 95% CI: 1.58–1.67, additive models: 0.2187, multiplicative models: 1.1265), renal and urinary disorders (RORcrude: 1.87, 95% CI: 1.80–1.95, RORadj: 1.72, 95% CI: 1.64–1.79, additive models: 0.3239, multiplicative models: 1.1799) and vascular disorders (RORcrude: 1.94, 95% CI: 1.87–2.02, RORadj: 1.87, 95% CI: 1.80–1.95, additive models: 0.5823, multiplicative models: 1.5676). Subset data analysis showed positive interaction signals for PDL-1/CTLA-4 inhibitors + AGI in hepatobiliary disorders, PD-1 inhibitors + AGI in investigations, or PD-1/PDL-1 inhibitors + AGI in renal and urinary/ vascular disorders. Based on FAERS data, four systemic disorders were identified as having DDIs related to the combined use of ICIs and AGIs. Pre-clinical trials are required to explore the mechanisms underlying these interactions.
Keywords
What’s new
This is the first study to identify interaction signals for the concomitant use of ICIs and AGIs through a pharmacovigilance analysis. Four systemic disorders, including drug interactions, hepatobiliary disorders, renal and urinary disorders and vascular disorders, were found to be associated with the concomitant use of ICIs and AGIs.
Introduction
Over the past decade, immune checkpoint inhibitors (ICIs), including programmed death-1 (PD-1), programmed death ligand-1 (PDL-1) and cytotoxic T-lymphocyte antigen-4 (CTLA-4) inhibitors, have revolutionised the strategic therapies for a wide variety of tumours. 1 The effectiveness of ICIs in treating various cancers has been demonstrated in several clinical trials, and immunotherapy has been positively correlated with the rate of a complete response to treatment.2–4 However, this approach does not meet expectations and encounters issues with response duration, resistance, treatment eligibility and safety because of tumour heterogeneity and the complexity of the tumour microenvironment.5,6 Consequently, immunotherapy and other therapies, such as chemotherapy and radiotherapy, are currently under investigation. 7 The combination of ICIs and angiogenesis inhibitors (AGIs) could have potential antitumor effects, given the growing understanding of the importance of the tumor microenvironment.8–10
Although the combination of ICIs and AGIs is highly effective, it has significantly more negative effects. 11 The risk of cardiovascular adverse events (AE) is higher with ICIs combined with AGIs than with ICIs alone. 12 Furthermore, combination therapy is an independent risk factor for adverse drug reactions (ADRs) associated with interstitial lung disease, hypertension and gastrointestinal bleeding. 13 The occurrence of high-grade AEs was significantly different between patients with renal cell carcinoma (RCC) treated with cabozantinib combined with nivolumab and ipilimumab and those treated with nivolumab and ipilimumab alone. 14 While combination therapies often seek to exploit complementary mechanisms of action to achieve superior antitumor efficacy, the concomitant use of multiple drugs may lead to drug–drug interactions (DDIs), resulting in drug toxicities, suboptimal therapy and treatment failure, all of which affect the full benefit of treatment. 15
However, there is currently no research on the DDIs through the combined use of ICIs and AGIs. To better understand the overall aspects of the concomitant use of ICIs and AGIs, it is crucial to obtain sufficient information on their DDIs from real-world data. We conducted descriptive and disproportionality analyses of DDIs to determine the possible harmful effects of combinations of ICIs and AGIs by examining the United States Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS) database. Our use of big data allowed us to summarise their interaction characteristics, confirm the intensity and occurrence patterns of their AEs, and provide a reference for clinical decision-making.
Methods
Data source
The FAERS database was used for this retrospective pharmacovigilance study. 16 The FAERS database comprises AE reports from health professionals, patients and manufacturers worldwide. The public has access to these data. Data were extracted from the first quarter (Q1) of 2015 to the fourth quarter (Q4) of 2023.
Procedures
Data were obtained from the REAC files according to the Medical Dictionary for Regulatory Activities (MedDRA, version 23.0) at the preferred term (PT) level based on all 27 Standardised MedDRA Query System Organ Class (SOC). 17 The 27 SOCs are listed in Table S1 in the supplemental material. The DRUG file contained generic name records of all drugs, with the role_code selected being either ‘PS’ (primary suspect) or ‘SS’ (secondary suspect). Combined immunotherapy drugs included anti-PD-1 drugs (pembrolizumab, nivolumab, cemiplimab, dostarlimab, prolongolimab, tislelizumab, toripalimab, sintilimab, camrelizumab, penpulimab, zimberelimab and serplulimab), anti-PDL-1 drugs (atezolizumab, durvalumab, avelumab, envafolimab and sugemalimab) and anti-CTLA4 drugs (ipilimumab and tremelimumab). AGIs included bevacizumab, ramucirumab, aflibercept, sorafenib, sunitinib, pazopanib, vandetanib, cabozantinib, regorafenib, axitinib, nintedanib/intedanib, lenvatinib, cediranib, tivozanib, erdafitinib, vatalanib, apatinib (China), anlotinib (China), fruquintinib (China) and endostatin (China). To remove duplicate reports, the last FDA_DT was selected when the CASEID was identical, and the PRIMARY_ID with the greater value was selected when the CASEID and FDA_DT were identical, as directed in the FAERS user instructions. 16 The clinical features of patients taking ICIs combined with AGIs were synthesised using descriptive analyses of data collected from the FAERS database.
Data analysis
In spontaneous reporting databases, the detection of possible interactions is based on the demonstration that a suspected AE is reported more frequently with a combination of two drugs than when they are used alone.18–20 Therefore, the reports were divided into three index groups: (i) reports of patients exposed to ICIs alone, but not AGIs; (ii) reports of patients exposed to AGIs alone, not to ICIs and (iii) reports of patients exposed to both ICIs and AGIs at the time of the event. The reference group comprised patients who were not exposed to ICIs or AGIs.
Disproportionality was calculated using the reporting odds ratio (ROR) with a relevant 95% confidence interval (95% CI), which was defined as statistically significant when the lower limit of the 95% CI exceeded 1, with at least three cases of interest reported. 21 To identify potential confounders in the database, crude RORs were recalculated using unconditional logistic regression analysis based on age, sex and reporting year. 22 The logistic model was as follows: Log (risk of the event) = β0 + β1 ICIs + β2 AGI + β3 ICIs × AGI + β4 age + β5 gender + β6 reporting year. An interaction can be claimed when the combination is linked to an enhanced ROR compared with other index groups. 23
To assess the consistency and reliability of DDIs, we used multiplicative and additive models. 22 The analysis provided a measurement of the threshold for detecting DDI signals. In the multiplicative model, the risk associated with a drug is multiplied by the background risk, whereas in the additive model, the risk associated with the drug is added to the background risk. The risk(drug 1 × drug 2)/((risk(drug 1) × risk(drug 2)) > 1 and risk(drug 1 × drug 2) – (risk(drug 1) + risk(drug 2)) > 0, respectively, indicate that the multiplicative and additive models generate a drug interaction signal. If the value (interaction term) goes beyond 0 or 1 in the additive or multiplicative models, it indicates a positive interaction. 24 Data management and analyses were performed using SPSS (version 22.0; IBM Corp., Armonk, NY, USA).
Results
Descriptive analysis
From January 2014 to December 2023, 51,336,49,755 reports were extracted from the FAERS database and 39,523,594 reports were included in the final analysis, of which 75,936 reports of ICIs combined with AGIs were identified. The clinical features of the patients are summarised in Table 1. The median patient age was 55 years (IQR: 64–72 years). The proportion of male patients (48.82%) was slightly higher than that of female patients (42.27%). The number of reports submitted by healthcare professionals was high (49.25%). Asia reported the highest rate of ADRs (31.74%), followed by North America (29.79%), Europe (20.01%), South America (1.01%), Oceania (1.00%) and Africa (0.04%; Figure 1a). The number of reports steadily increased over time, from 0.05% in 2014 to 31.20% in 2023 (Figure 1b). A Sankey diagram was used to display the distribution of the clinical characteristics (Figure 1c).
Characteristics of patients treated with immune checkpoint inhibitors combined with angiogenesis inhibitors in the FAERS database (January 1, 2014 to December 31, 2023).

(a) Geographical regions reporting immune checkpoint inhibitors (ICIs) combined with angiogenesis inhibitors (AGIs). (b) Annual reported adverse reactions of ICIs combined with AGI from 2014 to 2023. (c) The distribution of clinical characteristics of ICIs combined with AGIs using Sankey diagram.
Disproportionality analysis
The signal detection results of the DDIs with the concomitant use of ICIs and AGIs for all 27 SOCs are presented in Table 2. The crude/adjusted RORs and 95% CIs for all comparisons are presented using a disproportionality method. We estimated the DDIs according to additive/multiplicative models. Positive signals were detected in four SOCs: investigations, hepatobiliary, renal and urinary and vascular disorders (Table 3). For hepatobiliary disorders, the crude/adjusted ROR for the use of AGI alone was 1.91 (95% CI: 1.87–1.94)/1.73 (95% CI: 1.69–1.76), and the crude/adjusted ROR for ICIs alone was 4.11 (95% CI: 4.04–4.18)/3.95 (95% CI: 3.87–4.03). An increase in the signal for the concomitant use of ICIs and AGI emerged (RORcrude: 5.25, 95% CI: 5.07–5.44, RORadj: 5.01, 95% CI: 4.82–5.22). The positive interaction signal was supported by additive models (0.2323). For investigations, the crude/adjusted ROR for the use of AGI alone was 1.37 (95% CI: 1.36–1.38)/1.38 (95% CI: 1.36–1.39), and the crude/adjusted ROR for ICIs alone was 1.07 (95% CI: 1.06–1.09)/1.05 (95% CI: 1.04–1.07). An increase in the signal for the concomitant use of ICIs and AGI emerged (RORcrude: 1.66, 95% CI: 1.62–1.70, RORadj: 1.63, 95% CI: 1.58–1.67). The positive interaction signal was supported by additive (0.2187) and multiplicative (1.1265) models. For renal and urinary disorders, the crude/adjusted ROR for the use of AGI alone was 1.07 (95% CI: 1.05–1.08)/0.96 (95% CI: 0.94–0.98), and the crude/adjusted ROR for ICIs alone was 1.48 (95% CI: 1.45–1.51)/0.94 (95% CI: 1.35–1.41). An increase in the signal for the concomitant use of ICIs and AGI emerged (RORcrude: 1.87, 95% CI: 1.80–1.95, RORadj: 1.72, 95% CI: 1.64–1.79). The positive interaction signal was supported by additive (0.3239) and multiplicative (1.1799) models. For vascular disorders, the crude/adjusted ROR for the use of AGI alone was 1.57 (95% CI: 1.55–1.59)/1.36 (95% CI: 1.34–1.38), and the crude/adjusted ROR for ICIs alone was 0.79 (95% CI: 0.77–0.81)/0.74 (95% CI: 0.72–0.76). An increase in the signal for the concomitant use of ICIs and AGI emerged (RORcrude: 1.94, 95% CI: 1.87–2.02, RORadj: 1.87, 95% CI: 1.80–1.95). The positive interaction signal was supported by additive (0.5823) and multiplicative (1.5676) models.
Reporting odds ratios (RORs) and drug interaction approaches for immune checkpoint inhibitors (ICIs) and angiogenesis inhibitors (AGIs).
ICIs: immune checkpoint inhibitors; AGIs: angiogenesis inhibitors.
The boldfaced values represent meaningful data.
Summary of methods used to analyse drug-drug interactions for all SOCs.
Subset data analyses
For the four positive SOCs, we conducted a subset analysis of AGIs combined with PD-1, PDL-1 and CTLA-4 (Tables 4 and 5). For hepatobiliary disorders, both PDL-1 and CTLA-4 combined with AGIs revealed positive DDI signals, which were verified using additive or multiplicative models. For investigations, PD-1 combined with AGIs revealed positive DDI signal, which were verified using additive or multiplicative models. For renal and urinary disorders, PD-1 and PDL-1 in combination with AGIs showed positive DDI signals, which were verified using additive or multiplicative models. In vascular disorders, the combination of PD-1 and PDL-1 with AGIs revealed positive DDI signals, which were verified using additive or multiplicative models.
Disproportionality analyses and drug interaction approaches for the various drug combinations.
PD-1: programmed death-1 inhibitors; PDL-1: programmed death ligand-1 inhibitors; CTLA-4: cytotoxic T-lymphocyte antigen-4 inhibitors.
The boldfaced values represent meaningful data.
Summary of methods used to analyse drug-drug interactions in four SOCs.
PD-1: programmed death-1 inhibitors; PDL-1: programmed death ligand-1 inhibitors; CTLA-4: cytotoxic T-lymphocyte antigen-4 inhibitors.
Discussion
To the best of our knowledge, this is the first pharmacovigilance study to assess DDIs between ICIs and AGIs. Our study provides the most comprehensive information on the clinical characteristics and DDIs of ICIs combined with AGIs in a real-world setting. Several main findings emerged: (1) DDIs of ICIs combined with AGIs occurred in four systems: hepatobiliary disorders, investigations, renal and urinary disorders and vascular disorders. (2) PDL-1/CTLA-4 combined with AGIs is associated with hepatobiliary disorders, whereas PD-1 combined with AGIs is not. (3) PD-1/CTLA-4 combined with AGIs was associated with AEs, whereas PDL-1 combined with AGIs was not. (4) PD-1, PDL-1, and CTLA-4 combined with AGIs are associated with renal and urinary disorders. (5) PD-1/PDL-1 combined with AGIs is associated with vascular disorders, whereas CTLA-4 combined with AGIs is not.
The host immune response against tumours is activated by ICIs, which block negative regulatory immune signals and trigger a T-cell-mediated response.10,25 The formation of new blood vessels from the pre-existing vasculature, known as angiogenesis, is crucial for tumour growth. Angiogenesis is promoted by tumours secreting pro-angiogenic factors that aid in the provision of nutrients for their expansion and metastatic spread. 26 Vascular endothelial growth factor (VEGF) is crucial for angiogenesis. 27 Tumour cells hinder vessel maturation, leading to suboptimal blood flow, tumour hypoxia and an immunosuppressive tumour microenvironment.28,29 The inhibition of VEGF signalling results in the normalisation of tumour vasculature, which has significant positive effects on the infiltration of immune cells into tumours.30,31 The use of anti-angiogenic drugs, such as sorafenib, sunitinib, or bevacizumab, has been shown to increase PDL-1 expression in models of RCC and hepatocellular carcinoma.32,33 Tumour immune escape and treatment resistance can be attributed to the interactions between immunity and angiogenesis. 10 The synergistic antitumor effects of a combinatory VEGFI/ICI treatment approach have been verified in the clinical setting, displayed by promising results in various tumor types, including RCC and non-small cell lung cancer, and will become an important treatment strategy for various malignancies.34–36
Drug interactions can be classified into pharmacodynamic and pharmacokinetic interactions. Pharmacodynamic interactions can occur when different drugs have mechanisms of action that affect the same physiological processes, whereas pharmacokinetic interactions occur when a drug affects its absorption, distribution, metabolism and excretion. 37 DDIs can result in either increased or decreased therapeutic or AEs, or a unique response that is not caused by either agent alone. 15 This study examined drug interactions that lead to increased toxicity from the combination of ICIs and AGIs.
There are several methods for detecting interaction signals. The logistic regression model is a statistical method used to detect DDI signals from a spontaneous reporting system. The ROR, like the odds ratio, is a statistical model that can be adjusted for age and gender using the logistic regression model. The additive/multiplicative model is one of the methods to detect the signals of potential DDIs. According to the additive model, drug-related risks increase additively, while the multiplicative model assumes they increase synergistically. 22 Because there is no gold standard statistical method established for capturing DDI signals from spontaneous reporting data, we use adjusted ROR and additive/multiplicative model to detect the signals.
Abnormal liver function is one of the most common AEs associated with ICIs combined with bevacizumab in the FAERS database. 13 Liver toxicity is prevalent in ICI-treated patients and typically manifests as an asymptomatic increase in hepatic enzymes with or without hyperbilirubinemia, which can occur in 2–25% of patients.38,39 The proposed explanation involves increased autoimmunity of hepatocytes caused by T-cell activation from ICIs. The liver is among the most commonly involved. 40 The immune response is the cause of immunomediated hepatotoxicity caused by ICIs, whereas conventional drug-induced liver injury occurs from either direct or idiosyncratic effects. 41 The onset of immuno-related hepatitis typically occurs 8–12 weeks after the initiation of ICI therapy. 42 The role of VEGF in maintaining the structural and functional integrity is well-documented. 43 VEGF and VEGF receptor agonists exhibit more consistent growth, regenerative and cytoprotective effects than other agents used in the treatment of chemically induced hepatitis or partial hepatectomy.44,45 Under these conditions, transaminase elevations are due to a direct effect of the agent on hepatocytes, possibly partly due to the inhibition of the vascular endothelial growth factor receptor, or due to an effect on endothelial cells of small hepatic blood vessels, which is not fully clear. 46
Hypertension was the most frequent side effect experienced by patients with metastatic RCC who were treated with avelumab or pembrolizumab with axitinib as a first-line treatment.35,47 The risk of developing all-grade hypertension in RCC is higher when ICIs are used in combination than when sunitinib monotherapy is used. 48 Patients with advanced solid tumours were found to have a 67% incidence of hypertension when they received a combination of bevacizumab and sorafenib. 49 In patients with advanced lung cancer, there was an 8.41% rate of vascular disorders in those with combined ICIs and AGIs and an 18.37% rate in those with ICIs combined with AGI grade 3 or higher. 50 VEGF plays a crucial role in regulating vascular homeostasis and cardiac development and function. 51 Up to 36% of the patients reported hypertension while taking the humanised VEGF antibody bevacizumab; however, their blood pressure returned to normal after treatment discontinuation.52,53 Sunitinib-induced hypertension rates ranged from 16% to 47%.54–56 Both chemotherapy and angiogenesis inhibition increases the risk of venous and arterial thrombosis,57,58 and sporadic cases of arterial thromboembolism, particularly myocardial infarction, have been documented. 57 Thromboembolic events were more common among patients treated with ICIs, with an incidence rate of 2.6–18%.59,60 According to a pharmacovigilance study, ICIs are associated with an increased risk of venous and arterial thromboembolism. 61
ICIs plus anti-VEGF therapy was associated with a significant increase in the risk of all-grade proteinuria in RCC compared to sunitinib monotherapy. 48 The rate of renal and urinary disorders in patients with advanced lung cancer is 4.74% for all grades of ICIs combined with AGIs, and 4.08% for grades ⩾3. 50 ICI therapy leads to adverse renal events in 2.2% of patients. 62 Acute interstitial nephritis is the most frequently reported pathology. 63 The symptoms included oliguria, haematuria, peripheral oedema, worsening hypertension, electrolyte imbalance, altered urinary output and increased creatinine.63,64 Renal toxicity is a common side effect of angiogenesis inhibition and proteinuria is the most prominent manifestation. 65 VEGF plays a significant role in the normal functioning of the glomerulus. 66 The reason for VEGF inhibitor-induced proteinuria may be acute hypertension and the direct effects of VEGF antagonism on the glomeruli. 67 Proteinuria related to treatment with bevacizumab is a dose-dependent side effect and has been reported in up to 41–63% of patients, and the risk of proteinuria and hypertension related to bevacizumab treatment was not altered after the exclusion of patients with RCC. 68 Patients treated with sunitinib or sorafenib experienced significant proteinuria, which decreased or disappeared after treatment was reduced or stopped. 69
Our study has some limitations. First, FAERS is a spontaneous reporting system (SRS). The quantification of adverse reaction signals based on the total number of AEs was not possible using the collected data. The purpose was to provide a qualitative indicator using the signal intensity between the drugs and the reactions alone. Second, compared with clinical trials and cohort studies, SRS data are generally less reliable. The identification and reporting of AEs within a SRS are subject to less stringent controls. Third, it is difficult to identify significant risk factors between disorders and drugs because of deficiencies in preexisting disorders and comorbidities that may affect the disease. Furthermore, this study was not restricted to a particular disease, which is significantly different from clinical trials. Fourth, this study used SOC as the target reaction to encompass all AEs; however, its scope was too broad, and subgroup analyses were missing. Fifth, calculation, justification and power analyses for the selected sample size in this study were not conducted because the intention was to include all eligible ADRs. Sixth, FAERS database has reporting bias, including weber effect (reports increase in the first 2 years after launching, but then decline), ripple effect (the reporting of other drugs in the same class can be expedited due to the notoriety of a drug) and masking or cloaking effect (reports with the same adverse event connected to other drugs can suppress signal scores). For DDIs, the relative reporting rate of drug-induced AEs when either drug is used alone will be overestimated if there is a lack of information on one of the two drugs.70,71 Finally, data mining revealed imperfect reporting, with inaccuracies and incomplete entries, potentially causing analytical bias. Our findings, although limited by FAERS, shed light on DDIs related to ICIs combined with AGIs and provide a framework for rigorous research to validate the findings.
Conclusion
Our study identified four types of DDIs associated with the combined use of ICIs and AGIs using real-world FAERS data. Therefore, close attention should be paid when these two drugs are administered together. Preclinical trials are required to explore the mechanisms underlying these interactions, and further robust clinical studies are necessary to elucidate these relationships, understand the risk of DDIs associated with this combination therapy, and provide more granular details.
Supplemental Material
sj-docx-1-iji-10.1177_03946320241305390 – Supplemental material for Adverse reactions of immune checkpoint inhibitors combined with angiogenesis inhibitors: A pharmacovigilance analysis of drug–drug interactions
Supplemental material, sj-docx-1-iji-10.1177_03946320241305390 for Adverse reactions of immune checkpoint inhibitors combined with angiogenesis inhibitors: A pharmacovigilance analysis of drug–drug interactions by Xiayang Ren, Lei Deng, Xin Dong, Ying Bai, Guohui Li and Yanfeng Wang in International Journal of Immunopathology and Pharmacology
Footnotes
Acknowledgements
We thank Xin Zhou for his contributions regarding the data processing and image editing.
List of abbreviations
AE, adverse event; AGI, angiogenesis inhibitor; CI, confidence interval; CTLA-4, cytotoxic T-lymphocyte antigen-4; DDI, drug-drug reaction; FAERS, FDA Adverse Event Reporting System; FDA, US Food and Drug Administration; IC, information component; ICI, immune checkpoint inhibitor; IQR, interquartile range; MedDRA, Medical Dictionary for Regulatory Activities; PD-1, programmed death-1; PD-L1, programmed death ligand-1; PT, preferred term; RCC, renal cell carcinoma; ROR, reporting odds ratio; ROR025, lower 95% confidence limit of the ROR; SOC, System Organ Class; VEGF, vascular endothelial growth factor
Author contributions
XR drafted the manuscript. All authors participated in data analysis and interpretation, and manuscript revision. All authors have read and approved the manuscript.
Data availability statement
The data that support the findings of this study are available upon from the corresponding author, Yanfeng Wang, on reasonable request. The raw data can be obtained from the FAERS database at the following link: FAERS Quarterly Data Extract Files (fda.gov).
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This study was funded by Beijing Xisike Clinical Oncology Research FoundationY-HR2020MS-0671.
Ethical considerations
Not applicable. Ethical approval was not required for this study because we used the FAERS database, which is a free open-access database.
Consent to participate
Not applicable.
Consent for publication
Not applicable.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
