First-line treatments for advanced non-squamous non-small cell lung cancer with immune checkpoint inhibitors plus chemotherapy: a systematic review,network meta-analysis,and cost-effectiveness analysis

Abstract

Introduction:

The combination of immune checkpoint inhibitors (ICIs) and chemotherapy is a promising first-line therapy for patients with advanced non-squamous non-small cell lung cancer (NSCLC). The cost-effectiveness of combinations with different ICIs is yet to be compared.

Methods:

We utilized Bayesian network meta-analyses for the comparisons of overall survival, progression-free survival, and incidence of adverse events of the included treatments in the total population and subgroups with different programmed death-ligand 1 tumor proportional scores (TPS). The cost-effectiveness of the treatments from the perspectives of the US and Chinese healthcare systems was assessed using Markov models.

Results:

Three combinations, including pembrolizumab + chemotherapy (PembroC), nivolumab + ipilimumab + chemotherapy (NivoIpiC), and atezolizumab + chemotherapy (AteC), were included in our study. In terms of efficacy, PembroC was most likely to be ranked first for extending progression-free survival (PFS) (93.16%) and overall survival (OS) (90.73%). Nevertheless, from the US perspective, NivoIpiC and PembroC showed incremental cost-effectiveness ratios (ICERs) of $68,963.1/quality-adjusted life-years (QALY) and $179,355.6/QALY, respectively, compared with AteC. The one-way sensitivity analysis revealed that the results were primarily sensitive to the hazard ratios for OS or the cost of immunotherapy agents. At a willingness-to-pay (WTP) threshold of $150,000/QALY, NivoIpiC had the highest probability of being cost-effective (63%). As for the Chinese perspective, NivoIpiC and PembroC had ICERs of $145,983.4/QALY and $195,863.3/QALY versus AteC, respectively. The results were primarily sensitive to the HRs for OS. At a WTP threshold of $38,017/QALY, AteC had the highest probability of cost-effectiveness (94%).

Conclusion:

Although PembroC has the optimal efficacy, NivoIpiC and AteC were the most favorable treatments in terms of cost-effectiveness for patients with advanced non-squamous NSCLC from the US and Chinese perspectives, respectively.

Keywords

chemotherapy cost-effectiveness immune checkpoint inhibitor network meta-analysis non-squamous non-small cell lung cancer

Introduction

The 2022 Cancer Statistics report revealed that lung cancer was the most common cause of cancer-related death, accounting for almost a quarter of all cancer deaths.¹ The disease burden of lung cancer is also high, according to the Global Burden of Disease Study in 2019, which ranks lung cancer as one of the leading causes of disability-adjusted life-years (DALYs).² Non-small cell lung cancer (NSCLC) is the predominant histological subtype, representing about 85% of all lung cancer cases, with non-squamous cell carcinoma accounting for over 50% of cases.³ Due to the limited testing techniques, NSCLCs are usually diagnosed at an advanced stage, making effective first-line treatments crucial.⁴

Currently, for advanced NSCLC patients with negative driver gene mutations, first-line regimens include platinum-based doublet therapy plus immune checkpoint inhibitors (ICIs) with or without bevacizumab or ICIs monotherapy for patients with a programmed death-ligand 1 (PD-L1) tumor proportional score (TPS) ⩾50%.⁵ KEYNOTE 189, a phase III randomized controlled trial (RCT), paved the way for ICI plus chemotherapy (CT) as a standard first-line treatment for advanced NSCLCs without sensitizing targetable mutations. In this trial, pembrolizumab plus CT led to improvements in overall survival (OS) of non-squamous NSCLCs, especially in those who have PD-L1 TPS ⩾50% (27.7 versus 10.1 months).⁶ Similarly, IMpower 132, a randomized phase III trial showed that non-squamous NSCLC patients, regardless of PD-L1 expression, can benefit from the combination of atezolizumab and CT (median OS: 17.5 versus 13.6 months).⁷ Besides, the phase III CheckMate 9LA study revealed that nivolumab, a PD-1 inhibitor, plus ipilimumab, a cytotoxic T lymphocyte-associated antigen-4 inhibitor, plus CT as first-line treatments for non-squamous NSCLC cases also could prolong the OS compared with CT alone (CheckMate 9LA: 17.0 versus 11.9 months).⁸

In determining the most suitable treatment for patients, factors such as efficacy, safety, and cost-effectiveness must be considered. While previous studies have assessed the cost-effectiveness of ICIs plus CT versus CT alone, there is a lack of direct comparison of ICI combination therapies.^9–13 Therefore, this study aims to compare the cost-effectiveness, efficacy, and safety of different combination regimens with ICIs for untreated non-squamous NSCLCs by using a network meta-analysis (NMA) from the perspective of US healthcare payers. This NMA and cost-effectiveness analysis enables a comprehensive evaluation of the available evidence from clinical trials, which can inform healthcare decision-makers on the best treatment options.

Materials and methods

Network meta-analysis

Study selection

This NMA was finished following the Preferred Reporting Items for Systematic Review and Meta-Analyses extension statement and not registered online (Supplemental Material 2).¹⁴ Two authors (LN and WT) individually searched the Pubmed, Cochrane Library, EMBASE, and Web of Science for RCTs from 1 January 2000 to 24 February 2022, using terms of ‘Carcinoma, Non-Small-Cell Lung’, ‘Immunotherapy’, ‘Immune Checkpoint Inhibitors’, ‘Pembrolizumab’, ‘Nivolumab’, ‘Atezolizumab’, and ‘Durvalumab’. Detailed search strategies are provided in Supplemental Table S1. Literature titles, abstracts, and full text were screened by the two authors independently, and then we combined the results and deleted duplicated results. In case of disagreement, a third researcher was required. Details about the inclusion and exclusion criteria and study selection are in the Supplemental, respectively, in Figure 1.

Figure 1.

Flow diagram of study selection.

Data extraction and assessment of risk of bias

Two investigators (LN and WT) extracted data independently by browsing the full text of the studies. The following information was extracted: (1) authors and publication year; (2) median follow-up time; (3) interventions; (4) the number of patients with non-squamous NSCLC; (5) PD-L1 expression; (6) OS (the time from randomization to patient’s death of any cause); (7) progression-free survival (PFS) (the time from randomization to objective tumor progression or all-cause death); (8) the number of patients with treatment-related adverse events (TRAEs) or TRAEs of grade ⩾3 specifically. The methodological qualities (risk of bias) of included studies were assessed by the two authors independently in RevMan, version 5.4, according to the guidance provided in the Cochrane handbook.

Statistical analysis

We assessed the primary outcomes, including hazard ratios (HRs) for PFS and OS and odds ratio (OR) for AEs. Markov Chains Monte Carlo sampler was used to establish three Markov Chains that run 5000 burn-ins and 20,000 subsequent simulations to finish posterior summaries. The Brooks–Gelman–Rubin plots method was used to assess the model convergence. A fixed-effect model was used for the analysis due to the absence of data to assess the heterogeneity between trials. In order to meet the requirement of consistency, we included phase III RCTs with similar basic characteristics of patients and compared the distribution of clinical or methodological variables. The Bayesian method can provide treatment ranking probabilities by calculating surfaces under the cumulative ranking curves. Therefore, each treatment in a network can get a particular placement in ordering treatment effects from best to worst. Statistical analysis was performed using R software (version 4.0.4) with the package ‘gemtc’.

Cost-effectiveness analysis

The cost-effectiveness analysis follows the Consolidated Health Economic Evaluation Reporting Standards Value of Information guideline (Supplemental Material 3).¹⁵

Model structure

A Markov model was needed to simulate patient-level data to compare the cost-effectiveness of different ICIs as first-line treatments for advanced non-squamous NSCLC from the perspectives of the United States and China. This model was constructed using TreeAge Pro 2021 (TreeAge Software, LLC, Williamstown, Massachusetts) with a 21-day cycle length and three mutually independent health states: PFS, progressive disease (PD), and death (Figure 2). Initially, all patients were enrolled in the PFS health state and subsequently received various first-line ICI treatments until either disease progression or the development of intolerable toxicity. Patients who experienced disease progression during the first-line treatment were transitioned to the PD health state, where they received subsequent anticancer therapies. Ultimately, patients in any of the states had the potential to directly transition to the state of death.

Figure 2.

Framework of the decision tree and the Markov model.

In the model, the analytic time horizon was set to 15 years since 99% of the cohort died at this time point. The primary outcomes consisted of the life-years (LYs), quality-adjusted life-years (QALYs), overall costs, and incremental cost-effectiveness ratios (ICERs). We applied a half-cycle correction. A 3% discount rate per year and a 5% discount rate per year was used for both cost and health outcomes in the United States and in China, respectively.¹⁶ The willingness-to-pay (WTP) threshold in this study was set to $150,000 per QALY and $38,017 per QALY for the US perspective and the Chinese perspective, respectively. ¹⁷

Clinical data

The Kaplan–Meier (K–M) curves, including PFS and OS, were obtained from KEYNOTE 189, CheckMate 9LA, and IMpower 132. Subgroup analysis was conducted in patients categorized by PD-L1 expression level. Survival data points were extracted from curves using GetData Graph Digitizer (version 2.26, GetData Pty Ltd., Kogarah, NSW, Australia). The individual patient data were recreated using the method of Hoyle et al.¹⁸ For reconstructing the individual patient time-to-event data, these data points are used to fit parametric models, including exponential, Weibull, log-logistic, lognormal, genGamma, and Gompertz distributions. Subsequently, according to the values of the Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC) combined with visual inspections, the best-fit distribution models were chosen, and K–M curves were evaluated by using R software (version 4.0.4) with the package ‘survival’ and ‘flexsurv’. For some curves, the fitting curves with the lowest AIC/BIC were not selected because of tailing or PFS being higher than OS. The AIC and BIC values of each distribution are shown in Supplemental Table S2, and the comparisons between K–M and fitting curves of all non-squamous NSCLCs and patients with different levels of PD-L1 expression are shown in Supplemental Figure S5. The chosen estimated curves are displayed in Supplemental Figure S6. The HRs of PFS and OS among the three interventions were calculated using NMA and were needed to estimate the PFS and OS rates. According to the National Comprehensive Cancer Network (NCCN) guidelines, pembrolizumab plus CT is the preferred recommendation regardless of PD-L1 expression, so it was chosen as the baseline treatment. OS and PFS probabilities of the intention-to-treat (ITT), PD-L1 1–49%, and PD-L1 < 1% patients in the atezolizumab plus CT group were also calculated based on the those in the pembrolizumab plus CT group and the HRs.¹⁹ Due to the absence of PD-L1 subgroup survival data of non-squamous patients in CheckMate 9LA, we made reasonable assumptions based on relevant data and examined them in the sensitivity analyses.

Cost and utility data

Only direct medical costs were considered, which consisted of first-line and subsequent drug costs, costs of intravenous infusion, imaging trimonthly, routine follow-up, end-of-life (6 months), best supportive care (BSC), and management of AEs (Table 1). According to the trial design, the experimental groups were given pembrolizumab 200 mg triweekly combined with CT, nivolumab 360 mg triweekly plus ipilimumab 1 mg/kg every 6 weeks plus CT, or atezolizumab 1200 mg triweekly plus CT, respectively, in KEYNOTE 189, CheckMate 9LA, and IMpower 132. After induction treatment with ICIs plus CT, maintenance therapy was given until disease progression or unacceptable toxicity. Furthermore, patients would receive subsequent treatment and the BSC until death. All details of subsequent treatment options were derived from relevant clinical trials (Supplemental Table S3).^6–8 To estimate the dose of agents, we assumed a typical patient had 70 kg in weight and 1.79 m² in body surface area (BSA), and creatinine clearance rate is 70 ml/min.^20,21 All values of costs were from the Centers for Medicare & Medicaid Services and some published articles.^22–28 The utility-scale was from 0 (death) to 1 (perfect health), and different utilities represented particular health states. Our study’s survival and health state utilities were all obtained from previously published articles.^29–31 We also collected some utilities related to AEs, and the details of the utilities are shown in Table 1.

Table 1.

Parameters input to the model.

Parameters	Baseline value	Range		Distribution	Source
Parameters	Baseline value	Minimum	Maximum	Distribution	Source
Clinical
Body weight	70	52.5	87.5	Normal	Kohn et al., 2017²⁰
BSA	1.79	1.34	2.24	Normal	Kohn et al., 2017²⁰
Discount rate	0.03	ND	ND	Uniform	Kohn et al., 2017²⁰
HR of OS (AteT versus PembroC ITT)	1.53	1.15	2.04	ND	NMA
HR of OS (AteT versus PembroC NEG)	1.32	0.80	2.18	ND	NMA
HR of OS (AteT versus PembroC LOW)	1.78	1.04	3.07	ND	NMA
HR of OS (AteT versus PembroC HIGH)	1.24	0.48	3.16	ND	NMA
HR of OS (AteT versus PembroC ITT)	1.23	0.91	1.68	ND	NMA
HR of OS (NivoIpiC versus PembroC NEG)	1.22	0.76	1.93	ND	NMA
HR of OS (NivoIpiC versus PembroC LOW)	0.92	0.57	1.50	ND	NMA
HR of OS (NivoIpiC versus PembroC HIGH)	1.12	0.64	1.95	ND	NMA
HR of PFS (AteT versus PembroC ITT)	1.22	0.94	1.60	ND	NMA
HR of PFS (AteT versus PembroC NEG)	0.67	0.41	1.09	ND	NMA
HR of PFS (AteT versus PembroC LOW)	1.51	0.92	2.47	ND	NMA
HR of PFS (AteT versus PembroC HIGH)	1.32	0.58	2.97	ND	NMA
HR of PFS (NivoIpiC versus PembroC ITT)	1.51	1.14	2.00	ND	NMA
HR of PFS (NivoIpiC versus PembroC NEG)	1.06	0.69	1.63	ND	NMA
HR of PFS (NivoIpiC versus PembroC LOW)	1.30	0.83	2.04	ND	NMA
HR of PFS (NivoIpiC versus PembroC HIGH)	1.74	1.05	2.89	ND	NMA
Weibull OS survival model of PembroC (ITT)	λ: 0.022	ND	ND	ND	Model fitting
Weibull OS survival model of PembroC (ITT)	γ: 1.107	ND	ND	ND	Model fitting
Exponential OS survival model of PembroC (NEG)	λ: 0.038	ND	ND	ND	Model fitting
Weibull OS survival model of PembroC (LOW)	λ: 0.019	ND	ND	ND	Model fitting
	γ: 1.177
Exponential OS survival model of PembroC (HIGH)	λ: 0.025	ND	ND	ND	Model fitting
Exponential PFS survival model of PembroC (ITT)	λ: 0.066	ND	ND	ND	Model fitting
Exponential PFS survival model of PembroC (NEG)	λ: 0.098	ND	ND	ND	Model fitting
Exponential PFS survival model of PembroC (LOW)	λ: 0.066	ND	ND	ND	Model fitting
Exponential PFS survival model of PembroC (HIGH)	λ: 0.052	ND	ND	ND	Model fitting
Drug cost (per mg), $
Atezolizumab	7.96	5.97	9.95	Gamma	Medicare drug prices³²
Pembrolizumab	52.30	39.23	65.38	Gamma	Medicare drug prices³²
Nivolumab	29.25	21.94	36.56	Gamma	Medicare drug prices³²
Ipilimumab	160.70	120.53	200.88	Gamma	Medicare drug prices³²
Bevacizumab	6.79	5.09	8.49	Gamma	Medicare drug prices³²
Durvalumab	7.69	5.77	9.61	Gamma	Medicare drug prices³²
Daratumumab	5.72	4.29	7.15	Gamma	Medicare drug prices³²
Ramucirumab	12.71	9.53	15.89	Gamma	Medicare drug prices³²
Carboplatin	0.05	0.04	0.06	Gamma	Medicare drug prices³²
Cisplatin	0.19	0.14	0.24	Gamma	Medicare drug prices³²
Docetaxel	0.46	0.35	0.58	Gamma	Medicare drug prices³²
Paclitaxel albumin	14.04	10.53	17.55	Gamma	Medicare drug prices³²
Paclitaxel	0.12	0.09	0.15	Gamma	Medicare drug prices³²
Pemetrexed	7.60	5.70	9.50	Gamma	Medicare drug prices³²
Etoposide	1.51	1.13	1.89	Gamma	Medicare drug prices³²
Gemcitabine	0.02	0.01	0.03	Gamma	Medicare drug prices³²
Vinorelbine	0.86	0.65	1.08	Gamma	Medicare drug prices³²
Crizotinib	1.37	1.03	1.71	Gamma	Medicare drug prices³²
Osimertinib	6.62	4.97	8.28	Gamma	Medicare drug prices³²
Erlotinib	1.70	1.28	2.13	Gamma	Medicare drug prices³²
Tepotinib	1.70	1.28	2.13	Gamma	Medicare drug prices³²
Binimetinib	5.08	3.81	6.35	Gamma	Medicare drug prices³²
Nintedanib	1.34	1.01	1.68	Gamma	Medicare drug prices³²
Mitomycin	10.13	7.60	12.66	Gamma	Medicare drug prices³²
Alectinib	0.48	0.36	0.60	Gamma	Medicare drug prices³²
Palbociclib	9.27	6.95	11.59	Gamma	Medicare drug prices³²
Radiotherapy per event	8190.60	6142.95	10,238.25	Gamma	Medicare drug prices³²
Surgery per event	9608.07	7206.05	12,010.09	Gamma	Medicare drug prices³²
Drug administration costs, $
Chemotherapy infusion
First 1 h	140.16	105.12	175.2	Gamma	CMS (CPT 96413)³³
Additional 1 h	29.76	22.32	37.2	Gamma	CMS (CPT 96415)³³
Per hour for subsequent infusion	68.17	51.13	85.21	Gamma	CMS (CPT 96417)³³
Best supportive care	3489.98	2617.49	4362.48	Gamma	Criss et al., 2019²³
End of life	9982.78	7487.09	2495.70	Gamma	Aguiar Jr et al., 2018²²
Follow-up	510.56	293.39	727.73	Gamma	Klein et al., 2010²⁶
Three monthly imaging	116.62	87.47	145.78	Gamma	CMS (CPT 78816)³³
SAE management cost (per event), $
Anemia	1893.22	1419.92	2366.53	Gamma	Insinga et al., 2018²⁴
Neutropenia	1193.85	895.39	1492.31	Gamma	Insinga et al., 2018²⁴
Rash	308.65	231.49	385.81	Gamma	Zhang et al., 2021²⁸
Thrombocytopenia	2076.36	1557.27	2595.45	Gamma	Insinga et al., 2018²⁴
Utility
PD	0.58	0.50	0.66	Beta	Chouaid et al., 2013²⁹
PF	0.70	0.66	0.73	Beta	Chouaid et al., 2013²⁹
Disutility
Rash	0.15	0.11	0.19	Beta	Nafees et al., 2017³⁰
Thrombocytopenia	0.25	0.19	0.31	Beta	Nafees et al., 2017³⁰
Anemia	0.07	0.05	0.09	Beta	Wan et al., 2019³¹
Diarrhea	0.29	0.22	0.51	Beta	Nafees et al., 2017³⁰
Fatigue	0.29	0.22	0.36	Beta	Nafees et al., 2017³⁰
Neutropenia	0.35	0.26	0.44	Beta	Nafees et al., 2019³⁰
SAEs in PembroC group	0.10	0.08	0.13	Beta	ND
SAEs in NivoIpiC group	0.15	0.11	0.19	Beta	ND
SAEs in AteT group	0.0047	0.0035	0.0059	Beta	ND
Risk of SAEs in PembroC group
Anemia	0.15	0.11	0.19	Beta	Rodríguez-Abreu et al., 2021⁶
Fatigue	0.06	0.05	0.08	Beta	Rodríguez-Abreu et al., 2021⁶
Neutropenia	0.15	0.11	0.19	Beta	Rodríguez-Abreu et al., 2021⁶
Thrombocytopenia	0.08	0.06	0.10	Beta	Rodríguez-Abreu et al., 2021⁶
Risk of SAEs in NivoIpiC group
Anemia	0.21	0.16	0.26	Beta	Paz-Ares et al., 2021⁸
Diarrhea	0.14	0.11	0.18	Beta	Paz-Ares et al., 2021⁸
Neutropenia	0.24	0.18	0.30	Beta	Paz-Ares et al., 2021⁸
Thrombocytopenia	0.10	0.08	0.13	Beta	Paz-Ares et al., 2021⁸
Risk of SAEs in AteT group
Hepatitis	0.024	0.018	0.030	Beta	Nishio et al., 2021⁷
Pneumonitis	0.021	0.016	0.026	Beta	Nishio et al., 2021⁷
Rash	0.031	0.023	0.039	Beta	Nishio et al., 2021⁷

AteT, atezolizumab + chemotherapy; BSA, body surface area; CMS, Centers for Medicare & Medicaid Services; HIGH, PD-L1 TPS ⩾ 50% population; HR, hazard ratio; ITT, intention-to-treat population; LOW, PD-L1 TPS 1–49% population; NEG, PD-L1 TPS < 1% population; ND, not determined; NivoIpiC, nivolumab + ipilimumab + chemotherapy; NMA, network meta-analysis; OS, overall survival; PembroC, pembrolizumab + chemotherapy; PFS, progression-free survival; SAE, severe adverse event.

Sensitivity analysis

The model robustness was assessed using one-way sensitivity analyses and probabilistic sensitivity analyses (PSAs). In the one-way sensitivity analyses, the ranges of different variables were either based on the 95% confidence intervals (CIs) reported in the published articles or estimated by assuming ±25% from the baseline values (Table 1). PSAs were also performed through 10,000 Monte Carlo repetitions. The probability distribution of HRs (log-normal distribution), costs (gamma distribution), proportions (beta distribution), incidence rates (beta distribution), and utility values (beta distribution) were included. The PD-L1 subgroup sensitivity analyses were also conducted.

Results

Study characteristics and the risk of bias

We identified a total of 3791 studies through database searching (Figure 1) and finally included three studies (CheckMate 9LA, Keynote-189, and Impower132) (Supplemental Table S4), involving 1686 patients with advanced non-squamous NSCLC, according to the inclusion and exclusion criteria (Figure 1), and we summarized their characteristics in Supplemental Table S2. The patients with advanced non-squamous NSCLC were treated with nivolumab + ipilimumab + chemotherapy (NivoIpiC) (n = 248), pembrolizumab + chemotherapy (PembroC) (n = 410), atezolizumab + chemotherapy (AteC) (n = 292), or CT alone (n = 738).

The risks of the study bias were summarized in Supplemental Figure S1. Even though two studies had high risks of performance bias and one study had high risks of reporting bias, selection bias, detection bias, attrition bias, and other biases remained low or unknown among the included studies.

Efficacy and safety results

The NMAs for efficacy included four treatments for PFS and OS in the overall patients with non-squamous NSCLC and three treatments for the patients with negative (TPS < 1%), low (TPS 1–49%), or high (TPS ⩾ 50%) tumor PD-L1 expression (Supplemental Figure S2). Due to a lack of data on non-squamous PD-L1 subgroups in CheckMate 9LA, nivolumab + ipilimumab + CT was excluded in the PD-L1 subgroup analyses.

As for OS [Figure 3(a)], all the immunotherapy-based treatments, except atezolizumab + CT in the patients with high PD-L1 expression (HR 0.73, 95% CI 0.31–1.72) and atezolizumab + CT in the patients with low PD-L1 expression (HR 1.18, 95% CI 0.8–1.75), achieved significant OS prolongation in the total population and across subgroups with various PD-L1 expressions. Pembrolizumab + CT was most likely to be ranked first for extending OS in the total population (90.73%), patients with high PD-L1 expression (67.29%), patient with low PD-L1 expression (97.36%), and patients with negative PD-L1 expression (85.73%) [Supplemental Figure S3(A), (C), (E), and (G)]. Pembrolizumab + CT, as for the pooled HR profiles, significantly prolonged OS in the total population (HR 0.65, 95% CI 0.49–0.87) and in patients with low PD-L1 expression (HR 0.56, 95% CI 0.33–0.96) compared with atezolizumab + CT.

Figure 3.

The presented league table displays the pairwise comparisons of overall survival or progression-free survival, denoted as (a) and (b), respectively. The results are presented as HR along with their corresponding 95% CI. These ratios represent the treatment defined by the columns compared to the treatment defined by the rows.

As for PFS [Figure 3(b)], all the immunotherapy-based treatments, except atezolizumab + CT in the low PD-L1 expression group (HR 0.8, 95% CI 0.56–1.15; Figure 3), led to significant PFS prolongation compared with CT alone in the total population and across all the subgroups with various PD-L1 expressions. Among these immunotherapy treatments, pembrolizumab + CT was most likely to be ranked first for extending PFS in the total population (93.16%), patients with high PD-L1 expression (74.32%), and patients with low PD-L1 expression (94.94%) [Supplemental Figure S3(B), (D), and (F)]. However, atezolizumab + CT had the highest probability (94.55%) to be ranked first for extending PFS in paitents with negative PD-L1 expression. In line with the ranking profile, pembrolizumab + CT significantly prolonged PFS compared with nivolumab + ipilimumab + CT in the total population (HR 0.66, 95% 0.50–0.88).

The NMAs for safety included three treatments, excluding nivolumab + ipilimumab + CT due to a lack of the safety profile in patients with non-squamous NSCLC reported by CheckMate 9LA (Supplemental Figure S4) for all TRAEs [Supplemental Figure S4(A)] and TRAEs of grade ⩾ 3 [Supplemental Figure S4(B)]. The ranking profiles showed that CT alone was most likely to be ranked first in engendering the fewest AEs (77.41%) and AEs of grade ⩾ 3 (99.33%), while atezolizumab + CT had a high probability to be ranked first in causing the most grade ⩾ 3 AEs (77.86%). Based on the ORs, both immunotherapy-based treatments, including atezolizumab + CT (OR 1.86, 95% CI 1.33–2.6) and pembrolizumab + CT (OR 1.54, 95% CI 1.1–2.18), increased the probabilities of TRAEs of grade ⩾ 3 compared with CT alone.

Cost-effectiveness results

We summarized the base-case mean cost, LYs, QALYs, and ICERs of each treatment in Table 2. Since the median OS of patients with PD-L1 TPS ⩾50% treated with atezolizumab + CT was not reached, we did not include the data of this group in the cost-effectiveness analysis.

Table 2.

Baseline results.

Strategy	Cost ($)	Incremental cost ($)	QALYs	Incremental QALYs	LYs	Compared with	ICER ($/QALY)	Rank
The United States
ITT
AteC	271,718.22	NA	1.01	NA	1.61	NA	NA	2
NivoIpiC	284,176.50	12,458.28	1.19	0.18	1.95	AteC	68,963.1	1
PembroC	349,088.46	77,370.24	1.44	0.43	2.32	AteC	179,355.6	3
		64,911.96		0.25		NivoIpiC	258,894.4
PD-L1 < 1%
AteC	282,689.19	NA	1.00	NA	1.58	NA	Dominated	2
PembroC	272,344.16	−10,345.02	1.25	0.25	2.04	AteC	Dominate	1
PD-L1 1–49%
AteC	225,231.95	NA	1.32	NA	1.31	NA	NA	1
PembroC	336,030.44	110,798.49	0.82	0.50	2.12	AteC	222,605.8	2
China
ITT
AteC	85,686.76	NA	0.98	NA	1.57	NA	NA	2
NivoIpiC	110,386.59	24,699.83	1.15	0.17	1.88	AteC	145,983.4	1
PembroC	164,769.46	79,082.70	1.39	0.40	2.23	AteC	195,863.3	3
		54,382.88		0.23		NivoIpiC	231,842.1
PD-L1 <1%
AteC	89,856.14	NA	0.97	NA	1.53	NA	NA	1
PembroC	139,757.56	49,901.42	1.20	0.23	1.96	AteC	215,075.7	2
PD-L1 1–49%
AteC	73,557.28	NA	0.81	NA	1.28	NA	NA	1
PembroC	155,523.98	81,966.7	1.28	0.47	2.04	AteC	173,939.0	2

AteT, atezolizumab + chemotherapy; ICER, incremental cost-effectiveness ratio; ITT, intention-to-treat population; LYs, life-years; NA, not applicable; NivoIpiC, nivolumab + ipilimumab + chemotherapy; PembroC, pembrolizumab + chemotherapy; QALYs, quality-adjusted life-years.

From the perspective of the US health care payers, in terms of the total population, the mean costs of atezolizumab + CT, nivolumab + ipilimumab + CT, and pembrolizumab + CT were $271,718.22, $284,176.50, and $349,088.46, respectively, and the QALYs were 1.01, 1.19, and 1.44, respectively. Compared with atezolizumab + CT, the ICERs of nivolumab + ipilimumab + CT and pembrolizumab plus CT were $68,963.1 and $179,355.6 per QALY gained, respectively, and the ICER of pembrolizumab + CT compared with nivolumab + ipilimumab + CT was $258,894.4 per QALY gained. In patients with PD-L1 TPS <1%, the mean costs of combining CT with atezolizumab and pembrolizumab were $282,689.19 and $272,344.16, respectively, and the QALYs were 1.00 and 1.25, respectively. The cost-effectiveness of pembrolizumab + CT dominated that of atezolizumab + CT in this setting. In patients with PD-L1 TPS 1–49%, the mean costs of atezolizumab + CT and pembrolizumab + CT were $225,231.95 and $336,030.44, respectively. The QALYs were 1.32 and 0.82, respectively. Compared with atezolizumab + CT, the ICER for pembrolizumab plus CT was $222,605.8 per QALY gained. The one-way sensitivity analysis (Supplemental Figure S7) revealed that the ICER of pembrolizumab + CT versus atezolizumab + CT in the total population was mostly sensitive to cost of pembrolizumab [Supplemental Figure S7(A)], while the ICER of pembrolizumab + CT versus nivolumab + ipilimumab + CT in the total population was most sensitive to the HR for OS [Supplemental Figure S7(B)], and the ICER of atezolizumab + CT versus nivolumab + ipilimumab + CT was most sensitive to body weight [Supplemental Figure S6(C)]. In terms of patients with PD-L1 TPS 1–49%, the ICER of pembrolizumab + CT versus atezolizumab + CT was primarily sensitive to the HR for OS [Supplemental Figure S7(D)]. As for patients with PD-L1 TPS <1%, the ICER of pembrolizumab + CT versus atezolizumab + CT was primarily sensitive to HR for PFS [Supplemental Figure S7(E)]. At a WTP threshold of $150,000 per QALY, nivolumab + ipilimumab + CT had the highest probability of cost-effectiveness in the total population [63%; Figure 4(a)], while pembrolizumab + CT had the highest probability of being cost-effective in patients with PD-L1 1–49% [93%; Figure 4(b)]. In contrast, pembrolizumab + CT had the highest probability [96%; Figure 4(c)] of being cost-effective at a WTP threshold of $150,000 per QALY in patients with PD-L1 TPS <1%.

Figure 4.

The cost-effectiveness acceptability curves for pembrolizumab plus chemotherapy, atezolizumab plus chemotherapy, and nivolumab plus ipilimumab plus chemotherapy in (a–c) the United Sates and (d–f) China. (a and d) The intend-to-treatment population. (b and e) patients with PD-L1 TPS 1–49%. (c and f) Patients with PD-L1 TPS <1%.

From the China perspective, the mean costs of atezolizumab + CT, nivolumab + ipilimumab + CT, and pembrolizumab + CT were $85,686.76, $110,386.59, and $164,769.46, and the QALYs were 0.98, 1.15, and 1.39, respectively, in the total population. Compared with atezolizumab + CT, the ICERs of nivolumab + ipilimumab and pembrolizumab + CT were $145,983.4 and $195,863.3, respectively, in the total population. In patients with PD-L1 <1%, the costs of atezolizumab + CT and pembrolizumab + CT were $89,856.14 and $139,757.56, while the QALYs were, respectively, 0.97 and 1.20, yielding an ICER of $215,057.7. Similarly, in patients with PD-L1 1–49%, the costs of the two treatments were $73,557.28 and $155,523.98, while the QALYs were 0.81 and 1.28, leading to an ICER of $173,939.0 per QALY gained. The one-way sensitivity analysis (Supplemental Figure S8) identified that all the cost-effectiveness results in China were mostly sensitive to HR for OS. As for the PSA, atezolizumab + CT had the highest probability to be cost-effective in the total population [94%; Figure 4(d)] at the WTP threshold of $38,017 per QALY gained. Unlike the results from the US perspective, atezolizumab + CT also had higher probabilities of cost-effectiveness compared with pembrolizumab + CT in both patients with PD-L1 1–49% [100%; Figure 4(e)] and patients with PD-L1 <1% [100%; Figure 4(f)].

Discussion

The combination of ICIs with CT has been proven to be a promising first-line strategy in treating patients with advanced non-squamous NSCLC, the efficacy of which, however, varies with tumor PD-L1 expression. To date, several ICI + CT combination strategies have been recommended for the treatment of advanced or metastatic non-squamous NSCLC in the NCCN guideline.¹⁹ However, the cost-effectiveness of these therapies remains to be explored.

The results from the NMAs indicated that pembrolizumab + CT resulted in better survival benefits compared with the other two ICI-based treatments in the total population and in most PD-L1 expression subgroups, which was consistent with a former NMAs.³⁴ The evaluation of safety profiles demonstrated that all three ICI + CT combination treatments increased the incidences of TRAEs of grade ⩾3 compared with CT alone but had similar chances of causing TRAEs of any grades or TRAEs of grade ⩾3. Nevertheless, the cost-effectiveness analysis revealed that each strategy had distinct rankings in different PD-L1 expression subgroups from the perspective of the US healthcare payers. Specifically, nivolumab + ipilimumab + CT was the most favorable strategy in the total population, while atezolizumab + CT was more favorable in patients with PD-L1 1–49%, and pembrolizumab + CT was preferable in patients with PD-L1 <1%. The sensitivity analyses revealed that the results were generally sensitive to the HRs for OS or PFS of ICI + CT combinations, suggesting that the efficacies of the treatments were the most definitive factor in influencing the cost-effectiveness. Also, the costs of the ICIs account for one of the most significant factors, suggesting that the high price of ICIs is also an unneglectable factor. In contrast, from the Chinese perspective, atezolizumab + CT was the most cost-effective strategy in the total population, patients with PD-L1 1–49%, and patients with PD-L1 <1%, due to distinct cost inputs and the much lower WTP threshold. These results provide significant insights for patients and clinicians to choose which ICI to use. To be more specific, in the United States, nivolumab + ipilimumab + CT is more favorable for patients with non-squamous NSCLC, while atezolizumab + CT and pembrolizumab + CT should be recommended for patients with PD-L1 1–49% and patients with PD-L1 <1%, respectively, to meet the optimal cost-effectiveness. In China, atezolizumab + CT is an economic option despite PD-L1 expression.

Teng et al.³⁵ compared the cost-effectiveness of four ICI regimens, including nivolumab, atezolizumab, pembrolizumab, and durvalumab, as first-line monotherapy for Chinese patients with NSCLC, and the results showed that nivolumab ranked first in terms of cost-effectiveness. This study investigated the cost-effectiveness of several ICIs but could not answer the question of which combination of ICI and CT, which has better efficacy than ICI monotherapy, has the optimal cost-effectiveness. A study by Barbier et al.³⁶ showed that pembrolizumab + CT was not likely to be cost-effective in Swiss patients with PD-L1 TPS ⩾50%, and Jiang et al.³⁷ and Cai et al.¹⁰ also revealed that pembrolizumab + CT was not cost-effective for Chinese patients with non-squamous NSCLC. Additionally, from a US healthcare perspective, another study by Liu et al.⁹ showed that pembrolizumab + CT was not cost-effective for non-squamous NSCLC patients with PD-L1 TPS ⩾50% compared with pembrolizumab alone. Our acceptability analyses also showed low probabilities of pembrolizumab + CT being cost-effective in all non-squamous NSCLC patients and across subgroups of different PD-L1 expressions at a WTP threshold of $150,000 per QALY. As for atezolizumab + CT, Ding et al.³⁸ and Yang et al.¹¹ showed that the combination of atezolizumab and carboplatin plus nab-paclitaxel was not cost-effective for first-line treatment of advanced non-squamous NSCLC from the US payers’ perspective and the Chinese payers’ perspective, respectively. However, our results showed that the combination of atezolizumab and carboplatin/cisplatin plus pemetrexed had a high probability of being cost-effective for US patients (69%), suggesting that the CT regimens can also strongly affect the cost-effectiveness of ICI + CT. In 2023, a cost-effectiveness analysis investigated the cost-effectiveness of pembrolizumab + CT, atezolizumab + CT, nivolumab + ipilimumab + CT, nivolumab + ipilimumab, atezolizumab + bevacizumab + CT, pembrolizumab alone, and CT alone in treating advanced NSCLC in the United States.³⁹ The results showed that the nivolumab + ipilimumab, pembrolizumab + CT, and pembrolizumab alone were the preferred strategies for patients with PD-L1 <1%, 1–49% and ⩾50%, respectively, which differed from our results.³⁹ The difference may attribute to different survival estimating references (the study is based on CT, and our analysis is based on pembrolizumab + CT) and a more detailed cost consideration for patients of our study.³⁹

Based on our results and previous studies, we suggest administrating different ICIs to combine with CT regimens according to patients’ PD-L1 expression, as the cost-effectiveness ranks varied across different PD-L1 expression subgroups. PD-L1 testing is a recommended and routinely applied practice for advanced NSCLC patients, and our results highlight the importance of early detection of PD-L1 expression for non-squamous NSCLC patients with negative driver gene mutations to determine the most cost-effective treatment. Additionally, although pembrolizumab + CT has a favorable efficacy and has been listed as a preferred option in treating advanced NSCLC in the latest NCCN guideline,¹⁹ the cost-effectiveness of pembrolizumab + CT remains inferior. Our sensitivity analysis suggested that the cost of pembrolizumab accounts inserted an essential impact on the comparisons of cost-effectiveness, indicating that a discount on pembrolizumab price may significantly improve the cost-effectiveness of pembrolizumab + CT and benefit more patients with lower costs. In short, our study provides important information for clinicians and policymakers in making treatment decisions for advanced non-squamous NSCLC patients.

This study has the following limitations. First, since CheckMate 9LA enrolled both patients with squamous NSCLC and patients with non-squamous NSCLC, the available data of different PD-L1 expression subgroups in CheckMate 9LA involved patients with both cancer types. Accordingly, we excluded NivoIpiC in the subgroup analyses by PD-L1 expression. Second, although the regimens used in the three RCTs were similar, disparities, such as differences in patients’ baseline characteristics, duration of follow-up, and courses of CT, remain. Consequently, the discrepancies led to mild deviations of predicted OS and PFS curves of IMpower132 and CheckMate 9LA from the original K–M curves when we fitted the curves using NMA-generated HRs. Third, the cost-effectiveness of atezolizumab + CT in patients with PD-L1 TPS ⩾50% was not revealed in our study due to its relatively small sample size and the fact that the median OS of this group was not reached in the original data.

Conclusion

The NMAs revealed that pembrolizumab + CT seemed to have the best efficacy compared with nivolumab + ipilimumab + CT and atezolizumab + CT in the total population with advanced non-squamous NSCLC and most subgroups with different PD-L1 expressions, and the safety profiles of the ICI plus CT combinations were similar. However, the cost-effectiveness analyses showed that from the US perspective, nivolumab + ipilimumab + CT was most favorable in the total population, and pembrolizumab + CT and atezolizumab + CT had the best cost-effectiveness in patients with PD-L1 TPS <1% and 1–49%, respectively. However, from the Chinese perspective, atezolizumab + CT was the most cost-effective strategy in the total population, patients with PD-L1 TPS <1%, and patients with PD-L1 TPS 1–49%.

Supplemental Material

sj-docx-1-tam-10.1177_17588359241255613 – Supplemental material for First-line treatments for advanced non-squamous non-small cell lung cancer with immune checkpoint inhibitors plus chemotherapy: a systematic review, network meta-analysis, and cost-effectiveness analysis

Supplemental material, sj-docx-1-tam-10.1177_17588359241255613 for First-line treatments for advanced non-squamous non-small cell lung cancer with immune checkpoint inhibitors plus chemotherapy: a systematic review, network meta-analysis, and cost-effectiveness analysis by Wentao Tian, Lishui Niu, Yin Shi, Shuishi Li and Rongrong Zhou in Therapeutic Advances in Medical Oncology

Footnotes

Acknowledgements

None.

Declarations

ORCID iDs

Wentao Tian

Rongrong Zhou

Supplemental material

Supplemental material for this article is available online.

References

Siegel

Miller

Fuchs

, et al. Cancer statistics, 2022. CA Cancer J Clin 2022; 72: 7–33.

GBD 2019 Diseases and Injuries Collaborators. Global burden of 369 diseases and injuries in 204 countries and territories, 1990–2019: a systematic analysis for the Global Burden of Disease Study 2019. Lancet 2020; 396: 1204–1222.

Langer

Besse

Gualberto

, et al. The evolving role of histology in the management of advanced non-small-cell lung cancer. J Clin Oncol 2010; 28: 5311–5320.

The effect of PD-L1 categories-directed pembrolizumab plus chemotherapy for newly diagnosed metastatic non-small-cell lung cancer: a cost-effectiveness analysis. Transl Lung Cancer Res 2020; 9: 1770–1784.

Miller

Hanna

Advances in systemic therapy for non-small cell lung cancer. BMJ 2021; 375: n2363.

Rodriguez-Abreu

Powell

Hochmair

, et al. Pemetrexed plus platinum with or without pembrolizumab in patients with previously untreated metastatic nonsquamous NSCLC: protocol-specified final analysis from KEYNOTE-189. Ann Oncol 2021; 32: 881–895.

Nishio

Barlesi

West

, et al. Atezolizumab plus chemotherapy for first-line treatment of nonsquamous NSCLC: results from the randomized phase 3 IMpower132 trial. J Thorac Oncol 2021; 16: 653–664.

Paz-Ares

Ciuleanu

Cobo

, et al. First-line nivolumab plus ipilimumab combined with two cycles of chemotherapy in patients with non-small-cell lung cancer (CheckMate 9LA): an international, randomised, open-label, phase 3 trial. Lancet Oncol 2021; 22: 198–211.

Liu

Zhou

Luo

, et al. Cost-effectiveness of pembrolizumab plus chemotherapy versus pembrolizumab monotherapy in metastatic non-squamous and squamous NSCLC patients with PD-L1 expression >/=50. Front Pharmacol 2021; 12: 803626.

10.

Cai

Hui

Zhu

, et al. Cost-effectiveness analysis of pembrolizumab plus pemetrexed and platinum versus chemotherapy alone as first-line treatment in metastatic non-squamous non-small cell lung cancer: a reconstruction of partitioned survival model based on time dependent pricing mechanism of patient assistance program. Front Oncol 2021; 11: 768035.

11.

Yang

Zhu

Xiang

, et al. First-line atezolizumab plus chemotherapy in advanced non-squamous non-small cell lung cancer: a cost-effectiveness analysis from China. Expert Rev Pharmacoecon Outcomes Res 2021; 21: 1061–1067.

12.

Lin

Luo

Zhong

, et al. Cost-effectiveness of atezolizumab plus chemotherapy for advanced non-small-cell lung cancer. Int J Clin Pharm 2020; 42: 1175–1183.

13.

Criss

Mooradian

Watson

, et al. Cost-effectiveness of atezolizumab combination therapy for first-line treatment of metastatic nonsquamous non-small cell lung cancer in the United States. JAMA Netw Open 2019; 2: e1911952.

14.

Page

McKenzie

Bossuyt

, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021; 372: n71.

15.

Kunst

Siu

Drummond

, et al. Consolidated health economic evaluation reporting standards – value of information (CHEERS-VOI): explanation and elaboration. Value Health 2023; 26: 1461–1473.

16.

Sanders

Neumann

Basu

, et al. Recommendations for conduct, methodological practices, and reporting of cost-effectiveness analyses: second panel on cost-effectiveness in health and medicine. JAMA 2016; 316: 1093–1103.

17.

Neumann

Cohen

Weinstein

MC.

Updating cost-effectiveness – the curious resilience of the $50,000-per-QALY threshold. N Engl J Med 2014; 371: 796–797.

18.

Hoyle

Henley

Improved curve fits to summary survival data: application to economic evaluation of health technologies. BMC Med Res Methodol 2011; 11: 139.

19.

NCCN. The NCCNNCCN guidelines non-small cell lung cancer (Version 2.2023). https://www.nccn.org/guidelines/guidelines-detail?category=1&id=1450 (2023, accessed 03/10 2023).

20.

Kohn

Zeichner

Chen

, et al. Cost-effectiveness of immune checkpoint inhibition in BRAF wild-type advanced melanoma. J Clin Oncol 2017; 35: 1194–1202.

21.

Liu

Zhou

Luo

, et al. First-line ICI monotherapies for advanced non-small-cell lung cancer patients with PD-L1 of at least 50%: a cost-effectiveness analysis. Front Pharmacol 2021; 12: 788569.

22.

Aguiar

Jr Haaland

Park

, et al. Cost-effectiveness of osimertinib in the first-line treatment of patients with EGFR-mutated advanced non-small cell lung cancer. JAMA Oncol 2018; 4: 1080–1084.

23.

Criss

Mooradian

Sheehan

, et al. Cost-effectiveness and budgetary consequence analysis of durvalumab consolidation therapy vs no consolidation therapy after chemoradiotherapy in stage III non-small cell lung cancer in the context of the US Health Care System. JAMA Oncol 2019; 5: 358–365.

24.

Insinga

Vanness

Feliciano

, et al. Cost-effectiveness of pembrolizumab in combination with chemotherapy in the 1st line treatment of non-squamous NSCLC in the US. J Med Econ 2018; 21: 1191–1205.

25.

Insinga

Vanness

Feliciano

, et al. Cost-effectiveness of pembrolizumab in combination with chemotherapy versus chemotherapy and pembrolizumab monotherapy in the first-line treatment of squamous non-small-cell lung cancer in the US. Curr Med Res Opin 2019; 35: 1241–1256.

26.

Klein

Wielage

Muehlenbein

, et al. Cost-effectiveness of pemetrexed as first-line maintenance therapy for advanced nonsquamous non-small cell lung cancer. J Thorac Oncol 2010; 5: 1263–1272.

27.

Shi

Cost-effectiveness of atezolizumab plus bevacizumab vs sorafenib as first-line treatment of unresectable hepatocellular carcinoma. JAMA Netw Open 2021; 4: e210037.

28.

Zhang

Wang

Shi

, et al. Cost-effectiveness of atezolizumab plus bevacizumab vs sorafenib for patients with unresectable or metastatic hepatocellular carcinoma. JAMA Netw Open 2021; 4: e214846.

29.

Chouaid

Agulnik

Goker

, et al. Health-related quality of life and utility in patients with advanced non-small-cell lung cancer: a prospective cross-sectional patient survey in a real-world setting. J Thorac Oncol 2013; 8: 997–1003.

30.

Nafees

Lloyd

Dewilde

, et al. Health state utilities in non-small cell lung cancer: an international study. Asia Pac J Clin Oncol 2017; 13: e195–e203.

31.

Wan

, et al. First-line atezolizumab in addition to bevacizumab plus chemotherapy for metastatic, nonsquamous non-small cell lung cancer: a United States-based cost-effectiveness analysis. Cancer 2019; 125: 3526–3534.

32.

Centers for Medicare & Medicaid Services. ASP drug pricing files, https://www.cms.gov/medicare/payment/part-b-drugs/asp-pricing-files (2021, accessed 14 April 2022).

33.

Centers for Medicare & Medicaid Services. 2020 physician fee schedule search. https://www.cms.gov/medicare/physician-fee-schedule/search (2020, accessed 18 February 2020).

34.

Liu

Bai

Wang

, et al. Efficacy and safety of first-line immunotherapy combinations for advanced NSCLC: a systematic review and network meta-analysis. J Thorac Oncol 2021; 16: 1099–1117.

35.

Teng

Chen

Yang

, et al. Determining the optimal PD-1/PD-L1 inhibitors for the first-line treatment of non-small-cell lung cancer with high-level PD-L1 expression in China. Cancer Med 2021; 10: 6344–6353.

36.

Barbier

Pardo

Panje

, et al.; Swiss Group for Clinical Cancer Research (SAKK). A cost-effectiveness analysis of pembrolizumab with or without chemotherapy for the treatment of patients with metastatic, non-squamous non-small cell lung cancer and high PD-L1 expression in Switzerland. Eur J Health Econ 2021; 22: 669–677.

37.

Jiang

Wang

Cost-effectiveness analysis of pembrolizumab plus standard chemotherapy versus chemotherapy alone for first-line treatment of metastatic non-squamous non-small-cell lung cancer in China. Eur J Hosp Pharm 2022; 29: 139–144.

38.

Ding

Liao

, et al. Cost-effectiveness analysis of atezolizumab plus chemotherapy in the first-line treatment of metastatic non-squamous non-small cell lung cancer. Adv Ther 2020; 37: 2116–2126.

39.

Yang

, et al. Cost-effectiveness of first-line immunotherapies for advanced non-small cell lung cancer. Cancer Med 2023; 12: 8838–8850.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

11.81 MB