Estimating Treatment-Switching Bias in a Randomized Clinical Trial of Ovarian Cancer Treatment: Combining Causal Inference with Decision-Analytic Modeling
Restricted accessResearch articleFirst published online February, 2022
Estimating Treatment-Switching Bias in a Randomized Clinical Trial of Ovarian Cancer Treatment: Combining Causal Inference with Decision-Analytic Modeling
Bevacizumab is efficacious in delaying ovarian cancer progression and controlling ascites. The ICON7 trial showed a significant benefit in overall survival for bevacizumab, whereas the GOG-218 trial did not. GOG-218 allowed control group patients to switch to bevacizumab upon progression, which may have biased the results. Lack of data on switching behavior prevented the application of g-methods to adjust for switching. The objective of this study was to apply decision-analytic modeling to estimate the impact of switching bias on causal treatment-effect estimates.
Methods
We developed a causal decision-analytic Markov model (CDAMM) to emulate the GOG-218 trial and estimate overall survival. CDAMM input parameters were based on data from randomized clinical trials and the published literature. Overall switching proportion was based on GOG-218 trial information, whereas the proportion switching with and without ascites was estimated using calibration. We estimated the counterfactual treatment effect that would have been observed had no switching occurred by denying switching in the CDAMM.
Results
The survival curves generated by the CDAMM matched well with the ones reported in the GOG-218 trial. The survival curve correcting for switching showed an estimated bias such that 79% of the true treatment effect could not be observed in the GOG-218 trial. Results were most sensitive to changes in the proportion progressing with severe ascites and mortality.
Limitations
We used a simplified model structure and based model parameters on published data and assumptions. Robustness of the CDAMM was tested and model assumptions transparently reported.
Conclusions
Medical-decision science methods may be merged with empirical methods of causal inference to integrate data from other sources where empirical data are not sufficient. We recommend collecting sufficient information on switching behavior when switching cannot be avoided.
GoodmanMTCorreaCNTungKH, et al. Stage at diagnosis of ovarian cancer in the United States, 1992-1997. Cancer. 2003;97(10 suppl):2648–59.
4.
StirlingDEvansDGRPichertG, et al. Screening for familial ovarian cancer: failure of current protocols to detect ovarian cancer at an early stage according to the International Federation of Gynecology and Obstetrics system. J Clin Oncol. 2005;23(24):5588–96.
5.
National Comprehensive Cancer Network. Clinical Practice Guideline in Oncology (NCCN Guidelines): Ovarian Cancer Including Fallopian Tube Cancer and Primary Peritoneal Cancer. Version 3.2014. Available from: www.nccn.org/patient
6.
AghajanianCBlankSVGoffBA, et al. OCEANS: a randomized, double-blind, placebo-controlled phase III trial of chemotherapy with or without bevacizumab in patients with platinum-sensitive recurrent epithelial ovarian, primary peritoneal, or fallopian tube cancer. J Clin Oncol. 2012;30(17):2039–45.
7.
AghajanianCGoffBNycumLRWangYVHusainABlankSV.Final overall survival and safety analysis of OCEANS, a phase 3 trial of chemotherapy with or without bevacizumab in patients with platinum-sensitive recurrent ovarian cancer. Gynecol Oncol. 2015;139(1):10–6.
8.
BamiasAGibbsEKhoon LeeC, et al. Bevacizumab with or after chemotherapy for platinum-resistant recurrent ovarian cancer: exploratory analyses of the AURELIA trial. Ann Oncol. 2017;28(8):1842–8.
9.
BurgerRABradyMFBookmanMA, et al. Incorporation of bevacizumab in the primary treatment of ovarian cancer. N Engl J Med. 2011;365(26):2473–83.
10.
ColemanRLBradyMFHerzogTJ, et al. Bevacizumab and paclitaxel-carboplatin chemotherapy and secondary cytoreduction in recurrent, platinum-sensitive ovarian cancer (NRG Oncology/Gynecologic Oncology Group study GOG-0213): a multicentre, open-label, randomised, phase 3 trial. Lancet Oncol. 2017;18(6):779–1.
11.
OzaAMCookADPfistererJ, et al. Standard chemotherapy with or without bevacizumab for women with newly diagnosed ovarian cancer (ICON7): overall survival results of a phase 3 randomised trial. Lancet Oncol. 2015;16(8):928–36.
12.
PerrenTJSwartAMPfistererJ, et al. A phase 3 trial of bevacizumab in ovarian cancer. N Engl J Med. 2011;365(26):2484–96.
13.
Pujade-LauraineEHilpertFWeberB, et al. Bevacizumab combined with chemotherapy for platinum-resistant recurrent ovarian cancer: The AURELIA open-label randomized phase III trial. J Clin Oncol. 2014;32(13):1302–8.
14.
WuYSShuiLShenDChenX.Bevacizumab combined with chemotherapy for ovarian cancer: an updated systematic review and meta-analysis of randomized controlled trials. Oncotarget. 2017;8(6):10703–13.
15.
OzaAMCookADKaplanRParmarMKBPerrenTJ; ICON7 Investigators. Discordance between the results and conclusions of ICON7: authors’ reply. Lancet Oncol. 2015;16(13):e478–9.
16.
BellHLatimerNAmonkarMSwannS.Adjusting for treatment switching in RCTs: identifying, analysing and justifying appropriate methods: a case study in metastatic melanoma. Value Health. 2015;18(7):A338.
17.
LatimerNR.Treatment switching in oncology trials and the acceptability of adjustment methods. Expert Rev Pharmacoecon Outcomes Res. 2015;15(4):561–4.
18.
LatimerNRAbramsKRLambertPC, et al. Adjusting for treatment switching in randomised controlled trials - A simulation study and a simplified two-stage method. Stat Methods Med Res. 2017;26(2):724–51.
19.
LatimerNRAbramsKRAmonkarMMStapelkampCSwannRS, Adjusting for the confounding effects of treatment switching—the BREAK-3 Trial: dabrafenib versus dacarbazine. Oncologist. 2015;20(7):798–805.
20.
LatimerNRAbramsKRLambertPC, et al. Adjusting survival time estimates to account for treatment switching in randomized controlled trials: an economic evaluation context: methods, limitations, and recommendations. Med Decis Making. 2014;34(3):387–402.
21.
LatimerNRAbramsKRLambertPCMordenJPCrowtherMJ, Assessing methods for dealing with treatment switching in clinical trials: A follow-up simulation study. Stat Methods Med Res. 2018;27(3):765–84.
22.
LatimerNRHenshallCSiebertUBellH.Treatment switching: statistical and decision-making challenges and approaches. Int J Technol Assess Health Care. 2016;32(3):160–6.
23.
MordenJPLambertPCLatimerNAbramsKRWailooAJ.Assessing methods for dealing with treatment switching in randomised controlled trials: a simulation study. BMC Med Res Methodol. 2011;11:4.
24.
WatkinsCHuangXLatimerNTangYWrightEJ.Adjusting overall survival for treatment switches: commonly used methods and practical application. Pharm Stat. 2013;12(6):348–57.
25.
WhiteIR.Methods for handling treatment switching: rank-preserving structural nested failure time models, inverseprobability-of-censoring weighting, and marginal structural models. Workshop: Methods for Adjusting for Treatment Switches in Late-Stage Cancer Trials; London, UK; February2012.
26.
PazdurR.Endpoints for assessing drug activity in clinical trials. Oncologist. 2008;13(suppl 2):19–21.
27.
StuartGCKitchenerHBaconM, et al. 2010Gynecologic Cancer InterGroup (GCIG) consensus statement on clinical trials in ovarian cancer: report from the Fourth Ovarian Cancer Consensus Conference. Int J Gynecol Cancer. 2011;21(4):750–5.
28.
BullementALatimerNRGorrodHB.Survival extrapolation in cancer immunotherapy: a validation-based case study. Value Health. 2019;22(3):276–83.
29.
GorrodHBLatimerNRDamianDHettleRHartyGTWongSL.Impact of nonrandomized dropout on treatment switching adjustment in the relapsing-remitting multiple sclerosis CLARITY trial and the CLARITY extension study. Value Health. 2019;22(7):772–6.
30.
National Institute of Health and Care Excellence. Sunitinib for the treatment of gastrointestinal stromal tumours. NICE Technology Appraisals [TA179]. London: NICE; 2009.
31.
National Institute of Health and Care Excellence. Pazopanib for the first-line treatment of advanced renal cell carcinoma. NICE Technology Appraisals [TA215]. London: NICE; 2011.
32.
National Institute of Health and Care Excellence. Everolimus for the second-line treatment of advanced renal cell carcinoma. NICE Technology Appraisals [TA219]. London: NICE; 2011.
33.
JonssonLSandinREkmanM, et al. Analyzing overall survival in randomized controlled trials with crossover and implications for economic evaluation. Value Health. 2014;17(6):707–13.
34.
OuwensMMukhopadhyayPZhangYHuangMLatimerNBriggsA.Estimating lifetime benefits associated with immuno-oncology therapies: challenges and approaches for overall survival extrapolations. Pharmacoeconomics. 2019;37(9):1129–38.
35.
BullementAMengYCooperM, et al. A review and validation of overall survival extrapolation in health technology assessments of cancer immunotherapy by the National Institute for Health and Care Excellence: how did the initial best estimate compare to trial data subsequently made available?J Med Econ. 2019;22(3):205–14.
36.
LatimerNRWhiteIRTillingKSiebertU.Improved two-stage estimation to adjust for treatment switching in randomised trials: g-estimation to address time-dependent confounding. Stat Methods Med Res. 2020;29(10):2900–18.
37.
LatimerNRWhiteIRAbramsKRSiebertU.Causal inference for long-term survival in randomised trials with treatment switching: should re-censoring be applied when estimating counterfactual survival times?Stat Methods Med Res. 2019;28(8):2475–493.
38.
LatimerNRAbramsKRSiebertU.Two-stage estimation to adjust for treatment switching in randomised trials: a simulation study investigating the use of inverse probability weighting instead of re-censoring. BMC Med Res Methodol. 2019;19(1):69.
39.
HenshallCLatimerNRSansomLWardRL.Treatment switching in cancer trials: issues and proposals. Int J Technol Assess Health Care. 2016;32(3):167–74.
40.
SmolleETaucherVHaybaeckJ.Malignant ascites in ovarian cancer and the role of targeted therapeutics. Anticancer Res. 2014;34(4):1553–61.
41.
ArikanSKKasapBYetimalarHYildizASakaryaDKTatarS.Impact of prognostic factors on survival rates in patients with ovarian carcinoma. Asian Pac J Cancer Prev. 2014;15(15):6087–94.
42.
KoboldSHegewisch-BeckerSOechsleKJordanKBokemeyerCAtanackovicD.Intraperitoneal VEGF inhibition using bevacizumab: a potential approach for the symptomatic treatment of malignant ascites?Oncologist. 2009;14(12):1242–51.
43.
HuninkMWeinsteinMCWittenbergM, et al. Valuing outcomes. In: HuninkMWeinsteinMCWittenbergM, et al., eds. Decision Making in Health and Medicine: Integrating Evience and Values. New York: Cambridge University Press; 2012. p 88–127.
44.
SiebertU.When should decision analytic modeling be used in the economic evaluation of health care?Eur J Health Econ. 2003;4:143–50.
45.
SiebertUAlagozOBayoumiAM, et al. State-transition modeling: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3. Value Health. 2012;15(6):812–20.
46.
SiebertUAlagozOBayoumiAM, et al. State-transition modeling: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3. Med Decis Making. 2012;32(5):690–700.
47.
GrayAMClarkePMWolstenholmeJWordsworthS, Applied methods of cost-effectiveness analysis in health care. In: Handbooks in Health Economic Evaluation Series. New York: Oxford University Press; 2011.
48.
GreenlandSPearlJRobinsJM.Causal diagrams for epidemiologic research. Epidemiology. 1999;10(1):37–48.
49.
CaroJJBriggsAHSiebertUKuntzKM; ISPOR-SMDM Modeling Good Research Practices Task Force. Modeling good research practices—overview: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-1. Med Decis Making. 2012;32(5):667–77.
50.
CaroJJBriggsAHSiebertU, et al. Modeling good research practices—overview: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-1. Value Health. 2012;15:796–803.
51.
EddyD, DMHollingworthWCaroJJ, et al. Model transparency and validation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-7 [plus editorial]. Med Decis Making. 2012;32:733–43.
52.
EddyDHollingworthWCaroJJ, et al. Model transparency and validation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-7. Value Health. 2012;15:843–50.
53.
EskanderRNTewariKS.Emerging treatment options for management of malignant ascites in patients with ovarian cancer. Int J Womens Health. 2012;4:395–404.
54.
PovedaAMSelleFHilpertF, et al. Bevacizumab combined with weekly paclitaxel, pegylated liposomal doxorubicin, or topotecan in platinum-resistant recurrent ovarian cancer: analysis by chemotherapy cohort of the randomized phase III AURELIA trial. J Clin Oncol. 2015;33(32):3836–8.
55.
KristensenGBerton-RigaudDHilpertF, et al., editors. Effect of bevacizumab combined with chemotherapy for platinum-resistant recurrent ovarian cancer with ascites: Analysis of the AURELIA trial. International Gynecologic Cancer Society (IGCS) Conference; 2012 October 13-16; Vancouver, Canada.
56.
El-ShamiKElsaidAEl-KermY.Open-label safety and efficacy pilot trial of intraperitoneal bevacizumab as palliative treatment in refractory malignant ascites. J Clin Oncol. 2007;25(18 suppl):9043.
57.
HamiltonCAMaxwellGLChernofskyMRBernsteinSAFarleyJHRoseGS.Intraperitoneal bevacizumab for the palliation of malignant ascites in refractory ovarian cancer. Gynecol Oncol. 2008;111(3):530–2.
58.
KestersonJPMhawech-FaucegliaPLeleS.The use of bevacizumab in refractory ovarian granulosa-cell carcinoma with symptomatic relief of ascites: a case report. Gynecol Oncol. 2008;111(3):527–9.
59.
NumnumTMRocconiRPWhitworthJBarnesMN.The use of bevacizumab to palliate symptomatic ascites in patients with refractory ovarian carcinoma. Gynecol Oncol. 2006;102(3):425–8.
60.
SafraTRonIBoazM, et al. Heavily pretreated ovarian cancer patients treated by single-agent gemcitabine: a retrospective outcome comparison between platinum-sensitive and platinum-resistant patients. Acta Oncol. 2006;45(4):463–8.
61.
BeckJRPaukerSG.The Markov process in medical prognosis. Med Decis Making. 1983;3(4):419–58.
62.
BeckJRPaukerSGGottliebJEKleinKKassirerJP.A convenient approximation of life expectancy (the “DEALE”). II. Use in medical decision-making. Am J Med. 1982;73(6):889–97.
63.
LodiSPhillipsALundgrenJ, et al. Effect estimates in randomized trials and observational studies: comparing apples with apples. Am J Epidemiol. 2019;188(8):1569–77.
64.
SchomakerMKühneFSiebertU.RE: “Effect estimates in randomized trials and observational studies: comparing apples with apples.”Am J Epidemiol. 2020;189(1):77–8.
65.
MehtaDAHayJW.Cost-effectiveness of adding bevacizumab to first line therapy for patients with advanced ovarian cancer. Gynecol Oncol. 2014;132(3):677–83.
66.
BarnettJCSecordAACohnDELeathCAIIIMyersERHavrileskyLJ.Cost effectiveness of alternative strategies for incorporating bevacizumab into the primary treatment of ovarian cancer. Cancer. 2013;119(20):3653–61.
67.
HindeSEpsteinDCookAEmbletonAPerrenTSculpherM.The cost-effectiveness of bevacizumab in advanced ovarian cancer using evidence from the ICON7 trial. Value Health. 2016;19(4):431–9.
68.
DuongMWrightEYinLMartin-NunezIGhatagePFung-Kee-FungM.The cost-effectiveness of bevacizumab for the treatment of advanced ovarian cancer in Canada. Curr Oncol. 2016;23(5):e461–7.
69.
CohnDEBarnettJCWenzelL, et al. A cost-utility analysis of NRG Oncology/Gynecologic Oncology Group Protocol 218: incorporating prospectively collected quality-of-life scores in an economic model of treatment of ovarian cancer. Gynecol Oncol. 2015;136(2):293–9.
70.
BallGXieFTarrideJE.Economic evaluation of bevacizumab for treatment of platinum-resistant recurrent ovarian cancer in Canada. Pharmacoecon Open. 2018;2(1):19–29.
71.
ChanJKHerzogTJHuL, et al. Bevacizumab in treatment of high-risk ovarian cancer—a cost-effectiveness analysis. Oncologist. 2014;19(5):523–7.
72.
MonkBJPujade-LauraineEBurgerRA.Integrating bevacizumab into the management of epithelial ovarian cancer: the controversy of front-line versus recurrent disease. Ann Oncol. 2013;24(suppl 10):x53–8.
73.
RandallLMBurgerRANguyenH, et al., editors. Outcome differences in patients with stage IV epithelial ovarian, primary peritoneal and fallopian tube cancers treated with and without bevacizumab. Annual Meeting of the Society of Gynecology; 2013; Los Angeles, CA.
74.
MotzerRJEscudierBOudardS, et al. Phase 3 trial of everolimus for metastatic renal cell carcinoma: final results and analysis of prognostic factors. Cancer. 2010;116(18):4256–65.
75.
GreenlandSLanesSJaraM.Estimating effects from randomized trials with discontinuations: the need for intent-to-treat design and G-estimation. Clin Trials. 2008;5(1):5–13.
76.
RobinsJMFinkelsteinDM.Correcting for noncompliance and dependent censoring in an AIDS clinical trial with inverse probability of censoring weighted (IPCW) log-rank tests. Biometrics. 2000;56(3):779–88.
HernanMAHernandez-DiazS.Beyond the intention-to-treat in comparative effectiveness research. Clin Trials. 2012;9(1):48–55.
79.
HernanMAColeSRMargolickJCohenMRobinsJM.Structural accelerated failure time models for survival analysis in studies with time-varying treatments. Pharmacoepidemiol Drug Saf. 2005;14(7):477–91.
80.
HernanMABrumbackBRobinsJM.Marginal structural models to estimate the causal effect of zidovudine on the survival of HIV-positive men. Epidemiology. 2000;11(5):561–70.
81.
DanielRMCousensSNDe StavolaBLKenwardMGSterneJA.Methods for dealing with time-dependent confounding. Stat Med. 2013;32(9):. 1584–618.
82.
ColeSRHernanMA.Fallibility in estimating direct effects. Int J Epidemiol. 2002;31(1):163–5.
83.
BellamySLLinJYTen HaveTR.An introduction to causal modeling in clinical trials. Clin Trials. 2007;4(1):58–73.
84.
RobinsJMHernánMASiebertU.Estimations of the effects of multiple interventions. In: EzzatiMLopezADRodgersAAMurrayCJL, eds. Comparative Quantification of Health Risks: Global and Regional Burden of Disease Attributable to Selected Major Risk Factors, Geneva: World Health Organization; 2004. p 2191–230.
85.
KuehneFBeateJConrads-FrankA, et al. Guidance for a causal comparative effectiveness analysis emulating a target trial based on big real world evidence: when to start statin treatment. J Comp Eff Res. 2019;8(12):1013–25.
86.
HernanMASauerBCHernández-DíazSPlattRShrierI.Specifying a target trial prevents immortal time bias and other self-inflicted injuries in observational analyses. J Clin Epidemiol. 2016;79:70–5.
87.
HernanMARobinsJM.Per-protocol analyses of pragmatic trials. N Engl J Med. 2017;377(14):1391–8.
88.
HernanMARobinsJM.Using big data to emulate a target trial when a randomized trial is not available. Am J Epidemiol. 2016;183(8):758–64.
89.
HernanMARobinsJM.Estimating causal effects from epidemiological data. J Epidemiol Community Health. 2006;60(7):578–86.
90.
CainLESaagMSPetersenM, et al. Using observational data to emulate a randomized trial of dynamic treatment-switching strategies: an application to antiretroviral therapy. Int J Epidemiol. 2016;45(6):2038–49.
91.
CainLERobinsJMLanoyELoganRCostagliolaDHernánMA.When to start treatment? A systematic approach to the comparison of dynamic regimes using observational data. Int J Biostat. 2010;6(2):Article 18.
92.
CainLECanigliaECPhillipsA, et al. Efavirenz versus boosted atazanavir-containing regimens and immunologic, virologic, and clinical outcomes: a prospective study of HIV-positive individuals. Medicine (Baltimore). 2016;95(41):e5133.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
