Generalizing the per-protocol treatment effect: The case of ACTG A5095

Abstract

Background

Intention-to-treat comparisons of randomized trials provide asymptotically consistent estimators of the effect of treatment assignment, without regard to compliance. However, decision makers often wish to know the effect of a per-protocol comparison. Moreover, decision makers may also wish to know the effect of treatment assignment or treatment protocol in a user-specified target population other than the sample in which the trial was fielded. Here, we aimed to generalize results from the ACTG A5095 trial to the US recently HIV-diagnosed target population.

Methods

We first replicated the published conventional intention-to-treat estimate (2-year risk difference and hazard ratio) comparing a four-drug antiretroviral regimen to a three-drug regimen in the A5095 trial. We then estimated the intention-to-treat effect that accounted for informative dropout and the per-protocol effect that additionally accounted for protocol deviations by constructing inverse probability weights. Furthermore, we employed inverse odds of sampling weights to generalize both intention-to-treat and per-protocol effects to a target population comprising US individuals with HIV diagnosed during 2008–2014.

Results

Of 761 subjects in the analysis, 82 dropouts (36 in the three-drug arm and 46 in the four-drug arm) and 59 protocol deviations (25 in the three-drug arm and 34 in the four-drug arm) occurred during the first 2 years of follow-up. A total of 169 subjects incurred virologic failure or death. The 2-year risks were similar both in the trial and in the US HIV-diagnosed target population for estimates from the conventional intention-to-treat, dropout-weighted intention-to-treat, and per-protocol analyses. In the US target population, the 2-year conventional intention-to-treat risk difference (unit: %) for virologic failure or death comparing the four-drug arm to the three-drug arm was −0.4 (95% confidence interval: −6.2, 5.1), while the hazard ratio was 0.97 (95% confidence interval: 0.70, 1.34); the 2-year risk difference was −0.9 (95% confidence interval: −6.9, 5.3) for the dropout-weighted intention-to-treat comparison (hazard ratio = 0.95, 95% confidence interval: 0.68, 1.32) and −0.7 (95% confidence interval: −6.7, 5.5) for the per-protocol comparison (hazard ratio = 0.96, 95% confidence interval: 0.69, 1.34).

Conclusion

No benefit of four-drug antiretroviral regimen over three-drug regimen was found from the conventional intention-to-treat, dropout-weighted intention-to-treat or per-protocol estimates in the trial sample or target population.

Keywords

HIV/AIDS virologic failure clinical trial per-protocol effect generalizability antiretroviral therapy intention-to-treat effect external validity inverse probability weighting causality

Get full access to this article

View all access options for this article.

References

Pocock

SJ.

Clinical trials: a practical approach. New York: John Wiley and Sons, 1983.

Friedman

Furberg

DeMets

DL.

Fundamentals of clinical trials. 2nd ed.Boston, MA: PSG Inc., 1985.

Peto

Pike

Armitage

et al . Design and analysis of randomized clinical trials requiring prolonged observation of each patient. II. Analysis and examples. Br J Cancer 1977; 35: 1–39.

Hernán

Hernandez-Diaz

Beyond the intention-to-treat in comparative effectiveness research. Clin Trials 2012; 9: 48–55.

Hernán

Robins

JM.

Per-protocol analyses of pragmatic trials. N Engl J Med 2017; 377: 1391–1398.

Robins

Finkelstein

DM.

Correcting for noncompliance and dependent censoring in an AIDS Clinical Trial with inverse probability of censoring weighted (IPCW) log-rank tests. Biometrics 2000; 56: 779–788.

Murray

Hernán

MA.

Adherence adjustment in the Coronary Drug Project: a call for better per-protocol effect estimates in randomized trials. Clin Trials 2016; 13: 372–378.

Keiding

Louis

TA.

Perils and potentials of self-selected entry to epidemiological studies and surveys. J R Stat Soc Ser A Stat Soc 2016; 179: 319–376.

Lesko

Buchanan

Westreich

et al . Generalizing study results: a potential outcomes perspective. Epidemiology 2017; 28: 553–561.

10.

Cole

Stuart

EA.

Generalizing evidence from randomized clinical trials to target populations: the ACTG 320 trial. Am J Epidemiol 2010; 172: 107–115.

11.

Stuart

Cole

Bradshaw

et al . The use of propensity scores to assess the generalizability of results from randomized trials. J R Stat Soc Ser A Stat Soc 2011; 174: 369–386.

12.

Stuart

Bradshaw

Leaf

PJ.

Assessing the generalizability of randomized trial results to target populations. Prev Sci 2015; 16: 475–485.

13.

Pearl

Bareinboim

External validity: from do-calculus to transportability across populations. Stat Sci 2012; 29: 579–595.

14.

Buchanan

Hudgens

Cole

et al . Generalizing evidence from randomized trials using inverse probability of sampling weights. J R Stat Soc Ser A Stat Soc 2018; 181: 1193–1209.

15.

O’Muircheartaigh

Hedges

LV.

Generalizing from unrepresentative experiments: a stratified propensity score approach. J R Stat Soc Ser C Appl Stat 2014; 63: 195–210.

16.

Westreich

Edwards

Lesko

et al . Transportability of trial results using inverse odds of sampling weights. Am J Epidemiol 2017; 188: 1010–1014.

17.

Lumley

Complex surveys: a guide to analysis using R. Hoboken, NJ: Wiley, 2010.

18.

Gulick

Ribaudo

Shikuma

et al . Three- vs four-drug antiretroviral regimens for the initial treatment of HIV-1 infection: a randomized controlled trial. J Am Med Assoc 2006; 296: 769–781.

19.

Robins

Greenland

Identification of causal effects using instrumental variables: comment. J Am Stat Assoc 1996; 91: 456–458.

20.

Hernán

Alonso

Logan

et al . Observational studies analyzed like randomized experiments: an application to postmenopausal hormone therapy and coronary heart disease. Epidemiology 2013; 19: 766–779.

21.

Centers for Disease Control and Prevention. HIV surveillance report, vol. 28, 2016, http://www.cdc.gov/hiv/library/reports/hiv-surveillance.html (accessed 9 January 2018).

22.

Gulick

Ribaudo

Shikuma

et al . Triple-nucleoside regimens versus efavirenz-containing regimens for the initial treatment of HIV-1 infection. N Engl J Med 2004; 350: 1850–1861.

23.

Cox

DR.

Regression models and life-tables. J R Stat Soc Ser B Methodol 1972; 34: 187–220.

24.

Cox

DR.

Partial likelihood. Biometrika 1975; 62: 269–276.

25.

Efron

The efficiency of Cox’s likelihood function for censored data. J Am Stat Assoc 1977; 72: 557–565.

26.

Cain

Cole

SR.

Inverse probability-of-censoring weights for the correction of time-varying noncompliance in the effect of randomized highly active antiretroviral therapy on incident AIDS or death. Stat Med 2009; 28: 1725–1738.

27.

Division of AIDS, National Institute of Allergy and Infectious Diseases, National Institutes of Health, U.S. Department of Health and Human Services. Division of AIDS (DAIDS) table for grading the severity of adult and pediatric adverse events, corrected version 2.1, https://rsc.tech-res.com/docs/default-source/safety/division-of-aids-(daids)-table-for-grading-the-severity-of-adult-and-pediatric-adverse-events-corrected-v-2-1.pdf (accessed 9 January 2018).

28.

Howe

Cole

Westreich

et al . Splines for trend analysis and continuous confounder control. Epidemiology 2011; 22: 874–875.

29.

Harrell

FE.

Regression modeling strategies: with applications to linear models, logistic regression, and survival analysis (Springer series in statistics). New York: Springer, 2001.

30.

Cole

Hernán

MA.

Constructing inverse probability weights for marginal structural models. Am J Epidemiol 2008; 168: 656–664.

31.

Buchanan

Hudgens

Cole

et al . Worth the weight: using inverse probability weighted Cox models in AIDS research. AIDS Res Hum Retroviruses 2014; 30: 1170–1177.

32.

Westreich

Cole

SR.

Invited commentary: positivity in practice. Am J Epidemiol 2010; 171: 674–677.

33.

Hudgens

Halloran

ME.

Toward causal inference with interference. J Am Stat Assoc 2008; 103: 832–842.

34.

Tchetgen

EJT

Vanderweele

. On causal inference in the presence of interference. Stat Methods Med Res 2012; 21: 55–75.

35.

VanderWeele

TJ.

Concerning the consistency assumption in causal inference. Epidemiology 2009; 20: 880–883.

36.

Kaplan

Meier

Nonparametric estimation from incomplete observations. J Am Stat Assoc 1958; 53: 457–481.

37.

Cole

Hernán

MA.

Adjusted survival curves with inverse probability weights. Comput Methods Programs Biomed 2004; 75: 45–49.

38.

Efron

Tibshirani

RJ.

An introduction to the bootstrap. Boca Raton, FL: Chapman & Hall, 1993.

39.

Robins

Hernán

Brumback

Marginal structural models and causal inference in epidemiology. Epidemiology 2000; 11: 550–560.

40.

Cohen

Gray

Ocfemia

et al . The status of the National HIV Surveillance System, United States, 2013. Public Heal Rep 2014; 129: 335–341.

41.

Toh

Hernández-Díaz

Logan

et al . Estimating absolute risks in the presence of nonadherence: an application to a follow-up study with baseline randomization. Epidemiology 2010; 21: 528–539.

42.

Lodi

Sharma

Lundgren

et al . The per-protocol effect of immediate vs. deferred ART initiation in the START randomized trial. AIDS 2016; 30: 2659–2663.

43.

Greenland

An introduction to instrumental variables for epidemiologists. Int J Epidemiol 2000; 29: 722–729.

44.

Hernán

Robins

JM.

Instruments for causal inference: an epidemiologist’s dream?

Epidemiology 2006; 17: 360–372.

45.

Swanson

Hernán

Miller

et al . Partial identification of the average treatment effect using instrumental variables: review of methods for binary instruments, treatments, and outcomes. J Am Stat Assoc 2018; 113: 933–947.

46.

Lau

Cole

Gange

SJ.

Competing risk regression models for epidemiologic data. Am J Epidemiol 2009; 170: 244–256.

47.

Lunceford

Davidian

Stratification and weighting via the propensity score in estimation of causal treatment effects: a comparative study. Stat Med 2004; 23: 2937–2960.