Sage Journals: Discover world-class research

Abstract

Background/aims

Non-inferiority trials with time-to-event outcomes are becoming increasingly common. Designing non-inferiority trials is challenging, in particular, they require very large sample sizes. We hypothesized that the difference in restricted mean survival time, an alternative to the hazard ratio, could lead to smaller required sample sizes.

Methods

We show how to convert a margin for the hazard ratio into a margin for the difference in restricted mean survival time and how to calculate the required sample size under a Weibull survival distribution. We systematically selected non-inferiority trials published between 2013 and 2016 in seven major journals. Based on the protocol and article of each trial, we determined the clinically relevant time horizon of interest. We reconstructed individual patient data for the primary outcome and fit a Weibull distribution to the comparator arm. We converted the margin for the hazard ratio into the margin for the difference in restricted mean survival time. We tested for non-inferiority using the difference in restricted mean survival time and hazard ratio. We determined the required sample size based on both measures, using the type I error risk and power from the original trial design.

Results

We included 35 trials. We found evidence of non-proportional hazards in five (14%) trials. The hazard ratio and the difference in restricted mean survival time were consistent regarding non-inferiority testing, except in one trial where the difference in restricted mean survival time led to evidence of non-inferiority while the hazard ratio did not. The median hazard ratio margin was 1.43 (Q1–Q3, 1.29–1.75). The median of the corresponding margins for the difference in restricted mean survival time was −21 days (Q1–Q3, −36 to −8) for a median time horizon of 2.0 years (Q1–Q3, 1–3 years). The required sample size according to the difference in restricted mean survival time was smaller in 71% of trials, with a median relative decrease of 8.5% (Q1–Q3, 0.4%–38.0%). Across all 35 trials, about 25,000 participants would have been spared from enrollment using the difference in restricted mean survival time compared to hazard ratio for trial design.

Conclusion

The margins for the hazard ratio may seem large but translate to relatively small differences in restricted mean survival time. The difference in restricted mean survival time offers meaningful interpretation and can result in considerable reductions in sample size. Restricted mean survival time-based measures should be considered more widely in the design and analysis of non-inferiority trials with time-to-event outcomes.

Keywords

Randomized controlled trial research design sample size survival analysis

Get full access to this article

View all access options for this article.

References

Piaggio

Elbourne

Pocock

et al . Reporting of noninferiority and equivalence randomized trials: extension of the CONSORT 2010 statement. JAMA 2012; 308: 2594–2604.

Mulla

Scott

Jackevicius

et al . How to use a noninferiority trial: users’ guides to the medical literature. JAMA 2012; 308: 2605–2611.

Food and Drug Administration and Center for Drug Evaluation and Research. Non-inferiority clinical trials to establish effectiveness: Guidance for industry. Silver Spring, MD: U.S. Food and Drug Administration, 2016.

Noninferiority testing in clinical trials: issues and challenges. 1st ed.Boca Raton, FL: CRC Press, 2015, p. 184.

Rothmann

Wiens

Chan

ISF.

Design and analysis of non-inferiority trials. 1st ed.Boca Raton, FL: CRC Press.

Committee for Medicinal Products for Human Use (CHMP). Guideline on the choice of the non-inferiority margin. Stat Med 2006; 25: 1628–1638.

Snapinn

SM.

Noninferiority trials. Curr Control Trials Cardiovasc Med 2000; 1: 19–21.

Eichler

Bloechl-Daum

Abadie

et al . Relative efficacy of drugs: an emerging issue between regulatory agencies and third-party payers. Nat Rev Drug Discov 2010; 9: 277–291.

Gotzsche

PC.

Lessons from and cautions about noninferiority and equivalence randomized trials. JAMA 2006; 295: 1172–1174.

10.

Schulz

Grimes

DA.

Sample size calculations in randomised trials: mandatory and mystical. Lancet 2005; 365: 1348–1353.

11.

Spiegelhalter

Freedman

LS.

A predictive approach to selecting the size of a clinical trial, based on subjective clinical opinion. Stat Med 1986; 5: 1–13.

12.

Flacco

Manzoli

Ioannidis

JP.

Noninferiority is almost certain with lenient noninferiority margins. J Clin Epidemiol 2016; 71: 118.

13.

Flacco

Manzoli

Boccia

et al . Head-to-head randomized trials are mostly industry sponsored and almost always favor the industry sponsor. J Clin Epidemiol 2015; 68: 811–820.

14.

Schumi

Wittes

JT.

Through the looking glass: understanding non-inferiority. Trials 2011; 12: 106.

15.

Soonawala

Dekkers

Vandenbroucke

et al . Noninferiority is (too) common in noninferiority trials. J Clin Epidemiol 2016; 71: 118–120.

16.

Rothmann

Chen

et al . Design and analysis of non-inferiority mortality trials in oncology. Stat Med 2003; 22: 239–264.

17.

Case

Kimmick

Paskett

et al . Interpreting measures of treatment effect in cancer clinical trials. Oncologist 2002; 7: 181–187.

18.

Hernan

MA.

The hazards of hazard ratios. Epidemiology 2010; 21: 13–15.

19.

Royston

Parmar

MK.

Restricted mean survival time: an alternative to the hazard ratio for the design and analysis of randomized trials with a time-to-event outcome. BMC Med Res Methodol 2013; 13: 152.

20.

Trinquart

Jacot

Conner

et al . Comparison of treatment effects measured by the hazard ratio and by the ratio of restricted mean survival times in oncology randomized controlled trials. J Clin Oncol 2016; 34: 1813–1819.

21.

Uno

Claggett

Tian

et al . Moving beyond the hazard ratio in quantifying the between-group difference in survival analysis. J Clin Oncol 2014; 32: 2380–2385.

22.

A’Hern

RP.

Restricted mean survival time: an obligatory end point for time-to-event analysis in cancer trials?

J Clin Oncol 2016; 34: 3474–3476.

23.

Royston

Parmar

MK.

The use of restricted mean survival time to estimate the treatment effect in randomized clinical trials when the proportional hazards assumption is in doubt. Stat Med 2011; 30: 2409–2421.

24.

Uno

Wittes

et al . Alternatives to hazard ratios for comparing the efficacy or safety of therapies in noninferiority studies. Ann Intern Med 2015; 163: 127–134.

25.

Cheng

Pak

Wei

LJ.

Demonstrating noninferiority of accelerated radiotherapy with panitumumab vs standard radiotherapy with cisplatin in locoregionally advanced squamous cell head and neck carcinoma. JAMA Oncol 2017; 3: 1430–1431.

26.

Kim

Uno

Wei

LJ.

Restricted mean survival time as a measure to interpret clinical trial results. JAMA Cardiol 2017; 2: 1179–1180.

27.

Althunian

de Boer

Klungel

et al . Methods of defining the non-inferiority margin in randomized, double-blind controlled trials: a systematic review. Trials 2017; 18: 107.

28.

Food Drug Administration Center for Drug Evaluation Research. Diabetes mellitus- Evaluating cardiovascular risk in new antibiabetic therapies to treat type 2 diabetes: guidance for Industry. Silver Spring, MD: U.S. Food and Drug Administration, 2008.

29.

Guyot

Ades

Ouwens

et al . Enhanced secondary analysis of survival data: reconstructing the data from published Kaplan-Meier survival curves. BMC Med Res Methodol 2012; 12: 9.

30.

Zarin

Tse

Williams

et al . The ClinicalTrials.gov results database — update and key issues. N Engl J Med 2011; 364: 852–860.

31.

Chan

Tetzlaff

Gotzsche

et al . SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ 2013; 346: e7586.

32.

Moher

Hopewell

Schulz

et al . CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ 2010; 340: c869.

33.

Schemper

Smith

TL.

A note on quantifying follow-up in studies of failure time. Control Clin Trials 1996; 17: 343–346.

34.

Com-Nougue

Rodary

Patte

How to establish equivalence when data are censored: a randomized trial of treatments for B non-Hodgkin lymphoma. Stat Med 1993; 12: 1353–1364.

35.

Horiguchi

Pak

Mikami

et al . Issues of the hazard ratio estimate and application of the restricted mean survival time to a non-inferiority study (New Advances in Statistical Inference and Its Related Topics)2015; 1954: 1–14.

36.

Chow

Wang

Shao

Comparing time-to-event data. In: Chow

Wang

(eds) Sample size calculations in clinical research. 2nd ed.Boca Raton, FL: Chapman and Hall/CRC Press, 2008, pp. 163–186.

37.

Jung

Kang

McCall

et al . Sample size computation for two-sample noninferiority log-rank test. J Biopharm Stat 2005; 15: 969–979.

38.

Crisp

Curtis

Sample size estimation for non-inferiority trials of time-to-event data. Pharm Stat 2008; 7: 236–244.

39.

Spiegelman

VanderWeele

TJ.

Evaluating public health interventions: 6. Modeling ratios or differences? Let the data tell us. Am J Public Health 2017; 107: 1087–1091.

40.

Poole

Commentary: some thoughts on consequential epidemiology and causal architecture. Epidemiology 2017; 28: 6–11.

41.

Poole

On the origin of risk relativism. Epidemiology 2010; 21: 3–9.

42.

Dekkers

Cevallos

Buhrer

et al . Comparison of noninferiority margins reported in protocols and publications showed incomplete and inconsistent reporting. J Clin Epidemiol 2015; 68: 510–517.

43.

Garattini

Bertele

Non-inferiority trials are unethical because they disregard patients’ interests. Lancet 2007; 370: 1875–1877.

44.

Burotto

Prasad

Fojo

Non-inferiority trials: why oncologists must remain wary. Lancet Oncol 2015; 16: 364–366.

45.

Wangge

Roes

de Boer

et al . The challenges of determining noninferiority margins: a case study of noninferiority randomized controlled trials of novel oral anticoagulants. CMAJ 2013; 185: 222–227.

46.

Gayet-Ageron

Agoritsas

Rudaz

et al . The choice of the noninferiority margin in clinical trials was driven by baseline risk, type of primary outcome, and benefits of new treatment. J Clin Epidemiol 2015; 68: 1144–1151.

47.

Tian

Ruberg

et al . Efficiency of two sample tests via the restricted mean survival time for analyzing event time observations. Biometrics 2017; 74: 694–702.

48.

Gopal

Desai

Tse

et al . Reporting of noninferiority trials in ClinicalTrials.gov and corresponding publications. JAMA 2015; 313: 1163–1165.

49.

Le Henanff

Giraudeau

Baron

et al . Quality of reporting of noninferiority and equivalence randomized trials. JAMA 2006; 295: 1147–1151.

50.

Rehal

Morris

Fielding

et al . Non-inferiority trials: are they inferior? A systematic review of reporting in major medical journals. BMJ Open 2016; 6: e012594.

51.

Schiller

Burchardi

Niestroj

et al . Quality of reporting of clinical non-inferiority and equivalence randomised trials—update and extension. Trials 2012; 13: 214.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

2.08 MB

Design of non-inferiority randomized trials using the difference in restricted mean survival times