A comparative study of subgroup identification methods for differential treatment effect: Performance metrics and recommendations

Abstract

Subgroup identification with differential treatment effects serves as an important step towards precision medicine, as it provides evidence regarding how individuals with specific characteristics respond to a given treatment. This knowledge not only supports the tailoring of treatment strategies but also prompts the development of new treatments. This manuscript provides a brief overview of the issues associated with the methodologies aimed at identifying subgroups with differential treatment effects, and studies in depth the operational characteristics of five data-driven methods that have appeared recently in the literature. The performance of the methods under study to identify correctly the covariates affecting treatment effects is evaluated via simulation and under various conditions. Two clinical trial data sets are also used to illustrate the application of these methods. Discussion and recommendations pertaining to the use of these methods are provided, with emphasis on the relative performance of the methods under the conditions studied.

Keywords

Data-driven methods differential treatment effects predictive covariates prognostic covariates subgroup identification methods

Get full access to this article

View all access options for this article.

References

Rothwell

. Subgroup analysis in randomised controlled trials: importance, indications, and interpretation. Lancet 2005; 365: 176–186.

Pocock

Assmann

Enos

et al.

Subgroup analysis, covariate adjustment and baseline comparisons in clinical trial reporting: current practice and problems. Stat Med 2002; 21: 2917–2930.

Brookes

Whitley

Peters

et al.

Subgroup analyses in randomized controlled trials: quantifying the risks of false-positives and false-negatives. Health Technol Assess 2001; 5: 1–56.

Sun

Briel

Walter

et al.

Is a subgroup effect believable? Updating criteria to evaluate the credibility of subgroup analyses. BMJ 2010; 340: 850–854.

Sun

Ioannidis

Agoritsas

et al.

How to use a subgroup analysis: users guide to the medical literature. JAMA 2014; 311: 405–411.

Wang

Lagakos

Ware

et al.

Statistics in medicinereporting of subgroup analyses in clinical trials. N Engl J Med 2007; 357: 2189–2194.

Assmann

Pocock

Enos

et al.

Subgroup analysis and other (mis)uses of baseline data in clinical trials. Lancet 2000; 355: 1064–1069.

Guyatt

Wyer

Ioannidis

When to believe a subgroup analysis. In: Gordon

Drummond

Maureen

et al. (eds). Users’ guide to the medical literature. A manual for evidence-based clinical practice, Chicago, IL: American Medical Association, 2008, pp. 571–593.

Sun

Briel

Busse

et al.

Subgroup Analysis of Trials Is Rarely Easy (SATIRE): a study protocol for a systematic review to characterize the analysis, reporting, and claim of subgroup effects in randomized trials. Trials 2009, pp. 10–101.

10.

Imai

Ratkovic

. Estimating treatment effect heterogeneity in randomized program evaluation. Ann Appl Stat 2013; 7: 443–470.

11.

Zeileis

Hothorn

Hornik

. Model-based recursive partitioning. J Comp Graph Stat 2008; 17: 492–514.

12.

Dusseldorp

Conversano

Van Os

. Combining an additive and tree-based regression model simultaneously: STIMA. J Comp Graph Stat 2010; 19: 514–530.

13.

Loh

Man

. A regression tree approach to identifying subgroups with differential treatment effects. Stat Med 2015; 34: 1818–1833.

14.

Tsai

Wang

et al.

Subgroup analysis via recursive partitioning. J Mach Learn Res 2009; 10: 141–158.

15.

Lipkovich

Dmitrienko

Denne

et al.

Subgroup identification based on differential effect search-a recursive partitioning method for establishing response to treatment in patient subpopulations. Stat Med 2011; 30: 2601–2621.

16.

Foster

Taylor

Ruberg

. Subgroup identification from randomized clinical trial data. Stat Med 2011; 30: 2867–2880.

17.

Committee for Medicinal Products for Human Use. Guideline on the investigation of subgroups in confirmatory clinical trials, London: European Medicines Agency, 2014.

18.

Lipkovich

Dmitrienko

D’Agostino

. Tutorial in biostatistics: data driven subgroup identification and analysis in clinical trials. Stat Med 2017; 36: 136–196.

19.

Lagakos

. The challenge of subgroup analyses-reporting without distorting. N Engl J Med 2006; 354: 1667–1669.

20.

Telli

Hunt

Carlson

et al.

Trastuzumab-related cardiotoxicity: calling into question the concept of reversibility. J Clin Oncol 2007; 25: 3525–3533.

21.

Taylor

Ziesche

Yancy

et al.

Combination of isosorbide dinitrate and hydralazine in blacks with heart failure. N Engl J Med 2004; 351: 2049–2057.

22.

Italiano

. Prognostic or predictive? It’s time to get back to definitions!. J Clin Oncol 2011; 29: 4718–4718.

23.

Ondra

Dmitrienko

Friede

et al.

Methods for identification and confirmation of targeted subgroups in clinical trials: a systematic review. J Biopharm Stat 2016; 26: 99–119.

24.

Mayer

Lipkovich

Dmitrienko

. Survey results on industry practices and challenges in subgroup analysis in clinical trials. Stat Biopharm Res 2015; 7: 272–282.

25.

Wang

Hung

. A regulatory perspective on essential considerations in design and analysis of subgroups when correctly classified. J Biopharm Stat 2014; 24: 19–41.

26.

Patel

Hee

Mistry

et al.

Identifying back pain subgroups: developing and applying approaches using individual patient data collected within clinical trials. PGfAR 2016; 4: 1–314.

27.

Mori

Ikeda

Nakagawa

et al.

Pretreatment cognitive profile likely to benefit from donepezil treatment in dementia with Lewy bodies: pooled analyses of two randomized controlled trials. Dement Geriatr Cogn Disord 2016; 42: 58–68.

28.

Seibold

Zeileis

Hothorn

. Model-based recursive partitioning for subgroup analyses. Int J Biostat 2016; 12: 45–63.

29.

Meneses

McNees

et al.

Interaction trees: exploring the differential effects of an intervention programme for breast cancer survivors. J R Stat Soc Ser C 2011; 60: 457–474.

30.

Hardin

Rohwer

Curtis

et al.

Understanding heterogeneity in response to antidiabetes treatment: a post hoc analysis using SIDES, a subgroup identification algorithm. J Diabetes Sci Technol 2013; 7: 420–430.

31.

Zhou

Yan

et al.

Interaction trees with censored survival data. Int J Biostat 2008; 4: 1557–4679.

32.

Loh

Shih

. Split selection methods for classification trees. Stat Sin 1997; 7: 815–840.

33.

Lipkovich

Dmitrienko

. Strategies for identifying predictive biomarkers and subgroups with enhanced treatment effect in clinical trials using SIDES. J Biopharm Stat 2014; 24: 130–153.

34.

Zeileis

Hornik

. Generalized M fluctuation tests for parameter instability. Stat Neerl 2007; 61: 488–508.

35.

Tibshirani

. Regression shrinkage and selection via the lasso. J R Stat Soc Series B 1996; 58: 267–288.

36.

Wahba

. Spline models for observational data. In: CBMS-NSF regional conference series in applied mathematics 59, Philadelphia, PA: SIAM, 1990. . Available at: http://epubs.siam.org/doi/book/10.1137/1.9781611970128

37.

Friedman

Hastie

Tibshirani

. Regularization paths for generalized linear models via coordinate descent. J Stat Softw 2010; 33: 1–22.

38.

Loh

. Regression tress with unbiased variable selection and interaction detection. Stat Sin 2002; 12: 361–386.

39.

Carroll

Ball

Nich

et al.

Motivational interviewing to improve treatment engagement and outcome in individuals seeking treatment for substance abuse: a multisite effectiveness study. Drug Alcohol Depend 2006; 81: 301–312.

40.

McLellan

Luborsky

Cacciola

et al.

New data from the addiction severity index reliability and validity in three centers. J Nerv Ment Dis 1985; 173: 412–423.

41.

Doove

Dusseldorp

Van Deun

et al.

A comparison of five recursive partitioning methods to find person subgroups involved in meaningful treatmentsubgroup interactions. Adv Data Anal Classif 2014; 8: 403–425.

42.

Bogenschutz

Donovan

Mandler

et al.

Brief intervention for patients with problematic drug use presenting in emergency departments: a randomized clinical trial. JAMA Intern Med 2014; 174: 1736–1745.

43.

Skinner

. The drug abuse screening test. Addict Behav 1982; 7: 363–371.

44.

Bradley

DeBenedetti

Volk

et al.

AUDITC as a brief screen for alcohol misuse in primary care. Alcohol Clin Exp Res 2007; 31: 1208–1217.

45.

Zhou

Faries

. Estimating optimal treatment regimes via subgroup identification in randomized control trials and observational studies. Stat Med 2016; 35: 32853302–32853302.

46.

Zhao

et al.

Regularized outcome weighted subgroup identification for differential treatment effects. Biometrics 2015; 71: 645–653.

47.

Berger

Wang

Shen

. A Bayesian approach to subgroup identification. J Biopharm Stat 2014; 24: 110–129.

48.

Schnell

Tang

Offen

et al.

A Bayesian credible subgroups approach to identifying patient subgroups with positive treatment effects. Biometrics 2016; 72: 1026–1036.

49.

Yin

Lee

. Bayesian two-step Lasso strategy for biomarker selection in personalized medicine development for time toevent endpoints. Contemp Clin Trials 2013; 36: 642–650.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.28 MB