Controlling the false-discovery rate when identifying the subgroup benefiting from treatment

Abstract

One common goal of subgroup analyses is to determine which (if any) types of patients—sets of patients sharing a vector of baseline covariates—benefit from a particular treatment. Many approaches involve testing, implicitly or explicitly, hypotheses about many patient types which are nonexchangeable. Methods of controlling family-wise Type I error rate inflation in such approaches are available. Such methods are designed to control the rate of erroneously declaring at least one type of patient as benefiting and are, therefore, quite conservative. We present a method for instead controlling a weighted false discovery rate in the sense of controlling the expected proportion of patient types declared benefiting, weighted by their population prevalence, which do not in fact benefit from treatment. Such population-weighted false discovery rate control is analogous to maintaining the positive predictive value of a diagnostic test for expected benefit. We minimize power loss by using a resampling approach that accounts for correlation among test statistics corresponding to similar patient types. Simulation studies demonstrate successful control of the weighted false discovery rate by the proposed method, as well as anti-conservativeness in the absence of multiplicity corrections and conservativeness by methods controlling the false discovery rate without accounting for dependent test statistics or controlling the family-wise error rate. An analysis of a clinical trial of an Alzheimer’s disease treatment illustrates the approach on real data. Resampling-based methods allow weighted false discovery rate control without unnecessarily sacrificing power when treatment effect estimates are correlated among patient types, and admit useful interpretations in terms of bounding sets and positive predictive value.