Some Statistical Methods to Differentiate a Treatment Effect for Small Shifts in the Tail of a Distribution

Abstract

The intervention of most clinical therapies typically involves shifting the central tendency of a response variable. There are many analytical methods to summarize these shifts and provide inferential results for treatment differences. Differences between treatments, however, cannot always be discerned by central tendency shifts. Rare individual events of extreme deviations from the central tendency cannot be easily captured by standard test statistics and summary results. Instead, these extreme rare event instances need to be summarized by analytical methods to describe and test shifts between treatments that only occur in a small proportion of patients. In fact, in the case of extreme data, the achievement of significance based upon central tendency summaries may not be clinically meaningful. This paper will compare and contrast some well and lesser known analytical methods to differentiate a treatment effect for small shifts in the tail of a distribution. In particular, the following statistical methodologies will be compared and contrasted based upon clinical data from a Phase III study: Fisher's Exact Test based upon various cutpoints of the response variable, Ordered Categorical Analysis across the entire distribution, rank-based methods including the mixed normal and quantile tests in Johnson et al.'s paper (1), and a method from Conover et al. (2). Each of the above methodologies offers advantages and disadvantages in analyzing data with sparsely occurring extreme observations. Decisions toward the choice of which procedure to employ should be a function of the information deemed critical to the study sponsor and data analyst. Ultimately, the best strategy may be to use a combination of these procedures and attempt to reach a general corroborative conclusion about treatment effect.