Multiple Comparisons: When and How?

Abstract

The interpretation of clinical trials is a constant challenge to those involved in clinical research. The evaluation is a multidisciplinary process involving a statistical as well as a clinical interpretation of data. The statistical evaluation provides an assessment of the likelihood of the observed results. The clinical interpretation is based on the clinical relevance of what is observed in the trial. A point of debate in the statistical evaluation is the control of the Type I error rate. The Type I error is controlled by performing pairwise comparisons with procedures that either control on a comparisonwise or experimentwise basis. The following paper presents a discussion of the issues for and against either method of controlling the Type I error rate. The paper also attempts to illustrate through examples that consistent evidence of clinical benefit in trials which meet all the criteria of being adequate and well-controlled should suffice for concluding positive treatment effect. The basis of interpretation should be the magnitude of the treatment differences and variances, their corresponding interval estimates, and their clinical relevance.