In many clinical trials there are multiple treatments under consideration which require multiple comparisons. Investigators are faced with the problems of Type I error control (experimentwise, familywise, or comparisonwise), maintenance of reasonable power for the study, and the selection of appropriate multiple comparison techniques to achieve these. This paper reviews briefly the various multiple comparison techniques available and discusses strategies for aiding investigators in applying these in settings typical in clinical trials (eg, confirmatory studies with placebo or positive controls or both). In a confirmatory setting, the treatment comparisons can often be divided into sets or families. Appropriate multiple comparison techniques, possibly differing across families, can then be applied within and across these. Study objectives can be better achieved by employing the strategies suggested.
D'AgostinoRBHeerenTC. Multiple Comparisons in Over-the-Counter Drug Clinical Trials with Both Positive and Placebo Controls. Stat Med.1991;10(1):1–6.
2.
FisherRA. The Design of Experiments. Edinburgh: Oliver and Boyd; 1935.
3.
DunnOJ. Multiple Comparisons Among Means. J Am Stat Assoc. 1961;56(1):52–64.
4.
DunnOJ. Estimation of the Means of Dependent Variables. Ann Mathematical Stat. 1958;29(4):1095–1111.
5.
ŠidákZ. Rectangular Confidence Regions for the Means of Multivariate Normal Distributions. J Am Stat Assoc. 1967;62(2):626–633.
6.
TukeyJW. The Problem of Multiple Comparisons. Unpublished manuscript. 1953.
7.
ToothakerLE. Multiple Comparisons for Researchers. Newbury Park, California: Sage; 1991.
8.
KramerCY. Extension of Multiple Range Tests to Group Means with Unequal Numbers of Replications. Biometrics. 1956;12(3):307–310.
9.
MillerRGJr.Simultaneous Statistical Inference. (2nd ed.) New York: Springer-Verlag; 1981.
10.
DunnettCW. Pairwise Multiple Comparisons in the Homogeneous Variance, Unequal Sample Size Case. J Am Stat Assoc. 1980;75(4):789–795.
11.
DunnettCW. New Tables for Multiple Comparisons with a Control. Biometrics. 1964;20(3):482–491.
12.
NewmanD. The Distribution of the Range in Samples from a Normal Population, Expressed in Terms of an Independent Estimate of Standard Deviation. Biometrika. 1939;31: 20–30.
13.
KeulsM. The Use of the Studentized Range in Connection with an Analysis of Variance. Euphytica. 1952;1: 112–122.
14.
RyanTA. Significance Tests for Multiple Comparison of Proportions, Variance, and Other Statistics. Psychological Bulletin. 1960;57: 318–328.
15.
EinotIGabrielKR. A Study of the Powers of Several Methods of Multiple Comparisons. J Am Stat Assoc. 1975;70(3):574–583.
16.
WelschRE. Stepwise Multiple Comparison Procedures. J Am Stat Assoc. 1977;72(3):566–575.
17.
PeritzE. A Note on Multiple Comparisons. Unpublished Manuscript. Israel: Hebrew University; 1970.
18.
ShafferJP. Comparison of Means: An F Test Followed by a Modified Multiple Range Procedure. J Educational Stat. 1979;4: 14–23.
19.
DuncanDB. Multiple Range and Multiple F Tests. Biometrics. 1955;11(1):1–42.
20.
ScheffeH. A Method for Judging All Contrasts in the Analysis of Variance. Biometrika. 1953;40: 87–104.
21.
ScheffeH. The Analysis of Variance. New York: John Wiley & Sons; 1959.
22.
KruskalWHWallisWA. Use of Ranks in One-Criterion Variance Analysis. J Am Stat Assoc. 1952;47(4):583–621.
23.
FriedmanM. The Use of Ranks to Avoid the Assumption of Normality Implicit in Analysis of Variance. J Am Stat Assoc. 1937;32(4):675–701.
24.
DunnOJ. Multiple Comparisons Using Rank Sums. Technometrics. 1964;6: 241–252.
25.
WilliamsDA. A Test for Differences Between Treatment Means When Several Dose Levels are Compared with a Zero Dose Control. Biometrics. 1971;27(1):103–117.
26.
WilliamsDA. The Comparison of Several Dose Levels with a Zero Dose Control. Biometrics. 1972;28(2):519–531.
27.
DunnettCW. Pairwise Multiple Comparisons in the Unequal Variance Case. J Am Stat Assoc. 1980;75(4):796–800.
28.
GamesPAHowellJF. Pairwise Multiple Comparison Procedures with Unequal n's and/or Variances: A Monte Carlo Study. J Educational Stat. 1976;1: 113–125.
29.
HochbergYTamhaneAC. Multiple Comparison Procedures. New York: John Wiley & Sons; 1987.
30.
WinerBJ. Statistical Principles in Experimental Design. (2nd ed.) New York: McGraw-Hill; 1971.
31.
FleissJL. The Design and Analysis of Clinical Experiments. New York: John Wiley & Sons; 1986.
32.
CarmerSGSwansonMR. An Evaluation of Ten Pairwise Multiple Comparison Procedures by Monte Carlo Methods. J Am Stat Assoc. 1973;68(1):66–74.
33.
RamseyPH. Power Differences Between Pair-wise Multiple Comparisons. J Am Stat Assoc. 1978;73(3):479–487.
34.
BauerP. Multiple Testing in Clinical Trials. Stat Med.1991;10: 871–890.
35.
MarcusRPeritzEGabrielKR. On Closed Testing Procedures with Special Reference to Ordered Analysis of Variance. Biometrika. 1976;63: 655–660.
36.
HolmS. A Simple Sequentially Rejective Multiple Test Procedure. Scandinavian J Stat. 1979;6: 65–70.
37.
HochbergY. A Sharper Bonferroni Procedure for Multiple Tests of Significance. Biometrika. 1988;75: 800–803.
38.
HommelG. A Stagewise Rejective Multiple Test Procedure Based on a Modified Bonferroni Test. Biometrika. 1988;75(2):383–386.
39.
LaskaEMMeisnerMJ. Testing Whether an Identified Treatment is Best. Biometrics. 1989;45(4):1139–1151.
40.
KochGG. Comment. Stat Med.1991;10(1):13–16.
41.
HochbergYBenjaminiY. More Powerful Procedures for Multiple Significance Testing. Stat Med.1990;9: 811–818.
