This study considers the problem of testing the difference between treatment and control groups on m ≥ 2 measures when it is assumed a priori that the treatment group will perform better than the control group on all measures. Two procedures are investigated that do not rest on the assumptions of covariance homogeneity or multivariate normality: a likelihood ratio test based on a bootstrap critical value and a composite step-down procedure based on trimmed means. Type I error rates of both procedures are insensitive to assumption violations. Procedures that test a directional alternative hypothesis can be substantially more powerful than a procedure that tests a nondirectional hypothesis for certain configurations of the population mean vectors. The differences in average power of the investigated procedures are a function of the configuration of the population means, the magnitude of correlation among the outcome measures, and the shape of the population distribution.
Andrews, D.W.K. (2000). Inconsistency of the bootstrap when a parameter is on the boundary of the parameter space. Econometrica, 68, 399-405.
2.
Berkovits, I., Hancock, G.R., & Nevitt, J. (2000). Bootstrap resampling approaches for repeated measures designs: Relative robustness to sphericity and normality violations. Educational and Psychological Measurement, 60, 877-892.
3.
Blair, R.C., & Sawilowsky, S. (1993). Comparison of two tests useful in situations where treatment is expected to increase variability relative to controls. Statistics in Medicine, 12, 2233-2243.
4.
Bradley, J.V. (1978). Robustness? British Journal of Mathematical and Statistical Psychology, 31, 144-152.
5.
Brownie, C., Boos, D.D., & Hughes-Oliver, J. (1990). Modifying the t and ANOVA F tests when treatment is expected to increase variability relative to controls . Biometrics, 46, 259-266.
6.
Carpenter, J., & Bithell, J. (2000). Bootstrap confidence intervals: When, which, what? A practical guide for medical statisticians. Statistics in Medicine, 19, 1141-1164.
7.
Christensen, W.F., & Rencher, A.C. (1997). A comparison of Type I error rates and power levels for seven solutions to the multivariate Behrens-Fisher problem. Communications in Statistics: Simulation and Computation, 26, 1251-1273.
8.
Efron, B., & Tibshirani, R.J. (1993). An introduction to the bootstrap. New York: Chapman & Hall.
Fouladi, R.T., & Yockey, R.D. (2002). Type I error control of two-group multivariate tests on means under conditions of heterogeneous correlation structure and varied multivariate distributions. Communications in Statistics: Simulation and Computation, 31, 375-400.
11.
Harwell, M.R., Rubinstein, E.N., Hayes, W.S., & Olds, C.C. (1992). Summarizing Monte Carlo results in methodological research: The one- and two-factor fixed effects ANOVA cases. Journal of Educational and Behavioral Statistics, 17, 315-339.
12.
Holland, J.M., Neimeyer, R.A., Currier, J.M., & Berman, J.S. (2007). The efficacy of personal construct therapy: A comprehensive review. Journal of Clinical Psychology, 63, 93-107.
13.
Horton, N.J., & Lipsitz, S.R. (2001). Multiple imputation in practice: Comparison of software packages for regression models with missing variables. The American Statistician, 55, 244-254.
14.
Hotelling, H. (1931). The generalization of Student's ratio. Annals of Mathematical Statistics, 2, 360-378.
15.
Huang, P., Tilley, B.C., Woolson, R.F., & Lipsitz, S. (2005). Adjusting O'Brien's test to control Type I error for the generalized nonparametric Behrens-Fisher problem. Biometrics, 61, 532-539.
16.
Huber, P.J. (1981). Robust statistics. New York : John Wiley.
17.
Johansen, S. (1980). The Welch-James approximation to the distribution of the residual sum of squares in a weighted linear regression. Biometrika, 67, 85-92.
18.
Keefe, F.J., Dunsmore, J., & Burnett, R. (1992). Behavioral and cognitive-behavioral approaches to chronic pain: Recent advances and future directions. Journal of Consulting and Clinical Psychology, 60, 528-536.
19.
Kim, S.J. (1992). A practical solution to the multivariate Behrens-Fisher problem. Biometrika, 79, 171-176.
20.
Kudo, A. (1963). A multivariate analogue of the one-sided test . Biometrika, 50, 403-418.
21.
Lix, L.M., & Fouladi, R. (2007). Robust step-down tests for multivariate independent group designs. British Journal of Mathematical and Statistical Psychology, 60, 245-265.
22.
Lix, L.M., Keselman, H.J., & Hinds, A. (2005). Robust tests for the multivariate Behrens-Fisher problem. Computer Methods and Programs in Biomedicine, 77, 129-139.
23.
Micceri, T. (1989). The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin, 105, 156-166.
24.
Minhajuddin, A.T.M., Frawley, W.H., Schucany, W.R., & Woodward, W.A. (2007). Bootstrap tests for multivariate directional alternatives. Journal of Statistical Planning and Inference, 137, 2302-2315.
25.
Mudholkar, G.S., Kost, J., & Subbaiah, P. (2001). Robust tests for the significance of orthant restricted mean vector. Communications in Statistics: Theory and Methods , 30, 1789-1810.
26.
Mudholkar, G.S., & Srivastava, D.K. (2000). A class of robust stepwise alternatives to Hotelling's T2 tests. Journal of Applied Statistics , 27, 599-619.
27.
Nel, D.G., & van der Merwe, C.A. (1986). A solution to the multivariate Behrens-Fisher problem. Communications in Statistics: Simulation and Computation, 15, 3719-3735.
28.
Perlman, M.D. (1969). One-sided testing problems in multivariate analysis . Annals of Mathematical Statistics, 40, 549-567.
29.
Pocock, S.J., Geller, N.L., & Tsiatis, A.A. (1987). The analysis of multiple endpoints in clinical trials. Biometrics, 43, 487-498.
30.
Roy, S.N. (1958). Step down procedure in multivariate analysis. Annals of Mathematical Statistics, 29, 1177-1187.
31.
Roy, S.N., & Bargman, R.E. (1958). Tests of multiple independence and the associated confidence bounds. Annals of Mathematical Statistics, 29, 491-503.
32.
Sankoh, A.J., Huque, M.F., Russell, H.K., & D'Agostino, S. (1999). Global two-group multiple endpoint adjustment methods applied to clinical trials. Drug Information Journal, 33, 119-140.
33.
SAS Institute. (2004). SAS/IML user's guide, version 9. Cary, NC: Author.
34.
Schafer, J.L., & Graham, J.W. (2002). Missing data: Our view of the state of the art . Psychological Methods, 7, 147-177.
35.
Sen, P.K. (1983). A Fisherian detour of the step-down procedure . In P. K. Sen (Ed.), Contributions to statistics (pp. 367-377). Amsterdam: North-Holland.
36.
Srivastava, D.K., & Mudholkar, G.S. (2001). Trimmed T2 : A robust analog of Hotelling's T2 . Journal of Statistical Planning and Inference , 97, 343-358.
37.
Stevens, J.P. (1973). Step-down analysis and simultaneous confidence intervals in MANOVA. Multivariate Behavioral Research, 8, 391-402.
38.
Tamhane, A.C., & Logan, B.R. (2002). Accurate critical constants for the one-sided approximate likelihood ratio test of a normal mean vector when the covariance matrix is estimated. Biometrics, 58, 650-656.
39.
Tamhane, A.C., & Logan, B.R. (2003). Multiple endpoints: An overview and new developments (Tech. Rep. No. 43). Milwaukee: Medical College of Wisconsin, Division of Biostatistics.
40.
Tang, D., Gnecco, C., & Geller, N.L. (1989). An approximate likelihood ratio test for a normal mean vector with nonnegative components with application to clinical trials . Biometrika, 76, 577-583.
41.
Weisz, J.R., Doss, A.J., & Hawley, K.M. (2005). Youth psychotherapy outcome research: A review and critique of the evidence base. Annual Review of Psychology , 56, 337-363.
42.
Welch, B.L. (1938). The significance of the difference between two means when the population variances are unequal. Biometrika , 29, 350-362.
43.
Westfall, P.H., & Young, S.S. (1993). Resampling based multiple testing. New York: John Wiley.
44.
Wilcox, R.R. (1995). Simulation results on solutions to the multivariate Behrens-Fisher problem via trimmed means. The Statistician , 44, 213-225.
45.
Yao, Y. (1965). An approximate degrees of freedom solution to the multivariate Behrens-Fisher problem. Biometrika, 52, 139-147.
46.
Yuen, K.K. (1974). The two-sample trimmed t for unequal population variances. Biometrika, 61, 165-170.
47.
Yuen, K.K., & Dixon, W.J. (1973). The approximate behavior and performance of the two-sample trimmed t. Biometrika, 60, 369-374.
