This study evaluated the robustness of two recent methods for analyzing multivariate repeated measures when the assumptions of covariance homogeneity and multivariate normality are violated. Specifically, the authors' work compares the performance of the modified Brown—Forsythe (MBF) procedure and the mixed-model procedure adjusted by the Kenward—Roger solution available in SAS PROC MIXED. The authors found that, overall, the MBF procedure appeared to be the least sensitive to the factors examined in the present study; however, this is not necessarily the case for all data sets. As the results show, for tests of the between-subjects main effect, the MBF approach outperformed the mixed-model method when fitting either a patterned or nonpatterned covariance structure. But for tests of within-subjects effects, its Type I error control advantages decrease.
Allison, P.D. (2000). Multiple imputation for missing data: A cautionary tale. Sociological Methods & Research, 28, 301-309.
2.
Berkovits, I., Hancock, G.R., & Nevitt, J. (2000). Bootstrap resampling approaches for repeated measure designs: Relative robustness to sphericity and normality violations. Educational and Psychological Measurement, 60, 877-892.
3.
Boik, R.J. (1991). Scheffé's mixed model for multivariate repeated measures: A relative efficiency evaluation. Communication in Statistics: Theory and Methods, 20, 1233-1255.
4.
Bradley, J. (1978). Robustness?British Journal of Mathematical and Statistical Psychology, 31, 144-152.
5.
Brown, M.B., & Forsythe, A.B. (1974). The small sample behavior of some statistics which test the equality of several means. Technometrics, 16, 129-132.
6.
Chen, X., & Wei, L. (2003). A comparison of recent methods for the analysis of small-sample cross-over studies. Statistics in Medicine , 22, 2821-2833.
7.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
Gomez, V.E., Schaalje, G.B., & Fellingham, G.W. (2005). Performance of the Kenward—Roger method when the covariance structure is selected using AIC and BIC. Communications in Statistics: Simulation and Computation, 34, 377-392.
12.
Johansen, S. (1980). The Welch-James approximation to the distribution of the residual sum of squares in a weighted linear regression. Biometrika, 67, 85-92.
13.
Kenward, M.G. (2001). Comments on ``Parametric modelling of growth curve data: An overview,'' by D. L. Zimmerman & V. Núñez-Antón. Test, 10, 1-73.
14.
Kenward, M.G., & Roger, J.H. (1997). Small sample inference for fixed effects from restricted maximum likelihood. Biometrics, 53, 983-997.
15.
Keselman, H.J., & Lix, L.M. (1997). Analysing multivariate repeated measures designs when covariance matrices are heterogeneous. British Journal of Mathematical and Statistical Psychology, 50, 319-338.
16.
Kowalchuk, R.K., Keselman, H.J., Algina, J., & Wolfinger, R.D. (2004). The analysis of repeated measurements with mixed-model adjusted F tests. Educational and Psychological Measurement, 64, 224-242.
17.
Krishnamoorthy, K., & Yu, J. (2004). Modified Nel and Van der Merwe test for the multivariate Behrens-Fisher problem. Statistics & Probability Letters , 66, 161-169.
18.
Lix, L.M., Algina, J., & Keselman, H.J. (2003). Analysing multivariate repeated measures designs: A comparison of two approximate degrees of freedom procedures. Multivariate Behavioral Research, 38, 403-431.
19.
Lix, L.M., & Hinds, A.M. (2004). Multivariate contrasts for repeated measures designs under assumptions violations. Journal of Modern Applied Statistical Methods, 3, 333-343.
20.
Micceri, T. (1989). The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin,92, 778-785.
21.
Naik, D.N., & Rao, S.S. (2001). Analysis of multivariate repeated measures data with a Kronecker product structured covariance matrix. Journal ofApplied Statistics, 28, 91-105.
22.
Nel, D.G. (1997). Tests for equality of parameter matrices in two multivariate linear models. Journal of MultivariateAnalysis , 61, 29-37.
23.
Nel, D.G., & van der Merwe, C.A. (1986). A solution to the multivariate Behrens-Fisher problem. Communications in Statistics: Theory and Methods, 15, 3719-3735.
24.
SAS Institute, Inc. (1999). SAS/STAT user's guide, Version 7. Cary, NC: Author.
25.
SAS Institute, Inc. (2002). SAS/STAT software: Version 9.0 [Computer software]. Cary, NC: Author.
26.
Timm, N.H. (2002). Applied multivariate analysis. New York: Springer.
Vallejo, G., & Ato, M. (2006). Modified Brown-Forsythe procedure for testing interaction effects in split-plot designs. Multivariate Behavioral Research, 41, 549-578.
29.
Vallejo, G., Fidalgo, A., & Fernández, P. (2001). Effects of covariance heterogeneity on three procedures for analysing multivariate repeated measures designs. Multivariate Behavioral Research, 36, 1-27.
30.
Vallejo, G., & Livacic-Rojas, P. (2005). A comparison of two procedures for analyzing small sets of repeated measures data. Multivariate Behavioral Research, 40, 179-205.
31.
Vallejo, G., Moris, J., & Conejo, N.M. (2006). A SAS/IML program for implementing the modified Brown-Forsythe procedure in repeated measures designs. Computer Methods & Programs in Biomedicine, 83, 169-177.
32.
Wilks, S.S. (1932). Certain generalizations in the analysis of variance . Biometrika, 24, 471-494.
