The Effectiveness of Methods for Analyzing Multivariate Factorial Data

Abstract

A Monte Carlo simulation was used to examine the effectiveness of univariate analysis of variance (ANOVA), multivariate analysis of variance (MANOVA), and multiple indicator structural equation (MISE) modeling to analyze data from multivariate factorial designs. The MISE method yielded downwardly biased standard errors for the univariate parameter estimates in the small sample size conditions. In the large sample size data conditions, the MISE method outperformed MANOVA and ANOVA when the covariate accounted for variation in the dependent variable and variables were unreliable. With multivariate statistical tests, MANOVA outperformed the MISE method in the Type I error conditions and the MISE method outperformed MANOVA in the Type II error conditions. The Bonferroni methods were overly conservative in controlling Type I error rates for univariate tests, but a modified Bonferroni method had higher statistical power than the Bonferroni method. Both the Bonferroni and modified methods adequately controlled multivariate Type I error rates.

Get full access to this article

View all access options for this article.

References

Arbuckle, J. L., & Wothke, W. (1999). AMOS4.0 user’s guide. Chicago: SmallWaters.

Babbie, E. (1994). The practice of social research (7th ed.). Blemont, CA: Wadsworth.

Bagozzi, R. P. (1977). Structural equation models in experimental research. Journal of Marketing Research, 14, 209-226.

Bagozzi, R. P., & Yi, Y. (1989). On the use of structural equation models in experimental designs. Journal of Marketing Research, 26, 271-284.

Bagozzi, R. P., Yi, Y., & Singh, S. (1991). On the use of structural equation models in experimental designs: Two extensions. International Journal of Research in Marketing, 8, 125-140.

Bray, J. H., & Maxwell, S. E. (1982). Analyzing and interpreting significant MANOVAs. Review of Educational Research, 52, 340-367.

Castañeda, M. B., Levin, J. R., & Dunham, R. B. (1993). Using planned comparisons in management research: A case for the Bonferroni procedure. Journal of Management, 19, 707-724.

Everitt, B. S. (1979). A Monte Carlo investigation of the robustness of Hotelling’s one and two sample T2 tests. Journal of the American Statistical Association, 74, 48-51.

Fornell, C., & Bookstein, F. L. (1982). Two structural equation models: LISREL and PLS applied to consumer exit-voice theory. Journal of Marketing Research, 19, 440-452.

10.

Hakstian, A. R., Roed, J. C., & Lind, J. C. (1979). Two-sample T2 procedure and the assumption of homogeneous covariance matrices. Psychological Bulletin, 86, 1255-1263.

11.

Holland, B. S., & Copenhaver, M. (1988). Improved Bonferroni-type multiple testing procedures. Psychological Bulletin, 104, 145-149.

12.

Holloway, L. N., & Dunn, O. J. (1967). The robustness of Hotelling’s T2. Journal of the American Statistical Association, 62, 124-136.

13.

Holm, D. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6, 65-70.

14.

Hopkins, J. W., & Clay, P. P. (1963). Some empirical distributions of bivariate T2 and homoscedasticity criterion M under unequal variance and leptokurtosis. Journal of the American Statistical Association, 58: 1048-1053.

15.

Huberty, C. J., & Morris, J. D. (1989). Multivariate analysis versus multiple univariate analyses. Psychological Bulletin, 105, 302-308.

16.

Hummel, T. J., & Sligo, J. R. (1971). Empirical comparison of univariate and multivariate analysis of variance procedures. Psychological Bulletin, 76, 49-57.

17.

Jaccard, J. (1998). Interaction effects in factorial analysis of variance. Sage University Paper Series on Quantitative Applications in the Social Sciences, series no. 07-118. Thousand Oaks, CA: Sage.

18.

Jaccard, J., Becker, M., & Wood, G. (1984). Pairwise multiple comparisons: A review. Psychological Bulletin, 96: 589-596.

19.

Jaccard, J., & Wan, C. K. (1995). Measurement error in the analysis of interaction effects between continuous predictors using multiple regression: Multiple indicator and structural equation approaches. Psychological Bulletin, 117, 348-357.

20.

Jöreskog, K. G., & Sörbom, D. (1993). LISREL VIII user’s reference guide. Chicago: Scientific Software International.

21.

Kromery, J. D., & Dickinson, W. B. (1995). The use of an overall F test to control Type I error rates in factorial analysis of variance: Limitations and better strategies. Journal of Applied Behavioral Science, 31, 51-64.

22.

Kühnel, S. M. (1988). Testing MANOVA designs with LISREL. Sociological Methods & Research, 16, 504-523.

23.

Mardia, K. V. (1971). The effect of nonnormality on some multivariate and robustness to nonnormality in the linear model. Biometrika, 58(1), 105-121.

24.

Olson, C. L. (1974). Comparative robustness of six tests in multivariate analysis of variance. Journal of the American Statistical Association, 69, 894-908.

25.

Rosnow, R. L., & Rosenthal, R. (1989). Definition and interpretation of interaction effects. Psychological Bulletin, 105, 143-146.

26.

Spector, P. E. (1977). What to do with significant multivariate effects in multivariate analyses of variance. Journal of Applied Psychology, 62, 158-163.

27.

Tabachnick, B. G., & Fidell, L. S. (1996). Using multivariate statistics (3rd ed.). Northridge, CA: Harper Collins College Publishers.

28.

Wilkinson, L. (1975). Response variable hypotheses in the multivariate analysis of variance. Psychological Bulletin, 82, 408-412.