The Welch-James and Improved General Approximation tests were examined in between-subjects × within-subjects repeated measures designs for their rates of Type I error when data were nonnormal, nonspherical, and heterogeneous and when group sizes were unequal as well. The tests were computed with either least squares or robust estimators of central tendency and variability and assessed with critical values that were obtained either theoretically or through a bootstrapping method. Prior findings indicated that one could only obtain a robust test of the interaction effect with the Welch-James procedure when sample sizes were very large. This study’s results indicate that a robust test of the interaction effect can be obtained with reasonable sample sizes when the Welch-James test is computed with trimmed means and Winsorized covariance matrices.
Get full access to this article
View all access options for this article.
References
1.
Algina, J. (1994).Some alternative approximate tests for a split plot design. Multivariate Behavioral Research, 29,365-384.
2.
Algina, J. (1997).Generalization of improved general approximation tests to split-plot designs with multiple between-subjects factors and/or multiple within-subjects factors. British Journal of Mathematical and Statistical Psychology, 50,243-252.
3.
Algina, J. , & Keselman, H. J. (1997).Testing repeated measures hypotheses when covariances are heterogeneous: Revisiting the robustness of the Welch-James test. Multivariate Behavioral Research, 32,255-274.
4.
Algina, J. , & Keselman, H. J. (1998).A power comparison of the Welch-James and improved general approximation tests in the split-plot design. Journal of Educational and Behavioral Statistics, 23,152-169.
5.
Algina, J. , & Oshima, T. C. (1994).Type I error rates for Huynh’s general approximation and improved general approximation tests. British Journal of Mathematical and Statistical Psychology, 47,151-165.
6.
Algina, J. , & Oshima, T. C. (1995).An improved general approximation test for the main effect in a split plot design. British Journal of Mathematical and Statistical Psychology, 48,149-160.
7.
Bradley, J. V. (1978).Robustness?British Journal of Mathematical and Statistical Psychology, 31, 144-152.
8.
Efron, B. , & Tibshirani, R. J. (1993). An introduction to the bootstrap. New York: Chapman & Hall.
9.
Gross, A. M. (1976).Confidence interval robustness with longtailed symmetric distributions. Journal of the American Statistical Association, 71,409-416.
10.
Hall, P. (1986).On the number of bootstrap simulations required to construct a confidence interval. Annals of Statistics, 14,1431-1452.
11.
Hall, P. , & Padmanabhan, A. R. (1992).On the bootstrap and the trimmed mean. Journal of Multivariate Analysis, 41,132-153.
12.
Huber, P. J. (1981). Robust statistics. New York: Wiley.
13.
Huynh, H. (1978).Some approximate tests for repeated measurement designs. Psychometrika, 43,161-175.
14.
James, G. S. (1951).The comparison of several groups of observations when the ratios of the population variances are unknown. Biometrika, 38,324-329.
15.
James, G. S. (1954).Tests of linear hypotheses in univariate and multivariate analysis when the ratios of the population variances are unknown. Biometrika, 41,19-43.
16.
Johansen, S. (1980).The Welch-James approximation of the distribution of the residual sum of squares in weighted linear regression. Biometrika, 67,85-92.
17.
Johnson, M. F. , Ramberg, J. S., & Wang, C. (1982).The Johnson translation system in Monte Carlo studies. Communications in Statistics-Simulation and Computation, 11,521-525.
18.
Keselman, H. J. , & Algina, J. (1996). The analysis of higher-order repeated measures designs. In B. Thompson (Ed.), Advances in social science methodology, volume 4 (pp. 45-70). Greenwich, CT: JAI.
19.
Keselman, H. J. , Carriere, K. C., & Lix, L. M. (1993).Testing repeated measures hypotheses when covariance matrices are heterogeneous. Journal of Educational Statistics, 18,305-319.
20.
Keselman, H. J. , Huberty, C. J., Lix, L. M., Olejnik, S., Cribbie, R. A., Donahue, B., Kowalchuk, R. K., Lowman, L. L., Petoskey, M. D., Keselman, J. C., & Levin, J. R. (1998). Statistical practices of educational researchers: An analysis of their ANOVA, MANOVA and ANCOVA analyses. Review of Educational Research, 68(3), 350-386.
21.
Keselman, H. J. , Kowalchuk, R. K., Algina, J., Lix, L. M., & Wilcox, R. R. (in press). Testing treatment effects in repeated measures designs: Trimmed means and bootstrapping. British Journal of Mathematical and Statistical Psychology.
22.
Keselman, H. J. , Kowalchuk, R. K., & Lix, L. M. (1998).Robust nonorthogonal analyses revisited: An update based on trimmed means. Psychometrika, 63,145-163.
23.
Keselman, J. C. , & Keselman, H. J. (1990).Analysing unbalanced repeated measures designs. British Journal of Mathematical and Statistical Psychology, 43,265-282.
24.
Lecoutre, B. (1991).A correction for the approximate test in repeated measures designs with two or more independent groups. Journal of Educational Statistics, 16,371-372.
25.
Lix, L. M. , & Keselman, H. J. (1995).Approximate degrees of freedom tests: A unified perspective on testing for mean equality. Psychological Bulletin, 117,547-560.
26.
Lix, L. M. , & Keselman, H. J. (1998).To trim or not to trim: Tests of location equality under heteroscedasticity and nonnormality. Educational and Psychological Measurement, 58,409-429.
27.
Micceri, T. (1989).The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin, 105,156-166.
28.
Rosenberger, J. L. , & Gasko, M. (1983). Comparing location estimators: Trimmed means, medians, and trimean. In D. Hoaglin, F. Mosteller, & J. Tukey (Eds.), Understanding robust and exploratory data analysis (pp. 297-336). New York: Wiley.
29.
SAS Institute . (1989). SAS/IML software: Usage and reference, Version 6. Cary, NC: Author.
30.
Sawilowsky, S. S. , & Blair, R. C. (1992).A more realistic look at the robustness and Type II error probabilities of the t test to departures from population normality. Psychological Bulletin, 111,352-360.
31.
Welch, B. L. (1947).The generalization of Students’ problem when several different population variances are involved. Biometrika, 34,28-35.
32.
Welch, B. L. (1951).On the comparison of several mean values: An alternative approach. Biometrika, 38,330-336.
33.
Westfall, P. H. , & Young, S. S. (1993). Resampling-based multiple testing. New York: Wiley.
34.
Wilcox, R. R. (1994a).A one-way random effects model for trimmed means. Psychometrika, 59,289-306.
35.
Wilcox, R. R. (1994b).Some results on the Tukey-McLaughlin and Yuen methods for trimmed means when distributions are skewed. Biometrical Journal, 36,259-273.
36.
Wilcox, R. R. (1995a). ANOVA: A paradigm for low power and misleading measures of effect size?Review of Educational Research, 65(1), 51-77.
37.
Wilcox, R. R. (1995b).Three multiple comparison procedures for trimmed means. Biometrical Journal, 37,643-656.
38.
Wilcox, R. R. (1997a).Pairwise comparisons using trimmed means or M-estimators when working with dependent groups. Biometrical Journal, 39,677-688.
39.
Wilcox, R. R. (1997b). Introduction to robust estimation and hypothesis testing. New York: Academic Press.
40.
Wilcox, R.R. (1998).The goals and strategies of robust methods. British Journal of Mathematical and Statistical Psychology, 51,1-39.
41.
Wilcox, R. R. , Keselman, H. J., & Kowalchuk, R. K. (1998).Can tests for treatment group equality be improved?: The bootstrap and trimmed means conjecture. British Journal of Mathematical and Statistical Psychology, 51,123-134.
