Although confidence interval procedures for analysis of variance (ANOVA) have been available for some time, they are not well known and are often difficult to implement with statistical packages. This article discusses procedures for constructing individual and simultaneous confidence intervals on contrasts on parameters of a number of fixed-effects ANOVA models, including multivariate analysis of variance (MANOVA) models for the analysis of repeated measures data. Examples show how these procedures can be implemented with accessible software. Confidence interval inference on parameters of random-effects models is also discussed.
Get full access to this article
View all access options for this article.
References
1.
American Psychological Association . (2001). Publication manual of the American Psychological Association (5th ed.). Washington, DC: Author.
2.
Betz, M. A. , & Gabriel, K. R. (1978). Type IV errors and analysis of simple effects. Journal of Educational Statistics, 3, 121-143.
3.
Betz, M. A. , & Levin, J. R. (1982). Coherent analysis-of-variance hypothesis-testing strategies: A general model. Journal of Educational Statistics, 7, 193-206.
4.
Bird, K. D. , Hadzi-Pavlovic, D., & Isaac, A. P. (2000). PSY [Computer software]. Sydney, Australia: School of Psychology, University of New South Wales.
5.
Boik, R. J. (1986). Testing the rank of a matrix with applications to the analysis of interaction in ANOVA. Journal of the American Statistical Association, 81, 243-248.
6.
Boik, R. J. (1993). The analysis of two-factor interactions in fixed effects linear models. Journal of Educational Statistics, 18, 1-40.
7.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). New York: Academic Press.
8.
Cumming, G. , & Finch, S. (2001). A primer on the understanding, use, and calculation of confidence intervals that are based on central and noncentral distributions. Educational and Psychological Measurement, 61, 532-575.
9.
Dunlap, W. P. , Cortina, J. M., Vaslow, J. B., & Burke, M. J. (1996). Meta-analysis of experiments with matched groups or repeated measures designs. Psychological Methods, 1, 170-177.
10.
Dunnett, C. W. (1955). A multiple comparisons procedure for comparing several treatments with a control. Journal of the American Statistical Association, 50, 1096-1121.
11.
Fan, X. , & Thompson, B. (2001). Confidence intervals about score reliability coefficients, please: An EPM guidelines editorial. Educational and Psychological Measurement, 61, 517-531.
12.
Fidler, F. , & Thompson, B. (2001). Computing correct confidence intervals for ANOVA fixed- and random-effects effect sizes. Educational and Psychological Measurement, 61, 575-605.
13.
Finch, S. , Cumming, G., & Thomason, N. (2001). Reporting of statistical inference in the Journal of Applied Psychology: Little evidence of reform. Educational and Psychological Measurement, 61, 181-210.
14.
Harris, R. J. (1994). ANOVA: An analysis of variance primer. Itasca, IL: F. E. Peacock.
15.
Harris, R. J. (2001). A primer of multivariate statistics (3rd ed.). Mahwah, NJ: Lawrence Erlbaum.
16.
Hsu, J. C. (1996). Multiple comparisons: Theory and methods. London: Chapman & Hall.
17.
Kieffer, K. M. , Reese, R. J., & Thompson, B. (2001). Statistical techniques employed in AERJ and JCP articles from 1988 to 1997: A methodological review. Journal of Experimental Education, 69, 280-309.
18.
Klockars, A. J. , & Hancock, G. R. (1998). A more powerful post hoc multiple comparison procedure in analysis of variance. Journal of Educational and Behavioral Statistics, 23, 279-289.
19.
Levin, J. R. , & Marascuilo, L. A. (1972). Type IV errors and interactions. Psychological Bulletin, 78, 368-374.
20.
Marascuilo, L. A. , & Levin, J. R. (1970). Appropriate post hoc comparisons for interaction and nested hypotheses in analysis of variance designs: The elimination of Type IV errors. American Educational Research Journal, 7, 397-421.
21.
McGraw. K. O. , & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1, 30-46.
22.
O’Brien, R. G. , & Kaiser, M. K. (1985). MANOVA method for analyzing repeated measures designs: An extensive primer. Psychological Bulletin, 89, 570-574.
23.
Roy, S. N. (1953). On a heuristic method of test construction and its use in multivariate analysis. Annals of Mathematical Statistics, 24, 220-238.
24.
Scheffé, H. (1953). A method for judging all contrasts in the analysis of variance. Biometrika, 40, 87-104.
25.
Smithson, M. (2001). Correct confidence intervals for various regression effect sizes and parameters: The importance of noncentral distributions in computing intervals. Educational and Psychological Measurement, 61, 605-632.
26.
Steiger, J. H. (1999). STATISTICA power analysis. Tulsa, OK: StatSoft.
27.
Steiger, J. H. , & Fouladi, R. T. (1997). Noncentrality interval estimation and the evaluation of statistical models. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 221-257). Mahwah, NJ: Lawrence Erlbaum.
28.
Tukey, J. W. (1994). The problem of multiple comparisons. In H. I. Braun (Ed.), The collected works of John W. Tukey. Vol. VIII: Multiple comparisons: 1948-1983 (pp. 1-300). New York: Chapman & Hall. (Original unpublished paper dated 1953)
29.
Vacha-Haase, T. , Nilsson, J. E., Reetz, D. R., Lance, T. S., & Thompson, B. (2000). Reporting practices and APA editorial policies regarding statistical significance and effect size. Theory & Psychology, 10, 413-425.
30.
Wilkinson, L. , & American Psychological Association Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594-604.
31.
Wong, S. P. , & McGraw, K. O. (1999). Confidence intervals and F tests for intraclass correlations based on three-way random effects models. Educational and Psychological Measurement, 59, 270-288.
