The theory indicating that general linear model methods can be superior to ANOVA analogs is well known. However, the magnitudes of differences induced by different analysis choices has not been empirically investigated. The paper reports a Monte Carlo study of these differences over nine combinations of three sample sizes and three population parameter effect sizes. Generally, ANOVA methods tended somewhat to overestimate smaller effect sizes and to underestimate larger effect sizes.
Get full access to this article
View all access options for this article.
References
1.
Aiken, L. R. (1976). Update on attitudes and other affective variables in learning mathematics. Review of Educational Research, 46, 293-311.
2.
Baker, F. B. and Collier, R. O. (1968). An empirical study into factors affecting the F-test under permutation for the randomized block design. Journal of the American Statistical Association, 63, 902-911.
3.
Carver, R. P. (1978). The case against statistical significance testing. Harvard Educational Review, 48, 378-399.
4.
Cohen, J. (1968). Multiple regression as a general data-analytic system. Psychological Bulletin, 70, 426-443.
5.
Cohen, J. B. (1977). Statistical power analysis for the behavioral sciences (revised edition). New York: Academic Press.
6.
Collier, R. O. and Larson, R. C. (1969). Statistics: Monte Carlo methods. Review of Educational Research, 39, 734-739.
7.
Cronbach, L. J. (1957). The two disciplines of scientific psychology. American Psychologist, 12, 671-684.
8.
Cronbach, L. J. (1975). Beyond the two disciplines of scientific psychology. American Psychologist, 30, 116-127.
9.
Edgington, E. S. (1964). A tabulation of inferential statistics used in psychology journals. American Psychologist, 19, 202-203.
10.
Edgington, E. S. (1974). A new tabulation of statistical procedures used in APA journals. American Psychologist, 29, 25-26.
11.
Edwards, A. L. (1979). Multiple regression and the analysis of variance and covariance. San Francisco: Freeman and Company.
12.
Feldt, L. S. (1958). A comparison of the precision of three experimental designs employing a concomitant variable. Psychometrika, 23, 335-353.
13.
Glass, G. V. , McGaw, B., and Smith, M. L. (1981). Meta-analysis in social research. Beverly Hills, CA: SAGE.
14.
Glass, G. V. , Peckham, P. D., and Saunders, J. R. (1972). Consequences of failure to meet assumptions underlying the fixed effects analysis of variance and covariance. Review of Educational Research, 42, 237-288.
15.
Goodwin, L. D. and Goodwin, W. L. (1985). Statistical techniques in AERJ articles, 1979-1983: The preparation of graduate students to read the educational research literature. Educational Researcher, 14, 5-11.
16.
Hakstian, A. R. , Roed, J. C., and Lind, J. C. (1979). Two sample T2 procedure and the assumption of homogeneous covariance matrices. Psychological Bulletin, 86, 1255-1263.
17.
Hammersley, J. M. and Handscomb, D. C. (1964). Monte Carlo methods. London: Methven and Co.
18.
Huck, S. W. , Cormier, W. H., and Bounds, W. G., Jr. (1974). Reading statistics in research. New York: Harper and Row.
19.
Hummel, T. J. and Sligo, J. R. (1971). Empirical comparison of univariate and multivariate analysis of variance procedures. Psychological Bulletin, 76, 49-57.
20.
Kerlinger, F. N. (1973). Foundations of behavioral research (2nd ed.). New York: Holt, Rinehart and Winston.
21.
Kerlinger, F. N. and Pedhazur, E. J. (1973). Multiple regression in behavioral research. New York: Holt, Rinehart and Winston.
22.
Kirk, R. E. (1982). Experimental design: Procedures for the behavioral sciences (2nd ed.). Monterey, CA: Brooks/Cole.
23.
Knapp, T. R. (1978). Canonical correlation analysis: A general parametric significance testing system. Psychological Bulletin, 85, 410-416.
24.
Marsascuilo, L. A. and Levin, J. R. (1970). Appropriate post hoc comparisons for interaction and nested hypotheses in analysis of variance designs: The elimination of Type IV errors. American Educational Research Journal, 7, 397-421.
25.
Mood, A. M. (1971). Partitioning variance in multiple regression analyses as a tool for developing learning models. American Educational Research Journal, 8, 191-202.
26.
Morris, J. D. (1975). A computer program to create a population with any desired centroid and covariance matrix. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 35, 707-710.
27.
Nunnally, J. C. (1967). Psychometric theory. New York: McGraw-Hill.
28.
Olsen, C. L. (1976). On choosing a test statistic in MANOVA. Psychological Bulletin, 83, 579-586.
29.
Rubinstein, R. Y. (1981). Simulation and the Monte Carlo method. New York: Wiley and Sons.
30.
Thompson, B. (198la). The problem of OVAism. Paper presented at the annual meeting of the Mid-South Educational Research Association, Lexington, KY. [Order document #03980, National Auxiliary Publication Service, P.O. Box 3513, Grand Central Station, New York, NY 10017]
31.
Thompson, B. (1981b). Utility of invariance coefficients. Perceptual and Motor Skills, 52, 708-710.
32.
Thompson, B. (1984a). Canonical correlation analysis: Uses and interpretation. Beverly Hills: SAGE.
33.
Thompson, B. (1984b). Coding and commonality analysis: Non-ANOVA methods for analyzing data from experiments. Paper presented at the annual meeting of the Mid-South Educational Research Association, New Orleans.
34.
Thompson, B. (1985). Heuristics for teaching multivariate general linear model concepts. Paper presented at the annual meeting of the American Educational Research Association, Chicago. (ERIC Document Reproduction Service No. ED 262 073.)
35.
Thompson, B. and Borrello, G. M. (1985). The importance of structure coefficients in regression research. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 45, 203-209.
36.
Wick, J. W. , and Dirkes, C. (1973). Characteristics of current doctoral dissertations in education. Educational Researcher, 2, 20-22.
37.
Willson, V. L. (1980). Research techniques in AERJ articles: 1969 to 1978. Educational Researcher, 9, 5-10.
