Most textbooks explain how to compute confidence intervals for means, correlation coefficients, and other statistics using “central” test distributions (e.g., t, F) that are appropriate for such statistics. However, few textbooks explain how to use “noncentral” test distributions (e.g., noncentral t, noncentral F) to evaluate power or to compute confidence intervals for effect sizes. This article illustrates the computation of confidence intervals for effect sizes for some ANOVA applications; the use of intervals invoking noncentral distributions is made practical by newer software. Greater emphasis on both effect sizes and confidence intervals was recommended by the APA Task Force on Statistical Inference and is consistent with the editorial policies of the 17 journals that now explicitly require effect size reporting.
Get full access to this article
View all access options for this article.
References
1.
Aczel, A. D. (1995). Statistics: Concepts and applications. Chicago: Irwin.
2.
American Psychological Association . (1994). Publication manual of the American Psychological Association (4th ed.). Washington, DC: Author.
3.
Anderson, D. R. , Burnham, K. P., & Thompson, W. L. (2000). Null hypothesis testing: Problems, prevalence, and an alternative. Journal of Wildlife Management, 64, 912-923.
4.
Bagozzi, R. P. , Fornell, C., & Larcker, D. F. (1981). Canonical correlation analysis as a special case of a structural relations model. Multivariate Behavioral Research, 16, 437-454.
5.
Burdick, R. K. , & Graybill, F. A. (1992). Confidence intervals for variance components.New York: Marcel Dekker.
6.
Clark, H. H. (1973). The language-as-fixed-effect fallacy: A critique of language statistics in psychological research. Journal of Verbal Learning and Verbal Behavior, 12, 335-359.
7.
Cohen, J. (1968). Multiple regression as a general data-analytic system. Psychological Bulletin, 70, 426-443.
8.
Cohen, J. (1969). Statistical power analysis for the behavioral sciences. New York: Academic Press.
9.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
10.
Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49, 997-1003.
11.
Cumming, G. , & Finch, S. (2001). A primer on the understanding, use, and calculation of confidence intervals that are based on central and noncentral distributions. Educational and Psychological Measurement, 61, 532-574.
12.
Devore, J. , & Peck, R. (1997). Statistics: The exploration and analysis of data (3rd ed.). Pacific Grove, CA: Brooks/Cole.
13.
Edgington, E. S. (1964). A tabulation of inferential statistics used in psychology journals. American Psychologist, 19, 202-203.
14.
Edgington, E. S. (1974). A new tabulation of statistical procedures used in APA journals. American Psychologist, 29, 25-26.
15.
Elmore, P. B. , & Woehlke, P. L. (1988). Statistical methods employed in American Educational Research Journal, Educational Researcher, and Review of Educational Research from 1978 to 1987. Educational Researcher, 17(9), 19-20.
16.
Ezekiel, M. (1930). Methods of correlational analysis. New York: John Wiley.
17.
Fisher, R. A. (1931). Introduction. In J. R. Airey (Ed.), Table of Hh functions (pp. xxvi-xxxv). London: British Association.
18.
Frederick, B. N. (1999). Fixed-, random-, and mixed-effects ANOVA models: A user-friendly guide for increasing the generalizability of ANOVA results. In B. Thompson (Ed.), Advances in social science methodology (Vol. 5, pp. 111-122). Stamford, CT: JAI.
19.
Freedman, D. , Pisani, R., Purves, R., & Adhikari, A. (1991). Statistics (2nd ed.). New York: Norton.
20.
Friedman, H. (1968). Magnitude of experimental effect and a table for its rapid estimation. Psychological Bulletin, 70, 245-251.
21.
Glass, G. V (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5(10), 3-8.
22.
Goodwin, L. D. , & Goodwin, W. L. (1985). Statistical techniques in AERJ articles, 1979-1983: The preparation of graduate students to read the educational research literature. Educational Researcher, 14(2), 5-11.
23.
Hays, W. L. (1981). Statistics (3rd ed.). New York: Holt, Rinehart & Winston.
24.
Hedges, L. V. (1981). Distribution theory for Glass’s estimator of effect sizes and related estimators. Journal of Educational Statistics, 6, 107-128.
25.
Hinkle, D. E. , Wiersma, W., & Jurs, S. G. (1998). Applied statistics for the behavioral sciences (4th ed.). Boston: Houghton Mifflin.
26.
Howell, D. C. (1997). Statistical methods for psychology (4th ed.). Belmont, CA: Duxbury.
27.
Hyde, J. S. (2001). Reporting effect sizes: The roles of editors, textbook authors, and publication manuals. Educational and Psychological Measurement, 61, 225-228.
28.
Kirk, R. E. (1996). Practical significance: A concept whose time has come. Educational and Psychological Measurement, 56, 746-759.
29.
Kirk, R. E. (2001). Promoting good statistical practices: Some suggestions. Educational and Psychological Measurement, 61, 213-218.
30.
Knapp, T. R. (1978). Canonical correlation analysis: A general parametric significance testing system. Psychological Bulletin, 85, 410-416.
31.
Kromrey, J. D. , & Hines, C. V. (1996). Estimating the coefficient of cross-validity in multiple regression: A comparison of analytical and empirical methods. Journal of Experimental Education, 64, 240-266.
32.
Lindman, H. R. (1992). Analysis of variance in experimental design. New York: Springer Verlag.
33.
McLean, J. E. , & Kaufman, A. S. (2000). Editorial: Statistical significance testing and other changes to Research in the Schools. Research in the Schools, 7(2), 1-2.
34.
Meehl, P. E. (1967). Theory testing in psychology and physics: A methodological paradox. Philosophy of Science, 34, 103-115.
35.
Meehl, P. E. (1997). The problem is epistemology, not statistics: Replace significance tests by confidence intervals and quantify accuracy of risky numerical predictions. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 393-426). Mahwah, NJ: Lawrence Erlbaum.
36.
Moore, D. S. , & McCabe, G. P. (1993). Introduction to the practice of statistics (2nd ed.). New York: Freeman.
37.
Murphy, K. R. (1997). Editorial. Journal of Applied Psychology, 82, 3-5.
38.
Nickerson, R. S. (2000). Null hypothesis significance testing: A review of an old and continuing controversy. Psychological Methods, 5, 241-301.
39.
Owen, D. B. (1985). Noncentral t-distribution. In S. Kotz, N. L. Johnson, & C. B. Read (Eds.), Encyclopedia of statistical sciences (Vol. 6, pp. 286-290). New York: John Wiley.
40.
Pagano, R. R. (1990). Understanding statistics in the behavioral sciences (3rd ed.). St. Paul, MN: West.
41.
Pearson, E. S. , & Hartley, H. O. (1972). Biometrika tables for statisticians (Vol. 2). Cambridge, UK: Cambridge University Press.
42.
Rosenthal, R. (1994). Parametric measures of effect size. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 231-244). New York: Russell Sage.
43.
Schmidt, F. L. (1996). Statistical significance testing and cumulative knowledge in psychology: Implications for the training of researchers. Psychological Methods, 1, 115-129.
44.
Smithson, M. (2000a, November). Noncentral confidence intervals. Paper presented at a workshop for the School of Psychology, Latrobe University, Melbourne.
45.
Smithson, M. (2000b). Statistics with confidence. London: Sage.
46.
Smithson, M. (2001). Correct confidence intervals for various regression effect sizes and parameters: The importance of noncentral distributions in computing intervals. Educational and Psychological Measurement, 61, 605-632.
47.
Snyder, P. (2000). Guidelines for reporting results of group quantitative investigations. Journal of Early Intervention, 23, 145-150.
48.
Snyder, P. , & Lawson, S. (1993). Evaluating results using corrected and uncorrected effect size estimates. Journal of Experimental Education, 61, 334-349.
49.
Steiger, J. H. , & Fouladi, R. T. (1992). R2: A computer program for interval estimation, power calculation, and hypothesis testing for the squared multiple correlation. Behavior Research Methods, Instruments, and Computers, 4, 581-582.
50.
Steiger, J. H. , & Fouladi, R. T. (1997). Noncentrality interval estimation and the evaluation of statistical models. In L. L. Harlow, S. A. Mulaik & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 221-257). Mahwah, NJ: Lawrence Erlbaum.
51.
Thompson, B. (1994). Guidelines for authors. Educational and Psychological Measurement, 54, 837-847.
52.
Thompson, B. (1996). AERA editorial policies regarding statistical significance testing: Three suggested reforms. Educational Researcher, 25(2), 26-30.
53.
Thompson, B. (1998). In praise of brilliance: Where that praise really belongs. American Psychologist, 53, 799-800.
54.
Thompson, B. (1999a, April). Common methodology mistakes in educational research, revisited, along with a primer on both effect sizes and the bootstrap. Invited address presented at the annual meeting of the American Educational Research Association, Montreal. (ERIC Document Reproduction Service No. ED 429 110)
55.
Thompson, B. (1999b). Journal editorial policies regarding statistical significance tests: Heat is to fire as p is to importance. Educational Psychology Review, 11, 157-169.
56.
Thompson, B. (2000). Canonical correlation analysis. In L. Grimm & P. Yarnold (Eds.), Reading and understanding more multivariate statistics (pp. 285-316). Washington, DC: American Psychological Association.
57.
Thompson, B. (in press). “Statistical,”“practical,” and “clinical”: How many kinds of significance do counselors need to consider?Journal of Counseling and Development.
58.
Thompson, B. , & Kieffer, K. M. (2000). Interpreting statistical significance test results: A proposed new “What if” method. Research in the Schools, 7(2), 3-10.
59.
Tietjen, G. L. (1986). A topical dictionary of statistics. New York: Chapman and Hall.
60.
Vacha-Haase, T. , Nilsson, J. E., Reetz, D. R., Lance, T. S., & Thompson, B. (2000). Reporting practices and APA editorial policies regarding statistical significance and effect size. Theory & Psychology, 10, 413-425.
61.
Vaughn, G. M. , & Corballis, M. C. (1969). Beyond tests of significance: Estimating strengths of effects in selected ANOVA designs. Psychological Bulletin, 72, 204-223.
62.
Wilkinson, L. , & APA Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594-604. [reprint available through the APA Home Page: http://www.apa.org/journals/amp/amp548594.html]
63.
Willson, V. L. (1980). Research techniques in AERJ articles: 1969 to 1978. Educational Researcher, 9(6), 5-10.
