Sage Journals: Discover world-class research

Abstract

Because the widespread use of statistical significance testing has deleterious consequences for the development of a cumulative knowledge base, the American Psychological Association's Board of Scientific Affairs is in the process of appointing a Task Force whose charge includes the possibility of phasing out such testing in textbooks and journal articles. Just how popular is significance testing in psychology? This issue is examined in the present historical study, which uses data from randomly selected issues of the Journal of Applied Psychology for the period 1917-94. Results indicate that the practice of significance testing, at one time of restricted usage, has expanded to the point that it is virtually synonymous with empirical analysis. The data also lend support to Gigerenzer and Murray's (1987) allegation that an inference revolution occurred in psychology during the period 1940-55. Unfortunately, it is concluded that the ubiquity of significance testing constitutes a classic example of the overadoption of a methodology.

Keywords

inference revolution p values replication significance testing teaching statistics

Get full access to this article

View all access options for this article.

References

Bakan, D. (1966). The test of significance in psychological research. Psychological Bulletin, 66, 423-437.

Carver, R.P. (1978). The case against statistical significance testing. Harvard Educational Review, 48, 378-399.

Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45, 1304-1312.

Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49, 997-1003.

Falk, R. , & Greenbaum, C.W. (1995). Significance tests die hard: The amazing persistence of a probabilistic misconception. Theory & Psychology, 5, 75-98.

Gigerenzer, G. (1993). The superego, the ego, and the id in statistical reasoning. In G. Keren & C. A. Lewis (Eds.), A handbook for data analysis in the behavioral sciences-methodological issues (pp. 311-339). Hillsdale, NJ: Erlbaum.

Gigerenzer, G. , & Murray, D.J. (1987). Cognition as intuitive statistics. Hillsdale, NJ: Erlbaum.

Gigerenzer, G. , Swijtink, Z. , Porter, T. , Daston, L. , Beatty, J. , & Kruger, L. (1989). The empire of chance. New York: Cambridge University Press.

Goodman, S.N. (1993). p values, hypothesis tests, and likelihood: Implications for epidemiology of a neglected historical debate. American Journal of Epidemiology, 137, 485-496.

10.

Greenwald, A.G. (1975). Consequences of prejudice against the null hypothesis. Psychological Bulletin, 82, 1-20.

11.

Harris, M.J. (1991). Significance tests are not enough: The role of effect-size estimation in theory corroboration. Theory & Psychology, 1, 375-382.

12.

Hubbard, R. (1995). The earth is highly significantly round (p < .0001). American Psychologist, 50, 10-98.

13.

Hubbard, R. , & Armstrong, J.S. (1992). Are null results becoming an endangered species in marketing? Marketing Letters, 3, 127-136.

14.

Hubbard, R. , & Armstrong, J.S. (1994). Replications and extensions in marketing: Rarely published but quite contrary. International Journal of Research in Marketing, 11, 233-248.

15.

Hubbard, R. , & Vetter, D.E. (1996). An empirical comparison of published replication research in accounting, economics, finance, management, and marketing. Journal of Business Research, 35, 153-164.

16.

Huberty, C.J. (1993). Historical origins of statistical testing practices: The treatment of Fisher versus Neyman-Pearson views in textbooks. Journal of Experimental Education, 61, 317-333.

17.

Johnstone, D.J. (1986). Tests of significance in theory and practice (with comments). The Statistician, 35, 491-504.

18.

Kendall, M.G. (1942). On the future of statistics. Journal of the Royal Statistical Society, 105, 69-80.

19.

Kupfersmid, J. , & Fiala, M. (1991). A survey of attitudes and behaviors of authors who publish in psychology and education journals. American Psychologist, 46, 249-250.

20.

Lehmann, E.L. (1993). The Fisher, Neyman-Pearson theories of testing hypotheses: One theory or two? Journal of the American Statistical Association, 88, 1242-1249.

21.

Lindsay, R.M. , & Ehrenberg, A. S. C. (1993). The design of replicated studies. American Statistician, 47, 217-228.

22.

Meehl, P.E. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of Consulting and Clinical Psychology, 46, 806-834.

23.

Oakes, M. (1986). Statistical inference: A commentary for the social and behavioral sciences. New York: Wiley.

24.

Rogers, E.M. , & Shoemaker, F.F. (1971). Communication of innovations: A cross-cultural approach. New York: Free Press.

25.

Rosnow, R.L. , & Rosenthal, R. (1989). Statistical procedures and the justification of knowledge in psychological science. American Psychologist, 44, 1276-1284.

26.

Rucci, A.J. , & Tweney, R.D. (1980). Analysis of variance and the `second discipline' of scientific psychology: A historical account. Psychological Bulletin, 87, 166-184.

27.

Schmidt, F. (1996). Statistical significance testing and cumulative knowledge in psychology: Implications for the training of researchers. Psychological Methods, 1, 115-129.

28.

Shaver, J.P. (1993). What statistical significance testing is, and what it is not. Journal of Experimental Education, 61, 293-316.

The Spread of Statistical Significance Testing in Psychology

Abstract

Keywords

Get full access to this article

References