This book review first explores the treatment within the volume What If There Were No Significance Tests? of five selected major themes: (a) effect sizes, (b) the "nil" null hypothesis, (c) power and the "file drawer" problem, (d) language use, and (e) replicability evidence. Relevant literature is also cited in each area. Then, a brief review of the 14 chapters is presented, and some summary comments are offered.
Get full access to this article
View all access options for this article.
References
1.
Abelson, R. P. (1997). A retrospective on the significance test ban of 1999 (if there were no significance tests, they would be invented). In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 117-141). Mahwah, NJ: Lawrence Erlbaum.
2.
American Psychological Association (APA). (1994). Publication manual of the American Psychological Association (4th ed.). Washington, DC: Author.
3.
Carver, R. (1993). The case against statistical significance testing, revisited. Journal of Experimental Education, 61(4), 287-292.
4.
Cohen, J. (1994). The Earth is round (p .05). American Psychologist, 49, 997-1003.
5.
Cohen, J. (1997a). The Earth is round (p < .05). In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 22-35). Mahwah, NJ: Lawrence Erlbaum.
6.
Cohen, J. (1997b, August). Much ado about nothing. Lecture presented at the annual meeting of the American Psychological Association, Chicago.
7.
Glass, G. V. (1979). Policy for the unpredictable (uncertainty research and policy). Educational Researcher, 8(9), 12-14.
8.
Greenwald, A. G. (1975). Consequences of prejudice against the null hypothesis. Psychological Bulletin, 82, 1-20.
9.
Hagen, R. L. (1997). In praise of the null hypothesis statistical test. American Psychologist, 52, 15-24.
10.
Harlow, L. L. (1997). Significance testing introduction and overview. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 1-17). Mahwah, NJ: Lawrence Erlbaum.
11.
Harlow, L. L. , Mulaik, S. A., & Steiger, J. H. (1997). Preface. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. xiii-xiv). Mahwah, NJ: Lawrence Erlbaum.
12.
Harris, R. J. (1997). Reforming significance testing via three-valued logic. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 145-174). Mahwah, NJ: Lawrence Erlbaum.
13.
Heldref Foundation. (1997). Guidelines for contributors. Journal of Experimental Education, 65, 95-96.
14.
Huberty, C. J. (1993). Historical origins of statistical testing practices: The treatment of Fisher versus Neyman-Pearson views in textbooks. Journal of Experimental Education, 61, 317-333.
15.
Hunter, J. E. (1997). Needed: Aban on the significance test. Psychological Science, 8(1), 3-7.
16.
Kirk, R. (1996). Practical significance: A concept whose time has come. Educational and Psychological Measurement, 56, 746-759.
17.
McDonald, R. P. (1997). Goodness of approximation in the linear model. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 199-219). Mahwah, NJ: Lawrence Erlbaum.
18.
Meehl, P. E. (1997). The problem is epistemology, not statistics: Replace significance tests by confidence intervals and quantify accuracy of risky numerical predictions. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 393-426). Mahwah, NJ: Lawrence Erlbaum.
19.
Mulaik, S. A. , Raju, N. S., & Harshman, R. A. (1997). There is a time and place for significance testing. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 65-115). Mahwah, NJ: Lawrence Erlbaum.
20.
Murphy, K. R. (1997). Editorial. Journal of Applied Psychology, 82, 3-5.
21.
Nelson, N. , Rosenthal, R., & Rosnow, R. L. (1986). Interpretation of significance levels and effect size by psychological researchers. American Psychologist, 41, 1299-1301.
22.
Oakes, M. (1986). Statistical inference: A commentary for the social and behavioral sciences. New York: John Wiley.
23.
Olejnik, S. F. (1984). Planning educational research: Determining the necessary sample size. Journal of Experimental Education, 53, 40-48.
24.
Reichardt, C. S. , & Gollob, H. F. (1997). When confidence intervals should be used instead of statistical significance tests, and vice versa. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 259-284). Mahwah, NJ: Lawrence Erlbaum.
25.
Rindskopf, D. M. (1997). Testing "small," not null, hypotheses: Classical and Bayesian approaches. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 319-334). Mahwah, NJ: Lawrence Erlbaum.
26.
Rosenthal, R. (1979). The "file drawer problem" and tolerance for null results. Psychological Bulletin, 86, 638-641.
27.
Rossi, J. S. (1997). A case study in the failure of psychology as a cumulative science: The spontaneous recovery of verbal learning. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 176-197). Mahwah, NJ: Lawrence Erlbaum.
28.
Rozeboom, W.W. (1997). Good science is abductive, nothypothetico-deductive. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 335-392). Mahwah, NJ: Lawrence Erlbaum.
29.
Schmidt, F. (1996). Statistical significance testing and cumulative knowledge in psychology: Implications for the training of researchers. Psychological Methods, 1(2), 115-129.
30.
Schmidt, F. L. , & Hunter, J. E. (1997). Eight common but false objections to the discontinuation of significance testing in the analysis of research data. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 37-64). Mahwah, NJ: Lawrence Erlbaum.
31.
Shea, C. (1996). Psychologists debate accuracy of "significance test."Chronicle of Higher Education, 42(49), pp. A12, A16.
32.
Snyder, P. , & Lawson, S. (1993). Evaluating results using corrected and uncorrected effect size estimates. Journal of Experimental Education, 61(4), 334-349.
33.
Snyder, P. A. , & Thompson, B. (in press). Use of tests of statistical significance and other analytic choices in a school psychology journal: Review of practices and suggested alternatives. School Psychology Quarterly.
34.
Steiger, J. H. , & Fouladi, R. T. (1997). Noncentrality interval estimation and the evaluation of statistical models. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 221-257). Mahwah, NJ: Lawrence Erlbaum.
35.
Thompson, B. (1993). The use of statistical significance tests in research: Bootstrap and other alternatives. Journal of Experimental Education, 61, 361-377.
36.
Thompson, B. (1994). Guidelines for authors. Educational and Psychological Measurement, 54, 837-847.
37.
Thompson, B. (1996). AERA editorial policies regarding statistical significance testing: Three suggested reforms. Educational Researcher, 25(2), 26-30.
38.
Thompson, B. (1997a, August). If statistical significance tests are broken/misused, what practices should supplement or replace them?. Address presented at the annual meeting of the American Psychological Association, Chicago.
39.
Thompson, B. (1997b). The importance of structure coefficients in structural equation modeling confirmatory factor analysis. Educational and Psychological Measurement, 57, 5-19.
40.
Thompson, B. (1998, April). Common methodology errors in educational research: The pantheon of statistical significance and other faux pas. Address presented at the annual meeting of the American Educational Research Association, San Diego.
41.
Thompson, B. , & Borrello, G. M. (1985). The importance of structure coefficients in regression research. Educational and Psychological Measurement, 45, 203-209.
42.
Thompson, B. , & Snyder, P. A. (1997). Statistical significance testing practices in the Journal of Experimental Education. Journal of Experimental Education, 66, 75-83.
43.
Thompson, B. , & Snyder, P A. (in press). Statistical significance and reliability analyses in recent JCD research articles. Journal of Counseling and Development.
44.
Vacha-Haase, T. , & Nilsson, J. E. (in press). Statistical significance reporting: Current trends and usages within MECD. Measurement and Evaluation in Counseling and Development.
45.
Zuckerman, M. , Hodgins, H. S., Zuckerman, A., & Rosenthal, R. (1993). Contemporary issues in the analysis of data: A survey of 551 psychologists. Psychological Science, 4, 49-53.
