Continued discussion and debate regarding the appropriate use of null hypothesis significance testing (NHST) has led to greater reliance on effect size testing (EST) in published literature. This article examines the myth that uncritical replacement of NHST with EST will improve our science. The use of NHST and EST is described along with a summary of the arguments offered in support and against both. After addressing the veracity of these assertions, the article describes the concept of the translation mechanism and compares the success of NHST and EST as mechanisms. Finally, the authors suggest changes that may facilitate translation in future research.
Carnap, R. ( 1936). Testability and meaning. Philosophy of Science, 3, 419-447.
2.
Cohen, J. ( 1988). Statistical power analysis for the behavioral sciences . Hillsdale, NJ: Lawrence Erlbaum .
3.
Cohen, J. ( 1994). The Earth is round (p < .05). American Psychologist, 49, 997-1003.
4.
Cortina, J.M., & Chen, G. ( 2004, April). A framework for conceptualizing and supporting inferences from the observational design. Paper presented at the 19th Annual Conference of the Society for Industrial and Organizational Psychology, Chicago.
5.
Cortina, J.M., & Deshon, R.P. ( 1998). Determining relative importance of predictors with the observational design. Journal of Applied Psychology, 83, 798-804.
6.
Cortina, J.M., & Dunlap, W.P. ( 1997). On the logic and purpose of significance testing. Psychological Methods, 2, 161-172.
7.
Cortina, J.M., & Landis, R.S. ( 2009). When small effect sizes tell a big story, and when large effect sizes don’t. In C. E. Lance & R. J. Vandenberg (Eds.), Statistical and methodological myths and urban legends: Doctrine, verity, and fable in the organizational and social sciences (pp. 287-308). New York: Routledge.
8.
Cortina, J.M., & Nouri, H. ( 2000). Effect size for ANOVA designs. Thousand Oaks, CA: Sage.
9.
Cumming, G., & Fidler, F. ( 2009). Confidence intervals: Better answers to better questions . Zeitschrift fur psychologie, 217, 15-26.
10.
DeCoster, J., Iselin, A.M.R., & Gallucci, M. ( 2009). A conceptual and empirical examination of justifications for dichotomization. Psychological Methods, 14, 349-366.
11.
Edwards, J.R., & Berry, J.W. ( 2010). The presence of something or the absence of nothing: Increasing theoretical precision in management research. Organizational Research Methods, 13, 668-689.
12.
Falk, R., & Greenbaum, C.W. (1995). Significance tests die hard: The amazing persistence of a probabilistic misconception. Theory and Psychology, 5, 75-98.
13.
Frick, R.W. ( 1995). Accepting the null hypothesis. Memory and Cognition, 23, 132-138.
14.
Glass, G.V., McGaw, B., & Smith, M.L. ( 1981). Meta analysis in social research. Beverly Hills, CA: Sage.
15.
Hemphill, J.F. ( 2003). Interpreting the magnitude of correlation coefficients . American Psychologist, 58, 78-80.
16.
Hunter, J.E. ( 1997). Needed: A ban on the significance test. Psychological Science, 8, 3-7.
17.
Hunter, J.E., & Schmidt, F.L. ( 1990). Methods of meta-analysis: Correcting error and bias in research findings. Newbury Park, CA: Sage.
18.
Kant, I. ( 1781). Critique of pure reason. Riga, Latvia: Friedrich Hartknoch.
19.
Klemmert, H. ( 2004). Äquivalenz- und Effekttests in der psychologischen Forschung. [Equivalence and effect tests in psychological research]. Frankfurt/Main , Germany: Peter Lang.
20.
Kline, R.B. ( 2004). Beyond significance testing: Reforming data analysis methods in behavioral research. Washington, DC: American Psychological Association.
21.
Kuhn, T. ( 1962). The structure of scientific revolutions. Chicago: University of Chicago Press.
22.
Levine, T.R., Weber, R., Park, H.S., & Hullett, C.R. ( 2009). A communication researcher’s guide to null hypothesis significance testing and alternatives. Human Communication Research, 34, 188-209.
23.
Marewski, J.N., & Olsson, H. ( 2009). Beyond the null ritual: Formal modeling of psychological processes. Zeitschrift fur psychologie, 217, 15-26.
24.
Meehl, P. ( 1967). Theory-testing in psychology and physics: A methodological paradox. Philosophy of Science, 34, 103-115.
25.
Meehl, P.E. ( 1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of Consulting and Clinical Psychology, 46, 806-834.
26.
Mill, J.S. ( 1872). A system of logic: Ratiocinative and inductive (8th ed.). London: Longmans, Green, Reader, and Dyer.
27.
Miller, R.W. ( 1979). Reason and commitment in the social sciences. Philosophy and Public Affairs, 8, 241-266.
28.
Miller, R.W. ( 1987). Fact and method: Explanation, confirmation, and reality in the natural and the social sciences. Princeton, NJ: Princeton University Press.
29.
Murphy, K.R., & Myors, B. ( 1999). Testing the hypothesis that treatments have negligible effects: Minimum-effect tests in the general linear model. Journal of Applied Psychology, 84, 234-248.
30.
Neyman, J., & Pearson, E.S. ( 1928). On the use and interpretation of certain test criteria for the purposes of statistical inference. Biometrika, 20a, 175-263.
31.
Neyman, J., & Pearson, E.S. ( 1935). On the testing of statistical hypotheses in relation to probabilities a priori. Proceedings of the Cambridge Philosophical Society, 29, 492.
32.
Nickerson, R.S. ( 2000). Null hypothesis significance testing: A review of an old and continuing controversy. Psychological Methods, 5, 241-301.
33.
Quine, W.V. ( 1960). Word and object. Cambridge: MIT Press.
34.
Quine, W.V. ( 1968). Epistemology naturalized. Paper presented at the 14th International Congress of Philosophy, Vienna.
35.
Rodgers, J.L. ( 2010). The epistemology of mathematical and statistical modeling: A quiet methodological revolution. American Psychologist, 65, 1-12.
36.
Schmidt, F.L. ( 1996). Statistical significance testing and cumulative knowledge in psychology: Implications for training of researchers. Psychological Methods, 1, 115-129.
37.
Serlin, R.C., & Laspley, D.K. ( 1985). Rationality in psychological research: The good enough principle. American Psychologist, 40, 73-83.
38.
Shrout, P.E. ( 1997). Should significance tests be banned? Introduction to a special section exploring the pros and cons. Psychological Science, 8, 1-2.
39.
Stam, H.J. ( 2006). Pythagoreanism, meaning and the appeal to number. New Ideas in Psychology, 24, 240-251.
40.
Tanaka, J.S. ( 1993). Multifaceted conceptions of fit in structural equation models. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 10-39). Newbury Park, CA: Sage.
41.
Thompson, B. ( 2001). Significance, effect sizes, stepwise methods, and other issues: Strong arguments move the field. Journal of Experimental Education, 70, 80-93.
42.
Wilkinson, L. ( 1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594-604.
