The application of statistics to science is not a neutral act. Statistical tools have shaped and were also shaped by its objects. In the social sciences, statistical methods fundamentally changed research practice, making statistical inference its centerpiece. At the same time, textbook writers in the social sciences have transformed rivaling statistical systems into an apparently monolithic method that could be used mechanically. The idol of a universal method for scientific inference has been worshipped since the “inference revolution” of the 1950s. Because no such method has ever been found, surrogates have been created, most notably the quest for significant p values. This form of surrogate science fosters delusions and borderline cheating and has done much harm, creating, for one, a flood of irreproducible results. Proponents of the “Bayesian revolution” should be wary of chasing yet another chimera: an apparently universal inference procedure. A better path would be to promote both an understanding of the various devices in the “statistical toolbox” and informed judgment to select among these.
AltmannG.2007. Editorial journal policies and procedures. Cognition, 102: 1-6.
3.
American Psychological Association. 1974. Publication manual of the American Psychological Association, 2nd ed.Baltimore, MD: Garamond/Pridemark.
4.
ArbuthnottJ.1710. An argument for divine providence, taken from the constant regularity observ’d in the births of both sexes. Philosophical Transactions of the Royal Society, 27: 186-190.
5.
BaconF.1902. Novum organum. New York: P. F. Collier & Son. (Original work published 1620)
6.
BoringE. G.1963. History, psychology, and sciences: Selected papers. New York: Wiley.
7.
BozarthJ. D.RobertsR. R.Jr.1972. Signifying significant significance. American Psychologist, 27: 774-775.
8.
BreimanL.FriedmanJ. H.OlshenR. A.StoneC. J.1993. Classification and regression trees. New York: Chapman & Hall.
9.
DanzigerK.1990. Constructing the subject: Historical origins of psychological research. Cambridge, UK: Cambridge University Press.
10.
DastonL.1988. Classical probability in the Enlightenment. Princeton, NJ: Princeton University Press.
11.
DastonL.2005. Scientific error and the ethos of belief. Social Research, 72: 1-28.
12.
EarmanJ.1992. Bayes or bust? A critical examination of Bayesian confirmation theory. Cambridge, MA: MIT Press.
FienbergS. E.2006. When did Bayesian inference become “Bayesian”?Bayesian Analysis, 1: 1-40.
15.
FisherR. A.1955. Statistical methods and scientific induction. Journal of the Royal Statistical Society, 17 (Series B): 69-77.
16.
FisherR. A.1956. Statistical methods and scientific inference. Edinburgh, UK: Oliver & Boyd.
17.
FisherR. A.1971. The design of experiments, reprint of the 8th ed.New York: Hafner. (Original work published 1935)
18.
FrancoZ. E.BlauK.ZimbardoP. G.2011. Heroism: A conceptual analysis and differentiation between heroic action and altruism. Review of General Psychology, 15: 99-113.
19.
GemanS.BienenstockE.DoursatR.1992. Neural networks and the bias/variance dilemma. Neural Computation, 4: 1-58.
GigerenzerG.1987. Probabilistic thinking and the fight against subjectivity. In KrügerL.GigerenzerG.MorganM. (Eds.), The probabilistic revolution: Vol. 2. Ideas in the sciences, 11-33. Cambridge, MA: MIT Press.
22.
GigerenzerG.2002. Calculated risks: How to know when numbers deceive you. New York: Simon & Schuster.
23.
GigerenzerG.2004. Mindless statistics. Journal of Socio-Economics, 33: 587-606.
24.
GigerenzerG.2011. What are natural frequencies?British Medical Journal, 343: d6386.
25.
GigerenzerG.BrightonH.2009. Homo heuristicus: Why biased minds make better inferences. Topics in Cognitive Science, 1: 107-143.
26.
GigerenzerG.HellW.BlankH.1988. Presentation and content: The use of base rates as a continuous variable. Journal of Experimental Psychology: Human Perception and Performance, 14: 513-525.
27.
GigerenzerG.HertwigR.PachurT. (Eds.). 2011. Heuristics: The foundations of adaptive behavior. Oxford, UK: Oxford University Press.
28.
GigerenzerG.MurrayD. J.1987. Cognition as intuitive statistics. Hillsdale, NJ: Lawrence Erlbaum.
29.
GigerenzerG.SwijtinkZ.PorterT.DastonL.BeattyJ.KrügerL.1989. The empire of chance: How probability changed science and everyday life. Cambridge, UK: Cambridge University Press.
30.
GoodI. J.1971. 46656 varieties of Bayesianism. American Statistician, 25: 62-63.
31.
GoodP. I.HardingJ. W.2003. Common errors in statistics (and how to avoid them). New York: Wiley.
32.
GuilfordJ. P.1942. Fundamental statistics in psychology and education. New York: McGraw-Hill.
33.
HackingI.1987. Prussian numbers1860-1882. In KrügerL.DastonL.HeidelbergerM. (Eds.), The probabilistic revolution: Vol. 1. Ideas in history: 377-394. Cambridge, MA: MIT Press.
34.
HallerH.KraussS.2002. Misinterpretations of significance: A problem students share with their teachers?Methods of Psychological Research—Online, 7: 1-20.
35.
HubbardR.ArmstrongJ. S.1994. Replications and extensions in marketing: Rarely published but quite contrary. International Journal of Research in Marketing, 11: 233-248.
36.
Information for contributors. 2005. Psychological Science, 16: i.
37.
International Committee of Medical Journal Editors. 1988. Uniform requirements for manuscripts submitted to biomedical journals. British Medical Journal, 296: 401-405.
38.
IoannidisJ. P. A.GreenlandS.HlatkyM. A.KhouryM. J.MacleodM. R.MoherD. . . . TibshiraniR.2014. Increasing value and reducing waste in research design, conduct, and analysis. The Lancet, 383: 166-175.
39.
IversonG. J.LeeM. D.ZhangS.WagenmakersE. J.2009. Prep: An agony in five Fits. Journal of Mathematical Psychology, 53: 195-202.
40.
JeffreysH.1961. Theory of probability. Oxford, UK: Oxford University Press.
41.
JohnL.LoewensteinG. F.PrelecD.2012. Measuring the prevalence of questionable research practices with incentives for truth-telling. Psychological Science, 23: 524-532.
42.
KahnemanD.TverskyA.1972. Subjective probability: A judgment of representativeness. Cognitive Psychology, 3: 430-454.
43.
KassR. E.RafteryA. E.1995. Bayes factors. Journal of the American Statistical Association, 90: 773-795.
44.
KerrN. L.1998. HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review, 2: 196-217.
45.
KnightF. H.1921. Risk, uncertainty, and profit. New York: Houghton Mifflin.
46.
KoehlerJ. J.1992. Probabilities in the courtroom: An evaluation of the objections and policies. In KagehiroD. K.LauferW. S. (Eds.), Handbook of psychology and law: 167-184. New York: Springer-Verlag.
47.
KrügerL.DastonL. J.HeidelbergerM. (Eds.). 1987. The probabilistic revolution: Vol. 1. Ideas in history. Cambridge, MA: MIT Press.
48.
KrügerL.GigerenzerG.MorganM. S. (Eds.). 1987. The probabilistic revolution: Vol. 2. Ideas in the sciences. Cambridge, MA: MIT Press.
49.
LaplaceP. S.1774. Mémoire sur la probabilité des causes par les évènemens. Mémoires de mathématique et de physique, presentés a l’Académie Royale des Sciences, par divers savans, & lû dans ses assemblées, 6: 621-656. (English translation in Stigler, S. M. 1986. Laplace’s 1774 memoir on inverse probability. Statistical Science, 1: 359-378.)
50.
LaplaceP. S.1951. A philosophical essay on probabilities. New York: Dover. (Original work published 1814)
51.
LeibnizG. W.1951. Toward a universal characteristic. In WienerP. P. (Ed.), Selections: 17-25. New York: Scribner’s. (Original work published 1677)
52.
LindleyD. V.1975. The future of statistics: A Bayesian 21st century. Advances in Applied Probability (Supplement), 7: 106-115.
53.
LindleyD. V.1983. Theory and practice of Bayesian statistics. Journal of the Royal Statistical Society. Series D (The Statistician), 32: 1-11.
54.
LindleyD. V.1986. Comment. American Statistician, 40: 6-7.
55.
LuceR. D.1988. The tools-to-theory hypothesis. Review of G. Gigerenzer & D. J. Murray, “Cognition as Intuitive Statistics.”Contemporary Psychology, 33: 582-583.
56.
MaddenC. S.EasleyR. W.DunnM. G.1995. How journal editors view replication research. Journal of Advertising, 24: 77-79.
57.
MaslowA. H.1966. The psychology of science. New York: Harper & Row.
58.
MeltonA. W.1962. Editorial. Journal of Experimental Psychology, 64: 553-557.
59.
NewtonI.1952. Opticks. Or a treatise of the reflections, refractions, inflections, and colours of light. New York: Dover. (Original work published 1704)
60.
NeymanJ.1957. Inductive behavior as a basic concept of philosophy of science. International Statistical Review, 25: 7-22.
61.
NunnallyJ. C.1975. Introduction to statistics for psychology and education. New York: McGraw-Hill.
62.
OakesM.1986. Statistical inference: A commentary for the social and behavioral sciences. Chichester, UK: Wiley.
63.
OlbyR.2003. Quiet debut for the double helix. Nature, 276: 565-570.
SavageL. J.1972. The foundations of statistics. New York: Dover (Original work published 1954)
66.
SimkinM. V.RoychowdhuryV. P.2003. Read before you cite!Complex Systems, 14: 269-274.
67.
SimmonsJ. P.NelsonL. D.SimonsohnU.2011. False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 20: 1-8.
68.
SimonH. A.1979. Rational-decision making in business organizations (Nobel Memorial Lecture, 8December, 1978). American Economic Review, 69: 493-513.
69.
SimonH. A.1992. What is an “explanation” of behavior?Psychological Science, 3: 150-161.
70.
SorosG.2009. The crash of 2008 and what it means: The new paradigm for financial markets. New York: Public Affairs.
71.
StegmüllerW.1973. Jenseits von Popper und Carnap [Beyond Popper and Carnap]. Berlin, Germany: Springer.
72.
StevensS. S.1960. The predicament in design and significance. Contemporary Psychology, 9: 273-276.
73.
StiglerS. M.1983. Who discovered Bayes’s theorem?American Statistician, 37: 290-296.
74.
StiglerS. M.1999. Statistics on the table: The history of statistical concepts and methods. Cambridge, MA: Harvard University Press.
75.
Style guide for authors. 2011. Academy of Management Journal, 54: 1081-1084.
76.
TressoldiP. E.GiofréD.SellaF.CummingG.2013. High impact = high statistical standards? Not necessarily so. PLOS ONE, 8: 1-7.
77.
TrottaR.2007. Applications of Bayesian model selection to cosmological parameters. Monthly Notices of the Royal Astronomical Society, 378: 72-82.
78.
TukeyJ. W.1977. Exploratory data analysis. Reading, MA: Addison-Wesley.
79.
VulE.HarrisC.WinkielmanP.PashlerH.2009. Puzzlingly high correlations in fMRI studies of emotion, personality, and social cognition. Perspectives on Psychological Science, 4: 274-290.
80.
WeickK. E.1996. Drop your tools: An allegory for organizational studies. Administrative Science Quarterly, 41: 301-313.
81.
ZiliakS. T.McCloskeyD. N.2004. Size matters: The standard error of regressions in theAmerican Economic Review. Journal of Socio-Economics, 33: 527-546.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
