Science progresses by finding and correcting problems in theories. Good theories are those that help facilitate this process by being hard to vary: They explain what they are supposed to explain, they are consistent with other good theories, and they are not easily adaptable to explain anything. Here we argue that, rather than a lack of distinction between exploratory and confirmatory research, an abundance of flexible theories is a better explanation for the current replicability problems of psychology. We also explain why popular methods-oriented solutions fail to address the real problem of flexibility. Instead, we propose that a greater emphasis on theory criticism by argument might improve replicability.
AczelB.SzasziB.SarafoglouA.KekecsZ.KucharskýŠ.BenjaminD.. . . WagenmakersE.-J. (2020). A consensus-based transparency checklist. Nature Human Behaviour, 4, 4–6. https://doi.org/10.1038/s41562-019-0772-6
2.
BarghJ. A. (2006). What have we been priming all these years? On the development, mechanisms, and ecology of nonconscious social behavior. European Journal of Social Psychology, 36, 147–168. https://doi.org/10.1002/ejsp.336
3.
CarselT.DemosA. P.MotylM. (2018). Strong scientific theorizing is needed to improve replicability in psychological science. Behavioral and Brain Sciences, 41, Article e123. https://doi.org/10.1017/S0140525X1800078X
4.
DeutschD. (2011). The beginning of infinity: Explanations that transform the world. Allen Lane.
5.
DeutschD. (2016). The logic of experimental tests, particularly of Everettian quantum theory. Studies in History and Philosophy of Science Part B: Studies in History and Philosophy of Modern Physics, 55, 24–33. https://doi.org/10.1016/j.shpsb.2016.06.001
6.
DijksterhuisA. (2018). Reflection on the professor-priming replication report. Perspectives on Psychological Science, 13, 295–296. https://doi.org/10.1177/1745691618755705
7.
DijksterhuisA.van KnippenbergA. (1998). The relation between perception and behavior, or how to win a game of Trivial Pursuit. Journal of Personality and Social Psychology, 74, 865–877. https://doi.org/10.1037/0022-3514.74.4.865
DoyenS.KleinO.PichonC.-L.CleeremansA. (2012). Behavioral priming: It’s all in the mind, but whose mind?PLOS ONE, 7(1), Article e29081. https://doi.org/10.1371/journal.pone.0029081
FiedlerK. (2017). What constitutes strong psychological science? The (neglected) role of diagnosticity and a priori theorizing. Perspectives on Psychological Science, 12, 46–61. https://doi.org/10.1177/1745691616654458
12.
FiedlerK. (2018). The creative cycle and the growth of psychological science. Perspectives on Psychological Science, 13, 433–438. https://doi.org/10.1177/1745691617745651
13.
GrayK. (2017). How to map theory: Reliable methods are fruitless without rigorous theory. Perspectives on Psychological Science, 12, 731–741. https://doi.org/10.1177/1745691617691949
14.
HarrisC. R.CoburnN.RohrerD.PashlerH. (2013). Two failures to replicate high-performance-goal priming effects. PLOS ONE, 8(8), Article e72467. https://doi.org/10.1371/journal.pone.0072467
15.
MiguelE.CamererC.CaseyK.CohenJ.EsterlingK. M.GerberA.. . . Van der LaanM. (2014). Promoting transparency in social science research. Science, 343, 30–31. https://doi.org/10.1126/science.1245317
16.
NavarroD. J. (2018). Between the devil and the deep blue sea: Tensions between scientific judgement and statistical model selection. Computational Brain & Behavior, 2, 28–34. https://doi.org/10.1007/s42113-018-0019-z
17.
NewellB. R.ShawB. (2017). Priming risky choice: Do risk preferences need inferences?Journal of Behavioral Decision Making, 30, 332–346. https://doi.org/10.1002/bdm.1945
NosekB. A.EbersoleC. R.DeHavenA. C.MellorD. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences, USA, 115, 2600–2606. https://doi.org/10.1073/pnas.1708274114
O’DonnellM.NelsonL. D.AckermannE.AczelB.AkhtarA.AldrovandiS.. . . ZrubkaM. (2018). Registered Replication report: Dijksterhuis and van Knippenberg (1998). Perspectives on Psychological Science, 13, 268–294. https://doi.org/10.1177/1745691618755704
22.
PashlerH.CoburnN.HarrisC. R. (2012). Priming of social distance? Failure to replicate effects on social and food judgments. PLOS ONE, 7(8), Article e42510. https://doi.org/10.1371/journal.pone.0042510
23.
PashlerH.HarrisC. R. (2012). Is the replicability crisis overblown? Three arguments examined. Perspectives on Psychological Science, 7, 531–536. https://doi.org/10.1177/1745691612463401
24.
PopperK. (1959). The logic of scientific discovery. London, England: Routledge.
SanbonmatsuD. M.JohnstonW. A. (2019). Redefining science: The impact of complexity on theory development in social and behavioral research. Perspectives on Psychological Science, 14, 672–690. https://doi.org/10.1177/1745691619848688
ShiffrinR. M.BörnerK.StiglerS. M. (2018). Scientific progress despite irreproducibility: A seeming paradox. Proceedings of the National Academy of Sciences, USA, 115, 2632–2639. https://doi.org/10.1073/pnas.1711786114
SimonsD. J.HolcombeA. O.SpellmanB. A. (2014). An introduction to registered replication reports at perspectives on psychological science. Perspectives on Psychological Science, 9, 552–555. https://doi.org/10.1177/1745691614543974
32.
SzollosiA.DonkinC. (2019). Neglected sources of flexibility in psychological theories: From replicability to good explanations. Computational Brain & Behavior, 2, 190–192. https://doi.org/10.1007/s42113-019-00045-y
33.
SzollosiA.KellenD.NavarroD. J.ShiffrinR.van RooijI.Van ZandtT.DonkinC. (2020). Is preregistration worthwhile?Trends in Cognitive Sciences, 24, 94–95. https://doi.org/10.1016/j.tics.2019.11.009
WagenmakersE.-J.WetzelsR.BorsboomD.van der MaasH. L. J.KievitR. A. (2012). An agenda for purely confirmatory research. Perspectives on Psychological Science, 7, 632–638. https://doi.org/10.1177/1745691612463078
36.
ZwaanR. A.EtzA.LucasR. E.DonnellanM. B. (2018). Making replication mainstream. Behavioral and Brain Sciences, 41, Article e120. https://doi.org/10.1017/S0140525X17001972
