The Conceptual,Cunning,and Conclusive Experiment in Psychology

Abstract

The ideal experiment in physics must be conceptual, cunning, and conclusive. Adoption of these same standards in psychology has led to experiments that are uninformative and frivolous. We explain why we believe that psychology is fundamentally incompatible with hypothesis-driven theoretical science and conclude that this erodes the logic behind recent proposals to improve psychological research, such as stricter statistical standards, preregistration, and replication. The failure of psychology is not that it is somehow insufficiently scientific but rather that it makes inordinate use of methods that are a mismatch for the aspirations of researchers in the field, at the expense of valuable empirical research.

Keywords

philosophy history individual differences scientific methodology

Get full access to this article

View all access options for this article.

References

Benjamin

D. J.

Berger

J. O.

Johannesson

Nosek

B. A.

Wagenmakers

E.-J.

Berk

Bollen

K. A.

Brembs

Brown

Camerer

Cesarini

Chambers

C. D.

Clyde

Cook

T. D.

De Boeck

Dienes

Dreber

Easwaran

Efferson

. . . Johnson

V. E.

(2018). Redefine statistical significance. Nature Human Behaviour, 2, 6–10. https://doi.org/10.1038/s41562-017-0189-z

Blum

(2018, June 7). The lifespan of a lie. Gen. https://gen.medium.com/the-lifespan-of-a-lie-d869212b1f62

Borghi

A. M.

Fini

(2019). Theories and explanations in psychology. Frontiers in Psychology, 10, Article 958. https://doi.org/10.3389/fpsyg.2019.00958

Bullock

J. G.

Green

D. P.

S. E.

(2010). Yes, but what’s the mechanism? (Don’t expect an easy answer). Journal of Personality and Social Psychology, 98(4), 550–558. https://doi.org/10.1037/a0018933

Burghardt

Bodansky

A. N.

(2021). Why psychology needs to stop striving for novelty and how to move towards theory-driven research. Frontiers in Psychology, 12, Article 609802. https://doi.org/10.3389/fpsyg.2021.609802

Cahalan

(2019). The great pretender: The undercover mission that changed our understanding of madness. Grand Central Publishing.

Campbell

D. T.

(1979). A tribal model of the social system vehicle carrying scientific knowledge. Science Communication, 1(2), 181–201. https://doi.org/10.1177/107554707900100202

Chiang

H. H.-H.

(2008). Effecting science, affecting medicine: Homosexuality, the Kinsey reports, and the contested boundaries of psychopathology in the United States, 1948-1965. Journal of the History of the Behavioral Sciences, 44(4), 300–318. https://doi.org/10.1002/jhbs.20343

Coles

(2019). Einstein, Eddington and the 1919 eclipse. Nature, 568(7752), 306–307. https://doi.org/10.1038/d41586-019-01172-z

10.

Cronbach

L. J.

(1957). The two disciplines of scientific psychology. American Psychologist, 12(11), 671–684. https://doi.org/10.1007/s10440-009-9506-5

11.

D’Andrade

(1986). Three scientific world views and the covering law model. In Fiske

D. W.

Shweder

R. A.

(Eds.), Metatheory in social science: Pluralisms and subjectivities (pp. 19–41). University of Chicago Press.

12.

Diemand-Yauman

Oppenheimer

D. M.

Vaughan

E. B.

(2011). Fortune favors the bold (and the italicized): Effects of disfluency on educational outcomes. Cognition, 118(1), 111–115. https://doi.org/10.1016/j.cognition.2010.09.012

13.

Elster

(2015). Explaining social behavior: More nuts and bolts for the social sciences. Cambridge University Press.

14.

Engelhardt

C. R.

Bartholow

B. D.

Kerr

G. T.

Bushman

B. J.

(2011). This is your brain on violent video games: Neural desensitization to violence predicts increased aggression following violent video game exposure. Journal of Experimental Social Psychology, 47(5), 1033–1036. https://doi.org/10.1016/j.jesp.2011.03.027

15.

Eronen

M. I.

Bringmann

L. F.

(2021). The theory crisis in psychology: How to move forward. Perspectives on Psychological Science, 16(4), 779–788. https://doi.org/10.1177/1745691620970586

16.

Fried

E. I.

(2020). Lack of theory building and testing impedes progress in the factor and network literature. Psychological Inquiry, 31(4), 271–288. https://doi.org/10.1080/1047840X.2020.1853461

17.

Gigerenzer

(2010). Personal reflections on theory and psychology. Theory & Psychology, 20(6), 733–743. https://doi.org/10.1177/0959354310378184

18.

Gigerenzer

(2018). The bias bias in behavioral economics. Review of Behavioral Economics, 5, 303–336. https://doi.org/10.1561/105.00000092

19.

Greenwald

A. G.

Pratkanis

A. R.

Leippe

M. R.

Baumgardner

M. H.

(1986). Under what conditions does theory obstruct research progress? Psychological Review, 93(2), 216–229.

20.

Griggs

R. A.

Christopher

A. N.

(2016). Who’s who in introductory psychology textbooks: A citation analysis redux. Teaching of Psychology, 43(2), 108–119. https://doi.org/10.1177/0098628316636276

21.

Gurney

R. W.

Mott

N. F.

(1938). The theory of the photolysis of silver bromide and the photographic latent image. Proceedings of the Royal Society of London A: Mathematical and Physical Sciences, 164(917), 151–167.

22.

Henrich

Heine

S. J.

Norenzayan

(2010). The weirdest people in the world? The Behavioral and Brain Sciences, 33, 61–83; discussion 83–135. https://doi.org/10.1017/S0140525X0999152X

23.

Hossenfelder

(2018). Lost in math. Basic Books.

24.

Ioannidis

J. P. A.

(2014). How to make more published research true. PLOS Medicine, 11(10), Article e1001747. https://doi.org/10.1371/journal.pmed.1001747

25.

Judson

H. F.

(2013). The eighth day of creation. Cold Spring Harbor Laboratory Press.

26.

Laudan

(1981). Hume (and hacking) on induction. In Laudan

(Ed.), Science and hypothesis (pp. 72–85). Springer. https://doi.org/10.1007/978-94-015-7288-0_6

27.

Levitt

S. D.

List

J. A.

(2011). Was there really a Hawthorne effect at the Hawthorne plant? An analysis of the original illumination experiments. American Economic Journal: Applied Economics, 3(1): 224–238. https://doi.org/10.1257/app.3.1.224

28.

Lykken

D. T.

(1991). What’s wrong with psychology anyway? In Cicchetti

Grove

W. M.

(Eds.), Thinking clearly about psychology (Vol. 1, pp. 1–13). University of Minnesota Press.

29.

Mayr

(1963). The new versus the classical in science. Science, 141(3583), 765. https://doi.org/10.1126/science.141.3583.765

30.

McGuire

W. J.

(2013). An additional future for psychological science. Perspectives on Psychological Science, 8(4), 414–423. https://doi.org/10.1177/1745691613491270

31.

McPhetres

Albayrak-Aydemir

Barbosa Mendes

Chow

E. C.

Gonzalez-Marquez

Loukras

Maus

O’Mahony

Pomareda

Primbs

M. A.

Sackman

S. L.

Smithson

C. J. R.

Volodko

(2021). A decade of theory as reflected in Psychological Science (2009-2019). PLOS ONE, 16(3), Article e0247986. https://doi.org/10.1371/journal.pone.0247986

32.

Meehl

P. E.

(1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of Consulting and Clinical Psychology, 46(113), 806–834.

33.

Mischel

(2008). The toothbrush problem. APS Observer, 21(11). https://www.psychologicalscience.org/observer/the-toothbrush-problem

34.

Muthukrishna

Henrich

(2019). A problem in theory. Nature Human Behaviour, 3, 221–229. https://doi.org/10.1038/s41562-018-0522-1

35.

Nosek

B. A.

Ebersole

C. R.

DeHaven

A. C.

Mellor

D. T.

(2018). The preregistration revolution. Proceedings of the National Academy of Sciences, USA, 115(11), 2600–2606. https://doi.org/10.1073/pnas.1708274114

36.

Oberauer

Lewandowsky

(2019). Addressing the theory crisis in psychology. Psychonomic Bulletin and Review, 26(5), 1596–1618. https://doi.org/10.3758/s13423-019-01645-2

37.

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), Article aac4716. https://doi.org/10.1126/science.aac4716

38.

Perry

M. F.

(2007). Remembering the oil-drop experiment. Physics Today, 60(5), 56–60. https://doi.org/10.1063/1.2743125

39.

Platt

J. R.

(1964). Strong inference. Science, 146(3642), 347–353.

40.

Poincaré

(1902). La Science et l’hypothèse [Science and hypothesis]. Ernest Flammarion.

41.

Rautio

(2014, December 1). The long road to Maxwell’s equations. IEEE Spectrum. https://spectrum.ieee.org/tech-history/dawn-of-electronics/the-long-road-to-maxwells-equations

42.

Ring

(1967). Experimental social psychology: Some sober questions about some frivolous values. Journal of Experimental Social Psychology, 3, 113–123.

43.

Roberts

Pashler

(2000). How persuasive is a good fit? A comment on theory testing. Psychological Review, 107(2), 358–367. https://doi.org/10.1037//0033-295X.107.2.358

44.

Roethlisberger

F. J.

Dickson

W. J.

(2004). Management and the worker (Vol. 5). Routledge.

45.

Scheel

A. M.

Tiokhin

Isager

P. M.

Lakens

(2021). Why hypothesis testers should spend less time testing hypotheses. Perspectives on Psychological Science, 16(4), 744–755. https://doi.org/10.1177/1745691620966795

46.

Schimmack

(2012). The ironic effect of significant results on the credibility of multiple-study articles. Psychological Methods, 17(4), 551–566. https://doi.org/10.1037/a0029487

47.

Skinner

B. F.

(1987). Whatever happened to psychology as a science of behavior? American Psychologist, 42(8), 780–786. https://doi.org/10.1037/0003-066X.42.8.780

48.

Stoney

G. J.

(1894). Of the ‘electron,’ or atom of electricity. Philosophical Magazine, 38(233), 418–420. https://doi.org/10.1080/14786449408620653

49.

Terman

L. M.

(1925). Genetic studies of genius (Vol. 1). Stanford University Press.

50.

Tversky

Kahneman

(1982). Judgments of and by representativeness. In Kahneman

Slovic

Tversky

(Eds.), Judgment under uncertainty (pp. 84–98). Cambridge University Press. https://doi.org/10.1017/CBO9780511809477.007

51.

Wade

(1982, April 30). The editorial notebook: Smart apes, or dumb? The New York Times, A30.

52.

Wigner

E. P.

(1960). The unreasonable effectiveness of mathematics in the natural sciences. Richard Courant Lecture in Mathematical Sciences delivered at New York University, May 11, 1959. Communications on Pure and Applied Mathematics, 13, 1–14. https://doi.org/10.1002/cpa.3160130102

53.

Yarkoni

(2020). The generalizability crisis. Behavioral and Brain Sciences. Advance online publication. https://doi.org/10.1017/S0140525X20001685