Anatomy of a Psychological Theory: Integrating Construct-Validation and Computational-Modeling Methods to Advance Theorizing

Abstract

Discussions about the replicability of psychological studies have primarily focused on improving research methods and practices, with less attention paid to the role of well-specified theories in facilitating the production of reliable empirical results. The field is currently in need of clearly articulated steps to theory specification and development, particularly regarding frameworks that may generalize across different fields of psychology. Here we focus on two approaches to theory specification and development that are typically associated with distinct research traditions: computational modeling and construct validation. We outline the points of convergence and divergence between them to illuminate the anatomy of a scientific theory in psychology—what a well-specified theory should contain and how it should be interrogated and revised through iterative theory-development processes. We propose how these two approaches can be used in complementary ways to increase the quality of explanations and the precision of predictions offered by psychological theories.

Keywords

theory computational modeling construct validation

Get full access to this article

View all access options for this article.

References

Bechtel

Abrahamsen

(2005). Explanation: A mechanist alternative. Studies in History and Philosophy of Science Part C: Biological and Biomedical Sciences, 36, 421–441. https://doi.org/10.1016/j.shpsc.2005.03.010

Bechtel

Abrahamsen

(2010). Dynamic mechanistic explanation: Computational modeling of circadian rhythms as an exemplar for cognitive science. Studies in History and Philosophy of Science, 41, 321–333. https://doi.org/10.1016/j.shpsa.2010.07.003

Benjamin

D. J.

Berger

J. O.

Johannesson

Nosek

B. A.

Wagenmakers

E.-J.

Berk

Bollen

K. A.

Brembs

Brown

Camerer

Cesarini

Chambers

C. D.

Clyde

Cook

T. D.

De Boeck

Dienes

Dreber

Easwaran

Efferson

. . . Johnson

V. E.

(2018). Redefine statistical significance. Nature Human Behaviour, 2, 6–10. https://doi.org/10.1038/s41562-017-0189-z

Brandes

C. M

Reardon

K. W

Shields

A. N

Tackett

J. L

. (2021). Towards construct validity of relational aggression: An examination of the Children’s Social Behavior Scale. Psychological Assessment Advance online publication. https://doi.org/10.1037/pas0001005.

Clark

L. A.

Watson

(2019). Constructing validity: New developments in creating objective measuring instruments. Psychological Assessment, 31, 1412–1427. https://doi.org/10.1037/pas0000626

Cohen

(1994). The earth is round (p < .05). American Psychologist, 49, 997–1003. https://doi.org/10.1037/0003-066X.49.12.997

Craver

C. F.

(2006). When mechanistic models explain. Synthese, 153, 355–376. https://doi.org/10.1007/s11229-006-9097-x

Cronbach

L. J.

(1957). The two disciplines of scientific psychology. American Psychologist, 12, 671–684.

Cronbach

L. J.

Meehl

P. E.

(1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281–302. https://doi.org/10.4324/9781315128498

10.

Cummins

(2000). “How does it work?” versus “What are the laws?”: Two conceptions of psychological explanation. In Wilson

R. A.

Keil

F. C.

(Eds.), Explanation and Cognition (pp. 117–144). MIT Press.

11.

Daw

N. D.

Gershman

S. J.

Seymour

Dayan

Dolan

R. J.

(2011). Model-based influences on humans’ choices and striatal prediction errors. Neuron, 69, 1204–1215. https://doi.org/10.1016/j.neuron.2011.02.027

12.

DeYoung

C. G.

(2015). Cybernetic big five theory. Journal of Research in Personality, 56, 33–58. https://doi.org/10.1016/j.jrp.2014.07.004

13.

Dolan

R. J.

Dayan

(2013). Goals and habits in the brain. Neuron, 80, 312–325. https://doi.org/10.1016/j.neuron.2013.09.007

14.

Eldar

Rutledge

R. B.

Dolan

R. J.

Niv

(2016). Mood as representation of momentum. Trends in Cognitive Sciences, 20, 15–24. https://doi.org/10.1016/j.tics.2015.07.010

15.

Farell

Lewandowsky

(2018). Computational modeling of cognition and behavior. Cambridge University Press.

16.

Fechner

G. T.

(1987). My own viewpoint on mental measurement (1887). Psychological Research, 49, 213–219. https://doi.org/10.1007/BF00309029

17.

Fidler

Thorn

F. S.

Barnett

Kambouris

Kruger

(2018). The epistemic importance of establishing the absence of an effect. Advances in Methods and Practices in Psychological Science, 1, 237–244. https://doi.org/10.1177/2515245918770407

18.

Fiedler

(2017). What constitutes strong psychological science? The (neglected) role of diagnosticity and a priori theorizing. Perspectives on Psychological Science, 12, 46–61. https://doi.org/10.1177/1745691616654458

19.

Fischer

A. G.

Ullsperger

(2013). Real and fictive outcomes are processed differently but converge on a common adaptive mechanism. Neuron, 79, 1243–1255. https://doi.org/10.1016/j.neuron.2013.07.006

20.

Fletcher

J. A.

Doebeli

(2009). A simple and general explanation for the evolution of altruism. Proceedings of the Royal Society B: Biological Sciences, 276, 13–19. https://doi.org/10.1098/rspb.2008.0829

21.

Forstmann

B. U.

Wagenmakers

E. J.

(2015). Model-based cognitive neuroscience: A conceptual introduction. In Forstmann

B. U.

Wagenmakers

E. J.

(Eds.), An introduction to model-based cognitive neuroscience (pp. 139–156). Springer.

22.

Freud

(1959). Beyond the pleasure principle ( Strachey

, Trans.). Bantam. (Original work published 1920)

23.

Gawronski

Bodenhausen

G. V.

(2015). Theory evaluation. In Gawronski

Bodenhausen

G. V.

(Eds.), Theory and explanation in social psychology (pp. 3–23). Guilford Press.

24.

Gigerenzer

(1998). Surrogates for theories. Theory & Psychology, 8, 183–194.

25.

Grahek

Musslick

Shenhav

(2020). A computational perspective on the roles of affect in cognitive control. International Journal of Psychophysiology, 151, 25–34. https://doi.org/10.1016/j.ijpsycho.2020.02.001

26.

Grahek

Shenhav

Musslick

Krebs

R. M.

Koster

E. H. W.

(2019). Motivation and cognitive control in depression. Neuroscience and Biobehavioral Reviews, 102, 371–381. https://doi.org/10.1016/j.neubiorev.2019.04.011

27.

Gray

(2017). How to map theory: Reliable methods are fruitless without rigorous theory. Perspectives on Psychological Science, 12, 731–741. https://doi.org/10.1177/1745691617691949

28.

Greenwald

A. G.

(2012). There is nothing so theoretical as a good method. Perspectives on Psychological Science, 7, 99–108. https://doi.org/10.1177/1745691611434210

29.

Hammond

R. A.

Axelrod

(2006). Evolution of contingent altruism when cooperation is expensive. Theoretical Population Biology, 69, 333–338. https://doi.org/10.1016/j.tpb.2005.12.002

30.

Heathcote

Brown

S. D.

Wagenmakers

E. J.

(2015). An introduction to good practices in cognitive modeling. In Wagenmakers

E. J.

Forstmann

B. U.

(Eds.), An introduction to model-based cognitive neuroscience (pp. 25–48). Springer.

31.

Hedge

Powell

Sumner

(2018). The reliability paradox: Why robust cognitive tasks do not produce reliable individual differences. Behavior Research Methods, 50, 1166–1186. https://doi.org/10.3758/s13428-017-0935-1

32.

Helmholtz

(1962). Concerning the perceptions in general, treatise on physiological optics ( Southall

J. P. C.

, Trans.). Dover. (Original work published 1866)

33.

Higgins

E. T.

(1992). Increasingly complex but less interesting articles: Scientific progress or regulatory problem? Personality and Social Psychology Bulletin, 18, 489–492.

34.

Hommel

(2020). Pseudo-mechanistic explanations in psychology and cognitive neuroscience. Topics in Cognitive Science, 12, 1294–1305. https://doi.org/10.1111/tops.12448

35.

Jackson

J. C.

Rand

Lewis

Norton

M. I.

Gray

(2017). Agent-based modeling: A guide for social psychologists. Social Psychological and Personality Science, 8, 387–395.

36.

James

(1961). Psychology: The briefer course. Harper. (Original work published 1892)

37.

Klatzky

R. L.

Creswell

J. D.

(2014). An intersensory interaction account of priming effects—and their absence. Perspectives on Psychological Science, 9, 49–58. https://doi.org/10.1177/1745691613513468

38.

Klein

S. B.

(2014). What can recent replication failures tell us about the theoretical commitments of psychology? Theory & Psychology, 24, 326–338.

39.

Kotov

Krueger

R. F.

Watson

Achenbach

T. M.

Althoff

R. R.

Bagby

R. M.

Brown

T. A.

Carpenter

W. T.

Caspi

Clark

L. A.

Eaton

N. R.

Forbes

M. K.

Forbush

K. T.

Goldberg

Hasin

Hyman

S. E.

Ivanova

M. Y.

Lynam

D. R.

Markon

. . . Zimmerman

(2017). The Hierarchical Taxonomy of Psychopathology (HiTOP): A dimensional alternative to traditional nosologies. Journal of Abnormal Psychology, 126, 454–477. https://doi.org/10.1037/abn0000258

40.

Kruglanski

A. W.

(2001). That “vision thing”: The state of theory in social and personality psychology at the edge of the new millennium. Journal of Personality and Social Psychology, 80, 871–875. https://doi.org/10.1037/0022-3514.80.6.871

41.

Lakatos

(1976). Falsification and the methodology of scientific research programmes. In Harding

S. G.

(Ed.), Can theories be refuted (pp. 205–259). Springer.

42.

Lee

M. D.

Criss

A. H.

Devezer

Donkin

Etz

Leite

F. P.

Matzke

Rouder

J. N.

Trueblood

J. S.

White

C. N.

Vandekerckhove

(2019). Robust modeling in cognitive science. Computational Brain & Behavior, 2, 141–153. https://doi.org/10.1007/s42113-019-00029-y

43.

Marr

(1982). The philosophy and the approach. In Vision: A computational investigation into the human representation and processing of visual information (pp. 8–38). Henry Holt and Co.

44.

Marr

Poggio

(1979). A computational theory of human stereo vision. Proceedings of the Royal Society of London B: Biological Sciences, 204, 301–328. https://doi.org/10.1016/B978-1-4832-1446-7.50046-7

45.

Maul

(2017). Rethinking traditional methods of survey validation. Measurement, 15, 51–69. https://doi.org/10.1080/15366367.2017.1348108

46.

McDougall

(1908). An introduction to social psychology. Methuen.

47.

Meehl

P. E.

(1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of Consulting and Clinical Psychology, 46, 806–834.

48.

Meehl

P. E.

(1986). What social scientists don’t understand. In Fiske

D. W.

Shweder

R. A.

(Eds.), Metatheory in social science: Pluralisms and subjectivities (pp. 315–338). University of Chicago Press.

49.

Meehl

P. E.

(1990). Appraising and amending theories: The strategy of Lakatosian defense and two principles that warrant it. Psychological Inquiry, 1, 108–141.

50.

Morey

R. D.

Homer

Proulx

(2018). Beyond statistics: Accepting the null hypothesis in mature sciences. Advances in Methods and Practices in Psychological Science, 1, 245–258. https://doi.org/10.1177/2515245918776023

51.

Muthukrishna

Henrich

(2019). A problem in theory. Nature Human Behavior, 3(3), 221–229. https://doi.org/10.1038/s41562-018-0522-1

52.

Muthukrishna

Schaller

(2020). Are collectivistic cultures more prone to rapid transformation? Computational models of cross-cultural differences, social network structure, dynamic social influence, and cultural change. Personality and Social Psychology Review, 24, 103–120.

53.

Nesse

R. M.

(2013). Tinbergen’s four questions, organized: A response to Bateson and Laland. Trends in Ecology & Evolution, 28, 681–682.

54.

Nosek

B. A.

Ebersole

C. R.

Dehaven

A. C.

Mellor

D. T.

(2018). The preregistration revolution. Proceedings of the National Academy of Sciences, USA, 115, 2600–2606. https://doi.org/10.1073/pnas.1708274114

55.

Oberauer

Lewandowsky

(2019). Addressing the theory crisis in psychology. Psychonomic Bulletin & Review, 26, 1596–1618.

56.

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. 349(6251), Article aac4716. https://doi.org/10.1126/science.aac4716

57.

Ratcliff

(1978). A theory of memory retrieval. Psychological Review, 85, 59–108. https://doi.org/10.1037/0033-295X.85.2.59

58.

Ratcliff

(1981). A theory of order relations in perceptual matching. Psychological Review, 88, 552–572. https://doi.org/10.1037/0033-295X.88.6.552

59.

Rescorla

R. A.

Wagner

A. R.

(1972). A theory of pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In Black

A. H.

Prokasy

W. F.

(Eds.), Classical conditioning II: Current theory and research (pp. 64–99). Appleton-Century-Crofts.

60.

Rouder

J. N.

Haaf

J. M.

(2019). A psychometrics of individual differences in experimental tasks. Psychonomic Bulletin and Review, 26, 452–467. https://doi.org/10.3758/s13423-018-1558-y

61.

Rouhani

Niv

(2019). Depressive symptoms bias the prediction-error enhancement of memory towards negative events in reinforcement learning. Psychopharmacology, 236, 2425–2435. https://doi.org/10.1007/s00213-019-05322-z

62.

Schaller

(2016). The empirical benefits of conceptual rigor: Systematic articulation of conceptual hypotheses can reduce the risk of non-replicable results (and facilitate novel discoveries too). Journal of Experimental Social Psychology, 66, 107–115. https://doi.org/10.1016/j.jesp.2015.09.006

63.

Servedio

M. R.

Brandvain

Dhole

Fitzpatrick

C. L.

Goldberg

E. E.

Stern

C. A.

Van Cleve

Yeh

D. J.

(2014). Not just a theory—The utility of mathematical models in evolutionary biology. PLOS Biology, 12(12), Article e1002017. https://doi.org/10.1371/journal.pbio.1002017

64.

Simmons

J. P.

Nelson

L. D.

Simonsohn

(2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359–1366. https://doi.org/10.1177/0956797611417632

65.

Skinner

B. F.

(1965). Science and human behavior. Simon and Schuster.

66.

Smaldino

P. E.

(2017). Models are stupid, and we need more of them. In Vallacher

Read

Nowak

(Eds.), Computational social psychology (pp. 311–331). Routledge. https://doi.org/10.4324/9781315173726-14

67.

Smaldino

P. E.

(2020). How to translate a verbal theory into a formal model. Social Psychology, 51, 207–218.

68.

Smaldino

P. E.

Calanchini

Pickett

C. L.

(2015). Theory development with agent-based models. Organizational Psychology Review, 5, 300–317.

69.

Smaldino

P. E.

Lukaszewski

von Rueden

Gurven

(2019). Niche diversity can explain cross-cultural differences in personality structure. Nature Human Behaviour, 3(12), 1276–1283.

70.

Smith

E. R.

Conrey

F. R.

(2007). Agent-based modeling: A new approach for theory building in social psychology. Personality and Social Psychology Review, 11(1), 87–104.

71.

Sutton

R. S.

Barto

A. G.

(1981). Toward a modern theory of adaptive networks: Expectation and prediction. Psychological Review, 88, 135–170. https://doi.org/10.1037/0033-295X.88.2.135

72.

Tay

Jebb

A. T.

(2018). Establishing construct continua in construct validation: The process of continuum specification. Advances in Methods and Practices in Psychological Science, 1, 375–388.https://doi.org/10.1177/2515245918775707

73.

Thorndike

(1933). A proof of the law of effect. Science, 77, 173–175.

74.

Tolman

E. C.

(1925). Purpose and cognition: The determiners of animal learning. Psychological Review, 32, 285–297. https://doi.org/10.1037/h0072784

75.

Wilson

R. C.

Collins

A. G.

(2019). Ten simple rules for the computational modeling of behavioral data. eLife, 8, Article e49547. https://doi.org/10.7554/eLife.49547