Building Human-Like Artificial Agents: A General Cognitive Algorithm for Emulating Human Decision-Making in Dynamic Environments

Abstract

One of the early goals of artificial intelligence (AI) was to create algorithms that exhibited behavior indistinguishable from human behavior (i.e., human-like behavior). Today, AI has diverged, often aiming to excel in tasks inspired by human capabilities and outperform humans, rather than replicating human cogntion and action. In this paper, I explore the overarching question of whether computational algorithms have achieved this initial goal of AI. I focus on dynamic decision-making, approaching the question from the perspective of computational cognitive science. I present a general cognitive algorithm that intends to emulate human decision-making in dynamic environments, as defined in instance-based learning theory (IBLT). I use the cognitive steps proposed in IBLT to organize and discuss current evidence that supports some of the human-likeness of the decision-making mechanisms. I also highlight the significant gaps in research that are required to improve current models and to create higher fidelity in computational algorithms to represent human decision processes. I conclude with concrete steps toward advancing the construction of algorithms that exhibit human-like behavior with the ultimate goal of supporting human dynamic decision-making.

Keywords

Al algorithm decisions from experience choice memory learning instance-based learning theory

Get full access to this article

View all access options for this article.

References

Aggarwal

Thakoor

Jabbari

Cranford

E. A.

Lebiere

Tambe

Gonzalez

(2022). Designing effective masking strategies for cyberdefense through human experimentation and cognitive models. Computers & Security, 117, Article 102671. https://doi.org/10.1016/j.cose.2022.102671

Anderson

J. R.

Lebiere

C. J.

(1998). The atomic components of thought (1st ed.). Psychology Press. https://doi.org/10.4324/9781315805696

Ariely

(2009). The end of rational economics. Harvard Business Review, 87(7–8), 78–84.

Baumann

Lorenz-Spreen

Sokolov

I. M.

Starnini

(2020). Modeling echo chambers and polarization dynamics in social networks. Physical Review Letters, 124(4), Article 048301. https://doi.org/10.1103/PhysRevLett.124.048301

Bhatia

Svegliato

Zilberstein

(2021, August 2–13). Tuning the hyperparameters of anytime planning: A deep reinforcement learning approach [Paper presentation]. ICAPS 2021 Workshop on Heuristics and Search for Domain-Independent Planning, Guangzhou, China. https://openreview.net/forum?id=c7hpFp_eRCo

Botvinick

Ritter

Wang

J. X.

Kurth-Nelson

Blundell

Hassabis

(2019). Reinforcement learning, fast and slow. Trends in Cognitive Sciences, 23(5), 408–422.

Brehmer

(1980). In one word: Not from experience. Acta Psychologica, 45(1–3), 223–241.

Brehmer

(1992). Dynamic decision making: Human control of complex systems. Acta Psychologica, 81(3), 211–241.

Brehmer

Allard

(1991). Dynamic decision making: The effects of task complexity and feedback delay. In Rasmussen

Brehmer

Leplat

(Eds.), Distributed decision making: Cognitive models for cooperative work (pp. 319–334). John Wiley & Sons.

10.

Brehmer

Dörner

(1993). Experiments with computer-simulated microworlds: Escaping both the narrow straits of the laboratory and the deep blue sea of the field study. Computers in Human Behavior, 9(2–3), 171–184.

11.

Bugbee

E. H.

Gonzalez

(2022). Making predictions without data: How an instance-based learning model predicts sequential decisions in the balloon analog risk task. In Culbertson

Perfors

Rabagliati

Ramenzoni

(Eds.), Proceedings of the 44th Annual Conference of the Cognitive Science Society. Cognitive Science Society. https://escholarship.org/uc/item/5rj08314

12.

Busemeyer

J. R.

Diederich

(2010). Cognitive modeling. Sage.

13.

Chabert

J.-L.

(Ed.). (1999). Euclid’s algorithm. In A history of algorithms (pp. 113–138). Springer.

14.

Chase

W. G.

Simon

H. A.

(1973). Perception in chess. Cognitive Psychology, 4(1), 55–81.

15.

Chi

M. T.

Feltovich

P. J.

Glaser

(1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5(2), 121–152.

16.

Cohen

J. D.

McClure

S. M.

A. J.

(2007). Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Philosophical Transactions of the Royal Society B: Biological Sciences, 362(1481), 933–942.

17.

De Groot

A. D

. (2014). Thought and choice in chess. De Gruyter Mouton.

18.

Diehl

Sterman

J. D.

(1995). Effects of feedback complexity on dynamic decision making. Organizational Behavior and Human Decision Processes, 62(2), 198–215.

19.

Dörner

Güss

C. D.

(2022). Human error in complex problem solving and dynamic decision making: A taxonomy of 24 errors and a theory. Computers in Human Behavior Reports, 7, Article 100222. https://doi.org/10.1016/j.chbr.2022.100222

20.

Dutt

Gonzalez

(2015). Accounting for outcome and process measures and the effects of model calibration. Journal of Dynamic Decision Making, 1(2), 1–10. https://doi.org/10.1016/j.chb.2015.01.039

21.

Edwards

(1962). Dynamic decision theory and probabilistic information processings. Human Factors, 4(2), 59–74.

22.

Einhorn

H. J.

Hogarth

R. M.

(1981). Behavioral decision theory: Processes of judgement and choice. Annual Review of Psychology, 32(1), 53–88.

23.

Einhorn

H. J.

Hogarth

R. M.

(1978). Confidence in judgment: Persistence of the illusion of validity. Psychological Review, 85(5), 395.

24.

Erev

Ert

Plonsky

Cohen

(2017). From anomalies to forecasts: Toward a descriptive model of decisions under risk, under ambiguity, and from experience. Psychological Review, 124(4), 369–409. https://doi.org/10.1037/rev0000062

25.

Estes

W. K.

(1964). Probability learning. In Melton

A. W.

(Ed.), Categories of human learning (pp. 89–128). Academic Press.

26.

Estes

W. K.

(1976). The cognitive side of probability learning. Psychological Review, 83(1), 37–64. https://doi.org/10.1037/0033-295X.83.1.37

27.

Farrell

Lewandowsky

(2015). An introduction to cognitive modeling. In Forstmann

B. U.

Wagenmakers

E.-J.

(Eds.), An introduction to model-based cognitive neuroscience (pp. 3–24). Springer.

28.

Foglia

Wilson

R. A.

(2013). Embodied cognition. WIREs Cognitive Science, 4(3), 319–325.

29.

W. T.

Anderson

J. R.

(2006). From recurrent choice to skill learning: A reinforcement-learning model. Journal of Experimental Psychology: General, 135(2), 184–206. https://doi.org/10.1037/0096-3445.135.2.184

30.

Fuchs

Passarella

Conti

(2022). Modeling human behavior part II: Cognitive approaches and uncertainty. arXiv. https://doi.org/10.48550/arXiv.2205.06483

31.

Gershman

S. J.

Daw

N. D.

(2017). Reinforcement learning and episodic memory in humans and animals: An integrative framework. Annual Review of Psychology, 68, 101–128.

32.

Gigerenzer

G. E.

Hertwig

R. E.

Pachur

T. E.

(2011). Heuristics: The foundations of adaptive behavior. Oxford University Press.

33.

Gigerenzer

G. E.

Todd

P. M.

(1999). Fast and frugal heuristics: The adaptive toolbox. In Gigerenzer

Todd

P. M.

, & The ABC Research Group (Eds.), Simple heuristics that make us smart (pp. 3–34). Oxford University Press.

34.

Gluck

Bello

Busemeyer

(2008). Introduction to the special issue. Cognitive Science, 32(8), 1245–1247. https://doi.org/10.1080/03640210802473582

35.

Gobet

Simon

H. A.

(1996). Templates in chess memory: A mechanism for recalling several boards. Cognitive Psychology, 31(1), 1–40.

36.

Goldstein

D. G.

McAfee

R. P.

Suri

Wright

J. R.

(2020). Learning when to stop searching. Management Science, 66(3), 1375–1394.

37.

Goldstein

W. M.

Hogarth

(1997). Research on judgment and decision making. Cambridge University Press.

38.

Goldstone

R. L.

Son

J. Y.

(2012). Similarity. Oxford University Press.

39.

Gonzalez

(2004). Learning to make decisions in dynamic environments: Effects of time constraints and cognitive abilities. Human Factors, 46(3), 449–460.

40.

Gonzalez

(2005). Decision support for real-time, dynamic decision-making tasks. Organizational Behavior and Human Decision Processes, 96(2), 142–154.

41.

Gonzalez

(2013). The boundaries of instance-based learning theory for explaining decisions from experience. Progress in Brain Research, 202, 73–98.

42.

Gonzalez

(2017). Decision making: A cognitive science perspective. In Chipman

S. E. F.

(Ed.), The Oxford handbook of cognitive science (pp. 249–263). Oxford University Press.

43.

Gonzalez

(2022). Learning and dynamic decision making. Topics in Cognitive Science, 14(1), 14–30.

44.

Gonzalez

Aggarwal

(in press). Sequential decisions from sampling: Inductive generation of stopping decisions using instance-based learning theory. In Fiedler

Juslin

Denrell

(Eds.), Sampling theories continue to inspire novel judgment and decision research.

45.

Gonzalez

Aggarwal

Cranford

E. A.

Lebiere

(2023). Adaptive cyberdefense with deception: A human-AI cognitive approach. In Bao

Tambe

Wang

(Eds.), Cyber deception, techniques, strategies, and human aspects (Vol. 89, pp. 41–57). Springer. https://doi.org/10.1007/978-3-031-16613-6_3

46.

Gonzalez

Dutt

(2011). A generic dynamic control task for behavioral research and education. Computers in Human Behavior, 27(5), 1904–1914.

47.

Gonzalez

Dutt

(2016). Exploration and exploitation during information search and experimential choice. Journal of Dynamic Decision Making, 2, Article 2. https://doi.org/10.11588/jddm.2016.1.29308

48.

Gonzalez

Fakhari

Busemeyer

(2017). Dynamic decision making: Learning processes and new research directions. Human Factors, 59(5), 713–721.

49.

Gonzalez

Lerch

J. F.

Lebiere

(2003). Instance-based learning in dynamic decision making. Cognitive Science, 27(4), 591–635.

50.

Gonzalez

Quesada

(2003). Learning in dynamic decision making: The recognition process. Computational & Mathematical Organization Theory, 9(4), 287–304.

51.

Gonzalez

Vanyukov

Martin

M. K.

(2005). The use of microworlds to study dynamic decision making. Computers in Human Behavior, 21(2), 273–286.

52.

Griffiths

Kemp

C. B.

Tenenbaum

(2008). Bayesian models of cognition. Carnegie Mellon University. https://doi.org/10.1184/R1/6613682.v1

53.

Guan

Lee

M. D.

(2018). The effect of goals and environments on human performance in optimal stopping problems. Decision, 5(4), 339–361. https://doi.org/10.1037/dec0000081

54.

Harman

J. L.

Konstantinidis

Gonzalez

(2021). How to use a multi-criteria comparison procedure to improve modeling competitions. Psychological Review, 128(5), 995–1005. https://doi.org/10.1037/rev0000274

55.

Hastie

(2001). Problems for judgment and decision making. Annual Review of Psychology, 52(1), 653–683.

56.

Hertwig

(2015). Decisions from experience. In Keren

(Eds.), The Wiley Blackwell handbook of judgment and decision making (Vol. 1, pp. 240–267). John Wiley & Sons.

57.

Hertwig

Barron

Weber

E. U.

Erev

(2004). Decisions from experience and the effect of rare events in risky choice. Psychological Science, 15(8), 534–539.

58.

Hertwig

Erev

(2009). The description-experience gap in risky choice. Trends in Cognitive Sciences, 13(12), 517–523.

59.

Joslyn

Hunt

(1998). Evaluating individual differences in response to time-pressure situations. Journal of Experimental Psychology: Applied, 4(1), 16–43. https://doi.org/10.1037/1076-898X.4.1.16.

60.

Kahneman

Slovic

S. P.

Slovic

Tversky

(Eds.). (1982). Judgment under uncertainty: Heuristics and biases. Cambridge University Press.

61.

Kahneman

Tversky

(1979). On the interpretation of intuitive probability: A reply to Jonathan Cohen. Cognition, 7(4), 409–411.

62.

Kerstholt

J. H.

Raaijmakers

J. G.

(1997). Decision making: Cognitive models and explanations. Routledge.

63.

Klein

G. A.

(1993). A recognition-primed decision (RPD) model of rapid decision making. In Klein

G. A.

Orasanu

Calderwood

Zsambok

C. E.

(Eds.), Decision making in action: Models and methods (pp. 138–147). Ablex Publishing.

64.

Klein

G. A.

(1999). Sources of power: How people make decisions. MIT Press.

65.

Kochenderfer

M. J.

Wheeler

T. A.

Wray

K. H.

(2022). Algorithms for decision making. MIT Press.

66.

Lake

B. M.

Ullman

T. D.

Tenenbaum

J. B.

Gershman

S. J.

(2017). Building machines that learn and think like people. Behavioral and Brain Sciences, 40, Article E253. https://doi.org/10.1017/S0140525X16001837

67.

Langley

Simon

H. A.

(1981). The central role of learning in cognition. In Anderson

(Ed.), Cognitive skills and their acquisition (pp. 361–380). Lawrence Erlbaum Associates.

68.

Lebiere

(1999). The dynamics of cognition: An ACT-R model of cognitive arithmetic. Kognitionswissenschaft, 8(1), 5–19.

69.

Lebiere

Cranford

E. A.

Aggarwal

Cooney

Tambe

Gonzalez

(2023). Cognitive modeling for personalized, adaptive signaling for cyber deception. In Bao

Tambe

Wang

(Eds.), Cyber deception, techniques, strategies, and human aspects (Vol. 89, pp. 59–82). Springer. https://doi.org/10.1007/978-3-031-16613-6_4

70.

Lee

Seo

Jung

M. W.

(2012). Neural basis of reinforcement learning and decision making. Annual Review of Neuroscience, 35, 287–308.

71.

Lee

M. D.

(2006). A hierarchical Bayesian model of human decision-making on an optimal stopping problem. Cognitive Science, 30(3), 1–26.

72.

Lee

M. D.

Courey

K. A.

(2021). Modeling optimal stopping in changing environments: A case study in mate selection. Computational Brain & Behavior, 4, 1–17.

73.

Lee

M. D.

Newell

B. R.

Vandekerckhove

(2014). Modeling the adaptation of search termination in human decision making. Decision, 1(4), 223–251. https://doi.org/10.1037/dec0000019

74.

Lejarraga

Dutt

Gonzalez

(2012). Instance-based learning: A general model of repeated binary choice. Journal of Behavioral Decision Making, 25(2), 143–153. https://doi.org/10.1002/bdm.722

75.

Lejarraga

Hertwig

Gonzalez

(2012). How choice ecology influences search in decisions from experience. Cognition, 124(3), 334–342.

76.

Medin

D. L.

Schaffer

M. M.

(1978). Context theory of classification learning. Psychological Review, 85(3), 207–238.

77.

Mehlhorn

Newell

B. R.

Todd

P. M.

Lee

M. D.

Morgan

Braithwaite

V. A.

Hausmann

Fiedler

Gonzalez

(2015). Unpacking the exploration–exploitation tradeoff: A synthesis of human and animal literatures. Decision, 2(3), 191–215.

78.

Minsky

(1961). Steps toward artificial intelligence. Proceedings of the IRE, 49(1), 8–30.

79.

Nachshon

Cohen

Ben-Artzi

Maril

(2022). A model of similarity: Metric in a patch. PsyArXiv. https://doi.org/10.31234/osf.io/fyd4q

80.

Newell

(1992). Unified theories of cognition and the role of soar. In Michon

J. A.

Akyürek

(Eds.), Soar: A cognitive architecture in perspective: A tribute to Allen Newell (pp. 25–79). Springer.

81.

Nguyen

T. N.

McDonald

Gonzalez

(2023). Credit assignment: Challenges and opportunities in developing human-like AI agents. arXiv:2307.08171. https://doi.org/10.48550/arXiv.2307.08171

82.

Nguyen

T. N.

Phan

D. N.

Gonzalez

(2022). SpeedyIBL: A comprehensive, precise, and fast implementation of instance-based learning theory. arXiv. https://doi.org/10.48550/arXiv.2111.10268

83.

Niv

(2009). Reinforcement learning in the brain. Journal of Mathematical Psychology, 53(3), 139–154.

84.

Nosofsky

R. M.

(1986). Attention, similarity, and the identification–categorization relationship. Journal of Experimental Psychology: General, 115(1), 39–57.

85.

Payne

J. W.

Bettman

J. R.

Johnson

E. J.

(1993). The adaptive decision maker. Cambridge University Press.

86.

Pouncy

Tsividis

Gershman

S. J.

(2021). What is the model in model-based planning? Cognitive Science, 45(1), Article e12928. https://doi.org/10.1111/cogs.12928

87.

Powell

W. B.

(2022). Reinforcement learning and stochastic optimization: A unified framework for sequential decisions. John Wiley & Sons.

88.

Rai

Das

Ray

Dhal

K. G.

(2022). Human-inspired optimization algorithms: Theoretical foundations, algorithms, open-research issues and application for multi-level thresholding. Archives of Computational Methods in Engineering, 29(7), 5313–5352.

89.

Roughgarden

(2010). Algorithmic game theory. Communications of the ACM, 53(7), 78–86.

90.

Russell

S. J.

Norvig

(2010). Artificial intelligence: A modern approach (3rd ed.). Pearson Education.

91.

Seale

D. A.

Rapoport

(2000). Optimal stopping behavior with relative ranks: The secretary problem with unknown population size. Journal of Behavioral Decision Making, 13(4), 391–411.

92.

Shepard

R. N.

(1987). Toward a universal law of generalization for psychological science. Science, 237(4820), 1317–1323.

93.

Simon

H. A.

(1955). A behavioral model of rational choice. The Quarterly Journal of Economics, 69(1), 99–118.

94.

Simon

H. A.

(1957). Models of man; Social and rational. John Wiley & Sons.

95.

Simon

H. A.

(1983). Why should machines learn? In Carbonell

J. G.

Michalski

R. S.

Mitchell

T. M.

(Eds.), Machine learning: An artificial intelligence approach (pp. 25–37). Springer-Verlag.

96.

Sterman

J. D.

(1989). Modeling managerial behavior: Misperceptions of feedback in a dynamic decision making experiment. Management Science, 35(3), 321–339.

97.

Sterman

J. D.

(1994). Learning in and about complex systems. System Dynamics Review, 10(2–3), 291–330.

98.

Sutton

R. S.

Barto

A. G.

(2018). Reinforcement learning: An introduction. MIT Press.

99.

Thomson

Lebiere

Anderson

J. R.

Staszewski

(2015). A general instance-based learning framework for studying intuitive decision-making in a cognitive architecture. Journal of Applied Research in Memory and Cognition, 4(3), 180–190.

100.

Trueblood

J. S.

(2022). Theories of context effects in multialternative, multiattribute choice. Current Directions in Psychological Science, 31(5), 428–435. https://doi.org/10.1177/09637214221109587

101.

Trueblood

J. S.

Brown

S. D.

Heathcote

(2014). The multiattribute linear ballistic accumulator model of context effects in multialternative choice. Psychological Review, 121(2), 179–205.

102.

Turing

A. M.

(1950). I.—Computing machinery and intelligence. Mind, 59(236), 433–460.

103.

Tversky

(1977). Features of similarity. Psychological Review, 84(4), 327–352.

104.

Tversky

Kahneman

(1974). Judgment under uncertainty: Heuristics and biases: Biases in judgments reveal some heuristics of thinking under uncertainty. Science, 185(4157), 1124–1131.

105.

Walsh

M. M.

Anderson

J. R.

(2011). Modulation of the feedback-related negativity by instruction and experience. Proceedings of the National Academy of Sciences, USA, 108(47), 19048–19053.

106.

Walsh

M. M.

Anderson

J. R.

(2014). Navigating complex decision spaces: Problems and paradigms in sequential choice. Psychological Bulletin, 140(2), 466–486.

107.

Wulff

D. U.

Mergenthaler-Canseco

Hertwig

(2018). A meta-analytic review of two modes of learning and the description-experience gap. Psychological Bulletin, 144(2), 140–176.

108.

Yearsley

J. M.

Pothos

E. M.

Barque-Duran

Trueblood

J. S.

Hampton

J. A.

(2022). Context effects in similarity judgments. Journal of Experimental Psychology: General, 151(3), 711–717.

109.

Zhang

L. M.

Dahlmann

Zhang

(2009). Human-inspired algorithms for continuous function optimization. In 2009 IEEE international conference on intelligent computing and intelligent systems (Vol. 1, pp. 318–321). Institute of Electrical and Electronics Engineers.

110.

Zhou

Chen

(Eds.). (2018). Human and machine learning. Springer.

111.

Zsambok

C. E.

Klein

(Eds.). (2014). Naturalistic decision making. Psychology Press.