The success of methods based on artificial neural networks in creating intelligent machines seems like it might pose a challenge to explanations of human cognition in terms of Bayesian inference. We argue that this is not the case and that these systems in fact offer new opportunities for Bayesian modeling. Specifically, we argue that artificial neural networks and Bayesian models of cognition lie at different levels of analysis and are complementary modeling approaches, together offering a way to understand human cognition that spans these levels. We also argue that the same perspective can be applied to intelligent machines, in which a Bayesian approach may be uniquely valuable in understanding the behavior of large, opaque artificial neural networks that are trained on proprietary data.
AndersonJ. R. (1990). The adaptive character of thought. Lawrence Erlbaum & Associates.
3.
BayesT. (1958). Studies in the history of probability and statistics: IX. Thomas Bayes’s essay towards solving a problem in the doctrine of chances. Biometrika, 45, 296–315. (Original work published 1763)
4.
BishopC. (1995). Regularization and complexity control in feed-forward networks. In Proceedings of the International Conference on Artificial Neural Networks (pp. 141–148). Institute of Electrical and Electronics Engineers.
5.
DasguptaI.GershmanS. J. (2021). Memory as a computational resource. Trends in Cognitive Sciences, 25(3), 240–251.
6.
GoodmanN. D.UllmanT. D.TenenbaumJ. B. (2011). Learning a theory of causality. Psychological Review, 118, 110–119.
7.
GrantE.FinnC.LevineS.DarrellT.GriffithsT. L. (2018, April 30–May 3). Recasting gradient-based meta-learning as hierarchical Bayes [Conference session]. 6th International Conference on Learning Representations, Vancouver, BC, Canada.
8.
GriffithsT. L.ChaterN.KempC.PerforsA.TenenbaumJ. B. (2010). Probabilistic models of cognition: Exploring representations and inductive biases. Trends in Cognitive Sciences, 14(8), 357–364.
9.
GriffithsT. L.LiederF.GoodmanN. D. (2015). Rational use of cognitive resources: Levels of analysis between the computational and the algorithmic. Topics in Cognitive Science, 7(2), 217–229.
10.
GriffithsT. L.TenenbaumJ. B. (2006). Optimal predictions in everyday cognition. Psychological Science, 17, 767–773.
GriffithsT. L.VulE.SanbornA. N. (2012). Bridging levels of analysis for probabilistic models of cognition. Current Directions in Psychological Science, 21, 263–268.
13.
KingmaD. P.WellingM. (2013, May2–4). Auto-encoding variational Bayes [Conference session]. 1st International Conference on Learning Representations, Scottsdale, AZ, United States.
14.
LeCunY.BengioY.HintonG. (2015). Deep learning. Nature, 521(7553), 436–444.
15.
LiM. Y.GrantE.GriffithsT. L. (2023). Gaussian process surrogate models for neural networks. In EvansR. J.ShpitserP. (Eds.), UAI’23: Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence (pp. 1241–1252). Association for Computing Machinery.
16.
LucasC. G.GriffithsT. L.WilliamsJ. J.KalishM. L. (2015). A rational model of function learning. Psychonomic Bulletin & Review, 22(5), 1193–1215.
17.
MacKayD. (1995). Probable networks and plausible predictions -a review of practical Bayesian methods for supervised neural networks. Network: Computation in Neural Systems, 6, 469–505.
18.
MarrD. (1982). The philosophy and the approach. In Vision (pp. 8–37). W. H. Freeman and Company.
19.
McClellandJ. L.BotvinickM. M.NoelleD. C.PlautD. C.RogersT. T.SeidenbergM. S.SmithL. B. (2010). Letting structure emerge: Connectionist and dynamical systems approaches to understanding cognition. Trends in Cognitive Sciences, 14, 348–356.
20.
McCoyR. T.GriffithsT. L. (2023). Modeling rapid language learning by distilling Bayesian priors into artificial neural networks. arXiv. https://doi.org/10.48550/arXiv.2305.14701
21.
McCoyR. T.YaoS.FriedmanD.HardyM.GriffithsT. L. (2023). Embers of autoregression: Understanding large language models through the problem they are trained to solve. arXiv. https://doi.org/10.48550/arXiv.2309.13638
RibeiroM. T.SinghS.GuestrinC. (2016). “Why should I trust you?” Explaining the predictions of any classifier. In KDD’16: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 1135–1144). Association for Computing Machinery.
29.
RodriguezP.WilesJ.ElmanJ. L. (1999). A recurrent neural network that learns to count. Connection Science, 11(1), 5–40.
30.
RogersT.McClellandJ. (2004). Semantic cognition: A parallel distributed processing approach. MIT Press.
31.
RuleJ. S.TenenbaumJ. B.PiantadosiS. T. (2020). The child as hacker. Trends in Cognitive Sciences, 24(11), 900–915.
32.
SanbornA. N.GriffithsT. L.NavarroD. J. (2010). Rational approximations to rational models: Alternative algorithms for category learning. Psychological Review, 117, 1144–1167.
33.
ShepardR. N. (1987). Towards a universal law of generalization for psychological science. Science, 237, 1317–1323.
YangY.PiantadosiS. T. (2022). One model for the learning of language. Proceedings of the National Academy of Sciences, USA, 119(5), Article e2021865119. https://doi.org/10.1073/pnas.2021865119
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
