Very little is known about how individuals learn under uncertainty when other people are involved. We propose that humans are particularly tuned to social uncertainty, which is especially noisy and ambiguous. Individuals exhibiting less tolerance for uncertainty, such as those with anxiety, may have greater difficulty learning in uncertain social contexts and therefore provide an ideal test population to probe learning dynamics under uncertainty. Using a dynamic trust game and a matched nonsocial task, we found that healthy subjects (n = 257) were particularly good at learning under negative social uncertainty, swiftly figuring out when to stop investing in an exploitative social partner. In contrast, subjects with anxiety (n = 97) overinvested in exploitative partners. Computational modeling attributed this pattern to a selective reduction in learning from negative social events and a failure to enhance learning as uncertainty rises—two mechanisms that likely facilitate adaptive social choice.
AuerP.Cesa-BianchiN.FischerP. (2002). Finite-time analysis of the multiarmed bandit problem. Machine Learning, 47, 235–256. doi:10.1023/a:1013689704352
2.
AylwardJ.ValtonV.AhnW.-Y.BondR. L.DayanP.RoiserJ. P.RobinsonO. J. (2019). Altered learning under uncertainty in unmedicated mood and anxiety disorders. Nature Human Behaviour, 3, 1116–1123. doi:10.1038/s41562-019-0628-0
3.
BarreraT. L.NortonP. J. (2009). Quality of life impairment in generalized anxiety disorder, social phobia, and panic disorder. Journal of Anxiety Disorders, 23, 1086–1090. doi:10.1016/j.janxdis.2009.07.011
4.
BehrensT. E.HuntL. T.WoolrichM. W.RushworthM. F. (2008). Associative learning of social value. Nature, 456, 245–249. doi:10.1038/nature07538
5.
BehrensT. E.WoolrichM. W.WaltonM. E.RushworthM. F. (2007). Learning the value of information in an uncertain world. Nature Neuroscience, 10, 1214–1221. doi:10.1038/nn1954
6.
BerinskyA. J.HuberG. A.LenzG. S. (2012). Evaluating online labor markets for experimental research: Amazon.com’s Mechanical Turk. Political Analysis, 20, 351–368.
7.
BoelenP. A.ReijntjesA. (2009). Intolerance of uncertainty and social anxiety. Journal of Anxiety Disorders, 23, 130–135. doi:10.1016/j.janxdis.2008.04.007
8.
BrowningM.BehrensT. E.JochamG.O’ReillyJ. X.BishopS. J. (2015). Anxious individuals have difficulty learning the causal statistics of aversive environments. Nature Neuroscience, 18, 590–596. doi:10.1038/nn.3961
9.
CarletonR. N.MulvogueM. K.ThibodeauM. A.McCabeR. E.AntonyM. M.AsmundsonG. J. (2012). Increasingly certain about uncertainty: Intolerance of uncertainty across anxiety and depression. Journal of Anxiety Disorders, 26, 468–479. doi:10.1016/j.janxdis.2012.01.011
10.
ChangL. J.SmithA. C. (2015). Social emotions and psychological games. Current Opinion in Behavioral Sciences, 5, 133–140.
11.
CollinsA. G.FrankM. J. (2014). Opponent actor learning (OpAL): Modeling interactive effects of striatal dopamine on reinforcement learning and choice incentive. Psychological Review, 121, 337–366. doi:10.1037/a0037015
12.
DawN. D. (2014). Advanced reinforcement learning. In GlimcherP. W.FehrE. (Eds.), Neuroeconomics: Decision making and the brain (2nd ed., pp. 299–320). San Diego, CA: Academic Press.
13.
DayanP.NivY. (2008). Reinforcement learning: The good, the bad and the ugly. Current Opinion in Neurobiology, 18, 185–196. doi:10.1016/j.conb.2008.08.003
14.
DiaconescuA. O.MathysC.WeberL. A.DaunizeauJ.KasperL.LomakinaE. I.StephanK. E. (2014). Inferring on the intentions of others by hierarchical Bayesian learning. PLOS Computational Biology, 10(9), Article e1003810. doi:10.1371/journal.pcbi.1003810
15.
DiaconescuA. O.MathysC.WeberL. A. E.KasperL.MauerJ.StephanK. E. (2017). Hierarchical prediction errors in midbrain and septum during social learning. Social Cognitive and Affective Neuroscience, 12, 618–634. doi:10.1093/scan/nsw171
16.
EngW.HeimbergR. G.HartT. A.SchneierF. R.LiebowitzM. R. (2001). Attachment in individuals with social anxiety disorder: The relationship among adult attachment styles, social anxiety, and depression. Emotion, 1, 365–380.
17.
FaulF.ErdfelderE.LangA.-G.BuchnerA. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175–191.
18.
FeldmanHallO.ShenhavA. (2019). Resolving uncertainty in a social world. Nature Human Behaviour, 3, 426–435. doi:10.1038/s41562-019-0590-x
19.
FranklinN. T.FrankM. J. (2015). A cholinergic feedback circuit to regulate striatal population uncertainty and optimize reinforcement learning. eLife, 4, Article e12029. doi:10.7554/eLife.12029
20.
GershmanS. J. (2016). Empirical priors for reinforcement learning models. Journal of Mathematical Psychology, 71, 1–6. doi:10.1016/j.jmp.2016.01.006
21.
GillanC. M.DawN. D. (2016). Taking psychiatry research online. Neuron, 91, 19–23.
22.
HortonJ. J.RandD. G.ZeckhauserR. J. (2011). The online laboratory: Conducting experiments in a real labor market. Experimental Economics, 14, 399–425.
23.
MathysC.DaunizeauJ.FristonK. J.StephanK. E. (2011). A Bayesian foundation for individual learning under uncertainty. Frontiers in Human Neuroscience, 5, Article 39. doi:10.3389/fnhum.2011.00039
24.
McGuireJ. T.NassarM. R.GoldJ. I.KableJ. W. (2014). Functionally dissociable influences on learning rate in a dynamic environment. Neuron, 84, 870–881. doi:10.1016/j.neuron.2014.10.013
25.
MontagueP. R.DayanP.PersonC.SejnowskiT. J. (1995). Bee foraging in uncertain environments using predictive Hebbian learning. Nature, 377, 725–728. doi:10.1038/377725a0
26.
NassarM. R.WilsonR. C.HeaslyB.GoldJ. I. (2010). An approximately Bayesian delta-rule model explains the dynamics of belief updating in a changing environment. The Journal of Neuroscience, 30, 12366–12378. doi:10.1523/jneurosci.0822-10.2010
27.
NivY.DuffM. O.DayanP. (2005). Dopamine, uncertainty and TD learning. Behavioral and Brain Functions, 1, Article 6. doi:10.1186/1744-9081-1-6
28.
NivY.EdlundJ. A.DayanP.DohertyJ. P. (2012). Neural prediction errors reveal a risk-sensitive reinforcement-learning process in the human brain. The Journal of Neuroscience, 32, 551–562. doi:10.1523/JNEUROSCI.5498-10.2012
29.
O’ReillyJ. X. (2013). Making predictions in a changing world—inference, uncertainty, and learning. Frontiers in Neuroscience, 7, Article 105. doi:10.3389/fnins.2013.00105
30.
Payzan-LeNestourE.BossaertsP. (2011). Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings. PLOS Computational Biology, 7(1), Article e1001048. doi:10.1371/journal.pcbi.1001048
31.
PlattM. L.HuettelS. A. (2008). Risky business: The neuroeconomics of decision making under uncertainty. Nature Neuroscience, 11, 398–403. doi:10.1038/nn2062
32.
RangelA.CamererC.MontagueP. R. (2008). A framework for studying the neurobiology of value-based decision making. Nature Reviews Neuroscience, 9, 545–556. doi:10.1038/nrn2357
33.
RubinK. H.BurgessK. B. (2001). Social withdrawal and anxiety. In VaseyM. W.DaddsM. R. (Eds.), The developmental psychopathology of anxiety (pp. 407–434). New York, NY: Oxford University Press.
34.
RushworthM. F.BehrensT. E. (2008). Choice, uncertainty and value in prefrontal and cingulate cortex. Nature Neuroscience, 11, 389–397. doi:10.1038/nn2066
35.
ShapiroD. N.ChandlerJ.MuellerP. A. (2013). Using Mechanical Turk to study clinical populations. Clinical Psychological Science, 1, 213–220.
36.
SuttonR. S.BartoA. G. (1998). Reinforcement learning: An introduction. Cambridge, England: MIT Press.
37.
WehmeyerM. L. (2013). The Oxford handbook of positive psychology and disability. New York, NY: Oxford University Press.
38.
WeinsteinA.LittmanM. L. (2012, June). Bandit-based planning and learning in continuous-action Markov decision processes.Paper presented at the Twenty-Second International Conference on Automated Planning and Scheduling, Atibaia, São Paulo, Brazil.
39.
YamagishiT.YamagishiM. (1994). Trust and commitment in the United States and Japan. Motivation and Emotion, 18, 129–166. doi:10.1007/bf02249397
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
