Decisions result from an interaction between multiple functional systems acting in parallel to process information in very different ways, each with strengths and weaknesses. In this review, the authors address three action-selection components of decision-making: The Pavlovian system releases an action from a limited repertoire of potential actions, such as approaching learned stimuli. Like the Pavlovian system, the habit system is computationally fast but, unlike the Pavlovian system permits arbitrary stimulus-action pairings. These associations are a “forward’’ mechanism; when a situation is recognized, the action is released. In contrast, the deliberative system is flexible but takes time to process. The deliberative system uses knowledge of the causal structure of the world to search into the future, planning actions to maximize expected rewards. Deliberation depends on the ability to imagine future possibilities, including novel situations, and it allows decisions to be taken without having previously experienced the options. Various anatomical structures have been identified that carry out the information processing of each of these systems: hippocampus constitutes a map of the world that can be used for searching/imagining the future; dorsal striatal neurons represent situation-action associations; and ventral striatum maintains value representations for all three systems. Each system presents vulnerabilities to pathologies that can manifest as psychiatric disorders. Understanding these systems and their relation to neuroanatomy opens up a deeper way to treat the structural problems underlying various disorders.
AddisDRWongATSchacterDL. 2007. Remembering the past and imagining the future: common and distinct neural substrates during event construction and elaboration. Neuro-psychologia45(7):1363–77.
2.
AragonaBJDayJJRoitmanMFClevelandNAWightmanRMCarelliRM. 2009. Regional specificity in the real-time development of phasic dopamine transmission patterns during acquisition of a cue-cocaine association in rats. Eur J Neurosci2009;30(10):1889–99.
3.
AtallahEHLopeq-PaniaguaDRudyJWO’ReillyRC. 2007. Separate neural substrates for skill learning and performance in the ventral and dorsal striatum. Nature Neuroscience10(1):126–31.
BadianiABelinDEpsteinDCaluDShahamY. 2011. Opiate versus psychostimulant addiction: the differences do matter. Nat Rev Neurosci12(11):685–700.
6.
BarnesTDKubotaYHuDJinDZGraybielAM. 2005. Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories. Nature437:1158–61.
7.
BellmanR. 1958. On a routing problem. Quart J Appl Math16(1):87–90.
8.
BerkeJDEichenbaumH. 2009. Striatal versus hippocampal representations during win-stay maze performance. J Neurophysiol101(3):1575–87.
9.
BoutonME. 2007. Learning and behavior: a contemporary synthesis. Sunderland (MA): Sinauer Associates.
10.
BoysenSTBerntsonGG. 1995. Responses to quantity: Perceptual versus cognitive mechanisms in chimpanzees (Pan troglodytes). J Exp Psychol Anim Behav Process21(1):82–86.
11.
BrelandKBrelandM. 1961. The misbehavior of organisms. Am Psychol16(11):682–84.
12.
BucknerRLCarrollDC. 2007. Self-projection and the brain. Trends Cogn Sci11(2):49–57.
13.
BushongBKingLMCamererCFRangelA. 2010. Pavlovian processes in consumer choice: the physical presence of a good increases willingness-to-pay. Am Econ Rev100(4):1556–71.
14.
CardinalRNParkinsonJAHallJEverittBJ. 2002. Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex. Neurosci Biobehav Rev26(3):321–52.
15.
CisekPKalaskaJF. 2010. Neural mechanisms for interacting with a world full of action choices. Annu Rev Neurosci33:269–98.
16.
CohenNJEichenbaumH. 1993. Memory, amnesia, and the hippocampal system. Cambridge (MA): MIT Press.
17.
CraigAD. 2003. Interoception: the sense of the physiological condition of the body. Curr Opin Neurobiol13(4):500–5.
18.
D’ArdenneKMcClureSMNystromLECohenJD. 2008. BOLD responses reflecting dopaminergic signals in the human ventral tegmental area. Science319(5867):1264–67.
19.
DayanPNivY. 2008. Reinforcement learning: the good, the bad and the ugly. Curr Opin Neurobiol18(2):185–96.
20.
DayanPNivYSeymourBDawND. 2006. The misbehavior of value and the discipline of the will. Neural Netw19:1153–60.
21.
FentonAALyttonWWBarryJMLenck-SantiniPPZinyukLEKubikS, and others. 2010. Attention-like modulation of hippocampus place cell discharge. J Neurosci30(13):4613–25.
22.
FlagelSBClarkJJRobinsonTEMayoLCzujAWilluhnI, and others. 2011. A selective role for dopamine in stimulus-reward learning. Nature469(7328):53–57.
23.
FrankMJ. 2011. Computational models of motivated action selection in corticostriatal circuits. Curr Opin Neurobiol21(3):381–86.
HaberSN. 2009. Anatomy and connectivity of the reward circuit. In: DreherJCTremblayL, editors. Handbook of reward and decision making. New York: Academic Press. p. 327.
26.
HassabisDMaguireEA. 2011. The contruction system in the brain. In: BarM, editor. Predictions in the brain: using our past to generate a future. Oxford: Oxford University Press. p. 70–82.
27.
HershbergerWA. 1986. An approach through the looking glass. Learn Behav14(4):443–51.
28.
HeymanG. 2009. Addiction: a disorder of choice. Cambridgex (MA): Harvard University Press.
29.
HikosakaOSakamotoMUsuiS. 1989. Functional properties of monkey caudate neurons. J Neurophysiol61(4):780–832.
30.
HillAJ. 1978. First occurrence of hippocampal spatial firing in a new environment. Exp Neurol62:282–97.
31.
HillC. 2008. The rationality of preference construction (and the irrationality of rational choice). Min J Law Sci Technol9(2):689–742.
32.
HollupSAMoldenSDonnettJGMoserMBMoserEI. 2001. Accumulation of hippocampal place fields at the goal location in an annular watermaze task. J Neurosci21(5):1635–44.
33.
HuDAmselA. 1995. A simple test of the vicarious trial-and-error hypothesis of hippocampal function. Proc Natl Acad Sci U S A92:5506–9.
34.
HuysQJM. 2007. Reinforcers and control: towards a computational ætiology of depression. PhD thesis, Gatsby Computational Neuroscience Unit, University College London.
35.
HuysQJMMoutoussisMWilliamsJ. 2011. Are computational models of any use to psychiatry. Neural Netw24(6):544–51.
36.
JohnsonACroweDA. 2009. Revisiting Tolman, his theories and cognitive maps. Cogn Crit1:43–72.
37.
JohnsonAJacksonJRedishAD. 2008. Measuring distributed properties of neural representations beyond the decoding of local variables—implications for cognition. In: HölscherCMunkMHJ, editors. Mechanisms of information processing in the brain: encoding of information in neural populations and networks. Cambridge: Cambridge University Press. p. 95–119.
38.
JohnsonARedishAD. 2007. Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point. J Neurosci27(45):12176-12189.
39.
KentrosCGAgnihotriNTStreaterSHawkinsRDKandelER. 2004. Increased attention to spatial context increases both place field stability and spatial memory. Neuron42:283–95.
40.
KillcrossSCoutureauE. 2003. Coordination of actions and habits in the medial prefrontral cortex of rats. Cereb Cortex13(8):400–8.
41.
KimchiEYLaubachM. 2009. Dynamic encoding of action selection by the medial striatum. J Neurosci29(10):3148–59.
42.
KishidaKTKing-CasasBMontaguePR. 2010. Neuroeconomic approaches to mental disorders. Neuron67(4):543–54.
43.
Kurth-NelsonZRedishAD. 2012. Modeling decision-making systems in addiction. In: GutkinBAhmedSH, editors. Computational neuroscience of drug addiction. New York: Springer.
44.
LangerEJRothJ. 1975. Heads I win, tails it’s chance: the illusion of control as a function of the sequence of outcomes in a purely chance task. J Pers Soc Psychol32(6):951–5.
45.
LansinkCSGoltsteinPMLankelmaJVMcNaughtonBLPennartzCMA. 2009. Hippocampus leads ventral striatum in replay of place-reward information. PLoS Biol7(8):e1000173.
46.
LauBGlimcherPW. 2007. Action and outcome encoding in the primate caudate nucleus. J Neurosci27(52):14502–14.
47.
LavoieAMMizumoriSJY. 1994. Spatial-, movement- and reward-sensitive discharge by medial ventral striatum neurons in rats. Brain Res638:157–68.
48.
LedouxJ. 2002. The synaptic self. New York: Penguin.
49.
MaiaTVFrankMJ. 2011. From reinforcement learning models to psychiatric and neurological disorders. Nat Neurosci14(2):154–65.
50.
McDannaldMALucantonioFBurkeKANivYSchoenbaumG. 2011. Ventral striatum and orbitofrontal cortex are both required for model-based, but not model-free, reinforcement learning. J Neurosci31(7):2700–5.
51.
McHughPRSlavneyPR. 1998. The perspectives of psychiatry. Baltimore, MD: Johns Hopkins.
52.
MogensonGJJonesDLYimCY. 1980. From motivation to action: functional interface between the limbic system and the motor system. Prog Neurobiol14:69–97.
53.
MorrisRGMGarrudPRawlinsJNPO’KeefeJ. 1982. Place navigation impaired in rats with hippocampal lesions. Nature297:681–83.
54.
MuenzingerKFGentryE. 1931. Tone discrimination in white rats. J Comp Psychol12(2):195–206.
55.
NicolaSM. 2010. The flexible approach hypothesis: unification of effort and cue-responding hypotheses for the role of nucleus accumbens dopamine in the activation of reward-seeking behavior. J Neurosci30(49):16585–600.
56.
NivYJoelDDayanP. 2006. A normative perspective on motivation. Trends Cogn Sci10(8):375–81.
57.
O’DohertyJDayanPSchultzJDeichmannRFristonKDolanRJ. 2004. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science304(5669):452–54.
58.
O’KeefeJNadelL. 1978. The hippocampus as a cognitive map. Oxford: Clarendon Press.
59.
RedishAD. 1999. Beyond the cognitive map: from place cells to episodic memory. Cambridge (MA): MIT Press.
60.
RedishAD. 2004. Addiction as a computational process gone awry. Science306(5703):1944–47.
61.
RedishADJensenSJohnsonA. 2008. A unified framework for addiction: vulnerabilities in the decision process. Behav Brain Sci31:415–87.
62.
RedishADJensenSJohnsonAKurth-NelsonZ. 2007. Reconciling reinforcement learning models with behavioral extinction and renewal: Implications for addiction, relapse, and problem gambling. Psychol Rev114(3):784–805.
63.
RedishADTouretzkyDS. 1998. The role of the hippocampus in solving the Morris water maze. Neural Comput10(1):73–111.
64.
RescorlaRAWagnerAR. 1972. A theory of Pavlovian Conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In BlackAHProkesyWF editors. Classical Conditioning II: Current Research and Theory. New York: Appleton Century Crofts. p.64–99.
SamejimaKUedaYDoyaKKimuraM. 2005. Representation of action-specific reward values in the striatum. Science310(5752):1337–40.
67.
SamsonovichAVAscoliGA. 2005. A simple neural network model of the hippocampus suggesting its pathfinding role in episodic memory retrieval. Learn Mem12(2):193–208.
68.
SanfeyAGRillingJKAronsonJANystromLECohenJD. 2003. The neural basis of economic decision-making in the ultimatum game. Science300(5626):1755–58.
69.
SchacterDLAddisDR. 2011. On the nature of medial temporal lobe contributions to the constructive simulation of future events. In: BarM, editor. Predictions in the brain: using our past to generate a future. Oxford: Oxford University Press. p. 58–69.
70.
Schmitzer-TorbertNCRedishAD. 2004. Neuronal activity in the rodent dorsal striatum in sequential navigation: Separation of spatial and reward responses on the multiple-T task. J Neurophysiol91(5):2259–72.
71.
Schmitzer-TorbertNCRedishAD. 2008. Task-dependent encoding of space and events by striatal neurons is dependent on neural subtype. Neuroscience153(2):349–60.
72.
SchultzWDayanPMontagueR. 1997. A neural substrate of prediction and reward. Science275:1593–99.
73.
SinghTMcDannaldMAHaneyRZCerriDHSchoenbaumG. 2010. Nucleus accumbens core and shell are necessary for reinforcer devaluation effects on Pavlovian conditioned responding. Front Integr Neurosci4:126.
74.
StalnakerTCalhoonGGOgawaMRoeschMRSchoenbaumG. 2010. Neural correlates of stimulus-response and response-outcome associations in dorsolateral versus dorsomedial striatum. Front Integr Neurosci19:12.
75.
SutherlandGRMcNaughtonBL. 2000. Memory trace reactivation in hippocampal and neocortical neuronal ensembles. Curr Opin Neurobiol10(2):180–6.
76.
SuttonRSBartoAG. 1998. Reinforcement learning: an introduction. Cambridge (MA): MIT Press.
77.
SwansonLW. 2000. Cerebral hemisphere regulation of motivated behavior. Brain Res886(1-2):113–64.
78.
TahaSAFieldsHL. 2005. Encoding of palatability and appetitive behaviors by distinct neuronal populations in the nucleus accumbens. J Neurosci25(5):1193-1202.
79.
TalmiDSeymourBDayanPDolanRJ. 2008. Human pavlovian-instrumental transfer. J Neurosci28(2):360–8.
80.
ThornCAAtallahHHoweMGraybielAM. 2010. Differential dynamics of activity changes in dorsolateral and dorsomedial striatal loops during learning. Neuron66(5):781–95.
81.
TolmanEC. 1932. Purposive behavior in animals and men. New York: Appleton-Century-Crofts.
82.
TseDLangstonRFKakeyamaMBethusISpoonerPAWoodER, and others. 2007. Schemas and memory consolidation. Science316(5821):76–82.
83.
van der MeerMAAJohnsonASchmitzer-TorbertNCRedishAD. 2010. Triple dissociation of information processing in dorsal striatum, ventral striatum, and hippocampus on a learned spatial decision task. Neuron67(1):25–32.
84.
van der MeerMAARedishAD. 2009. Covert expectation-of-reward in rat ventral striatum at decision points. Front Integr Neurosci3(1):1-15.
85.
van der MeerMAARedishAD. 2011. Theta phase precession in rat ventral striatum links place and reward information. J Neurosci31(8):2843–54.
86.
WikenheiserAMRedishAD. 2011. Changes in reward contingency modulate the trial-to-trial variability of hippocampal place cells. J Neurophysiol106(2):589–98.
87.
YinHHKnowltonBJ. 2004. Contributions of striatal subregions to place and response learning. Learn Mem11(4):459–63.
