The Michelin red guide of the brain: role of dopamine in goal-oriented navigation
Aude Retailleau
Thomas Boraud- 1Sagol Department of Neurobiology, University of Haifa, Haifa, Israel
- 2Institut des Maladies Neurodegeneratives UMR 5293, University of Bordeaux, Bordeaux, France
- 3Institut des Maladies Neurodegeneratives UMR 5293, CNRS, Bordeaux, France
Spatial learning has been recognized over the years to be under the control of the hippocampus and related temporal lobe structures. Hippocampal damage often causes severe impairments in the ability to learn and remember a location in space defined by distal visual cues. Such cognitive disabilities are found in Parkinsonian patients. We recently investigated the role of dopamine in navigation in the 6-Hydroxy-dopamine (6-OHDA) rat, a model of Parkinson’s disease (PD) commonly used to investigate the pathophysiology of dopamine depletion (Retailleau et al., 2013). We demonstrated that dopamine (DA) is essential to spatial learning as its depletion results in spatial impairments. Our results showed that the behavioral effect of DA depletion is correlated with modification of the neural encoding of spatial features and decision making processes in hippocampus. However, the origin of these alterations in the neural processing of the spatial information needs to be clarified. It could result from a local effect: dopamine depletion disturbs directly the processing of relevant spatial information at hippocampal level. Alternatively, it could result from a more distributed network effect: dopamine depletion elsewhere in the brain (entorhinal cortex, striatum, etc.) modifies the way hippocampus processes spatial information. Recent experimental evidence in rodents, demonstrated indeed, that other brain areas are involved in the acquisition of spatial information. Amongst these, the cortex—basal ganglia (BG) loop is known to be involved in reinforcement learning and has been identified as an important contributor to spatial learning. In particular, it has been shown that altered activity of the BG striatal complex can impair the ability to perform spatial learning tasks. The present review provides a glimpse of the findings obtained over the past decade that support a dialog between these two structures during spatial learning under DA control.
Introduction: of Parkinson’s Disease, Disorientation, Dopamine and Hippocampus
Parkinson disease (PD) has long been characterized as a motor disease (Agid, 1991). Emphasis has been stressed for long on the so-called triad of akinesia, rigidity and tremor. However classic drug and surgical therapies are now able to control more or less these symptoms, often at the cost of disabling side effects such as drug-induced dyskinesia (Boraud et al., 2001). Nevertheless, the increased autonomy acquired by the Parkinsonian patients unmasked cognitive symptoms previously underestimated (Sawamoto et al., 2002). It was previously thought that cognitive alteration was a late feature of the disease but, in fact, it occurs early in about 15–20% of the patients (Weintraub et al., 2011; Svenningsson et al., 2012). Amongst cognitive symptoms we can identify spatial disorientation, which has been described as an early landmark 25 years ago but left unexplored for a long time (Hovestadt et al., 1987; Taylor et al., 1989) despite its impact on life quality and public health problem (Crizzle et al., 2012).
Spatial cognition includes all the processes that allow animals to acquire process, memorize and use spatial information to perform goal-directed movements. Animals perceive space and extract pertinent information relevant to their spatial behavior using two types of sensory information: external cues supplied by the environment and internal cues supplied by their own movements. Navigation strategies are based on internal cues (path integration), external cues (taxon navigation) or on both (local navigation; O’Keefe and Conway, 1978; Gallistel, 1990; Trullier et al., 1997) Navigation could be represented in space by two systems of coordinates: an external coordinate system (allocentric representation) or an internal frame (egocentric representation; Berthoz, 1991; Poucet and Benhamou, 1997; Benhamou and Poucet, 1998). Several structures are considered to be important for building the neural representation of these spatial coordinates often called cognitive maps: amongst them the hippocampus is considered as the corner stone (O’Keefe and Dostrovsky, 1971; O’Keefe, 1979). Dopaminergic afferents to the hippocampal formation arise from both the ventral tegmental area (VTA, A10) and substantia nigra pars compacta (SNc, A8–A9) dopaminergic cell groups (Scatton et al., 1980). In addition, intra-hippocampal injections of D1 agonists and D2 antagonists improve memory (Packard and White, 1991; Wilkerson and Levin, 1999), while 6-Hydroxy-dopamine (6-OHDA) lesions of dopaminergic inputs to hippocampus induce spatial working memory deficits (Gasbarri et al., 1996). The dorsal part of Cornu Ammonis areas 3 (CA3), considered to be involved in the rapid acquisition of new memory (Kesner, 2007), is the target of dopamine (DA) projections from VTA and SNc (Scatton et al., 1980; Luo et al., 2011). DA innervation together with a higher liability of place fields as compared to those of CA1 (Barnes et al., 1990; Mizumori, 2006), makes CA3 a good candidate for the detection of the contextual significance of spatial features (Penner and Mizumori, 2012).
These facts raise the question of what happens in this brain structure when dopamine is depleted in Parkinsonian patient. We recently addressed this issue in the 6-OHDA rat, a model of PD commonly used to investigate the pathophysiology of dopamine depletion (Retailleau et al., 2013). We demonstrated that hippocampus played not only a role in the identification of locations in an environment and in the planning of the trajectories to goals and path selection as previously showed (Morris et al., 1990; Bannerman et al., 1995; Whishaw and Jarrard, 1995; Whishaw and Tomie, 1996; Hollup et al., 2001; Johnson and Redish, 2007; Rolls, 2007), but also in goal encoding. Behavioral analysis showed that sham rats are able to maximize their behavior in a baited Y-maze in order to increase the total amount of reward income over the course of the session. This behavior is correlated to an increase in the mean firing rate of neurons in CA3 at both decision point and reward location. Moreover, the lesion of the dopaminergic neurons of Substantia Nigra disrupted this ability and uncorrelated firing rate of CA3 neurons from decision and reward location (Figure 1).
Figure 1. Dopamine depletion modifies the encoding of decision process and reward location in CA3 (adapted form Retailleau et al., 2013). (A) In a baited Y-maze, the rat has to choose one of the arms where the reward is located. Rat with a 6-OHDA lesion in the right medial forebrain bundle had difficulties to find the reward when it is in the right arm as it is shown by its low ratio of good choice (i.e., the baited arm)/the number of exploration. We correlated the behavior in the maze with population responses in the right CA3 (A, right). The time spent by the animal in the decision zone (B) is correlated with the population firing rate of CA3 neurons of Sham, but not 6-OHDA rats (C). The ratio of good choice/total number of exploration is correlated with the population firing rate of CA3 neurons of Sham, but not 6-OHDA rats (E) when the animal is in the reward location (D).
However, the origin of these alterations in the neural processing of the spatial features needs to be clarified. It could result from a local effect: dopamine depletion disturbs directly the processing of relevant spatial information at hippocampal level. Alternatively, it could result from a more distributed network effect: dopamine depletion elsewhere in the brain (entorhinal cortex, striatum, etc.) modifies the way hippocampus processes spatial information. In this mini-review we address as survey of arguments in the literature that can support eachtheory.
Influence of DA in the Hippocampus
Dopamine from SNc and VTA provides a reward prediction error signal to the dorsal and ventral striatum (Schultz, 2006). It is tempting to assume that similar mechanisms are involved in the hippocampus itself. There are strong elements to support this assumption. Hippocampus is one of the rare brain areas where the 5 subtypes of DA receptors are found, D2-like (Brouwer et al., 1992) and D1-like families (Gingrich et al., 1992; Laurier et al., 1994; Sokoloff and Schwartz, 1995). In addition, intra-hippocampal injections of D1 agonists and D2 antagonists improves memory (Packard and White, 1991; Wilkerson and Levin, 1999), while 6-OHDA lesions of dopamine input to hippocampus result in spatial working memory deficits (Gasbarri et al., 1996). Exposure to a novel context increases hippocampal dopamine release (Ihalainen et al., 1999), which in turn facilitates long term potentiation (LTP) induction (Li et al., 2003; Lemon and Manahan-Vaughan, 2006). Detection of a context change by hippocampus can be used to update memory systems, which in turn may signal dopamine neurons to increase dopamine release. The subsequent increase in dopamine release in hippocampus seems to excite hippocampal neurons and so increase the duration of neural responses to glutamatergic input (Smialowski, 1987; Smialowski and Bijak, 1987) contributing as such to the increased stabilization of place fields typically observed as rats become familiar with new environmental conditions (Frank et al., 2004).
All of these data strengthened the hypothesis of a local effect of DA in the hippocampus, but there is a major drawback, which is that dopaminergic direct input to hippocampus, is very sparse and poorly described despite intensive search. As far as we know, we found only one study which described that a few fibers arise from both the VTA, A10 and SNc, A8–A9 dopaminergic cell groups (Scatton et al., 1980). But sparseness doesn’t mean inefficiency and the contrast with the reasonably high density of DA receptors in the hippocampus pleads for an actual functional effect.
Basal Ganglia and Spatial Navigation
Experimental approaches in rodents have provided behavioral evidence that pharmacological manipulations of the nucleus accumbens (NAc—part of the ventral striatum) impair acquisition and/or performance in spatial learning tasks (Annett et al., 1989; Wiener, 1993; Ploeger et al., 1994; Gal et al., 1997; Smith-Roe et al., 1999; Atallah et al., 2007; Ferretti et al., 2007). Packard and McGaugh’s seminal experiments (Packard and McGaugh, 1996) introduced the hypothesis that the striatum contributed to egocentric learning. However, recent data have generated controversy and show that the striatum is also involved in other aspects of spatial learning. Various studies examined the involvement of different sub-regions of the striatum in allocentric and egocentric learning (Voorn et al., 2004). Their results suggest that the Dorso Lateral Striatum (DLS) plays a role in egocentric spatial learning (Yin and Knowlton, 2004) based on cue-action association (van der Meer et al., 2010) and both the NAc and the Dorso Medial Striatum (DMS) plays a role in allocentric learning (Lavoie and Mizumori, 1994; Mizumori et al., 2004; Ferretti et al., 2007). The central role of the ventral striatum in spatial decision making has been confirmed by electrophysiological approach (Lansink et al., 2009; van der Meer et al., 2010; van der Meer and Redish, 2011).
These findings pave the way for integration between allocentric and egocentric process in the striatum. Striatum is a highly convergent structure where huge overlaps exist between the different functional territories (Parent and Hazrati, 1995). Thorn et al. (2010) recently demonstrated that DMS and DLS display contrasting patterns of activity during task performance that developed concurrently with sharply different dynamics. It suggests that this integration can be a cooperative (additive) process or a competition process that would contribute to multimodal decision making using mechanisms similar to those that have been modeled for action selection (Leblois et al., 2006; Guthrie et al., 2013).
Influence of DA in the Basal Ganglia
The regulation of striatal input by tonic and phasic dopamine release is so well known and described that there is not enough space in this mini-review to detail and we report the reader to more comprehensive review (Goto et al., 2007; Schultz, 2007; Humphries et al., 2012). Briefly, dopamine presysnaptic button connect the base of dendritic spines of the Medium Spiny Neurons of the striatum. From this key position they control the effect of the other inputs connecting the head of the spine (Parent and Hazrati, 1993) and play a key role in synaptic plasticity.
The role of DA at striatal level in the control of spatial navigation has been evidenced both in its ventral and dorsal part. In the ventral part, evidences are accumulated since the late 1980s. Mogenson was the first to highlight that the hippocampal signal transmission to the pedunculopontine tegmental nucleus is regulated by dopamine receptors of the NAc (Mogenson et al., 1987). The integrative role of the NAc and subpallidal area in relaying hippocampal signals to the mesencephalic locomotor region in the brainstem was investigated electrophysiologically in urethane-anaesthetized rats. A behavioral study of the functional connections was also performed in freely moving rats. In the electrophysiological experiments, subpallidal output neurons to the pedunculopontine nucleus were first identified by their antidromic responses to electrical stimulation of the pedunculopontine nucleus. Hippocampal stimulation was then shown to inhibit orthodromically some of these subpallidal neurons. The inhibitory response was attenuated following microinjection of a dopamine D2 agonist, but not a D1 agonist, into the NAc. This suggests that signal transmission from the hippocampus to the subpallidal output neurons to the pedunculopontine nucleus is modulated by a D2 receptor-mediated mechanism in the NAc. Injections of N-methyl-D-aspartate into the ventral subiculum of the hippocampus resulted in a threefold increase in locomotor responses. Injection of a D2 agonist into the accumbens reduced the hyperkinetic response dose-dependently and suggests that D2 receptors regulate locomotor responses initiated by the hippocampal-accumbens pathway. These results provide evidence of limbic (e.g., hippocampus) influences on locomotor activity by way of NAc-subpallidal-pedunculopontine nucleus connections that may contribute to adaptive behavior.
Concerning the dorsal part (DMS and DLS), the role of DA in controlling spatial navigation has been partially demonstrated only much more recently (for review: Penner and Mizumori, 2012). What is known so far is that, DA depletion in the DLS interfere with strategy shifting, but no experiments has been conducted yet in order to assess if it induced disruption of the egocentric learning.
Thus despite it is highly probable that DA depletion in the striatum could explain the effect on spatial behavior, there is no definitive evidences yet.
Towards an Integrative Representation
This quick review of the literature highlighted the multilevel implication of DA in the process of spatial information by the nervous system, however, it remains to determine at which level(s) the disruption of the dopaminergic function impacts spatial navigation in order to unravel the origin of spatial disorientation in PD patients (Hovestadt et al., 1987; Taylor et al., 1989).
The classical segregation between allocentric and egocentric learning seems too schematic and does not take into account the different modalities of perception of the environment. Hippocampus and striatum both contribute to encoding the spatial representation and seem to be involved in various modalities of perception. The hippocampus is related to the building of a spatial map (where am I?) but recent study evidenced that many place cells recorded from rats performing place or cue navigation tasks also discharged when they are at the goal location (Hok et al., 2007). The striatum provides reinforcing values to several aspects of the environment (where is my reward?), the ventral part to the pure localization aspects while the medial part encodes external cues. Meanwhile the dorsal striatum encodes the procedure (from here, which sequence of actions do I have to take in order to get my reward? (Retailleau et al., 2012). This taxonomy overlaps with the one recently proposed by Khamassi and Humphries (2012), which proposed that allocentric framework encompassed a model-based learning process while egocentric is more related to model free associative learning. It seems obvious that DA plays a key role in the striatum and that striatal DA depletion partly disrupts these processes. It is also highly probable that the rich density of DA receptors together with the response to reward location and decision processes we evidenced in CA3 (Retailleau et al., 2013) argue for also a local effect. So both mechanisms should be involved in the spatial disorientation observed in PD patients, but what remains to be established is in which proportion.
How Emerges a Spatial Strategy?
It has been shown that different spatial strategies can be used and that lesion of the hippocampus or one of the striatal territories may shift the dominant strategy used by the subject (Packard and McGaugh, 1996; Mizumori et al., 2004; Penner and Mizumori, 2012) It strongly suggests that the neural mechanism of the specific selection of one of these modalities is based on competition mechanisms.
We demonstrated that action selection and decision making are emerging properties of the architecture of the frontal and dorsal part of the cortex-basal ganglia (BG) loop (Leblois et al., 2006; Guthrie et al., 2013). It makes it a perfect analogous of the actor in the actor-critic model of Sutton and Barto (1998), working under the supervision of a critic. Actor and critic are interfaced by DA at striatal level. The exact role of DA is still discussed and may be different in different structures (Khamassi and Humphries, 2012). While it is almost evident that it acts as error prediction signal in dorsal territories it may be simple reward signal elsewhere. In fact the neural substrate underlying the critic is still unknown and different sub-population of the actor structures can be involved (Lansink et al., 2009; Gläscher et al., 2010). For a comprehensive review of the involvement of the structures in different modalities of critic processing, we report the reader to Khamassi and Humphries (2012). We propose here to extend our model of the actor to the modalities of spatial navigation. The three modalities (external coordinates, external cues and internal coordinates) are supported by three different loops. The first loop passing through the ventral BG and the hippocampus, the second supported by the medial BG and thalamus respectively, and the third loop passing through the dorsal BG (Figure 2). The competition mechanism would allow the activation of one of the three modalities. During learning, dopamine would induce plasticity at various levels (ventral/medial or dorsal striatum, hippocampus) and modify the respective weight of each loop. According to the phase of learning and the reinforcing value of one of the aspects, one of the three networks would win the competition and therefore take over the control of the spatial navigation. This theory has the advantage of taking into account both the competitive and the collaborative aspects of their interactions and explains why, when all loops are functional, one can take over from another during the course of learning while, when one is disrupted, a different loop is able to take over the spatial navigation function. At the beginning of learning the system switches from one modality to another randomly due to the system noise level. Meanwhile, each system learns in parallel from the outcomes. From Packard and McGaugh (1996) work, we can infer that the more ventral system learns first and drives animal behavior after a moderate period of learning. After longer practice, the dorsal system is strengthened, takes over from the ventral system and drives the animal behavior. Inhibition of the stronger partner of the system (the ventral in the early phase, the dorsal in the later phase) allows the weaker system to take control of the behavior. According to the topography of the lesion, dopamine depletion as in PD may disrupt one or several of these systems. Where this disruption occurs and can it be compensated for by another system is still an open question to be investigated.
Figure 2. Schematic representation of the three loops involved in spatial lnavigation. The loops are embedded in the fronto-cortical structures (SMC: sensorimotor cortex, PLC: prelimbic cortex, EC: entorhinal cortex), the basal ganglia (BG) segregated roughly into ventral (VS: ventral striatum, VP: ventral pallidum), dorsomedial (DMS: dorsomedial striatum, EP: entopeduncular nucleus) and dorsolateral (DLS: dorsolat- eral striatum, SNr: substantia nigra pars reticulata) parts, the anterior thalamus and the dorsal part of the hippocampus (Dorsal HPC). Dopamine projections from the SNc and ventral tegmantal area (VTA) are shown in green. The three functional loops are shown in blue for allocentric navigation by localization, in purple for allocentric navigation by external cues and in red for egocentric navigation. Convergence at BG level allows the system to perform selection of one of the modalities through a competition mechanism similar to those of the action selection loop (adapted form Retailleau et al., 2012).
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
Agid, Y. (1991). Parkinson’s disease: pathophysiology. Lancet 337, 1321–1324. doi: 10.1016/0140-6736(91)92989-F
Annett, L. E., McGregor, A., and Robbins, T. W. (1989). The effects of ibotenic acid lesions of the nucleus accumbens on spatial learning and extinction in the rat. Behav. Brain Res. 31, 231–242. doi: 10.1016/0166-4328(89)90005-3
Atallah, H. E., Lopez-Paniagua, D., Rudy, J. W., and O’Reilly, R. C. (2007). Separate neural substrates for skill learning and performance in the ventral and dorsal striatum. Nat. Neurosci. 10, 126–131. doi: 10.1038/nn1817
Bannerman, D. M., Good, M. A., Butcher, S. P., Ramsay, M., and Morris, R. G. (1995). Distinct components of spatial learning revealed by prior training and NMDA receptor blockade. Nature 378, 182–186. doi: 10.1038/378182a0
Barnes, C. A., McNaughton, B. L., Mizumori, S. J., Leonard, B. W., and Lin, L. H. (1990). Comparison of spatial and temporal characteristics of neuronal activity in sequential stages of hippocampal processing. Prog. Brain Res. 83, 287–300. doi: 10.1016/s0079-6123(08)61257-1
Benhamou, S., and Poucet, B. (1998). Landmark use by navigating rats: contrasting geometrical and featural information. J. Comp. Psychol. 112, 317–322. doi: 10.1037/0735-7036.112.3.317
Berthoz, A. (1991). “Reference frames for the perception and control of movement,” in Brain and Space, ed J. Paillard (Oxford, UK: Oxford University Press), 81–111.
Boraud, T., Bezard, E., Bioulac, B., and Gross, C. E. (2001). Dopamine agonist-induced dyskinesias are correlated to both firing pattern and frequency alterations of pallidal neurones in the MPTP-treated monkey. Brain 124, 546–557. doi: 10.1093/brain/124.3.546
Brouwer, N., Van Dijken, H., Ruiters, M. H., Van Willigen, J. D., and Ter Horst, G. J. (1992). Localization of dopamine D2 receptor mRNA with non-radioactive in situ hybridization histochemistry. Neurosci. Lett. 142, 223–227. doi: 10.1016/0304-3940(92)90378-k
Crizzle, A. M., Classen, S., and Uc, E. Y. (2012). Parkinson disease and driving: an evidence-based review. Neurology 79, 2067–2074. doi: 10.1212/WNL.0b013e3182749e95
Ferretti, V., Sargolini, F., Oliverio, A., Mele, A., and Roullet, P. (2007). Effects of intra-accumbens NMDA and AMPA receptor antagonists on short-term spatial learning in the Morris water maze task. Behav. Brain Res. 179, 43–49. doi: 10.1016/j.bbr.2007.01.009
Frank, M. J., Seeberger, L. C., and O’Reilly, R. C. (2004). By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 306, 1940–1943. doi: 10.1126/science.1102941
Gal, G., Joel, D., Gusak, O., Feldon, J., and Weiner, I. (1997). The effects of electrolytic lesion to the shell subterritory of the nucleus accumbens on delayed non-matching-to-sample and four-arm baited eight-arm radial-maze tasks. Behav. Neurosci. 111, 92–103. doi: 10.1037/0735-7044.111.1.92
Gallistel, C. R. (1990). Representations in animal cognition: an introduction. Cognition 37, 1–22. doi: 10.1016/0010-0277(90)90016-d
Gasbarri, A., Sulli, A., Innocenzi, R., Pacitti, C., and Brioni, J. D. (1996). Spatial memory impairment induced by lesion of the mesohippocampal dopaminergic system in the rat. Neuroscience 74, 1037–1044. doi: 10.1016/s0306-4522(96)00202-3
Gingrich, J. A., Dearry, A., Falardeau, P., Bates, M. D., Fremeau, R. T. Jr., and Caron, M. G. (1992). Location and molecular cloning of D1 dopamine receptor. Neurochem. Int. 20, 9S–15S. doi: 10.1016/0197-0186(92)90204-5
Gläscher, J., Daw, N., Dayan, P., and O’Doherty, J. P. (2010). States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron 66, 585–595. doi: 10.1016/j.neuron.2010.04.016
Goto, Y., Otani, S., and Grace, A. A. (2007). The Yin and Yang of dopamine release: a new perspective. Neuropharmacology 53, 583–587. doi: 10.1016/j.neuropharm.2007.07.007
Guthrie, M., Leblois, A., Garenne, A., and Boraud, T. (2013). Interaction between cognitive and motor cortico-basal ganglia loops during decision making: a computational study. J. Neurophysiol. 109, 3025–3040. doi: 10.1152/jn.00026.2013
Hok, V., Lenck-Santini, P. P., Roux, S., Save, E., Muller, R. U., and Poucet, B. (2007). Goal-related activity in hippocampal place cells. J. Neurosci. 27, 472–482. doi: 10.1523/jneurosci.2864-06.2007
Hollup, S. A., Kjelstrup, K. G., Hoff, J., Moser, M. B., and Moser, E. I. (2001). Impaired recognition of the goal location during spatial navigation in rats with hippocampal lesions. J. Neurosci. 21, 4505–4513.
Hovestadt, A., de Jong, G. J., and Meerwaldt, J. D. (1987). Spatial disorientation as an early symptom of Parkinson’s disease. Neurology 37, 485–487. doi: 10.1212/WNL.37.3.485
Humphries, M. D., Khamassi, M., and Gurney, K. (2012). Dopaminergic control of the exploration-exploitation trade-off via the basal ganglia. Front. Neurosci. 6:9. doi: 10.3389/fnins.2012.00009
Ihalainen, J. A., Riekkinen, P. Jr., and Feenstra, M. G. (1999). Comparison of dopamine and noradrenaline release in mouse prefrontal cortex, striatum and hippocampus using microdialysis. Neurosci. Lett. 277, 71–74. doi: 10.1016/s0304-3940(99)00840-x
Johnson, A., and Redish, A. D. (2007). Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point. J. Neurosci. 27, 12176–12189. doi: 10.1523/jneurosci.3761-07.2007
Kesner, R. P. (2007). Behavioral functions of the CA3 subregion of the hippocampus. Learn. Mem. 14, 771–781. doi: 10.1101/lm.688207
Khamassi, M., and Humphries, M. D. (2012). Integrating cortico-limbic-basal ganglia architectures for learning model-based and model-free navigation strategies. Front. Behav. Neurosci. 6:79. doi: 10.3389/fnbeh.2012.00079
Lansink, C. S., Goltstein, P. M., Lankelma, J. V., McNaughton, B. L., and Pennartz, C. M. (2009). Hippocampus leads ventral striatum in replay of place-reward information. PLoS Biol. 7:e1000173. doi: 10.1371/journal.pbio.1000173
Laurier, L. G., O’Dowd, B. F., and George, S. R. (1994). Heterogeneous tissue-specific transcription of dopamine receptor subtype messenger RNA in rat brain. Brain Res. Mol. Brain Res. 25, 344–350. doi: 10.1016/0169-328x(94)90170-8
Lavoie, A. M., and Mizumori, S. J. (1994). Spatial, movement- and reward-sensitive discharge by medial ventral striatum neurons of rats. Brain Res. 638, 157–168. doi: 10.1016/0006-8993(94)90645-9
Leblois, A., Boraud, T., Meissner, W., Bergman, H., and Hansel, D. (2006). Competition between feedback loops underlies normal and pathological dynamics in the basal ganglia. J. Neurosci. 26, 3567–3583. doi: 10.1523/jneurosci.5050-05.2006
Lemon, N., and Manahan-Vaughan, D. (2006). Dopamine D1/D5 receptors gate the acquisition of novel information through hippocampal long-term potentiation and long-term depression. J. Neurosci. 26, 7723–7729. doi: 10.1523/jneurosci.1454-06.2006
Li, S., Cullen, W. K., Anwyl, R., and Rowan, M. J. (2003). Dopamine-dependent facilitation of LTP induction in hippocampal CA1 by exposure to spatial novelty. Nat. Neurosci. 6, 526–531. doi: 10.1038/nn1049
Luo, A. H., Tahsili-Fahadan, P., Wise, R. A., Lupica, C. R., and Aston-Jones, G. (2011). Linking context with reward: a functional circuit from hippocampal CA3 to ventral tegmental area. Science 333, 353–357. doi: 10.1126/science.1204622
Mizumori, S. J. (2006). Hippocampal place fields: a neural code for episodic memory? Hippocampus 16, 685–690. doi: 10.1002/hipo.20209
Mizumori, S. J., Yeshenko, O., Gill, K. M., and Davis, D. M. (2004). Parallel processing across neural systems: implications for a multiple memory system hypothesis. Neurobiol. Learn. Mem. 82, 278–298. doi: 10.1016/j.nlm.2004.07.007
Mogenson, G. J., Ciriello, J., Garland, J., and Wu, M. (1987). Ventral pallidum projections to mediodorsal nucleus of the thalamus: an anatomical and electrophysiological investigation in the rat. Brain Res. 404, 221–230. doi: 10.1016/0006-8993(87)91373-4
Morris, R. G., Schenk, F., Tweedie, F., and Jarrard, L. E. (1990). Ibotenate lesions of hippocampus and/or subiculum: dissociating components of allocentric spatial learning. Eur. J. Neurosci. 2, 1016–1028. doi: 10.1111/j.1460-9568.1990.tb00014.x
O’Keefe, J. (1979). A review of the hippocampal place cells. Prog. Neurobiol. 13, 419–439. doi: 10.1016/0301-0082(79)90005-4
O’Keefe, J., and Conway, D. H. (1978). Hippocampal place units in the freely moving rat: why they fire where they fire. Exp. Brain Res. 31, 573–590. doi: 10.1007/bf00239813
O’Keefe, J., and Dostrovsky, J. (1971). The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. Brain Res. 34, 171–175. doi: 10.1016/0006-8993(71)90358-1
Packard, M. G., and McGaugh, J. L. (1996). Inactivation of hippocampus or caudate nucleus with lidocaine differentially affects expression of place and response learning. Neurobiol. Learn. Mem. 65, 65–72. doi: 10.1006/nlme.1996.0007
Packard, M. G., and White, N. M. (1991). Dissociation of hippocampus and caudate nucleus memory systems by posttraining intracerebral injection of dopamine agonists. Behav. Neurosci. 105, 295–306. doi: 10.1037//0735-7044.105.2.295
Parent, A., and Hazrati, L. N. (1993). Anatomical aspects of information processing in primate basal ganglia. Trends Neurosci. 16, 111–116. doi: 10.1016/0166-2236(93)90135-9
Parent, A., and Hazrati, L. N. (1995). Functional anatomy of the basal ganglia. I. The cortico-basal ganglia-thalamo-cortical loop. Brain Res. Brain Res. Rev. 20, 91–127. doi: 10.1016/0165-0173(94)00007-C
Penner, M. R., and Mizumori, S. J. (2012). Neural systems analysis of decision making during goal-directed navigation. Prog. Neurobiol. 96, 96–135. doi: 10.1016/j.pneurobio.2011.08.010
Ploeger, G. E., Spruijt, B. M., and Cools, A. R. (1994). Spatial localization in the Morris water maze in rats: acquisition is affected by intra-accumbens injections of the dopaminergic antagonist haloperidol. Behav. Neurosci. 108, 927–934. doi: 10.1037/0735-7044.108.5.927
Poucet, B., and Benhamou, S. (1997). The neuropsychology of spatial cognition in the rat. Crit. Rev. Neurobiol. 11, 101–120. doi: 10.1615/critrevneurobiol.v11.i2-3.10
Retailleau, A., Etienne, S., Guthrie, M., and Boraud, T. (2012). Where is my reward and how do I get it? Interaction between the hippocampus and the basal ganglia during spatial learning. J. Physiol. Paris 106, 72–80. doi: 10.1016/j.jphysparis.2011.10.002
Retailleau, A., Dejean, C., Fourneaux, B., Leinekugel, X., and Boraud, T. (2013). Why am I lost without dopamine? Effects of 6-OHDA lesion on the encoding of reward and decision process in CA3. Neurobiol. Dis. 59, 151–164. doi: 10.1016/j.nbd.2013.07.014
Rolls, E. T. (2007). An attractor network in the hippocampus: theory and neurophysiology. Learn. Mem. 14, 714–731. doi: 10.1101/lm.631207
Sawamoto, N., Honda, M., Hanakawa, T., Fukuyama, H., and Shibasaki, H. (2002). Cognitive slowing in Parkinson’s disease: a behavioral evaluation independent of motor slowing. J. Neurosci. 22, 5198–5203.
Scatton, B., Simon, H., Le Moal, M., and Bischoff, S. (1980). Origin of dopaminergic innervation of the rat hippocampal formation. Neurosci. Lett. 18, 125–131. doi: 10.1016/0304-3940(80)90314-6
Schultz, W. (2006). Behavioral theories and the neurophysiology of reward. Annu. Rev. Psychol. 57, 87–115. doi: 10.1146/annurev.psych.56.091103.070229
Schultz, W. (2007). Behavioral dopamine signals. Trends Neurosci. 30, 203–210. doi: 10.1016/j.tins.2007.03.007
Smialowski, A. (1987). Repeated imipramine enhances sensitivity of the brain dopaminergic system related to exploratory behavior. J. Neural. Transm. 69, 201–209. doi: 10.1007/bf01244341
Smialowski, A., and Bijak, M. (1987). Excitatory and inhibitory action of dopamine on hippocampal neurons in vitro. Involvement of D2 and D1 receptors. Neuroscience 23, 95–101. doi: 10.1016/0306-4522(87)90274-0
Smith-Roe, S. L., Sadeghian, K., and Kelley, A. E. (1999). Spatial learning and performance in the radial arm maze is impaired after N-methyl-D-aspartate (NMDA) receptor blockade in striatal subregions. Behav. Neurosci. 113, 703–717. doi: 10.1037/0735-7044.113.4.703
Sokoloff, P., and Schwartz, J. C. (1995). Novel dopamine receptors half a decade later. Trends Pharmacol. Sci. 16, 270–275. doi: 10.1016/s0165-6147(00)89044-6
Sutton, R. S., and Barto, A. G. (1998). Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press.
Svenningsson, P., Westman, E., Ballard, C., and Aarsland, D. (2012). Cognitive impairment in patients with Parkinson’s disease: diagnosis, biomarkers, and treatment. Lancet Neurol. 11, 697–707. doi: 10.1016/S1474-4422(12)70152-7
Taylor, A., Lang, A. E., and Saint-Cyr, J. A. (1989). Spatial disorientation in PD. Neurology 39, 879–880. doi: 10.1212/WNL.39.6.879-a
Thorn, C. A., Atallah, H., Howe, M., and Graybiel, A. M. (2010). Differential dynamics of activity changes in dorsolateral and dorsomedial striatal loops during learning. Neuron 66, 781–795. doi: 10.1016/j.neuron.2010.04.036
Trullier, O., Wiener, S. I., Berthoz, A., and Meyer, J. A. (1997). Biologically based artificial navigation systems: review and prospects. Prog. Neurobiol. 51, 483–544. doi: 10.1016/s0301-0082(96)00060-3
van der Meer, M. A., Kalenscher, T., Lansink, C. S., Pennartz, C. M., Berke, J. D., and Redish, A. D. (2010). Integrating early results on ventral striatal gamma oscillations in the rat. Front. Neurosci. 4:300. doi: 10.3389/fnins.2010.00300
van der Meer, M. A., and Redish, A. D. (2011). Theta phase precession in rat ventral striatum links place and reward information. J. Neurosci. 31, 2843–2854. doi: 10.1523/JNEUROSCI.4869-10.2011
Voorn, P., Vanderschuren, L. J., Groenewegen, H. J., Robbins, T. W., and Pennartz, C. M. (2004). Putting a spin on the dorsal-ventral divide of the striatum. Trends Neurosci. 27, 468–474. doi: 10.1016/j.tins.2004.06.006
Weintraub, D., Doshi, J., Koka, D., Davatzikos, C., Siderowf, A. D., Duda, J. E., et al. (2011). Neurodegeneration across stages of cognitive decline in Parkinson disease. Arch. Neurol. 68, 1562–1568. doi: 10.1001/archneurol.2011.725
Whishaw, I. Q., and Jarrard, L. E. (1995). Similarities vs. differences in place learning and circadian activity in rats after fimbria-fornix section or ibotenate removal of hippocampal cells. Hippocampus 5, 595–604. doi: 10.1002/hipo.450050610
Whishaw, I. Q., and Tomie, J. (1996). Of mice and mazes: similarities between mice and rats on dry land but not water mazes. Physiol. Behav. 60, 1191–1197. doi: 10.1016/s0031-9384(96)00176-x
Wiener, S. I. (1993). Spatial and behavioral correlates of striatal neurons in rats performing a self-initiated navigation task. J. Neurosci. 13, 3802–3817.
Wilkerson, A., and Levin, E. D. (1999). Ventral hippocampal dopamine D1 and D2 systems and spatial working memory in rats. Neuroscience 89, 743–749. doi: 10.1016/s0306-4522(98)00346-7
Keywords: dopamine, hippocampus, basal ganglia, spatial navigation, striatum
Citation: Retailleau A and Boraud T (2014) The Michelin red guide of the brain: role of dopamine in goal-oriented navigation. Front. Syst. Neurosci. 8:32. doi: 10.3389/fnsys.2014.00032
Received: 23 October 2013; Accepted: 18 February 2014;
Published online: 18 March 2014.
Edited by:
Ahmed A. Moustafa, University of Western Sydney, AustraliaReviewed by:
Mehdi Khamassi, CNRS (Centre National de la Recherche Scientifique), FranceAhmed A. Moustafa, University of Western Sydney, Australia
Copyright © 2014 Retailleau and Boraud. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Aude Retailleau, Neural Representations Lab, Sagol Department of Neurobiology, University of Haifa, 99 Aba Khoushy Ave, Mount Carmel, Haifa 3498838, Israel e-mail: retailleau@campus.haifa.ac.il