Social modulation of learned behavior by dopamine in the basal ganglia: Insights from songbirds
Highlights
► Songbirds are a good model to study the function of dopamine in the basal ganglia. ► Dopamine modulates information transmission through the song-related basal ganglia. ► Dopaminergic input to the basal ganglia modulates song with social context. ► The tonic dopaminergic input may regulate the exploration–exploitation balance.
Introduction
The neuromodulator dopamine (DA) is delivered to a variety of forebrain structures by neurons located in the substantia nigra par compacta (SNc) and ventral tegmental area (VTA) (Smith and Kieval, 2000). Dysfunction of the dopaminergic system in Parkinson’s disease or drug addiction leads to motor, cognitive, and motivational symptoms (Barbeau, 1967, Hoehn and Yahr, 1967, Wise, 1987, Sulzer, 2011). Although it has long been identified with motor function, DA is therefore also essential for cognitive and motivational processes (Salamone, 1992, Wise, 2004). In particular, the involvement of DA in motivation and reward learning has been emphasized in the last decades.
Behaviors resulting in the obtaining rewards, such as sexual behavior, social behavior, feeding, and foraging, are likely to be repeated, reflecting a high degree of motivation of the subject. Such rewards are advantageous or necessary for the survival of individuals, and reward learning is central to the implementation of the interactions of individuals between themselves (social interactions) and with their environment. Because rewarding events, such as food consumption, copulation, and drug uptake, are associated with strong DA release (Schultz et al., 1993), DA neurons are thought to code for the rewarding value of events. In the framework of reinforcement learning theory (Sutton and Barto, 1990), DA delivery in the striatum reflects reward prediction errors and provides a reinforcement signal guiding motor learning toward optimal behaviors, eventually leading to maximal reward uptakes (Montague et al., 1996, Phillips et al., 2003, Wise, 2004). It important to note, however, that DA is neither necessary nor sufficient to mediate the hedonic impact of reward (Berridge, 2007), and that other brain regions may encode reward and its prediction (Gonon, 2009), questioning the simplistic view of DA as a reward signal.
The extrasynaptic distribution of DA receptors indicates that DA neurons convey information by volume transmission (one-to-many) in the extracellular space rather than by one-to-one synaptic transmission (Zoli et al., 1999, Gonon et al., 2000). In the view of DA as a reward signal, fast phasic changes in extracellular DA concentration activity on a subsecond to second timescale encode the probability of reward availability, its uncertainty, and more generally the prediction error (Schultz et al., 1997, Fiorillo et al., 2003). But importantly, DA neurons can operate in distinct temporal modes (Hauber, 2010). On one hand, DA neurons display phasic firing in bursts of action potentials in relation to behaviorally relevant events. Such phasic bursts are mediated by glutamatergic inputs originating in particular in the ventral tegmentum, gated by inhibitory afferents from the ventral pallidum (VP) (Grace et al., 2007). Burst firing triggers high amplitude, transient DA release. On the other hand, DA neurons display slow and irregular spontaneous activity (2–10 Hz). Although it is driven by an intrinsic pacemaker, spontaneous activity is regulated by afferent input, in particular the inhibitory connections from the VP (Grace and Bunney, 1984). The modulation of this tonic firing of DA neurons underlies slow changes in the concentration of tonic extracellular DA on much slower time scales (seconds to minutes) (Floresco et al., 2003). Spillover from synapses following phasic DA release also contributes to modulate extrasynaptic DA concentrations and is tightly regulated by DA transporters (Cragg and Rice, 2004).
These temporally distinct modes of DA signaling seem to be related to different kinds of information and may serve dissociable behavioral functions. Slower changes in tonic DA levels in the basal ganglia (BG) and cortex likely serve functions other than signaling reinforcement in motor learning (Salamone et al., 2005, Costa, 2007). Interestingly, reward processing and social interaction processes share common neural substrates (Caldú and Dreher, 2007). One might consider the possibility that phasic DA release in the striatum signals reward prediction error, and may provide a reinforcement signal to the BG (Schultz et al., 1993, Phillips et al., 2003), while tonic dopaminergic signaling would be involved in the representation of motivational state related to social cues and contexts (Wang et al., 1999, Anstrom et al., 2009, Aragona and Wang, 2009). Another possibility is for tonic DA signals to be involved in the regulation of inter-individual differences in exploration and exploitation behaviors (Frank et al., 2009). The function of tonic DA levels in the BG, outside of specific pathophysiology, remains largely unexplored.
Songbirds use learned vocalizations to communicate during courtship or aggressive behaviors. These vocalizations, called song, represent a complex sensorimotor task and require fast coordination of the numerous laryngeal and respiratory muscles. Moreover, song learning in young birds relies on a variety of complex social cues which may serve to open some attentional or arousal gate, which then permits sensory learning (Doupe and Kuhl, 1999). For example, songbirds have been shown to learn alien songs from live tutors when they would reject the same songs presented by tape playback (Petrinovich and Baptista, 1987). In adult birds, song performance is also modulated by social context, and male birds sing a more stereotyped song when singing to a female than when singing alone (Sossinka and Böhner, 1980, Kao et al., 2005). As DA has been involved in the control of both complex motor (Wickens et al., 2003, Joshua et al., 2009) and social behaviors (Young and Wang, 2004, Skuse and Gallagher, 2009), songbirds offer a unique opportunity as a model system to study the role of DA signaling in the social modulation of a learned sensorimotor task.
Over the past 15 years, advances in anatomical, physiological and histochemical characterization of avian BG neurons and circuitry have revealed evolutionary homologies to mammalian BG networks (Fig. 1; Bottjer and Johnson, 1997, Reiner et al., 1998, Luo et al., 2001, Reiner et al., 2004, Doupe et al., 2005). The song-related BG nucleus Area X contains neuron types homologue to both the striatal and pallidal components of the mammalian BG circuit (Farries and Perkel, 2002, Goldberg and Fee, 2010, Goldberg et al., 2010). It receives dense dopaminergic innervation from the ventral tegmental area (VTA; Lewis et al., 1981, Bottjer, 1993, Gale et al., 2008) which neurons display physiological properties very similar to those of mammalian SNc and VTA (Gale and Perkel, 2006). This specialized BG–thalamo-cortical loop devoted to a complex, social, naturally learned sensorimotor task is particularly tractable for elucidating the interwoven sensory, motor and reward signals carried by DA in the BG, and the function of DA signaling in skill learning and execution (Doupe et al., 2005).
Section snippets
DA receptors in the song system
The two families of dopaminergic receptors (D1-like and D2-like) are present in the songbird brain, with particularly high expression levels in the striatum (Casto and Ball, 1994). Interestingly, the song-related BG nucleus Area X displays even higher concentration of both DA receptor subtypes compared to the surrounding striatum (Casto and Ball, 1994). D1-like DA receptor are present to a lesser extent in cortical song-related nuclei such as HVC (used as a proper name), the robust nucleus of
DA effects on mammalian striatal neurons
DA is known for several decades to modulate synaptic transmission and neuronal activity in the mammalian BG (Akaike et al., 1987, Hu and Wang, 1988, Nicola et al., 2000). Because the striatum receives the largest DA input in the BG in mammals (Smith and Villalba, 2008), previous physiological studies have concentrated on DA effects on striatal neurons. While early studies reported an excitatory effect of DA through D1-like receptor and inhibitory through D2 (Albin et al., 1989), the complex
A loop circuit between striatal and dopaminergic neurons in birds
In birds as in mammals, the BG and in particular song-related nucleus Area X receive dense dopaminergic input from the SNc and the VTA (Lewis et al., 1981, Bottjer, 1993, Soha et al., 1996). At first glance, Area X differs from mammalian BG in its gross anatomical structure. Spiny neurons in Area X, homologue to the mammalian striatal projection neurons, do not project outside of Area X (Farries and Perkel, 2002). Instead, Area X contains a class of neurons homologous to mammalian pallidal
Social context-modulated song-related DA signals in the avian BG
Several experimental findings show that the dopaminergic signal is important for song modulation by social context. First, interfering with DA transmission induces profound decrements in female-directed singing and in copulatory behavior in males (Harding, 2004), and on female responses to male songs (Riters and Pawlisch, 2007, Pawlisch and Riters, 2010). Secondly, immunocytochemical data suggest that DA in both song control and motivation brain regions is tightly linked to the regulation of
D1-receptor mediate social context modulation of song spectral variability
As stated in the introduction, adult birds modulate trial-to-trial variability in their song with social context (Sossinka and Böhner, 1980, Kao et al., 2005). More precisely, the spectral variability of syllables from rendition to rendition, measured as the spread of the distribution of fundamental frequencies of syllables displaying a harmonic spectral structure, is decreased when the bird sings courtship songs directed toward a female (Kao et al., 2005, Kao and Brainard, 2006). The output
Tonic DA in the BG regulates exploration–exploitation trade-off
The song-related BG–thalamo-cortical loop is critical for song learning in juvenile birds (Scharff and Nottebohm, 1991), and for song plasticity in adults (Brainard and Doupe, 2000, Andalman and Fee, 2009). Based on all the experimental data summarized in this review, can we postulate a function for the role of the tonic dopaminergic signal in the song-related BG nucleus Area X in song learning? And can this role be extrapolated to other vertebrates?
What about D2?
While I have proposed a function for the tonic dopaminergic signaling through D1-like receptors in the BG of songbird, acting to regulate the balance between exploration and exploitation depending on social context, the function of tonic signaling through D2-like receptors, also present in large amounts in the songbird BG, remains to be elucidated. Although it has been proposed that two different populations of MSN neurons in Area X would express D1-like and D2-like dopaminergic receptors (
Acknowledgements
I am grateful to David J. Perkel, Thomas Boraud and David Hansel for their valuable comments on this manuscript, and to Samuel D Gale for fruitful discussions. This work was supported by the French Agency for Research, Grant ANR-10-PDOC-016-01 and by the European Union, Grant PIRG06-GA-2009-256488.
References (143)
- et al.
Excitatory and inhibitory effects of dopamine on neuronal activity of the caudate nucleus neurons in vitro
Brain Res.
(1987) - et al.
The functional anatomy of basal ganglia disorders
Trends Neurosci.
(1989) - et al.
Increased phasic dopamine signaling in the mesolimbic pathway during social defeat in rats
Neuroscience
(2009) The role of dopamine in locomotor activity and learning
Brain Res.
(1983)- et al.
DAncing past the DAT at a DA synapse
Trends Neurosci.
(2004) - et al.
Birdbrains could teach basal ganglia research a new song
Trends Neurosci.
(2005) The dopaminergic hypothesis of attention-deficit/hyperactivity disorder needs re-examining
TINS
(2009)- et al.
Geometry and kinetics of dopaminergic transmission in the rat striatum and in mice lacking the dopamine transporter
Prog. Brain Res.
(2000) - et al.
Regulation of firing of dopaminergic neurons and control of goal-directed behaviors
Trends Neurosci.
(2007) The primate basal ganglia: parallel and integrative networks
J. Chem. Neuroanat.
(2003)
Evidence that dopamine within motivation and song control brain regions regulates birdsong context-dependently
Physiol. Behav.
The dynamics of dopamine in control of motor behavior
Curr. Opin. Neurobiol.
Striatal plasticity and basal ganglia circuit function
Neuron
Neuromodulatory control of striatal plasticity and behavior
Curr. Opin. Neurobiol.
Selective behavioral responses to male song are affected by the dopamine agonist GBR-12909 in female European starlings (Sturnus vulgaris)
Brain Res.
Unitary IPSPs drive precise thalamic spiking in a circuit required for learning
Neuron
Anim. Behav.
Seasonal changes in courtship behavior, plasma androgen levels and in hypothalamic aromatase immunoreactivity in male free-living European starlings (Sturnus vulgaris)
Gen. Comp. Endocrinol.
Structural and functional evolution of the basal ganglia in vertebrates
Brain Res. Brain Res. Rev.
Lesions to the medial preoptic nucleus differentially affect singing and nest box-directed behaviors within and outside of the breeding season in European starlings (Sturnus vulgaris)
Behav. Neurosci.
Cocaine-induced stereotypy is linked to an imbalance between the medial prefrontal and sensorimotor circuits of the basal ganglia
Eur. J. Neurosci.
A basal ganglia–forebrain circuit in the songbird biases motor output to avoid vocal errors
Proc. Natl. Acad. Sci. USA
Dopamine regulation of social choice in a monogamous rodent species
Front. Behav. Neurosci.
The “pink spot”, 3,4-dimethoxyphenylethylamine and dopamine. Relationship to Parkinson’s disease and to schizophrenia
Rev. Can. Biol.
The debate over dopamine’s role in reward: the case for incentive salience
Psychopharmacology
The distribution of tyrosine hydroxylase immunoreactivity in the brains of male and female zebra finches
J. Neurobiol.
Parallel pathways for vocal learning in basal ganglia of songbirds
Nat. Neurosci.
Circuits, hormones, and learning: vocal behavior in songbirds
J. Neurobiol.
Interruption of a basal ganglia-forebrain circuit prevents plasticity of learned vocalizations
Nature
Hormonal and genetic influences on processing reward and social information
Ann. N. Y. Acad. Sci.
A measure of striatal function predicts motor stereotypy
Nat. Neurosci.
Is the songbird Area X striatal, pallidal, or both? An anatomical study
J. Comp. Neurol.
Characterization and localization of D1 dopamine receptors in the sexually dimorphic vocal control nucleus, Area X, and the basal ganglia of European starlings
J. Neurobiol.
Chronic microsensors for longitudinal, subsecond dopamine detection in behaving animals
Nat. Methods
Neuron-type-specific signals for reward and punishment in the ventral tegmental area
Nature
Plastic corticostriatal circuits for action learning: what’s dopamine got to do with it?
Ann. N. Y. Acad. Sci.
Dopamine modulates excitability of spiny neurons in the avian basal ganglia
J. Neurosci. Off. J. Soc. Neurosci.
Long-term potentiation in an avian basal ganglia nucleus essential for vocal learning
J. Neurosci. Off. J. Soc. Neurosci.
Presynaptic depression of glutamatergic synaptic transmission by D1-like dopamine receptor activation in the avian basal ganglia
J. Neurosci. Off. J. Soc. Neurosci.
Birdsong and human speech: common themes and mechanisms
Annu. Rev. Neurosci.
A telencephalic nucleus essential for song learning contains neurons with physiological characteristics of both striatum and globus pallidus
J. Neurosci. Off. J. Soc. Neurosci.
A hypothesis for basal ganglia-dependent reinforcement learning in the songbird
Neuroscience
Ventral tegmental area neurons in learned appetitive behavior and positive reinforcement
Annu. Rev. Neurosci.
Spike-timing dependent plasticity in the striatum
Front. Synap. Neurosci.
Discrete coding of reward probability and uncertainty by dopamine neurons
Science
Afferent modulation of dopamine neuron firing differentially regulates tonic and phasic dopamine transmission
Nat. Neurosci.
Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation
Nat. Neurosci.
Properties of dopamine release and uptake in the songbird basal ganglia
J. Neurophysiol.
Physiological properties of zebra finch ventral tegmental area and substantia nigra pars compacta neurons
J. Neurophysiol.
A basal ganglia pathway drives selective auditory responses in songbird dopaminergic neurons via disinhibition
J. Neurosci. Off. J. Soc. Neurosci.
Cited by (12)
Capturing the Effects of Domestication on Vocal Learning Complexity
2021, Trends in Cognitive SciencesCitation Excerpt :In male zebra finches, neuronal firing in the primarily dopaminergic midbrain VTA, which projects to Area X (see Figure I in Box 1), correlates with the singing of a more stereotyped female-directed song [57]. Social-context–dependent alterations to dopaminergic signaling regulate the change from juvenile exploratory vocalizations to the adult crystallization of a more stereotyped song [58]. Independently of the influences of social context on song learning, stimulation or inhibition of midbrain (VTA) dopaminergic projections to the songbird striatum, driven by auditory feedback, is sufficient to guide syllable pitch learning [59].
Social context-dependent singing alters molecular markers of synaptic plasticity signaling in finch basal ganglia Area X
2021, Behavioural Brain ResearchCitation Excerpt :Akt protein levels trended downward with more time spent singing UD song (Fig. 8C), but there was no relationship between the number of UD motifs to Akt protein levels (Fig. 8D). UD song has more acoustic variability from rendition-to-rendition compared to FD song [32,33]. These song output differences are supported by studies measuring neural activity of neurons in Area X.
Dopamine depletion affects communicative intentionality in Parkinson's disease patients: Evidence from action kinematics
2016, CortexCitation Excerpt :As a final point, the present findings on the possible relationship between dopamine systems and social behaviour are also in line with animal studies showing that in rats D2-R-related functions balance pro- and antisocial behaviours (Aragona et al., 2006; Couppis, Kennedy, & Stanwood, 2008). In songbirds, individuals which show communicative ‘disorders’ can be motivated to communicate through pharmacological manipulations of dopamine receptors, suggesting dopamine as a candidate neurotransmitter system involved in stimulating social communication (Leblois, 2013). In conclusion, our results support the view that specific kinematic patterns characterize and distinguish communicative actions from actions executed with a purely individual intent.
Social intentions in Parkinson's disease patients: A kinematic study
2015, CortexCitation Excerpt :The fact that the VS may be activated during social interaction implies that the basal ganglia (BG), the main recipient of VS outputs (Draganski et al., 2008), are involved in these endeavours. The BG are implicated in sensorimotor learning and receive a strong dopaminergic signal, which has been shown to play an important role in social interactions (Leblois, 2013). Despite this evidence, however, how the BG works to integrate social cues and how a dysfunction of the dopaminergic system can affect the ability to plan and to execute actions in a social context remains largely unexplored.
A neural network model for timing control with reinforcement
2022, Frontiers in Computational NeuroscienceNeural correlates of vocal initiation in the VTA/SNc of juvenile male zebra finches
2021, Scientific Reports