Reward and Behavioral Factors Contributing to the Tonic Activity of Monkey Pedunculopontine Tegmental Nucleus Neurons during Saccade Tasks
Ken-ichi Okada1,2 
Yasushi Kobayashi1,2,3*- 1Laboratories for Neuroscience, Visual Neuroscience Group, Graduate School of Frontier Biosciences, Osaka University, Osaka, Japan
- 2Center for Information and Neural Networks, National Institute of Information and Communications Technology, Osaka University, Osaka, Japan
- 3Research Center for Behavioral Economics, Osaka University, Osaka, Japan
The pedunculopontine tegmental nucleus (PPTg) in the brainstem plays a role in controlling reinforcement learning and executing conditioned behavior. We previously examined the activity of PPTg neurons in monkeys during a reward-conditioned, visually guided saccade task, and reported that a population of these neurons exhibited tonic responses throughout the task period. These tonic responses might depend on prediction of the upcoming reward, successful execution of the task, or both. Here, we sought to further distinguish these factors and to investigate how each contributes to the tonic neuronal activity of the PPTg. In our normal visually guided saccade task, the monkey initially fixated on the central fixation target (FT), then made saccades to the peripheral saccade target and received a juice reward after the saccade target disappeared. Most of the tonic activity terminated shortly after the reward delivery, when the monkey broke fixation. To distinguish between reward and behavioral epochs, we then changed the task sequence for a block of trials, such that the saccade target remained visible after the reward delivery. Under these visible conditions, the monkeys tended to continue fixating on the saccade target even after the reward delivery. Therefore, the prediction of the upcoming reward and the end of an individual trial were separated in time. Regardless of the task conditions, half of the tonically active PPTg neurons terminated their activity around the time of the reward delivery, consistent with the view that PPTg neurons might send reward prediction signals until the time of reward delivery, which is essential for computing reward prediction error in reinforcement learning. On the other hand, the other half of the tonically active PPTg neurons changed their activity dependent on the task condition. In the normal condition, the tonic responses terminated around the time of the reward delivery, while in the visible condition, the activity continued until the disappearance of the saccade target (ST) after reward delivery. Thus, for these neurons, the tonic activity might be related to maintaining attention to complete fixation behavior. These results suggest that, in addition to the reward value information, some PPTg neurons might contribute to the execution of conditioned task behavior.
Introduction
Humans and other animals select and execute appropriate behavior moment by moment, based on the prediction of the upcoming reward. In this context, when we obtain or loose a reward, we must acquire and renew our behavioral policy. In such learning situations, the dopaminergic neurons in the midbrain substantia nigra pars compacta (SNc) and ventral tegmental area (VTA) are thought to play a pivotal role by encoding a reward prediction error signal (Schultz, 2002; Bromberg-Martin et al., 2010). The pedunculopontine tegmental nucleus (PPTg, also known as PPTN or PPN) in the brainstem is one structure that projects to these dopaminergic neurons, and regulates the firing of dopaminergic neurons (Mena-Segovia et al., 2008). Neurons in the PPTg are involved in reward processing and learning. Lesions of the PPTg in rats blocked the positive reinforcing effects of morphine and amphetamine (Bechara and van der Kooy, 1989) and disrupted learning (Dellu et al., 1991), and PPTg neuronal responses to reward were reported in various species, including rats (Norton et al., 2011), mice (Thompson and Felsen, 2013), cats (Dormont et al., 1998) and monkeys (Kobayashi et al., 2002; Okada et al., 2009; Hong and Hikosaka, 2014). We previously reported that a population of PPTg neurons exhibited tonic responses throughout the task period of a conditioning task, and some of them showed a significant dependency on the magnitude of the predicted reward (Okada et al., 2009). This property of the signal matches that of the reward prediction signal that is necessary for the computation of reward prediction error as represented by dopaminergic neurons.
While the PPTg preferentially projects to the dopaminergic neurons of the SNc (Watabe-Uchida et al., 2012), other studies suggested that neurons in the PPTg also carry motor and reward signals to the dopaminergic neurons of the VTA (Dautan et al., 2016). The PPTg also connects with various limbic and motor structures (Nakano, 2000; Mena-Segovia et al., 2004), and is postulated to be an integral component of the limbic-motor interface (Winn et al., 1997). Single PPTg neurons respond to various modalities of task events, including sensory, motor, and reward (Dormont et al., 1998; Kobayashi et al., 2002; Thompson and Felsen, 2013). The classical literature regarded the PPTg as a locomotor center (Garcia-Rill and Skinner, 1988), but more recent studies suggest that the PPTg is related to the execution of conditioned behavior. Dysfunction of the PPTg did not disrupt locomotion or feeding behavior, but impaired performance on several conditioned task behaviors (Inglis et al., 1994; Condé et al., 1998; MacLaren et al., 2014). We previously reported conditioned behavior-related activity in PPTg neurons, as some PPTg neurons showed activity modulation only for conditioned task saccades and not for spontaneous saccades outside the task (Okada and Kobayashi, 2009). The tonic activity of PPTg neurons also related to the behavioral performance of monkeys (Kobayashi et al., 2002; Okada et al., 2009), and it started prior to the appearance of the initial stimulus and were related to the anticipatory fixation behavior (Okada and Kobayashi, 2013). The tonic activity of PPTg neurons might play a role in executing conditioned task behavior by maintaining the motivational and/or attentional state of the animal (Steriade, 1996).
While the tonic activity of PPTg neurons was related to prediction of upcoming reward and execution of fixation behavior, the functional relationship between those activities remains unclear. Here, we sought to further distinguish these factors and to investigate how each factor contributes to the activity of neurons in the PPTg. To clarify the functional significance of tonic activity for behavior and learning, we examined individual, tonically active PPTg neurons during two task conditions that distinguished between reward and behavioral epochs.
Materials and Methods
General
We recorded the neuronal activity of neurons in the PPTg in three Japanese monkeys (Macaca fuscata; monkey 1 and 2, male; monkey 3, female) while they performed a visually guided saccade task. All experimental procedures were approved by the Committee for Animal Experiments at Okazaki National Research Institutes and Osaka University and are in accordance with the National Institutes of Health Guidelines for the Care and Use of Laboratory Animals.
Information on the experimental procedures was published previously (Kobayashi et al., 2002). In short, a head holding device, a recording chamber, and a scleral search coil were implanted under general anesthesia. The position of the recording chamber was determined based on MRI data (2.2 T; Hitachi; Okada et al., 2009). The task was controlled and behavioral and neuronal data were stored in a personal computer-based, real-time data acquisition system (TEMPO; Reflective Computing). Eye position was sampled using a search coil method at a spatial resolution of 0.1° and time resolution of 1 ms. Neuronal activity of single neurons was recorded with tungsten microelectrodes (impedance of 1–6 MΩ, FHC) and was isolated by the shape of action potentials using a template matching algorithm (MSD; Alpha Omega).
Behavioral Task
During the experiment, the monkeys were seated in a primate chair and placed in front of the screen of a 21″ cathode ray tube monitor in a dark, sound-attenuated room. The monkeys performed a visually guided saccade task, during which they made saccades to a peripheral visual target for a juice reward. Initially, the fixation target (FT, a circle of 0.8°) appeared at the center of the screen and the monkey had to fixate on it within 3000 ms to a precision of ±2° and maintain the fixation for a variable duration (400–1500 ms). Then, a saccade target (ST, a circle of 0.8°) appeared at an eccentricity of 10° from the FT in 1 of 2 (left or right) or 8 (0, 45, 90, 135, 180, 225, 270, and 315°) possible directions. Monkeys were required to saccade to the ST within 80–500 ms to a precision of ±2°. The trials that the monkeys maintained fixation on the ST for 300 ms were regarded as successful. If the monkey broke fixation at any time during the fixation period, failed to make a saccade to the ST, or broke fixation to the ST, an error tone sounded and the trial was aborted. In the normal saccade task, the ST disappeared 400 ms after the saccade (ST remained visible 100 ms after performance inspection), and after a 400-ms delay, rewards (some drops of juice) were presented together with a tone. The next trial started after an intertrial interval of 1.5–2 s.
To distinguish between reward and behavioral epochs, we changed the above task sequence for a block of trials (20–30 trials) with no apparent cue. In the normal task condition described above, ST disappeared before the reward delivery, while in the visible task condition, the ST remained visible even after the reward delivery (until 500 ms after the reward, Figure 1). Under these visible task conditions, the monkeys tended to maintain fixation on the ST even after the reward (see “Results” Section). Thus, the two task sequences enabled us to separate the prediction of upcoming reward and execution of fixational behavior in time.
Figure 1. Schematic illustration of the normal and visible tasks. In the normal task, the saccade target (ST) disappeared before the reward delivery, while in the visible task, the ST remained visible even after the reward delivery.
Data Analysis
Our database consisted of 296 neurons (216, 27, and 53 neurons for monkeys 1–3, respectively). These 296 neurons all showed tonic increases in activity during the normal visually guided saccade task, as we previously reported (Okada and Kobayashi, 2013). Within this sample of neurons, 148 were recorded from under both normal and visible task conditions.
To analyze and display neuronal activity around the time of reward, the normalized activity of each neuron was calculated as a receiver operating characteristic (ROC) value, comparing the firing rate of the neuron collected in a 200-ms window centered on that time vs. the firing rate collected during a post-reward period represented by a 500–1000 ms after the reward delivery. Termination of the tonic activity was estimated using a method based on the cumulative sum of the ROC values (Falzett et al., 1985).
For the analysis of monkey behavior during the normal and visible tasks, we extracted the fixation break behavior when they stopped watching the ST (i.e., when the gaze exited a 3° window around the ST). A fixation break <0 ms implies that the monkey broke fixation before reward delivery, while a fixation break >0 ms implies that the monkey maintained fixation even after reward delivery.
Because there were normal/visible task-related differences in fixation behavior during the reward period (0–600 ms after the onset of the reward, see “Results” Section), task-related changes in neuronal activity were defined based on their significant increase, decrease, or no significant change in activity during that reward period for the normal vs. visible task conditions (p < 0.05, Wilcoxon rank-sum test). To examine the changes in the neuronal activity after the task change, we compared the firing rates during the reward period for the five trials before and after task change (Scheffé test, p < 0.05), and compared the response in the first trial after task change with the previous five trials (Scheffé test, p < 0.05). We defined the normal/visible task-related modulation as the ROC value comparing the firing rate in the reward period between normal vs. visible trials. For the analysis of anticipatory behavior-related modulation, we used the reaction time to fixate on the FT (RTFT). We calculated behavior-related modulation by comparing the firing rate in the short vs. long RTFT trials for the pre-fixation period (0–600 ms before the appearance of the FT) using ROC analysis. The activity correlations with the initial fixation behavior-related modulation and normal/visible task-related modulation were assessed using Spearman’s rank correlation (Okada and Kobayashi, 2013).
Results
We previously examined the neuronal activity of the PPTg during a reward conditioned, visually guided saccade task, and reported that a population of PPTg neurons showed tonic responses throughout the task period (Okada et al., 2009). Some tonic changes in activity started prior to the appearance of the initial stimulus and were related to the anticipatory fixation behavior (Okada and Kobayashi, 2013). Here, we first examined the termination timing of tonic increases in activity during the normal task condition (Figure 1). Many tonic increases in activity terminated shortly after the reward delivery (peaked around 200 ms after reward), while the tonic activity of other neurons terminated shortly before the reward (Figure 2).
Figure 2. (A) Activity of the tonic excitatory neurons of the PPTg during the normal saccade task. The activity of each of the 296 PPTg neuron is presented as a row of pixels. The data are aligned to the time of reward delivery. The neurons were sorted in the order of their termination of tonic firing. The color of each pixel indicates the receiver operating characteristic (ROC) value based on a comparison of the firing rate during a post-reward period (500–1000 ms after the reward delivery) and a test window centered on the pixel (200-ms duration). The warm colors (ROC value >0.5) indicate higher firing rate relative to the post-reward period, whereas the cool colors (ROC value <0.5) indicate lower firing rate. (B) Histograms for the termination of tonic activity.
To examine whether the termination of tonic activity was related to the prediction of reward or execution of fixation behavior, we analyzed the monkeys’ fixation breaking behavior in successful trials. In the normal saccade task, there were basically two types of fixation breaks; in one (peak at about −200 ms) the monkeys broke fixation shortly after the disappearance of the ST but before the reward delivery possibly in response to the disappearance of the ST, in the other (peak at 500 ms) the monkeys broke fixation shortly after the reward delivery (Figure 3, normal condition). Thus, in the normal task condition, the end of an individual trial occurred almost at the same time as the delivery of reward and the end of fixation. Therefore, we could not distinguish neuronal activity related to prediction of upcoming reward from that associated with successful execution of the task behavior.
Figure 3. Histograms for the fixation break times under the normal (magenta) and visible (green) task conditions.
To distinguish between reward and behavioral factors in time, we changed the task sequence for a block of trials, such that the ST remained visible after the reward delivery (visible task, Figure 1). Note that, we did not change the performance inspection criteria; the monkeys were not required to fixate on the ST after reward delivery (rewards were already given). However, in the visible trials, the monkeys tended to maintain fixation even after reward delivery (Figure 3, visible condition). Comparing the normal task condition, one peak before the reward delivery is diminished possibly because there is no visual cue, while the other peak after the reward delivery is shifted later in time, yielding significantly longer fixation durations than under the normal task condition (p < 0.001, Wilcoxon rank-sum test). Therefore, in the visible task condition, the prediction of the reward and the end of an individual trial were separated in time. We recorded the activities of 67 neurons during both normal and visible task conditions and compared their activities under the two conditions.
Figure 4 shows examples of neuronal activities recorded during the normal and visible task conditions. In the normal trials, the activity of the neuron in Figure 4A increased around the time of the FT appearance, remained increased during the saccade task, and terminated shortly after the reward delivery. The monkey broke fixation around the time of the reward delivery. In the visible condition, the tonic activity also terminated shortly after the reward, while the ST remained visible after the reward delivery and the monkey tended to maintain fixation even after reward delivery. The modulation profiles relative to the reward delivery were almost identical across the two conditions. Thus, the data suggest that, regardless of the task condition or fixation behavior, this neuron might send information regarding prediction of the upcoming reward until the time of reward delivery.
Figure 4. Examples of neuronal activities during normal (magenta) and visible (green) task conditions displayed as rastergrams (upper) and spike density functions (lower). The black circles indicate appearance of the fixation target (FT), black solid lines indicate disappearance of the ST, black squares indicate fixation break, and yellow bars indicate reward delivery period. The trials are sorted by fixation break timing relative to reward delivery. (A) This representative neuron showed almost identical responses during the normal and visible task conditions. The data are aligned to the time of reward delivery. (B,C) Examples of neuronal activity during the same set of trials are shown. The data are aligned to the time of reward delivery (B) and the end of an individual trial (C). In the visible condition, tonic responses were sustained until the disappearance of the ST after the reward delivery.
Figure 4B illustrates another example of neuronal activity. This neuron exhibited an anticipatory increase in activity before the FT appeared, and maintained this activity until shortly after the reward delivery in the normal task condition. On the other hand, during the visible task condition, tonic responses were sustained until the disappearance of the ST after the reward delivery. Note that even trials that the monkey broke fixation before the reward delivery, the tonic activity was maintained until the disappearance of the ST. Figure 4C shows the same set of trials as shown in Figure 4B, aligned to the end of an individual trial such that the reward delivery in the normal trials and the disappearance of the ST in the visible trials. For this neuron, the modulation profiles relative to the end of an individual trial were almost identical across the two conditions. Thus, the tonic activity of this neuron might be related to the execution and completion of task behavior.
Some of the neurons (37%) showed almost identical response patterns relative to the reward delivery during the normal and visible task conditions, like the example neuron in Figure 4A (unchanged-type neurons, N = 25, p > 0.05, Wilcoxon rank-sum test, Figure 5A). While most of the others (58%) exhibited sustained activity during the visible condition, and thus, were significantly more active during the visible than during the normal condition (sustained-type neurons, N = 39, p < 0.05, Wilcoxon rank-sum test, Figures 4B, 5B). These neurons showed almost identical response patterns relative to the end of an individual trial, both for pre- and post-reward fixation break trials (Figure 5C, data are shown for 25 sustained-type neurons that have sufficient data for four conditions). Thus, these tonic activity was not related to actual fixation break itself. Only a minority of neurons exhibited lower activity during the visible condition (N = 3). As we mentioned above, during the normal task condition, many tonic increases in activity terminated shortly after the reward delivery, while other neuronal activity terminated shortly before that (see Figure 2B). Both types of neuronal groups included unchanged- and sustained-types of activity modulations during the visible condition. Some tonic active PPTg neurons (10 unchanged- and 16 sustained-type neurons) showed saccade-related phasic changes in activity, but the saccade-direction dependent differential responses ceased before the reward period. Another subset of the tonic active PPTg neurons (4 unchanged- and 16 sustained-type neurons) showed higher activity during the task period in the visible condition than the normal condition, as seen in Figure 5B. One possible explanation is that the visible task condition requires higher motivational demands than normal task condition, and the tonic activity reflects motivation of the monkey. It is also possible that this group of sustained-type neurons simply takes longer for their activity to decay following reward delivery in visible trials. There were no significant differences in recording location (p = 0.3, Wilcoxon rank-sum test) or spike duration (p = 0.06, Wilcoxon rank-sum test) between the unchanged- and sustained-type of neurons.
Figure 5. Population average activity during normal (magenta) and visible (green) tasks are shown separately for unchanged- (A) and sustained- (B,C) type neurons. The data are aligned to the time of reward delivery (A,B) and the end of an individual trial (C). (C) The data are further separated into pre- (dotted line) and post-reward (solid line) fixation break trials.
We then analyzed fixation behavior and neuronal responses during the transition phases between the two different task conditions for unchanged- and sustained-type neurons (Figures 6A,B). In the first trial after changes in task condition, monkeys could not predict the condition change because there was no apparent cue, but the fixation behavior changed immediately with actual task condition (p < 0.001 for the normal to visible condition and p = 0.048 for the visible to normal condition, Scheffé test, comparing the response in the first changed trial with the previous 5 trials). The unchanged-type neurons exhibited similar activities before and after the task condition change (p = 0.95 for the normal to visible and p = 0.99 for the visible to normal condition, Scheffé test). On the other hand, the activities of the sustained-type neurons changed after the task condition change (p < 0.001 for both conditions, Scheffé test), even in the first trial after the task change (p < 0.001 for the normal to visible and p = 0.029 for the visible to normal condition, Scheffé test). Thus, the activity of the sustained-type neurons changed with the actual task event and fixation behavior, which is different from the reward predicting activity of PPTg neurons (Okada et al., 2009).
Figure 6. Effects of task change on the response of unchanged- (black) and sustained- (red) type neurons and on the fixation behavior (cyan) in the change from the normal to visible (A) and the visible to normal (B) conditions. The responses represent the average firing frequencies collected from 0 ms to 600 ms after the reward delivery, normalized to the peak responses of the individual neurons.
As we previously reported, some tonic changes in activity were related to anticipatory fixation behavior (Okada and Kobayashi, 2013). Thus, one working hypothesis is that, the activity of one group of neurons changed solely related to prediction of reward, while that of others were related solely to execution of fixation behavior. If the tonic activity of some PPTg neurons reflected fixational behavior both in the initial and last phase of the task, these neurons showed higher value both for normal/visible task- and initial fixation-related modulations. We analyzed the relationship between the normal/visible task-related modulation and initial fixation-related modulation (Figure 7), but there was only a poor correlation (r = 0.13, p = 0.15, Spearman’s rank correlation). Thus, PPTg neurons encode behavioral and reward information in a rather complex manner.
Figure 7. A plot of normal/visible task-related modulation (x-axis) vs. initial fixation-related modulation (y-axis). Task-related activity modulation at the last phase of the saccade task was measured by comparing average firing rates at the reward period between normal vs. visible trials using ROC analysis. Behavior-related modulation at the initial phase of the task was measured by comparing average firing rates at 0–600 ms before the appearance of the initial FT between short vs. long reaction times to FT trials using ROC analysis.
After much training and many recording sessions, monkey 1 changed its fixation behavior and tended to break fixation before reward delivery, even during the visible task condition (Figure 8A). In this later period, the fixation-break behavior of monkey 1 was identical for the two task conditions (p = 0.99 for the normal to visible and p = 0.24 for the visible to normal conditions, Scheffé test). Thus, the monkey 1 might change behavioral policy and ignore the ST in visible condition. We then questioned whether, if tonic neuronal activity were related to the conditioned task event and monkey’s fixation behavior, the proportion of sustained-type neurons might decrease in the later period. During this later period, neuronal activity was recorded from 81 neurons during both normal and visible task conditions. The proportion of sustained-type neurons was indeed smaller during this period compared to the early period (Figure 8B, χ2 = 9.9, df = 2, p < 0.01, χ2 analysis). Thus, at the population level, the tonic activity of PPTg neurons was related to task event and fixation behavior.
Figure 8. Learning-related changes in behavior and proportions of neuron types. (A) Median time of fixation break relative to reward delivery are plotted against recording sessions for monkey 1. (B) Proportion of sustained-type neurons during the early and later periods. The proportion of sustained-type neurons was smaller in the later period than in the early period. *p < 0.01, χ2 analysis.
Discussion
These experiments were designed to determine whether the tonic activity of PPTg neurons was related to the prediction of reward or execution of task behavior. By using a modified version of a visually guided saccade task, we could distinguish between reward epoch and the end of an individual trial in time. Nearly 40% of the PPTg neurons seem to send information about the prediction of the upcoming reward until the time of reward delivery, while the activity of nearly 60% of them seemed to be related to the execution of task behavior. These findings suggest that in addition to the reward value information, which is essential for the computation of reward prediction error in reinforcement learning, some PPTg neurons contribute to the execution of the conditioned task behavior.
PPTg Encodes Reward Prediction Signal
The tonic activity of one group of PPTg neurons ceased shortly after reward delivery, both for the normal trials, when the monkeys had already broken their fixation, and for the visible trials, when the monkeys’ gazes remained fixed (Figures 4A, 5A). Thus, regardless of the task condition or monkey’s fixation behavior, this type of PPTg neurons might send the information regarding prediction of the upcoming reward until the time of reward delivery. The reinforcement learning theory assumes that the key information necessary to associate action and outcome is the reward prediction error signal that is implemented in dopaminergic neurons (Schultz, 2002; Bromberg-Martin et al., 2010). The PPTg is one of the excitatory input sources to dopaminergic neurons (Mena-Segovia et al., 2008; Watabe-Uchida et al., 2012; Dautan et al., 2016). The tonic activity of PPTg neurons that was maintained until reward delivery regardless of the monkeys’ behavior, matches the requirement of the reward prediction signal that is necessary to compute the reward prediction error in dopaminergic neurons.
Lesioning the PPTg impaired probabilistic reversal learning by reducing the sensitivity to positive reward feedback (Syed et al., 2016), and specifically, after inactivation of the posterior PPTg, rats showed no sign of omission learning (MacLaren et al., 2013). Moreover, lesioning the PPTg attenuated lever pressing for d-amphetamine in rats, but if the rats had already learned the task before the lesion, some of the deficits were ameliorated (Alderson et al., 2004). The results of these studies suggest that PPTg neurons might play a role in learning, especially in acquiring new action-outcome associations. Moreover, recent studies reported that selective lesions of cholinergic PPTg neurons did not impact responding and learning (Steidl et al., 2014; MacLaren et al., 2016), suggesting that non-cholinergic PPTg neurons are responsible for reward processing and learning. However, in our current experiments, it was difficult to determine the neurotransmitter of the recorded PPTg neurons (Boucetta et al., 2014); further studies are needed to identify the relationship between the neurochemical identity and response property of these neurons.
PPTg Contributes to the Execution of Conditioned Task Behavior
The tonic activity of another group of PPTg neurons sustained until the end of an individual trial. In the visible task, the ST remained visible and the monkeys tended to maintain fixation even after reward delivery, and in these cases, some neuronal activity was also sustained after reward delivery compared to that during the normal task condition (Figures 4B, 5B). Furthermore, the tonic activity sustained until the end of an individual trial, both for early and later fixation break trials (Figures 4C, 5C). These changes in activity occurred even in the first trial after task change, implying that it was related to the actual task event and fixation behavior and not to the prediction of upcoming events (Figure 6B). Additionally, after several recording sessions when monkey 1 began to perform the normal and visible tasks similarly, the proportion of the sustained-type neurons was less (Figure 8B). These observations suggest that this type of PPTg neuron contributes to the execution of conditioned task behavior, which is consistent with a similar view of Gut and Winn (2016). Classically, the PPTg was regarded as a locomotor center (Garcia-Rill and Skinner, 1988) because stimulation of the PPTg area induced locomotion (Garcia-Rill et al., 1987). However, recent studies reported that lesioning the PPTg did not disrupt locomotion, but affected conditioned task behavior (Inglis et al., 1994; Condé et al., 1998; MacLaren et al., 2014). Another line of studies viewed the PPTg as part of the ascending reticular activating system that maintains the motivational and attentional state of animals (Steriade, 1996). Lesioning the PPTg did produce deficits in sustained attention (Kozak et al., 2005), especially selective lesions of cholinergic neurons (Cyr et al., 2015). The behavior-related tonic activity of PPTg neurons might maintain the motivational and/or attentional state of the monkey and contribute to the successful completion of task behavior.
Involvement of the PPTg in Parkinson’s Disease
Recently, the PPTg was highlighted by its relation to Parkinson’s disease (PD), a neurodegenerative disorder and whose main pathophysiology is a loss of dopaminergic neurons, but the neuronal loss was also reported in the PPTg in PD patients (Pahapill and Lozano, 2000). Recent imaging studies revealed that, in PD patients with freezing of gate, fractional anisotropy was reduced and mean diffusivity values were increased in the PPTg (Youn et al., 2015) and structural connectivity with the PPTg was also altered (Schweder et al., 2010; Fling et al., 2013). Freezing of gait is not a simple motor symptom, but rather a deficit in initiation and execution of movement (Shine et al., 2011). PD patients with freezing of gate also exhibit disturbances in upper limb movement (Nieuwboer et al., 2009), voluntary saccades (Walton et al., 2015), and speech (Park et al., 2014). The findings of a recent study suggested roles of the PPTg in the initial analysis of sensory data and in rapid decision making (Gut and Winn, 2016). Moreover, PD patients with freezing of gate showed no pre-cue effect, whereas healthy controls exhibited faster reaction times in response to loud auditory stimuli (Thevathasan et al., 2011). The freezing of gait and postural instability in PD are resistant to dopaminergic medication (Bloem et al., 2004), and deep brain stimulation of the PPTg has emerged as an effective treatment for these symptoms (Ferraye et al., 2010; Moro et al., 2010). Deep brain stimulation of the PPTg might restore dopaminergic and non-dopaminergic function in PD patients and contribute to the execution of conditioned behavior.
Author Contributions
KO and YK: conceived and designed the experiments, performed the experiments, analyzed the data and wrote the article.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
This research is partially supported by the “Integrated research on neuropsychiatric disorders and Development of BMI Technologies for Clinical Application” and “Construction of System for Spread of Primate Model Animals” carried out under the Strategic Research Program for Brain Sciences from Japan Agency for Medical Research and development, AMED. This work was also supported by Grants-in-aid for scientific research (14431458, 15H02669) from the Japan Society for the Promotion of Science.
References
Alderson, H. L., Latimer, M. P., Blaha, C. D., Phillips, A. G., and Winn, P. (2004). An examination of d-amphetamine self-administration in pedunculopontine tegmental nucleus-lesioned rats. Neuroscience 125, 349–358. doi: 10.1016/j.neuroscience.2004.02.015
Bechara, A., and van der Kooy, D. (1989). The tegmental pedunculopontine nucleus: a brain-stem output of the limbic system critical for the conditioned place preferences produced by morphine and amphetamine. J. Neurosci. 9, 3400–3409.
Bloem, B. R., Hausdorff, J. M., Visser, J. E., and Giladi, N. (2004). Falls and freezing of gait in Parkinson’s disease: a review of two interconnected, episodic phenomena. Mov. Disord. 19, 871–884. doi: 10.1002/mds.20115
Boucetta, S., Cissé, Y., Mainville, L., Morales, M., and Jones, B. E. (2014). Discharge profiles across the sleep-waking cycle of identified cholinergic, GABAergic and glutamatergic neurons in the pontomesencephalic tegmentum of the rat. J. Neurosci. 34, 4708–4727. doi: 10.1523/JNEUROSCI.2617-13.2014
Bromberg-Martin, E. S., Matsumoto, M., and Hikosaka, O. (2010). Dopamine in motivational control: rewarding, aversive and alerting. Neuron 68, 815–834. doi: 10.1016/j.neuron.2010.11.022
Condé, H., Dormont, J. F., and Farin, D. (1998). The role of the pedunculopontine tegmental nucleus in relation to conditioned motor performance in the cat. II. Effects of reversible inactivation by intracerebral microinjections. Exp. Brain Res. 121, 411–418. doi: 10.1007/s002210050475
Cyr, M., Parent, M. J., Mechawar, N., Rosa-Neto, P., Soucy, J.-P., Clark, S. D., et al. (2015). Deficit in sustained attention following selective cholinergic lesion of the pedunculopontine tegmental nucleus in rat, as measured with both post-mortem immunocytochemistry and in vivo PET imaging with [18F]fluoroethoxybenzovesamicol. Behav. Brain Res. 278, 107–114. doi: 10.1016/j.bbr.2014.09.021
Dautan, D., Souza, A. S., Huerta-Ocampo, I., Valencia, M., Assous, M., Witten, I. B., et al. (2016). Segregated cholinergic transmission modulates dopamine neurons integrated in distinct functional circuits. Nat. Neurosci. 19, 1025–1033. doi: 10.1038/nn.4335
Dellu, F., Mayo, W., Cherkaoui, J., Le Moal, M., and Simon, H. (1991). Learning disturbances following excitotoxic lesion of cholinergic pedunculo-pontine nucleus in the rat. Brain Res. 544, 126–132. doi: 10.1016/0006-8993(91)90893-Z
Dormont, J. F., Condé, H., and Farin, D. (1998). The role of the pedunculopontine tegmental nucleus in relation to conditioned motor performance in the cat. I. Context-dependent and reinforcement-related single unit activity. Exp. Brain Res. 121, 401–410. doi: 10.1007/s002210050474
Falzett, M., Moore, R. K., Petry, H. M., and Powers, M. K. (1985). A method for determining threshold from single-unit neural activity. Brain Res. 347, 127–131. doi: 10.1016/0006-8993(85)90898-4
Ferraye, M. U., Debû, B., Fraix, V., Goetz, L., Ardouin, C., Yelnik, J., et al. (2010). Effects of pedunculopontine nucleus area stimulation on gait disorders in Parkinson’s disease. Brain 133, 205–214. doi: 10.1093/brain/awp229
Fling, B. W., Cohen, R. G., Mancini, M., Nutt, J. G., Fair, D. A., and Horak, F. B. (2013). Asymmetric pedunculopontine network connectivity in parkinsonian patients with freezing of gait. Brain 136, 2405–2418. doi: 10.1093/brain/awt172
Garcia-Rill, E., Houser, C. R., Skinner, R. D., Smith, W., and Woodward, D. J. (1987). Locomotion-inducing sites in the vicinity of the pedunculopontine nucleus. Brain Res. Bull. 18, 731–738. doi: 10.1016/0361-9230(87)90208-5
Garcia-Rill, E., and Skinner, R. D. (1988). Modulation of rhythmic function in the posterior midbrain. Neuroscience 27, 639–654. doi: 10.1016/0306-4522(88)90295-3
Gut, N. K., and Winn, P. (2016). The pedunculopontine tegmental nucleus—A functional hypothesis from the comparative literature. Mov. Disord. 31, 615–624. doi: 10.1002/mds.26556
Hong, S., and Hikosaka, O. (2014). Pedunculopontine tegmental nucleus neurons provide reward, sensorimotor and alerting signals to midbrain dopamine neurons. Neuroscience 282, 139–155. doi: 10.1016/j.neuroscience.2014.07.002
Inglis, W. L., Dunbar, J. S., and Winn, P. (1994). Outflow from the nucleus accumbens to the pedunculopontine tegmental nucleus: a dissociation between locomotor activity and the acquisition of responding for conditioned reinforcement stimulated by d-amphetamine. Neuroscience 62, 51–64. doi: 10.1016/0306-4522(94)90314-X
Kobayashi, Y., Inoue, Y., Yamamoto, M., Isa, T., and Aizawa, H. (2002). Contribution of pedunculopontine tegmental nucleus neurons to performance of visually guided saccade tasks in monkeys. J. Neurophysiol. 88, 715–731.
Kozak, R., Bowman, E. M., Latimer, M. P., Rostron, C. L., and Winn, P. (2005). Excitotoxic lesions of the pedunculopontine tegmental nucleus in rats impair performance on a test of sustained attention. Exp. Brain Res. 162, 257–264. doi: 10.1007/s00221-004-2143-3
MacLaren, D. A. A., Santini, J. A., Russell, A. L., Markovic, T., and Clark, S. D. (2014). Deficits in motor performance after pedunculopontine lesions in rats—impairment depends on demands of task. Eur. J. Neurosci. 40, 3224–3236. doi: 10.1111/ejn.12666
MacLaren, D. A. A., Wilson, D. I. G., and Winn, P. (2013). Updating of action-outcome associations is prevented by inactivation of the posterior pedunculopontine tegmental nucleus. Neurobiol. Learn. Mem. 102, 28–33. doi: 10.1016/j.nlm.2013.03.002
MacLaren, D. A. A., Wilson, D. I. G., and Winn, P. (2016). Selective lesions of the cholinergic neurons within the posterior pedunculopontine do not alter operant learning or nicotine sensitization. Brain Struct. Funct. 221, 1481–1497. doi: 10.1007/s00429-014-0985-4
Mena-Segovia, J., Bolam, J. P., and Magill, P. J. (2004). Pedunculopontine nucleus and basal ganglia: distant relatives or part of the same family? Trends Neurosci. 27, 585–588. doi: 10.1016/j.tins.2004.07.009
Mena-Segovia, J., Winn, P., and Bolam, J. P. (2008). Cholinergic modulation of midbrain dopaminergic systems. Brain Res. Rev. 58, 265–271. doi: 10.1016/j.brainresrev.2008.02.003
Moro, E., Hamani, C., Poon, Y.-Y., Al-Khairallah, T., Dostrovsky, J. O., Hutchison, W. D., et al. (2010). Unilateral pedunculopontine stimulation improves falls in Parkinson’s disease. Brain 133, 215–224. doi: 10.1093/brain/awp261
Nakano, K. (2000). Neural circuits and topographic organization of the basal ganglia and related regions. Brain Dev. 22, S5–S16. doi: 10.1016/S0387-7604(00)00139-X
Nieuwboer, A., Vercruysse, S., Feys, P., Levin, O., Spildooren, J., and Swinnen, S. (2009). Upper limb movement interruptions are correlated to freezing of gait in Parkinson’s disease. Eur. J. Neurosci. 29, 1422–1430. doi: 10.1111/j.1460-9568.2009.06681.x
Norton, A. B. W., Jo, Y. S., Clark, E. W., Taylor, C. A., and Mizumori, S. J. Y. (2011). Independent neural coding of reward and movement by pedunculopontine tegmental nucleus neurons in freely navigating rats. Eur. J. Neurosci. 33, 1885–1896. doi: 10.1111/j.1460-9568.2011.07649.x
Okada, K.-I., and Kobayashi, Y. (2009). Characterization of oculomotor and visual activities in the primate pedunculopontine tegmental nucleus during visually guided saccade tasks. Eur. J. Neurosci. 30, 2211–2223. doi: 10.1111/j.1460-9568.2009.07009.x
Okada, K.-I., and Kobayashi, Y. (2013). Reward prediction-related increases and decreases in tonic neuronal activity of the pedunculopontine tegmental nucleus. Front. Integr. Neurosci. 7:36. doi: 10.3389/fnint.2013.00036
Okada, K., Toyama, K., Inoue, Y., Isa, T., and Kobayashi, Y. (2009). Different pedunculopontine tegmental neurons signal predicted and actual task rewards. J. Neurosci. 29, 4858–4870. doi: 10.1523/JNEUROSCI.4415-08.2009
Pahapill, P. A., and Lozano, A. M. (2000). The pedunculopontine nucleus and Parkinson’s disease. Brain 123, 1767–1783. doi: 10.1093/brain/123.9.1767
Park, H. K., Yoo, J. Y., Kwon, M., Lee, J.-H., Lee, S. J., Kim, S. R., et al. (2014). Gait freezing and speech disturbance in Parkinson’s disease. Neurol. Sci. 35, 357–363. doi: 10.1007/s10072-013-1519-1
Schultz, W. (2002). Getting formal with dopamine and reward. Neuron 36, 241–263. doi: 10.1016/S0896-6273(02)00967-4
Schweder, P. M., Hansen, P. C., Green, A. L., Quaghebeur, G., Stein, J., and Aziz, T. Z. (2010). Connectivity of the pedunculopontine nucleus in parkinsonian freezing of gait. Neuroreport 21, 914–916. doi: 10.1097/WNR.0b013e32833ce5f1
Shine, J. M., Naismith, S. L., and Lewis, S. J. G. (2011). The pathophysiological mechanisms underlying freezing of gait in Parkinson’s disease. J. Clin. Neurosci. 18, 1154–1157. doi: 10.1016/j.jocn.2011.02.007
Steidl, S., Wang, H., and Wise, R. A. (2014). Lesions of cholinergic pedunculopontine tegmental nucleus neurons fail to affect cocaine or heroin self-administration or conditioned place preference in rats. PLoS One 9:e84412. doi: 10.1371/journal.pone.0084412
Steriade, M. (1996). Arousal: revisiting the reticular activating system. Science 272, 225–226. doi: 10.1126/science.272.5259.225
Syed, A., Baker, P. M., and Ragozzino, M. E. (2016). Pedunculopontine tegmental nucleus lesions impair probabilistic reversal learning by reducing sensitivity to positive reward feedback. Neurobiol. Learn. Mem. 131, 1–8. doi: 10.1016/j.nlm.2016.03.010
Thevathasan, W., Pogosyan, A., Hyam, J. A., Jenkinson, N., Bogdanovic, M., Coyne, T. J., et al. (2011). A block to pre-prepared movement in gait freezing, relieved by pedunculopontine nucleus stimulation. Brain 134, 2085–2095. doi: 10.1093/brain/awr131
Thompson, J. A., and Felsen, G. (2013). Activity in mouse pedunculopontine tegmental nucleus reflects action and outcome in a decision-making task. J. Neurophysiol. 110, 2817–2829. doi: 10.1152/jn.00464.2013
Walton, C. C., O’Callaghan, C., Hall, J. M., Gilat, M., Mowszowski, L., Naismith, S. L., et al. (2015). Antisaccade errors reveal cognitive control deficits in Parkinson’s disease with freezing of gait. J. Neurol. 262, 2745–2754. doi: 10.1007/s00415-015-7910-5
Watabe-Uchida, M., Zhu, L., Ogawa, S. K., Vamanrao, A., and Uchida, N. (2012). Whole-brain mapping of direct inputs to midbrain dopamine neurons. Neuron 74, 858–873. doi: 10.1016/j.neuron.2012.03.017
Winn, P., Brown, V. J., and Inglis, W. L. (1997). On the relationships between the striatum and the pedunculopontine tegmental nucleus. Crit. Rev. Neurobiol. 11, 241–261. doi: 10.1615/CritRevNeurobiol.v11.i4.10
Keywords: reward, saccade, reinforcement learning, Parkinson’s disease, movement execution
Citation: Okada K and Kobayashi Y (2016) Reward and Behavioral Factors Contributing to the Tonic Activity of Monkey Pedunculopontine Tegmental Nucleus Neurons during Saccade Tasks. Front. Syst. Neurosci. 10:94. doi: 10.3389/fnsys.2016.00094
Received: 28 September 2016; Accepted: 03 November 2016;
Published: 11 November 2016.
Edited by:
Hisao Nishijo, University of Toyama, JapanReviewed by:
Gidon Felsen, University of Colorado Denver School of Medicine, USAPhilip Winn, University of Strathclyde, UK
Copyright © 2016 Okada and Kobayashi. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Yasushi Kobayashi, yasushi@fbs.osaka-u.ac.jp