Altered activation in association with reward-related trial-and-error learning in patients with schizophrenia
Introduction
An increasing number of studies indicate that reinforcement learning is impaired in patients with schizophrenia (Morris et al., 2008, Premkumar et al., 2008, Waltz et al., 2007). The ability to learn from reinforcement, feedback or reward and to optimize behavior accordingly is essential for normal functioning in everyday life. This impairment must therefore be regarded as strongly debilitating. Several lines of evidence indicate that disorder-related alterations in the dopamine (DA) system might underlie this deficit (Holcomb and Rowland, 2007, Meisenzahl et al., 2007).
Dopaminergic processes are assumed to constitute the basis of the so-called prediction error (PE). In primate studies (Schultz, 2000), firing in midbrain dopamine neurons has been found to be elevated after an unpredicted reward (positive prediction error) and decreased when the predicted reward was omitted (negative prediction error). Especially the positive prediction error is thought to constitute a relevant constituent of reinforcement-based learning (Hikosaka et al., 2008, Schultz, 2006). Medial and lateral prefrontal as well as dorsal and ventral striatal regions constitute central dopaminergic projection fields. Accordingly, phasic DA releases and accompanying activation increases in fronto-striatal areas shortly after a positive prediction error have been demonstrated in a number of primate studies (Schultz et al., 1993, Schultz and Romo, 1990).
In line with this notion, a number of studies on healthy volunteers found activation increases in predominantly medial and lateral frontal regions to be inversely related to the likelihood to receive positive feedback or reward (McClure et al., 2003, O'Doherty et al., 2003, Pagnoni et al., 2002). Accordingly, increased activation in these regions in association with increased uncertainty has also been reported in healthy volunteers during feedback- or reward-based probabilistic learning (Fiorillo et al., 2003, Koch et al., 2008, Schlösser et al., 2009, Volz et al., 2003).
There is strong empirical support for an altered dopaminergic state in patients with schizophrenia (Abi-Dargham, 2004, Abi-Dargham et al., 2002, Goldman-Rakic et al., 2004). Findings on a decreased activation in the ventral striatum during reward anticipation (Kirsch et al., 2007) and processing (Schlagenhauf et al., 2008) corroborate this frequently discussed “dopamine hypothesis”. Hence, investigating reward-based learning with its putatively strong dependence on the fronto-striatal system should reveal disorder-related alterations within this system. It is surprising, therefore, that only few studies investigated the neural correlates of reward- or reinforcement-related learning in patients (Murray et al., 2008, Waltz et al., 2009, Weickert et al., 2009). Initial evidence by Shepard and Holcomb (2006) pointed to lacking habenular and dorsal striatal activation in association with impaired feedback-based learning in schizophrenia patients. Recent results by Murray and colleagues (2008) who applied a reward-based trial-and-error learning task revealed hypoactivations in mainly dorsal striatum, midbrain and frontal regions in first episode patients.
The present study explored reward-related trial-and-error learning in a dynamic environment by varying the predictability of a consequence. Reward was provided by an indication of a monetary gain. As most patients exhibited a high degree of negative symptoms we implemented a monetary loss (instead of a mere omission of a predicted reward) to intensify the negative emotion accompanying the omission of a positive consequence. Assuming that it is the absence of an expected positive consequence and its accompanying negative emotion which characterizes the negative PE the additional implementation of a monetary loss or punishment was an attempt to enhance this negative emotional effect. It should be mentioned, however, that our implementation of the negative prediction error is strictly speaking not in accordance with its original concept which implies the absence or lack of reinforcement when it was expected.
To investigate the direct relation between impaired learning performance and brain activation, we assessed individual trial learning by modeling an individual learning rate parameter and relating this to learning-associated activation patterns. As mentioned above, previous studies on healthy subjects found a negative relation between activation intensity and predictability in task-relevant areas. Against the background of the findings mentioned above (Kirsch et al., 2007, Schlagenhauf et al., 2008, Waltz et al., 2009) indicating disorder-related alterations within the fronto-striatal system which is known to be involved in successful reinforcement learning, we expected patients to show an impaired learning performance. Accordingly, we expected the negative relation between activation intensity and predictability in task-relevant areas to be missing in patients. We furthermore anticipated this to be reflected by an altered relation between individual trial learning efficiency and brain activation.
Section snippets
Subjects
Initially, 20 patients and 20 controls had been included in the study. One patient was excluded due to excessive movement. Hence, 19 right-handed (Annett, 1967) patients (12 male, 7 female) with a DSM-IV diagnosis of schizophrenia and 20 right-handed healthy subjects (12 male, 8 female) were finally included in the study. On average, patients were 35.2 ± 11.5 years old and had a mean education of 10.58 ± 2.06 years. In the healthy controls, mean age was 29.7 ± 9.1 and mean education 12.70 ± 0.92 years.
Behavioral data
The two-way repeated measures ANOVA on the percentage of correct responses yielded a significant main effect of probability condition (F(2,74) = 52.52, p < 0.001), a significant main effect of group (F(1,37) = 24.24, p < 0.001) and a significant probability-by-group interaction (F(2,74) = 6.82, p < 0.002). Planned post hoc t-tests revealed no significant difference in the percentage of correct reactions between the groups for the 50% condition (t(37) = 0.99, n.s.) but a significant difference for the 81%
Discussion
Present findings indicate that in patients with schizophrenia reward-related probabilistic trial-and-error learning is significantly impaired. They show this impairment to go along with altered activation patterns in mainly frontal, cingulate and striatal regions.
There were no significant performance differences between the groups under conditions which allowed no learning (i.e. the 50% condition). However, patients exhibited significantly less correct responses in association with moderate
Acknowledgment
This work was supported by the German Research Foundation (Deutsche Forschungsgemeinschaft [KO 3744/1-1 to K.K.]).
References (71)
- et al.
Prediction error as a linear function of reward probability is coded in human nucleus accumbens
Neuroimage
(2006) - et al.
The influence of feedback valence in associative learning
Neuroimage
(2009) - et al.
Task-induced deactivation of midline cortical regions in schizophrenia assessed with fMRI
Schizophr. Res.
(2007) - et al.
Decision-making impairments in the context of intact reward sensitivity in schizophrenia
Biol. Psychiatry
(2008) - et al.
New insights on the subcortical representation of reward
Curr. Opin. Neurobiol.
(2008) - et al.
Temporal changes in neural activation during practice of information retrieval from short-term memory: an fMRI study
Brain Res.
(2006) - et al.
Temporal modeling demonstrates preserved overlearning processes in schizophrenia: an fMRI study
Neuroscience
(2007) - et al.
Functional effects of antipsychotic drugs: comparing clozapine with haloperidol
Biol. Psychiatry
(2003) - et al.
Functional effects of single dose first- and second-generation antipsychotic administration in subjects with schizophrenia
Psychiatry Res.
(2005) Prefrontal cortex dysfunction during working memory performance in schizophrenia: reconciling discrepant findings
Schizophr. Res.
(2003)
Temporal prediction errors in a passive learning task activate human striatum
Neuron
Learning-related changes in brain activity following errors and performance feedback in schizophrenia
Schizophr. Res.
Dialogues on prediction errors
Trends Cogn. Sci.
Temporal difference models and reward-related learning in the human brain
Neuron
Emotional decision-making and its dissociable components in schizophrenia and schizoaffective disorder: a behavioural and MRI investigation
Neuropsychologia
Distinct portions of anterior cingulate cortex and medial prefrontal cortex are activated by reward processing in separable phases of decision-making cognition
Biol. Psychiatry
Getting formal with dopamine and reward
Neuron
Getting connected in the dopamine system
Prog. Neurobiol.
The anterior cingulate as a conflict monitor: fMRI and ERP studies
Physiol. Behav.
Predicting events of varying probability: uncertainty investigated by fMRI
Neuroimage
Selective reinforcement learning deficits in schizophrenia support predictions from computational models of striatal-cortical dysfunction
Biol. Psychiatry
Progressive deformation of deep brain nuclei and hippocampal-amygdala formation in schizophrenia
Biol. Psychiatry
The neural implementation of multi-attribute decision making: a parametric fMRI study with human subjects
Neuroimage
Do we still believe in the dopamine hypothesis? New data bring new evidence
Int. J. Neuropsychopharmacol.
Prefrontal dopamine D1 receptors and working memory in schizophrenia
J. Neurosci.
Neural correlates of frustration
Neuroreport
The binomial distribution of right, mixed and left handedness
Q. J. Exp. Psychol.
Chlorpromazine equivalents: a consensus of opinion for both clinical and research applications
Psychiatri. Bull.
The problem of functional localization in the human brain
Nat. Rev. Neurosci.
Physiological dysfunction of the dorsolateral prefrontal cortex in schizophrenia revisited
Cereb. Cortex
Theoretical Neuroscience Computational and Mathematical Modeling of Neural Systems
Discrete coding of reward probability and uncertainty by dopamine neurons
Science
Structured Clinical Interview for DSM-IV (SCID)
Reward processing in schizophrenia: a deficit in the representation of value
Schizophr. Bull.
Targeting the dopamine D1 receptor in schizophrenia: insights for cognitive dysfunction
Psychopharmacology (Berl.)
Cited by (86)
Integrative Brain Network and Salience Models of Psychopathology and Cognitive Dysfunction in Schizophrenia
2023, Biological PsychiatryCitation Excerpt :Across-species, midbrain dopamine signaling is considered a correlate of the mismatch between expectation and experience, i.e., prediction errors (83-85). Computational work on prediction errors has been most extensively carried out in the context of reward processing (86), from both model-based and model-free reinforcement learning perspectives (76, 87-90). Reward prediction error signals generated by striatal neurons are a key neural signal that facilitates optimal choice behavior and allows learning from past experiences (85, 91).
Neural Circuitry of Salience and Reward Processing in Psychosis
2023, Biological Psychiatry Global Open ScienceMemory-Based Prediction Deficits and Dorsolateral Prefrontal Dysfunction in Schizophrenia
2023, Biological Psychiatry: Cognitive Neuroscience and NeuroimagingRetention of Value Representations Across Time in People With Schizophrenia and Healthy Control Subjects
2021, Biological Psychiatry: Cognitive Neuroscience and Neuroimaging