Elsevier

NeuroImage

Volume 50, Issue 1, March 2010, Pages 223-232
NeuroImage

Altered activation in association with reward-related trial-and-error learning in patients with schizophrenia

https://doi.org/10.1016/j.neuroimage.2009.12.031Get rights and content

Abstract

In patients with schizophrenia, the ability to learn from reinforcement is known to be impaired. The present fMRI study aimed at investigating the neural correlates of reinforcement-related trial-and-error learning in 19 schizophrenia patients and 20 healthy volunteers. A modified gambling paradigm was applied where each cue indicated a subsequent number which had to be guessed. In order to vary predictability, the cue-number associations were based on different probabilities (50%, 81%, 100%) which the participants were not informed about. Patients' ability to learn contingencies on the basis of feedback and reward was significantly impaired. While in healthy volunteers increasing predictability was associated with decreasing activation in a fronto-parietal network, this decrease was not detectable in patients. Analysis of expectancy-related reinforcement processing yielded a hypoactivation in putamen, dorsal cingulate and superior frontal cortex in patients relative to controls. Present results indicate that both reinforcement-associated processing and reinforcement learning might be impaired in the context of the disorder. They moreover suggest that the activation deficits which patients exhibit in association with the processing of reinforcement might constitute the basis for the learning deficits and their accompanying activation alterations.

Introduction

An increasing number of studies indicate that reinforcement learning is impaired in patients with schizophrenia (Morris et al., 2008, Premkumar et al., 2008, Waltz et al., 2007). The ability to learn from reinforcement, feedback or reward and to optimize behavior accordingly is essential for normal functioning in everyday life. This impairment must therefore be regarded as strongly debilitating. Several lines of evidence indicate that disorder-related alterations in the dopamine (DA) system might underlie this deficit (Holcomb and Rowland, 2007, Meisenzahl et al., 2007).

Dopaminergic processes are assumed to constitute the basis of the so-called prediction error (PE). In primate studies (Schultz, 2000), firing in midbrain dopamine neurons has been found to be elevated after an unpredicted reward (positive prediction error) and decreased when the predicted reward was omitted (negative prediction error). Especially the positive prediction error is thought to constitute a relevant constituent of reinforcement-based learning (Hikosaka et al., 2008, Schultz, 2006). Medial and lateral prefrontal as well as dorsal and ventral striatal regions constitute central dopaminergic projection fields. Accordingly, phasic DA releases and accompanying activation increases in fronto-striatal areas shortly after a positive prediction error have been demonstrated in a number of primate studies (Schultz et al., 1993, Schultz and Romo, 1990).

In line with this notion, a number of studies on healthy volunteers found activation increases in predominantly medial and lateral frontal regions to be inversely related to the likelihood to receive positive feedback or reward (McClure et al., 2003, O'Doherty et al., 2003, Pagnoni et al., 2002). Accordingly, increased activation in these regions in association with increased uncertainty has also been reported in healthy volunteers during feedback- or reward-based probabilistic learning (Fiorillo et al., 2003, Koch et al., 2008, Schlösser et al., 2009, Volz et al., 2003).

There is strong empirical support for an altered dopaminergic state in patients with schizophrenia (Abi-Dargham, 2004, Abi-Dargham et al., 2002, Goldman-Rakic et al., 2004). Findings on a decreased activation in the ventral striatum during reward anticipation (Kirsch et al., 2007) and processing (Schlagenhauf et al., 2008) corroborate this frequently discussed “dopamine hypothesis”. Hence, investigating reward-based learning with its putatively strong dependence on the fronto-striatal system should reveal disorder-related alterations within this system. It is surprising, therefore, that only few studies investigated the neural correlates of reward- or reinforcement-related learning in patients (Murray et al., 2008, Waltz et al., 2009, Weickert et al., 2009). Initial evidence by Shepard and Holcomb (2006) pointed to lacking habenular and dorsal striatal activation in association with impaired feedback-based learning in schizophrenia patients. Recent results by Murray and colleagues (2008) who applied a reward-based trial-and-error learning task revealed hypoactivations in mainly dorsal striatum, midbrain and frontal regions in first episode patients.

The present study explored reward-related trial-and-error learning in a dynamic environment by varying the predictability of a consequence. Reward was provided by an indication of a monetary gain. As most patients exhibited a high degree of negative symptoms we implemented a monetary loss (instead of a mere omission of a predicted reward) to intensify the negative emotion accompanying the omission of a positive consequence. Assuming that it is the absence of an expected positive consequence and its accompanying negative emotion which characterizes the negative PE the additional implementation of a monetary loss or punishment was an attempt to enhance this negative emotional effect. It should be mentioned, however, that our implementation of the negative prediction error is strictly speaking not in accordance with its original concept which implies the absence or lack of reinforcement when it was expected.

To investigate the direct relation between impaired learning performance and brain activation, we assessed individual trial learning by modeling an individual learning rate parameter and relating this to learning-associated activation patterns. As mentioned above, previous studies on healthy subjects found a negative relation between activation intensity and predictability in task-relevant areas. Against the background of the findings mentioned above (Kirsch et al., 2007, Schlagenhauf et al., 2008, Waltz et al., 2009) indicating disorder-related alterations within the fronto-striatal system which is known to be involved in successful reinforcement learning, we expected patients to show an impaired learning performance. Accordingly, we expected the negative relation between activation intensity and predictability in task-relevant areas to be missing in patients. We furthermore anticipated this to be reflected by an altered relation between individual trial learning efficiency and brain activation.

Section snippets

Subjects

Initially, 20 patients and 20 controls had been included in the study. One patient was excluded due to excessive movement. Hence, 19 right-handed (Annett, 1967) patients (12 male, 7 female) with a DSM-IV diagnosis of schizophrenia and 20 right-handed healthy subjects (12 male, 8 female) were finally included in the study. On average, patients were 35.2 ± 11.5 years old and had a mean education of 10.58 ± 2.06 years. In the healthy controls, mean age was 29.7 ± 9.1 and mean education 12.70 ± 0.92 years.

Behavioral data

The two-way repeated measures ANOVA on the percentage of correct responses yielded a significant main effect of probability condition (F(2,74) = 52.52, p < 0.001), a significant main effect of group (F(1,37) = 24.24, p < 0.001) and a significant probability-by-group interaction (F(2,74) = 6.82, p < 0.002). Planned post hoc t-tests revealed no significant difference in the percentage of correct reactions between the groups for the 50% condition (t(37) = 0.99, n.s.) but a significant difference for the 81%

Discussion

Present findings indicate that in patients with schizophrenia reward-related probabilistic trial-and-error learning is significantly impaired. They show this impairment to go along with altered activation patterns in mainly frontal, cingulate and striatal regions.

There were no significant performance differences between the groups under conditions which allowed no learning (i.e. the 50% condition). However, patients exhibited significantly less correct responses in association with moderate

Acknowledgment

This work was supported by the German Research Foundation (Deutsche Forschungsgemeinschaft [KO 3744/1-1 to K.K.]).

References (71)

  • McClureS.M. et al.

    Temporal prediction errors in a passive learning task activate human striatum

    Neuron

    (2003)
  • MorrisS.E. et al.

    Learning-related changes in brain activity following errors and performance feedback in schizophrenia

    Schizophr. Res.

    (2008)
  • NivY. et al.

    Dialogues on prediction errors

    Trends Cogn. Sci.

    (2008)
  • O'DohertyJ.P. et al.

    Temporal difference models and reward-related learning in the human brain

    Neuron

    (2003)
  • PremkumarP. et al.

    Emotional decision-making and its dissociable components in schizophrenia and schizoaffective disorder: a behavioural and MRI investigation

    Neuropsychologia

    (2008)
  • RogersR.D. et al.

    Distinct portions of anterior cingulate cortex and medial prefrontal cortex are activated by reward processing in separable phases of decision-making cognition

    Biol. Psychiatry

    (2004)
  • SchultzW.

    Getting formal with dopamine and reward

    Neuron

    (2002)
  • Van den HeuvelD.M. et al.

    Getting connected in the dopamine system

    Prog. Neurobiol.

    (2008)
  • van VeenV. et al.

    The anterior cingulate as a conflict monitor: fMRI and ERP studies

    Physiol. Behav.

    (2002)
  • VolzK.G. et al.

    Predicting events of varying probability: uncertainty investigated by fMRI

    Neuroimage

    (2003)
  • WaltzJ.A. et al.

    Selective reinforcement learning deficits in schizophrenia support predictions from computational models of striatal-cortical dysfunction

    Biol. Psychiatry

    (2007)
  • WangL. et al.

    Progressive deformation of deep brain nuclei and hippocampal-amygdala formation in schizophrenia

    Biol. Psychiatry

    (2008)
  • ZyssetS. et al.

    The neural implementation of multi-attribute decision making: a parametric fMRI study with human subjects

    Neuroimage

    (2006)
  • Abi-DarghamA.

    Do we still believe in the dopamine hypothesis? New data bring new evidence

    Int. J. Neuropsychopharmacol.

    (2004)
  • Abi-DarghamA. et al.

    Prefrontal dopamine D1 receptors and working memory in schizophrenia

    J. Neurosci.

    (2002)
  • AblerB. et al.

    Neural correlates of frustration

    Neuroreport

    (2005)
  • AnnettM.

    The binomial distribution of right, mixed and left handedness

    Q. J. Exp. Psychol.

    (1967)
  • AtkinsM. et al.

    Chlorpromazine equivalents: a consensus of opinion for both clinical and research applications

    Psychiatri. Bull.

    (1997)
  • BrettM. et al.

    The problem of functional localization in the human brain

    Nat. Rev. Neurosci.

    (2002)
  • CallicottJ.H. et al.

    Physiological dysfunction of the dorsolateral prefrontal cortex in schizophrenia revisited

    Cereb. Cortex

    (2000)
  • DayanP. et al.

    Theoretical Neuroscience Computational and Mathematical Modeling of Neural Systems

    (2001)
  • FiorilloC.D. et al.

    Discrete coding of reward probability and uncertainty by dopamine neurons

    Science

    (2003)
  • FirstM.B. et al.

    Structured Clinical Interview for DSM-IV (SCID)

    (1996)
  • GoldJ.M. et al.

    Reward processing in schizophrenia: a deficit in the representation of value

    Schizophr. Bull.

    (2008)
  • Goldman-RakicP.S. et al.

    Targeting the dopamine D1 receptor in schizophrenia: insights for cognitive dysfunction

    Psychopharmacology (Berl.)

    (2004)
  • Cited by (86)

    • Integrative Brain Network and Salience Models of Psychopathology and Cognitive Dysfunction in Schizophrenia

      2023, Biological Psychiatry
      Citation Excerpt :

      Across-species, midbrain dopamine signaling is considered a correlate of the mismatch between expectation and experience, i.e., prediction errors (83-85). Computational work on prediction errors has been most extensively carried out in the context of reward processing (86), from both model-based and model-free reinforcement learning perspectives (76, 87-90). Reward prediction error signals generated by striatal neurons are a key neural signal that facilitates optimal choice behavior and allows learning from past experiences (85, 91).

    • Neural Circuitry of Salience and Reward Processing in Psychosis

      2023, Biological Psychiatry Global Open Science
    • Memory-Based Prediction Deficits and Dorsolateral Prefrontal Dysfunction in Schizophrenia

      2023, Biological Psychiatry: Cognitive Neuroscience and Neuroimaging
    View all citing articles on Scopus
    View full text