Abstract
One prevalent theory of learning states that dopamine neurons signal mismatches between expected and actual outcomes, called temporal difference errors (TDEs). Evidence indicates that dopamine system dysfunction is involved in negative symptoms of schizophrenia (SZ), including avolition and anhedonia. As such, we predicted that brain responses to TDEs in dopamine midbrain nuclei and target areas would be abnormal in SZ. A total of 18 clinically stable patients with chronic SZ and 18 controls participated in an fMRI study, which used a passive conditioning task. In the task, the delivery of a small amount of juice followed a light stimulus by exactly 6 s on approximately 75% of 78 total trials, and was further delayed by 4–7 s on the remaining trials. The delayed juice delivery was designed to elicit the two types of TDE signals, associated with the recognition that a reward was omitted at the expected time, and delivered at an unexpected time. Main effects of TDE valence and group differences in the positive–negative TDE contrast (unexpected juice deliveries–juice omissions) were assessed through whole-brain and regions of interest (ROI) analyses. Main effects of TDE valence were observed for the entire sample in the midbrain, left putamen, left cerebellum, and primary gustatory cortex, bilaterally. Whole-brain analyses revealed group differences in the positive–negative TDE contrast in the right putamen and left precentral gyrus, whereas ROI analyses revealed additional group differences in the midbrain, insula, and parietal operculum, on the right, the putamen and cerebellum, on the left, and the frontal operculum, bilaterally. Further, these group differences were generally driven by attenuated responses in patients to positive TDEs (unexpected juice deliveries), whereas responses to negative TDEs (unexpected juice omissions) were largely intact. Patients also showed reductions in responses to juice deliveries on standard trials, and more blunted reinforcer responses in the left putamen corresponded to higher ratings of avolition. These results provide evidence that SZ patients show abnormal brain responses associated with the processing of a primary reinforcer, which may be a source of motivational deficits.
Similar content being viewed by others
INTRODUCTION
In addition to positive, or psychotic symptoms, schizophrenia (SZ) is characterized by negative symptoms, reflecting deficits in areas such as motivation, emotional expression, and speech production. These negative symptoms have been shown to relate closely to functional impairments exhibited by patients (Green et al, 2000; Tamminga et al, 1998), particularly with regards to engagement in goal-directed behavior. The fact that patients also show deficits on experimental tasks of reinforcement learning suggests that dysfunctional reinforcement processing mechanisms may contribute to negative symptoms. This idea is further supported by evidence that brain dopamine (DA) systems, which are known to be important in modulating reinforcement learning (Montague et al, 2004), are disrupted in SZ (Abi-Dargham et al, 2000; Breier et al, 1997; Laruelle et al, 1996; Okubo et al, 1997). As such, the purpose of our study was to investigate whether disrupted reward processing mechanisms contribute to deficits in motivated behavior in SZ by examining the function of DA in reward processing.
One prevalent theory of DA system function is that DA cells signal, through phasic modulations of their firing rates, the registration of mismatches between reinforcer expectations and outcomes (Schultz et al, 1997), known as ‘reward prediction errors,’ or ‘temporal difference errors’ (TDEs; if they pertain to the timing of reinforcement). Neuroimaging studies with humans have observed activation changes in both (presumably) dopaminergic midbrain nuclei (Aron et al, 2004) and dopamine target structures (Berns et al, 2001; McClure et al, 2003; Seymour et al, 2004) in association with the perception of prediction errors. McClure et al (2003) found that an area of the left putamen showed significantly decreased activation, relative to baseline, when a juice reinforcer was omitted at the expected time (corresponding to a negative TDE). This area also showed increased activation when the reinforcer was unexpectedly delivered 10 s after the light cue (corresponding to a positive TDE).
The results of several studies (Jensen et al, 2008; Murray et al, 2007), in fact, suggest that SZ patients have specific impairment in the perception and representation of prediction errors. Murray et al (2007) found differential activations between psychotic patients and controls in the dopaminergic midbrain in response to unexpected monetary outcomes in an operant learning paradigm. In learning to choose between stimuli predicting monetary outcomes, patients showed attenuated neural responses to outcomes that were better or worse than expected, whereas their responses to neutral outcomes were somewhat elevated, relative to those of controls. Corlett et al (2007) found that prediction-error-evoked fMRI activity in prefrontal cortex (PFC) was strongly predictive of clinical ratings of delusional thought content, as measured by the Brief Psychiatric Rating Scale (BPRS; Overall and Gorman, 1962).
STUDY OBJECTIVES AND PREDICTIONS
Multiple studies (Corlett et al, 2007; Murray et al, 2007) have identified abnormalities in prediction signaling in the context of higher-level cognitive tasks using a symbolic reinforcer (eg money). Determining whether or not there are alterations in responses prediction errors regarding a primary reinforcer (eg a food reinforcer) can make an important contribution to the interpretation of the above findings, by helping to establish whether these findings generalize to more elementary forms of learning. To address this issue, we implemented a version of the Pavlovian classical conditioning paradigm employed by McClure et al (2003), which used a juice reinforcer, in conjunction with fMRI. This paradigm enabled us to separately investigate neural responses to positive and negative prediction errors in SZ patients. Furthermore, our study was specifically designed to examine the signaling of errors in predictions about the timing of a reinforcer (TDEs) in SZ. Finally, because we hypothesized that faulty TDE processing is at the root of deficits in reward-driven learning and behavior, our goal was to investigate relationships between reinforcer responses and ratings of negative symptoms.
On the basis of evidence of both disrupted DA function in SZ (Abi-Dargham et al, 2000; Breier et al, 1997; Laruelle et al, 1996; Okubo et al, 1997) and deficits in reinforcement learning driven by both positive and negative feedback (Prentice et al, 2008; Waltz et al, 2007; Waltz and Gold, 2007), we hypothesized that patients would show abnormal brain correlates of both positive and negative TDEs. On the basis of our formulation that negative symptoms in SZ reflect a reduced ability of rewards and punishments to modulate learning and motivate behavior in SZ patients, we further hypothesized that patients would show systematic relationships between ratings of negative symptoms, such as avolition, and brain responses to the presence and absence of rewards.
PARTICIPANTS AND METHODS
Recruiting and Screening of Participants
A total of 18 patients and 18 demographically matched healthy controls participated in the study (Table 1). All participants provided written informed consent to a protocol approved by the institutional review boards at the National Institute on Drug Abuse and the University of Maryland School of Medicine. All participants were right-handed, as determined by the Edinburgh Handedness Inventory (Oldfield, 1971).
All patients were on stable antipsychotic medication regimens (no changes for 4 weeks), almost all with second-generation antipsychotic drugs (Supplementary Table 1). The diagnosis of SZ or schizoaffective disorder in patients was confirmed using the SCID-I (First et al, 1997), as was the absence of Axis I diagnoses in control participants. Control participants diagnosed with Axis II personality disorders (based on screening with the SIDP-R; Pfohl et al, 1989) were also excluded. Participants in both experimental groups underwent medical screening, involving a medical history and physical exam, and those with evidence of any neurological or medical condition that might confound data interpretation were excluded (such as significant head injury, stroke or severe vascular problems, chronic untreated diabetes, etc). Further exclusionary criteria included: pregnancy, current illegal drug use (both verified by urine screens), and admission of past substance dependence.
Procedures
Events before MRI scanning
Participants were instructed to abstain from alcohol for 24 h before each visit, which was verified by a breathalyzer test before each session. Subjects were deprived of fluid for 3 h before the actual MRI session to increase the value of the juice, which served as a primary reinforcer (see below).
Experimental task (TDE paradigm)
Participants underwent a classical conditioning task (Berns et al, 2001; McClure et al, 2003; Figure 1). Before actual MRI scanning, participants were trained in a mock scanner to associate the receipt of a juice reward with the presentation of a light stimulus (a large yellow disc on a black background, which was displayed on a screen behind the subject's head and viewed through a mirror attached to the head coil). During training, participants completed three runs of 26 ‘standard’ trials each, during which the light-juice interval was always 6 s. Following training, subjects performed the same task in the MRI scanner. During scanning, however, a number of ‘catch’ events were introduced. On catch trials the juice was not delivered at the ‘expected’ time point, but rather 4–7 s after the expected delivery. On the basis of McClure et al (2003), we predicted that this type of catch event would induce a negative prediction error at the time when juice was not delivered as expected, and that a subsequent positive prediction error would be incurred by the ‘unexpected’ delivery of juice. The number of paired training events outnumbered the catch events by a ratio of approximately 3:1 in scanning sessions. The time between individual trials ranged from 4 to 12 s.
Before training, participants chose one type of commercially available juice from three options: apple, grape, or fruit punch. Juice was delivered through small-bore IV tubing connected to syringes set into an MR-compatible syringe pump (Harvard Apparatus, Boston, MA). The end of the tubing was placed in the participant's mouth, attached to a commercial sports mouthguard to stabilize the tube. A 600-μl squirt of juice was delivered on each trial, at a rate of 1 ml/s (duration=0.6 s). Between runs of the TDE task, subjects rated the pleasantness of the juice stimuli, by moving a cursor on a visual-analog scale using a wheel manipulandum, with a rating of 8 representing extreme liking, and rating of 0 representing extreme aversion.
Interval timing test
The capacity of an individual to learn the association between the light cue and the juice reward is reliant upon their ability to accurately estimate the time at which they should (based on previous learning) expect receipt of the reward. To assess time estimation abilities, participants completed a short test of timing function, based on the work of Rao and co-workers (Harrington et al, 2004; Hinton et al, 2004), outside of the MRI scanner and following all other experimental phases. On each trial, subjects were presented with three temporal intervals, defined by four beeps, and asked to judge whether the third interval was shorter than, longer than, or the same as the two preceding intervals. The first two intervals were 6 s in all cases, whereas the third ranged from 4.5 to 7.5 s, in increments of 0.5 s. Thus, the timing paradigm assessed the ability to estimate a similar time interval as was used in the TDE/classical conditioning paradigm, albeit demanding much greater precision.
Other psychological assessments
Cognitive function was assessed in all participants using three standard measures: the Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler, 1999), the Wechsler Test of Adult Reading (WTAR; Wechsler, 2001), and the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS; Randolph et al, 1998; Wilk et al, 2005). To quantify the ability of study participants to experience pleasure both physically and in social contexts, we had all subjects complete the Scales for Physical and Social Anhedonia (Chapman et al, 1976). Standard symptom ratings were obtained for all patients using the Brief Psychiatric Rating Scale (BPRS; Overall and Gorman, 1962), the Scale for the Assessment of Negative Symptoms (SANS; Andreasen, 1984), and the Calgary Depression Scale (CDS; Addington et al, 1992).
Acquisition and Analysis of fMRI Data
MRI scanning
A 3-T Siemens Allegra scanner (Erlangen, Germany) acquired whole-brain functional EPI images for measurement of T2*-weighted BOLD effects (4-mm oblique axial slices, 30° axial to coronal; 64 × 64 matrix; FOV=22 × 22 cm; TR=2 s; TE=27 ms; FA=80°). In each scanning session, a whole-brain oblique axial T1-weighted structural image (MPRAGE) was acquired for anatomical reference (1-mm3 isotropic voxels; TR=2.5 s; TE=4.38 ms; FA=80°).
Whole-brain analyses of MRI data
All preprocessing and first-level analyses of MRI data were performed using the AFNI software package (Analysis of Functional NeuroImages; Cox, 1996). Preprocessing steps included volume registration for motion correction, slice-timing correction, and temporal normalization. To generate statistical parametric maps for individual subjects, we used an approach similar to McClure et al (2003) and D'Ardenne et al (2008). We used three types of events as regressors in regression analyses: juice deliveries on standard trials, and positive and negative TDEs (juice deliveries and omissions on catch trials, respectively). Regressors were modeled by an idealized hemodynamic response function and its temporal derivative, time locked to the event onsets. Additional regressors of no interest included the six motion-correction curves.
For the purposes of this study, we examined responses only after learning because it provides the strongest method for studying TDE responses in isolation, while avoiding dependencies on exact learning-model parameters. That is, when learning is complete, positive and negative prediction errors evoked by changing the time of reward can be fully modeled without dependence on how effective TDEs were in driving learning.
Beta weights from the first-level analysis were spatially normalized to Talairach space and smoothed with a 4.2 mm FWHM Gaussian kernel before second-level (ie group) analyses. The main second-level analyses were two separate repeated-measures ANOVAs, performed to determine brain areas showing significant main effects of group, event type, and group × event type interactions: one with factors of group (patients vs control) and TDE valence (positive vs negative), and one with factors of group and trial type (catch vs standard, to compare responses to ‘unexpected’ and ‘expected’ deliveries of the reinforcer). Correction for multiple comparisons was accomplished using a Monte Carlo simulation, which determined a minimum cluster size required for a given voxel-wise threshold. This simulation determined that, for our chosen voxel-wise threshold (p<0.002), a minimum cluster size of 11 voxels (424 μl) was required to achieve an overall significance level of p<0.05.
Analyses of group differences in regions of interest
To compare the performance of the two groups in regions involved in the processing of the reinforcer, as well as prediction errors, we selected all significant clusters showing significant main effects of TDE valence and all clusters showing significant group × TDE valence interactions. This criterion yielded 11 regions of interest (ROIs), described below. On the basis of the results of McClure et al (2003), which used the same paradigm to examine prediction-error-related brain activity in healthy volunteers, we selected two additional regions of interest a priori, in the left and right putamen by drawing spheres of 5 mm radius around their Talairach coordinates (±18, 2, 8). Thus, we performed further analyses on 13 total ROIs. Reported p-values based on this group analysis for a priori regions of interest were corrected for the number of comparisons made within each region.
Correlation analyses
We used Spearman correlation analyses to assess relationships among timing task performance, clinical ratings of avolition (to assess reductions in goal-directed behavior in SZ), and BOLD responses to juice deliveries in the 11 ROIs identified functionally, and the 2 ROIs identified a priori. Clinical ratings of avolition were determined by summing the 4 items from the avolition subscale of the 25-item SANS.
RESULTS
Brain Regions Distinguishing Positive From Negative TDEs
A two-way ANOVA with factors of group and TDE valence was performed to determine the brain regions distinguishing positive from negative TDEs in the entire sample (N=36; Table 2(a)). This ANOVA produced five brain regions showing main effects of TDE valence, all of which resulted from activations to positive TDEs and deactivations to negative TDEs. Regions showing main effects of TDE valence included left putamen and the right midbrain, consistent with previous findings (D’Ardenne et al, 2008; McClure et al, 2003; Murray et al, 2007; Figure 2a–c). Large areas also emerged, centered on the left and right frontal and parietal operculum, extending into insular cortex. These areas correspond to primary gustatory cortex, as identified in previous studies (Lee et al, 1998; Small et al, 1999). For the purpose of ROI analysis, we divided each of the large areas into three components, based on their overlap with precentral gyrus, postcentral gyrus, and insula, as identified by the Talairach daemon (Lancaster et al, 2000). Together with an additional cluster emerging in the declive of the cerebellum, this yielded 9 ROIs for further analysis. Note that a main effect of group, without respect to TDE valence, is difficult to interpret, because we expected (and observed) activations for positive TDEs, and deactivations for negative TDEs; thus, we do not report this effect.
Two brain regions, the right putamen and in the left frontal operculum, showed significant group × TDE valence interactions (Table 2(b); Figure 2d–f). In these two areas, controls showed BOLD activations to positive TDEs and deactivations to negative TDEs, whereas patients showed the opposite (aberrant) pattern (Figure 3b and c; ROIs 10 and 11).
ROI Analyses of Group Differences in the Positive–Negative TDE Contrast
With 11 regions emerging from whole-brain analyses, and 2 regions chosen a priori, we performed additional analyses on 13 total ROIs. In 9 of the 13 ROIs investigated, the two subject groups differed in the magnitude of the (positive–negative) TDE contrast (the difference trended toward significance in ROI 13, the right putamen; Figure 3a). This was true despite the fact that in nine of these regions (ROIs 1–9), the magnitude of the (positive–negative) TDE contrast was significantly different from zero within the patient group (all t's >2.75; Figure 3a).
Further analyses revealed that group differences in the (positive–negative) TDE contrast were generally driven by attenuated responses to positive TDEs in the patient group (all t's >3.10; Figure 3b). Whereas controls showed significant activations to positive TDEs, relative to baseline, in all 13 ROIs, patients showed significant activations to positive TDEs only in the pre- and postcentral gyrus, bilaterally (ROIs 5–8). In response to negative TDEs, however, patients and controls did not differ significantly in their responses to negative TDEs in any of the ROIs (Figure 3c), with patients showing robust deactivations, relative to baseline, in nine regions (ROIs 1–9), including the midbrain and left putamen.
Together, these analyses indicate that group differences in BOLD signal modulations, due to the valence of prediction errors, generally result from attenuated responses in patients to unexpected administrations of the reinforcer. To determine whether patients’ responses to positive TDEs were selectively disrupted, or if patients showed more general dysfunction in neural processing of the reinforcer, we performed analyses comparing subjects’ responses to unexpected (catch) vs expected (standard) juice deliveries.
MRI Responses to Non-TDE Events
Juice deliveries on standard vs catch trials
A two-way ANOVA of group × trial type was performed to compare unexpected deliveries of reward (positive TDEs) with expected reward delivery (null TDEs). This analysis revealed significant main effects of group in numerous areas, regardless of whether they occurred on standard or catch trials (Table 3). Controls showed more positive responses to juice delivery than patients, regardless of trial type, in all of these areas, which overlapped with many of the ROIs identified above, including midbrain, left and right putamen, left and right frontal operculum, and left insular cortex.
Also in our previously identified ROIs, participants within each group showed similar responses to juice deliveries on standard and catch trials, with controls showing significant activations to both in all regions (relative to baseline; compare Figure 3b and d). In 3 of the 13 ROIs, in fact, BOLD responses to expected juice deliveries and unexpected juice deliveries correlated significantly in the patient group (Supplementary Table 2).
No region showed a significant main effect of trial type, and no brain region showed a significant group × trial type interaction. Thus, we did not find evidence of an enhanced neural response to (presumably unexpected) juice deliveries on catch trials, relative to juice deliveries on standard trials.
MRI responses to the light cue
It should be noted that patients did not show attenuated activations, or deactivations to all event types; brain responses to the reward-predicting cue (the conditioned stimulus) were largely intact in SZ patients (Supplementary Table 3; Supplementary Figure 1).
Behavioral and Correlation Analyses
Timing task performance
Patients and controls differed significantly in their performance on the interval timing task performed outside the scanner (t(31)=3.38; p=0.002). Whereas patients correctly estimated only 45.8% (SD=12.1%) of test intervals, controls correctly estimated 61.6% (SD=14.7%) of test intervals.
When we computed Spearman correlation coefficients for relationships between scores on the interval timing task and MRI responses to standard juice deliveries, we observed a single significant correlation: timing task performance in patients related systematically to responses to standard juice deliveries in the left frontal operculum/precentral gyrus (Table 4).
Anhedonia and avolition
Exploratory analyses of relationships between BOLD data and rating of clinical symptoms revealed that avolition subscores from the SANS correlated negatively with MRI responses to standard juice deliveries in the left frontal and parietal operculum (pre- and postcentral gyrus; Table 4). Furthermore, significant negative correlations were observed between avolition ratings and MRI responses to standard juice deliveries in the left and right a priori putamen ROIs, and six additional ROIs showed correlations of medium effect size (d>0.3; Cohen, 1992). In all of these ROIs, patients with the highest ratings of avolition showed the least positive BOLD responses to juice deliveries. Supplementary Figure 2 illustrates this pattern in the left frontal operculum and the left putamen ROIs.
By contrast, avolition ratings bore no relationship to self-reports of reinforcer enjoyment (Spearman's ρ=−0.023), which did not differ between patients and controls (mean rating of 5.8 (1.4) for patients vs mean of 5.3 (1.1) for controls; t(34)=1.25; p>0.10). Avolition ratings also bore no relationship to self-reports of physical (ρ=−0.287) and social anhedonia (ρ=−0.037) from the Chapman scales. Patients and controls also did not differ significantly on these measures (t(34)<1.6; p>0.10 in both cases; Table 1).
DISCUSSION
In this functional MRI study of prediction error-related activity, our data led us to draw the following conclusions: (1) multiple components of reward circuits showed responses that distinguished between positive and negative TDEs in the entire sample of participants; (2) patients with SZ showed reduced contrasts in brain activity evoked by positive and negative TDEs in multiple brain areas, driven largely by attenuated responses to positive TDEs; (3) patients showed attenuated responses in numerous brain regions to delivery of the juice reinforcer on standard trials, as well; and (4) clinical ratings of avolition correlated significantly with brain activity evoked by standard juice delivery in the primary gustatory cortex and putamen. Each of these findings will be addressed in turn.
Contrasts between positive and negative TDEs
When combined across groups, neural activity distinguished between positive and negative TDEs in multiple brain regions. Consistent with the results of McClure and co-workers (D’Ardenne et al, 2008; McClure et al, 2003), these areas included the midbrain and the left putamen. Both ventral and dorsal areas of the neostriatum have been identified as components of circuits for reward-based learning (Haber et al, 2006; Voorn et al, 2004), and a function for these areas in outcome processing, in particular, is supported by the results of several previous neuroimaging studies (Delgado et al, 2005, 2000; McClure et al, 2003). Our results provide further evidence that projections from the midbrain to the basal ganglia comprise a critical component of circuits for processing outcomes, and mismatches between expectations and outcomes.
In addition, significant contrasts between unexpected juice delivery and omissions were observed, bilaterally, in the insula and the frontal and parietal operculum, consistent with the results of previous studies that have linked these areas to gustation (Lee et al, 1998), general interoception (Craig, 2003; Critchley et al, 2004), and the processing of outcomes (Paulus et al, 2005; Seymour et al, 2004).
Group differences in responses to TDEs
Whole-brain analyses revealed that patients showed a reduced BOLD contrast between positive and negative TDEs in right putamen and the left frontal operculum. Additional ROI analyses revealed group differences in the (positive–negative) TDE contrast in the right midbrain, the left putamen, right primary gustatory cortex, and left cerebellum.
Evidence of abnormal prediction-error responses in SZ suggests a possible factor underlying reinforcement learning deficits commonly observed in patients (Kemali et al, 1987; Keri, 2008; Schwartz et al, 2003). Our finding of attenuated prediction-error-related activity in SZ is consistent with the results of a recent neuroimaging study (Murray et al, 2007), which found evidence of abnormal prediction error responses in the midbrain, among other brain regions, in SZ patients performing an operant learning paradigm. Several other studies have identified attenuated responses in the striatum in SZ patients (Kumari et al, 2002; Reiss et al, 2006), associated with the feedback-driven learning of procedures.
We observed group differences between patients and controls mainly for positive prediction errors (unexpected juice delivery), rather than negative prediction errors (unexpected juice omissions). This finding suggests that sensitivity to outcomes that are worse than expected (negative TDEs) might be relatively preserved in SZ. The relatively intact response to negative TDEs in patients in this study was somewhat unexpected, in light of the behavioral evidence pointing to impaired online error-correction behavior in SZ (Prentice et al, 2008; Waltz and Gold, 2007).
It is possible, however, that the type of the learning involved, specifically in terms of whether subjects need to explicitly represent feedback and use it to make rapid adjustments in behavior, may influence the extent to which brain signals associated with negative prediction errors are abnormal in SZ. Despite relatively clear evidence of deficits in the ability to use feedback to make rapid adjustments in behavior, other behavioral results indicate that punishment-driven learning may be unimpaired in SZ patients if the learning task is of a probabilistic or procedural nature (Waltz et al, 2007). The fact that the passive conditioning task in this study has been shown to depend largely on striatal regions, implicated in procedural learning, might explain our finding of largely intact neural responses to negative TDEs in patients. It is possible, furthermore, that the negative TDE error signal is intact in SZ, but that the ability of target brain areas to use error information to modulate response selection and learning is disrupted.
Responses to reinforcer deliveries on standard trials
Contrary to our expectations, patients also showed attenuated responses in numerous brain regions to delivery of the juice reinforcer on standard trials. Furthermore, we observed that neuronal activity evoked by juice delivery in the left primary gustatory cortex and bilateral putamen correlated significantly with clinical ratings of avolition. These findings, together with our observation that BOLD responses to expected and unexpected juice deliveries correlated significantly in three ROIs in the patient group, suggest that abnormal processing of the juice reinforcer was a contributing factor to the attenuated positive prediction error signals in SZ patients, and that reductions in goal-directed behavior frequently observed in SZ patients may be influenced by abnormal processing of rewards.
Findings regarding the experience of rewards in SZ are mixed. Evidence from behavioral studies (Cohen and Minor, 2008; Gard et al, 2007; Germans and Kring, 2000) supports, in large part, the idea that patients with SZ have intact self-reported experience of rewards. Indeed, patients in this study reported finding the juice just as pleasant as controls did. These findings, however, appear to be contradicted by the results of multiple neuroimaging studies (Crespo-Facorro et al, 2001; Paradiso et al, 2003; Plailly et al, 2006), which have found evidence of abnormal neural responses to pleasant stimuli in SZ (especially primary reinforcers, such as pleasant odors). Furthermore, multiple studies have found evidence of reduced gray matter volume in components of reward processing circuitry, such as insula and ventral PFC (Crespo-Facorro et al, 2000; Davatzikos et al, 2005).
One possible explanation for this apparent dichotomy is that, even if patients report normal hedonic experiences, the physiology underlying those hedonic experiences might still be abnormal. Several previous studies have found differential brain responses between SZ patients and controls, despite similar behavioral performance on learning tasks (Murray et al, 2007; Zedkova et al, 2006), as well as similar reports of emotional experience (Takahashi et al, 2004). A disconnect between the reported experience of rewards, and the associated physiology, may partly explain the reduced ability of rewards and punishments to motivate behavior in SZ. In short, patients who do not report a reduced experienced of pleasure might function like those who do, if the physiology underlying the experience of pleasure is abnormal. This view is supported by our finding that ratings of avolition in patients correlate most strongly with neural responses to the juice reinforcer, whereas avolition ratings bore no relationship to self-reports of reinforcer enjoyment.
It is possible that the strong response to juice delivery on standard trials, in components of reward circuits in controls, stemmed from the frequency of ‘catch’ trials in our paradigm (roughly one-fourth of total trials). A much lower proportion of nonstandard events has been used in some paradigms where infrequency was used to enhance the salience of an event (see, eg Zink et al, 2006, 2004). In our study, expectations may have changed across the course of the session, causing MRI responses to the reinforcer on standard and catch trials to become more similar.
Our observation of a significant correlation between timing task performance in patients, and neural responses to standard juice delivery in the left frontal operculum, suggests that it is also possible that group differences in interval timing abilities contributed to group differences in brain responses to TDEs. However, the fact that SZ patients showed largely intact responses to omissions of the reinforcer (negative TDEs) contradicts this, suggesting that patients had developed a relatively normal expectation as to the timing of standard juice deliveries. Thus, poor interval estimation is likely to make only a minor contribution to attenuated physiological responses to prediction errors in SZ patients.
Might the altered neurophysiological response to juice rewards reflect the impact of antipsychotic medications that attenuate dopamine signaling rather than an effect that can be considered to be a consequence of the illness? Previous studies have not produced a clear picture of what the impact of antipsychotic medications on reward processing might be. At least two studies using pharmacological challenge in normal volunteers have found that antipsychotic drugs modulate feedback-related brain activity (Abler et al, 2007; Zirnheld et al, 2004). On the other hand, results from studies of reward processing in patients with SZ are mixed. Juckel et al (2006a, 2006b) found that unmedicated SZ patients and patients on first-generation (typical) antipsychotics showed attenuated activity in ventral striatum, relative to controls. However, treatment with olanzapine appears to produce a relative normalization of reward anticipation BOLD response in the ventral striatum (relative to activity observed in patients treated with typical antipsychotics; Juckel et al, 2006a; Schlagenhauf et al, 2008). Thus, at least in the case of a symbolic reinforcer, some antipsychotic medications may actually enhance brain activity related to reward anticipation in SZ patients.
The design of this study does not allow us to address this question in a straightforward fashion. In the absence of random assignment to drug type and dose, any post hoc analysis of medication effects is confounded. That is, it is not possible to evaluate drug dose and type independently of the illness features and treatment history that led to the choice of that drug and dose. This may be a particular concern in the patient group studied here, where one-third of individuals were receiving clozapine—evidence of treatment resistance. Thus, in principle, our findings of abnormal reward processing physiology may be limited to treated patients. To answer the critical question of whether these abnormalities are intrinsic to the illness would require a very ambitious study design where patients are evaluated off medication and then again on medication, with the need to randomly assign drug type and dose to address this issue comprehensively.
The fact that our findings may only generalize to treated patients does not diminish the clinical importance of the results, because the great majority of patients with SZ take antipsychotic medications. Such patients demonstrate altered reward processing, and the correlations between brain reward signal modulation and negative symptoms suggest that this physiological response is related to core features of the illness—features that are highly treatment resistant and are highly predictive of functional disability.
References
Abi-Dargham A, Rodenhiser J, Printz D, Zea-Ponce Y, Gil R, Kegeles LS et al (2000). Increased baseline occupancy of D2 receptors by dopamine in schizophrenia. Proc Natl Acad Sci USA 97: 8104–8109.
Abler B, Erk S, Walter H (2007). Human reward system activation is modulated by a single dose of olanzapine in healthy subjects in an event-related, double-blind, placebo-controlled fMRI study. Psychopharmacology (Berl) 191: 823–833.
Addington D, Addington J, Maticka-Tyndale E, Joyce J (1992). Reliability and validity of a depression rating scale for schizophrenics. Schizophr Res 6: 201–208.
Andreasen NC (1984). The Scale for the Assessment of Negative Symptoms (SANS). University of Iowa: Iowa City, IA.
Aron AR, Shohamy D, Clark J, Myers C, Gluck MA, Poldrack RA (2004). Human midbrain sensitivity to cognitive feedback and uncertainty during classification learning. J Neurophysiol 92: 1144–1152.
Berns GS, McClure SM, Pagnoni G, Montague PR (2001). Predictability modulates human brain response to reward. J Neurosci 21: 2793–2798.
Breier A, Su TP, Saunders R, Carson RE, Kolachana BS, de Bartolomeis A et al (1997). Schizophrenia is associated with elevated amphetamine-induced synaptic dopamine concentrations: evidence from a novel positron emission tomography method. Proc Natl Acad Sci USA 94: 2569–2574.
Chapman LJ, Chapman JP, Raulin ML (1976). Scales for physical and social anhedonia. J Abnorm Psychol 85: 374–382.
Cohen AS, Minor KS (2008). Emotional experience in patients with schizophrenia revisited: meta-analysis of laboratory studies. Schizophr Bull. [PMID: 18562345; Epub ahead of print: 2008 Jun 17].
Cohen J (1992). A power primer. Psychol Bull 112: 155–159.
Corlett PR, Murray GK, Honey GD, Aitken MR, Shanks DR, Robbins TW et al (2007). Disrupted prediction-error signal in psychosis: evidence for an associative account of delusions. Brain 130: 2387–2400.
Cox RW (1996). AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. Comput Biomed Res 29: 162–173.
Craig AD (2003). Interoception: the sense of the physiological condition of the body. Curr Opin Neurobiol 13: 500–505.
Crespo-Facorro B, Kim J, Andreasen NC, O′Leary DS, Magnotta V (2000). Regional frontal abnormalities in schizophrenia: a quantitative gray matter volume and cortical surface size study. Biol Psychiatry 48: 110–119.
Crespo-Facorro B, Paradiso S, Andreasen NC, O′Leary DS, Watkins GL, Ponto LL et al (2001). Neural mechanisms of anhedonia in schizophrenia: a PET study of response to unpleasant and pleasant odors. JAMA 286: 427–435.
Critchley HD, Wiens S, Rotshtein P, Ohman A, Dolan RJ (2004). Neural systems supporting interoceptive awareness. Nat Neurosci 7: 189–195.
D’Ardenne K, McClure SM, Nystrom LE, Cohen JD (2008). BOLD responses reflecting dopaminergic signals in the human ventral tegmental area. Science 319: 1264–1267.
Davatzikos C, Shen D, Gur RC, Wu X, Liu D, Fan Y et al (2005). Whole-brain morphometric study of schizophrenia revealing a spatially complex set of focal abnormalities. Arch Gen Psychiatry 62: 1218–1227.
Delgado MR, Miller MM, Inati S, Phelps EA (2005). An fMRI study of reward-related probability learning. Neuroimage 24: 862–873.
Delgado MR, Nystrom LE, Fissell C, Noll DC, Fiez JA (2000). Tracking the hemodynamic responses to reward and punishment in the striatum. J Neurophysiol 84: 3072–3077.
First MB, Spitzer RL, Gibbon M, Williams JBW (1997). Structured Clinical Interview for DSM-IV- Axis I Disorders (SCID-I). American Psychiatric Press: Washington, DC.
Gard DE, Kring AM, Gard MG, Horan WP, Green MF (2007). Anhedonia in schizophrenia: distinctions between anticipatory and consummatory pleasure. Schizophr Res 93: 253–260.
Germans MK, Kring AM (2000). Hedonic deficit in anhedonia: support for the role of approach motivation. Pers Indiv Differ 28: 659–672.
Green MF, Kern RS, Braff DL, Mintz J (2000). Neurocognitive deficits and functional outcome in schizophrenia: are we measuring the ‘right stuff’? Schizophr Bull 26: 119–136.
Haber SN, Kim KS, Mailly P, Calzavara R (2006). Reward-related cortical inputs define a large striatal region in primates that interface with associative cortical connections, providing a substrate for incentive-based learning. J Neurosci 26: 8368–8376.
Harrington DL, Boyd LA, Mayer AR, Sheltraw DM, Lee RR, Huang M et al (2004). Neural representation of interval encoding and decision making. Brain Res Cogn Brain Res 21: 193–205.
Hinton SC, Harrington DL, Binder JR, Durgerian S, Rao SM (2004). Neural systems supporting timing and chronometric counting: an FMRI study. Brain Res Cogn Brain Res 21: 183–192.
Jensen J, Willeit M, Zipursky RB, Savina I, Smith AJ, Menon M et al (2008). The formation of abnormal associations in schizophrenia: neural and behavioral evidence. Neuropsychopharmacology 33: 473–479.
Juckel G, Schlagenhauf F, Koslowski M, Filonov D, Wustenberg T, Villringer A et al (2006a). Dysfunction of ventral striatal reward prediction in schizophrenic patients treated with typical, not atypical, neuroleptics. Psychopharmacology (Berl) 187: 222–228.
Juckel G, Schlagenhauf F, Koslowski M, Wustenberg T, Villringer A, Knutson B et al (2006b). Dysfunction of ventral striatal reward prediction in schizophrenia. Neuroimage 29: 409–416.
Kemali D, Maj M, Galderisi S, Monteleone P, Mucci A (1987). Conditional associative learning in drug-free schizophrenic patients. Neuropsychobiology 17: 30–34.
Keri S (2008). Interactive memory systems and category learning in schizophrenia. Neurosci Biobehav Rev 32: 206–218.
Kumari V, Gray JA, Honey GD, Soni W, Bullmore ET, Williams SC et al (2002). Procedural learning in schizophrenia: a functional magnetic resonance imaging investigation. Schizophr Res 57: 97–107.
Lancaster JL, Woldorff MG, Parsons LM, Liotti M, Freitas CS, Rainey L et al (2000). Automated Talairach atlas labels for functional brain mapping. Hum Brain Mapp 10: 120–131.
Laruelle M, Abi-Dargham A, van Dyck CH, Gil R, D’Souza CD, Erdos J et al (1996). Single photon emission computerized tomography imaging of amphetamine-induced dopamine release in drug-free schizophrenic subjects. Proc Natl Acad Sci USA 93: 9235–9240.
Lee BC, Hwang SH, Rison R, Chang GY (1998). Central pathway of taste: clinical and MRI study. Eur Neurol 39: 200–203.
McClure SM, Berns GS, Montague PR (2003). Temporal prediction errors in a passive learning task activate human striatum. Neuron 38: 339–346.
Montague PR, Hyman SE, Cohen JD (2004). Computational roles for dopamine in behavioural control. Nature 431: 760–767.
Murray GK, Corlett PR, Clark L, Pessiglione M, Blackwell AD, Honey G et al (2007). Substantia nigra/ventral tegmental reward prediction error disruption in psychosis. Mol Psychiatry 13: 267–276.
Okubo Y, Suhara T, Suzuki K, Kobayashi K, Inoue O, Terasaki O et al (1997). Decreased prefrontal dopamine D1 receptors in schizophrenia revealed by PET. Nature 385: 634–636.
Oldfield RC (1971). The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 9: 97–113.
Overall JE, Gorman DR (1962). The brief psychiatric rating scale. Psychol Rep 10: 799–812.
Paradiso S, Andreasen NC, Crespo-Facorro B, O’Leary DS, Watkins GL, Boles Ponto LL et al (2003). Emotions in unmedicated patients with schizophrenia during evaluation with positron emission tomography. Am J Psychiatry 160: 1775–1783.
Paulus MP, Feinstein JS, Leland D, Simmons AN (2005). Superior temporal gyrus and insula provide response and outcome-dependent information during assessment and action selection in a decision-making situation. Neuroimage 25: 607–615.
Pfohl B, Blum N, Zimmerman M, Stangl D (1989). Structured Interview for DSM-III-R Personality Disorders (SIDP-R). Department of Psychiatry, University of Iowa: Iowa City, IA.
Plailly J, d’Amato T, Saoud M, Royet JP (2006). Left temporo-limbic and orbital dysfunction in schizophrenia during odor familiarity and hedonicity judgments. Neuroimage 29: 302–313.
Prentice KJ, Gold JM, Buchanan RW. (2008). The Wisconsin Card Sorting impairment in schizophrenia is evident in the first four trials. Schizophr Res 106: 81–87.
Randolph C, Tierney MC, Mohr E, Chase TN (1998). The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS): preliminary clinical validity. J Clin Exp Neuropsychol 20: 310–319.
Reiss JP, Campbell DW, Leslie WD, Paulus MP, Ryner LN, Polimeni JO et al (2006). Deficit in schizophrenia to recruit the striatum in implicit learning: a functional magnetic resonance imaging investigation. Schizophr Res 87: 127–137.
Schlagenhauf F, Juckel G, Koslowski M, Kahnt T, Knutson B, Dembler T et al (2008). Reward system activation in schizophrenic patients switched from typical neuroleptics to olanzapine. Psychopharmacology (Berl) 196: 673–684.
Schultz W, Dayan P, Montague PR (1997). A neural substrate of prediction and reward. Science 275: 1593–1599.
Schwartz BL, Howard DV, Howard Jr JH, Hovaguimian A, Deutsch SI (2003). Implicit learning of visuospatial sequences in schizophrenia. Neuropsychology 17: 517–533.
Seymour B, O’Doherty JP, Dayan P, Koltzenburg M, Jones AK, Dolan RJ et al (2004). Temporal difference models describe higher-order learning in humans. Nature 429: 664–667.
Small DM, Zald DH, Jones-Gotman M, Zatorre RJ, Pardo JV, Frey S et al (1999). Human cortical gustatory areas: a review of functional neuroimaging data. Neuroreport 10: 7–14.
Takahashi H, Koeda M, Oda K, Matsuda T, Matsushima E, Matsuura M et al (2004). An fMRI study of differential neural response to affective pictures in schizophrenia. Neuroimage 22: 1247–1254.
Tamminga CA, Buchanan RW, Gold JM (1998). The role of negative symptoms and cognitive dysfunction in schizophrenia outcome. Int Clin Psychopharmacol 13 (Suppl 3): S21–S26.
Voorn P, Vanderschuren LJ, Groenewegen HJ, Robbins TW, Pennartz CM (2004). Putting a spin on the dorsal–ventral divide of the striatum. Trends Neurosci 27: 468–474.
Waltz JA, Frank MJ, Robinson BM, Gold JM (2007). Selective reinforcement learning deficits in schizophrenia support predictions from computational models of striatal-cortical dysfunction. Biol Psychiatry 62: 756–764.
Waltz JA, Gold JM (2007). Probabilistic reversal learning impairments in schizophrenia: further evidence of orbitofrontal dysfunction. Schizophr Res 93: 296–303.
Wechsler D (1999). Wechsler Abbreviated Scale of Intelligence (WASI). The Psychological Corporation: San Antonio, TX.
Wechsler D (2001). Wechsler Test of Adult Reading (WTAR). The Psychological Corporation: San Antonio, TX.
Wilk CM, Gold JM, McMahon RP, Humber K, Iannone VN, Buchanan RW (2005). No, it is not possible to be schizophrenic yet neuropsychologically normal. Neuropsychology 19: 778–786.
Zedkova L, Woodward ND, Harding I, Tibbo PG, Purdon SE (2006). Procedural learning in schizophrenia investigated with functional magnetic resonance imaging. Schizophr Res 88: 198–207.
Zink CF, Pagnoni G, Chappelow J, Martin-Skurski M, Berns GS (2006). Human striatal activation reflects degree of stimulus saliency. Neuroimage 29: 977–983.
Zink CF, Pagnoni G, Martin-Skurski ME, Chappelow JC, Berns GS (2004). Human striatal responses to monetary reward depend on saliency. Neuron 42: 509–517.
Zirnheld PJ, Carroll CA, Kieffaber PD, O′Donnell BF, Shekhar A, Hetrick WP (2004). Haloperidol impairs learning and error-related negativity in humans. J Cogn Neurosci 16: 1098–1112.
Acknowledgements
This research was made possible by Grant no. 1 K12 RR023250-01, no. P30 MH068580-01, no. 1 R24 MH72647-01A1 from the National Institutes of Mental Health, and the Intramural Research Program of the National Institute on Drug Abuse. Sharon August and Kimberly Warren assisted with screening and neuropsychological assessments.
Author information
Authors and Affiliations
Corresponding author
Additional information
These data were presented, in part, at the 13th annual meeting of the Organization for Human Brain Mapping, in Chicago, IL, and at the 37th annual meeting of the Society for Neuroscience, in San Diego, California.
DISCLOSURE/CONFLICT OF INTEREST
Dr Gold receives royalties from sales of the Brief Assessment of Cognition in Schizophrenia (BACS) battery and has served as a consultant or an advisory board member for Pfizer, AstraZeneca, and GlaxoSmithKline. All other authors declare that, except for income received from their primary employers, no financial support or compensation has been received from any individual or corporate entity over the past 3 years for research or professional service, and there are no personal financial holdings that could be perceived as constituting a potential conflict of interest.
Supplementary Information accompanies the paper on the Neuropsychopharmacology website (http://www.nature.com/npp)
Supplementary information
Rights and permissions
About this article
Cite this article
Waltz, J., Schweitzer, J., Gold, J. et al. Patients with Schizophrenia have a Reduced Neural Response to Both Unpredictable and Predictable Primary Reinforcers. Neuropsychopharmacol 34, 1567–1577 (2009). https://doi.org/10.1038/npp.2008.214
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/npp.2008.214
Keywords
This article is cited by
-
Schizophrenia: from neurochemistry to circuits, symptoms and treatments
Nature Reviews Neurology (2024)
-
Reduced learning bias towards the reward context in medication-naive first-episode schizophrenia patients
BMC Psychiatry (2022)
-
The prediction-error hypothesis of schizophrenia: new data point to circuit-specific changes in dopamine activity
Neuropsychopharmacology (2022)
-
Learning and Motivation for Rewards in Schizophrenia: Implications for Behavioral Rehabilitation
Current Behavioral Neuroscience Reports (2020)
-
Neural basis of self-initiative in relation to apathy in a student sample
Scientific Reports (2017)