Reward history impacts attentional orienting and inhibitory control on untrained tasks

Abstract

It has been robustly shown that stimuli with reward history receive attentional priority. However, the majority of this research tests reward history effects on attentional bias using similar tasks for both the reward learning phase and the unrewarded testing phase, which limits our understanding of how the effects of reward history generalize beyond the trained tasks and mental sets. Across two new experiments, the current study addresses these issues by first associating reward with a stimulus in a visual search paradigm, and then testing value-driven effects of that stimulus in untrained and unrewarded tasks, including a cueing paradigm, a go/no-go task, and a delay discounting task. Results of Experiment 1 demonstrate that history of reward association in a visual search task generalizes to value-driven attentional bias in a different attention paradigm (i.e., cueing), indicating these effects are indeed attributable to imbued value that can transfer to other tasks beyond that in which the reward was trained. The results of Experiment 2 demonstrate that in addition to eliciting attentional orienting on untrained tasks, reward history can lead to better inhibitory control in the go/no-go task. We find no evidence for reward history effects in the delay discounting task. Together, these experiments demonstrate that when the reward association task is in the attention domain, reward history modulates attentional priority, and this effect generalizes to untrained and unrewarded tasks that utilize both spatial and nonspatial attention.

Reward is crucial for shaping cognition and behavior. Rewards influence what we approach, choose, crave, and pay attention to (Berridge & Robinson, 2016; Guitart-Masip, Talmi, & Dolan, 2010; Krieglmeyer, Deutsch, De Houwer, & De Raedt, 2010; Pessoa & Engelmann, 2010). It has been well established that enhanced visual processing and attentional engagement are exhibited for stimuli predicting reward receipt (Bourgeois, Chelazzi, & Vuilleumier, 2016; Della Libera & Chelazzi, 2009; Pessoa & Engelmann, 2010; Serences, 2008; van den Berg, Krebs, Lorist, & Woldorff, 2014). More recent work suggests that after repeated association with reward, a stimulus can be imbued with value such that it receives attentional priority even after the prospect of reward is no longer available, a phenomenon termed value-driven attentional priority (Anderson, Laurent, & Yantis, 2011b; Anderson & Yantis, 2013; Failing & Theeuwes, 2014, 2018). These effects of reward history on attentional priority cannot be explained by salience-driven or goal-driven attention alone (Anderson et al., 2011b; Le Pelley, Pearson, Griffiths, & Beesley, 2015). Instead, it has been suggested that an alternative mechanism must be considered through which history influences attentional priority (Awh, Belopolsky, & Theeuwes, 2012; Theeuwes, 2019). Whether selection history (i.e., previous experience attending to a stimulus) and reward history influence attentional priority through the same mechanism is still the subject of debate (Anderson & Britton, 2019; Anderson, Chiu, DiBartolo, & Leal, 2017; Kim & Anderson, 2019a, 2019b).

Value-driven attentional priority has been the focus of much research in part because of its potential translational relevance; however, the critical issue of the generalizability of the reward associations across tasks is left largely untested. Testing generalizability is an important step both in characterizing the specific impact on cognition and in establishing its translational applicability. For example, attentional bias to drug cues, which has been theorized to be a specific instance of this value-driven attentional priority (Anderson, 2016), is exhibited widely across an array of tasks (Field & Cox, 2008) and is thought to lead to automatic approach behaviors (Berridge & Robinson, 2016). However, the generalizability of value-driven attentional priority requires further investigation, and this study aims to address this critical gap. First, because the unrewarded test phases often match the task goals of the reward-association training phases, it cannot be differentiated whether findings of value-driven attentional priority are specific to the attentional sets adopted during reward training or can be generalized to novel, untrained tasks. This will be addressed across Experiments 1 and 2 by implementing an unrewarded test phase that differs in task goals and target stimuli from the earlier reward-association training phase. Second, it is not yet determined the manner in which reward history manipulated in a visual search task might affect motor inhibition or impulsive choice. As such, Experiment 2 serves to test how patterns of behavior are altered by reward history in go/no-go and delay discounting tasks.

Experiment 1

Introduction

The majority of studies that have examined value-driven effects on attention have used similar stimuli, goals, and paradigms in both reward association training and later testing (e.g., Anderson et al., 2011b; Failing & Theeuwes, 2014). Importantly, it is possible that the impact of history of reward association on attention is moderated by top-down attentional sets. Top-down attentional sets are developed for particular task goals, and these attentional sets moderate the attentional priority given to stimuli (Folk, Remington, & Johnston, 1992; Folk, Remington, & Wright, 1994). When the task goals are identical between reward association training and testing, it is unclear whether (a) increased attentional priority for stimuli with reward history is specific to the type of attentional set adopted from the reward association training or (b) attentional priority to the stimulus with a reward history is driven by the persistence of value association that would transfer to dissimilar tasks. As such, it is possible that value-driven attentional priority is displayed most robustly in tasks that share similar stimuli, task goals, and targets in the attentional bias testing, as was used in the reward association training.

There are relatively few studies that have tested whether reward association training in one task leads to attentional bias on tasks with unrelated task goals. One study addressing this created a reward association to a color by pairing money with a red or green circle in a visual search task (Anderson, Laurent, & Yantis, 2012). In a later flanker task with no monetary reward, participants performed worse when the distractors on the flanker task appeared in the previously high-value color. Other work that has implemented a similar visual search reward association paradigm has subsequently tested effects of reward history on visual memory. Results from these studies demonstrate that performance on visual memory tasks, which require participants to identify the orientation of a line in a visual search array after some delay, is enhanced when the target has a reward history (Gong & Li, 2014) and interferes with visual memory of adjacent stimuli (Infanti, Hickey, & Turatto, 2015). These studies are consistent with an increased attentional priority to stimuli with a reward association history that subsequently leads to an increased priority in visual memory. Other work has examined how effects of reward history on attention might generalize to untrained stimuli or scenes. For example, some have investigated how reward history affects category representation, and found that when reward is conferred for an object in a given category, attentional priority is given to other objects in that category that have not been associated with reward (Failing & Theeuwes, 2015; Hickey, Kaiser, & Peelen, 2015). Another study found that conferring reward for eye movements on the left side of one scene led to increased eye movements to the left side in an untrained scene, exhibiting some generalization of spatial attentional selection (Bourgeois, Badier, Baron, Carruzzo, & Vuilleumier, 2018).

These studies provide preliminary evidence that attentional bias to stimuli with a history of reward association can be exhibited across some tasks with disparate goals from the task employed for reward association. However, these studies cannot speak to what form this enhanced attentional priority takes. For example, while these findings demonstrate that the historical value of a stimulus will receive higher attentional priority, neither study could address whether attention is oriented to the stimulus with a reward history. This is because those studies examined attentional costs of distractors (with a reward-associated history) without examining attentional benefits (the enhancement of processing at the specific location of a previously rewarded stimuli). In this regard, spatial cueing paradigms are particularly helpful in examining attentional costs and benefits following the orienting of attention in space (Posner & Cohen, 1984; Yantis & Jonides, 1990). The current study aims to first train reward association to a stimulus in a visual search paradigm (adapted from Anderson et al., 2011b) and subsequently test value-driven attentional bias to that stimulus in a spatial cueing paradigm that allows both costs and benefits of attention to be measured (adapted from Failing & Theeuwes, 2014). The reward association training phase and the subsequent unrewarded testing phase have different target stimuli and different goals, which minimizes the degree that participants might adopt similar attentional sets between the training and testing tasks.

In addition to investigating the generalizability of attentional costs and benefits across paradigms, this study investigates the potential relationship between value-driven attentional bias and impulsivity. Impulsivity is a neurocognitive endophenotype, characterized as a tendency to behave in a risky, premature manner that can lead to negative consequences, and it is associated with various disorders, such as some types of attention-deficit/hyperactivity disorder, substance use, and eating disorders (Daruna & Barnes, 1993; Robbins, Gillan, Smith, de Wit, & Ersche, 2012). Given that impulsivity involves both elements of biasing behavior toward proximal rewards and reduced cognitive control (Robbins et al., 2012), it is plausible that value-driven attentional bias, which is characterized by a failure to engage attentional control to overcome a reward-driven bias, might be related to impulsivity. Findings thus far, however, have been largely mixed. Value-driven attentional bias to reward has been found to be exaggerated in populations shown to be higher on impulsivity, such as adolescents and individuals with a history of substance use (Anderson, Faulkner, Rilee, Yantis, & Marvel, 2013; Roper, Vecera, & Vaidya, 2014), and some experiments have found relationships between impulsivity and attentional bias to reward (Anderson, Kronemer, Rilee, Sacktor, & Marvel, 2016; Anderson et al., 2011b, Experiment 1). However, other studies have found either no relation (Anderson et al., 2011b, Experiment 3; Pearson, Donkin, Tran, Most, & Pelley, 2015, Experiment 2) or even an inverse relationship (Pearson et al., 2015, Experiment 1). In the current study, we examine both whether impulsivity is related to reward biasing during the initial reward association (training) phase and whether impulsivity is related to the value-driven attentional bias in the later attention task (testing). The former tests whether a potential relationship to impulsivity is primarily driven by reward sensitivity involved in the acquisition phase, while the latter tests whether attentional control in the context of value-associated stimuli drives this relationship. If the relationship to reward-driven attentional bias is driven primarily by reward sensitivity, it would be expected to see a relationship to impulsivity in both the reward association phase and testing phase. This is possible, as other studies have highlighted that measures related to reward-driven attentional bias are related to self-report of reward sensitivity (Hickey, Chelazzi, & Theeuwes, 2010; Hickey & Peelen, 2015, 2017), though not always reaching significance (Barbaro, Peelen, & Hickey, 2017). However, because attentional bias to reward and attentional control have both been linked with an aspect of executive functioning, specifically working memory capacity (Anderson et al., 2011b; Fukuda & Vogel, 2011), we anticipate a relationship to impulsivity to be specifically evidenced for behavior in the unrewarded cueing phase.

Overall, this study aims to test how reward learning in one task generalizes to attentional costs and benefits in a novel task with differing goals and target stimuli. Additionally, this study will examine the degree to which value-driven attentional bias is associated with impulsivity over and above reward-learning during the reward association phase.

Method

Participants

Participants were 41 healthy adults (ages 18–23 years, 56% female) with self-reported normal or corrected-to-normal eyesight and no colorblindness, which was confirmed using the Ishihara color vision test (Ishihara, 1973). Participants were recruited via flyers and a university LISTSERV. Monetary compensation was provided based on participant performance (mean = $14.92; range: $8.88–$17.64). All tasks and questionnaires were completed in a 1-hour study visit. First, reward association was manipulated in a reward training paradigm, and subsequently value-driven attentional bias was tested using a cueing paradigm. All tasks were programmed using OpenSesame (Mathôt, Schreij, & Theeuwes, 2012), and stimuli were presented on a 22-inch color monitor with a refresh rate of 60 frames per second. All tasks were completed in a sound-attenuated booth. Participants were seated in a chair such that their eyes were approximately 75 cm from the screen. All procedures were approved by the UNC Chapel Hill Institutional Review Board.

Reward training

Reward was associated with a given colored circle using a training paradigm adapted from Anderson et al. (2011b). Training trials consisted of a fixation screen, followed by a search array, and then a feedback screen on a gray background (see Fig. 1). The white fixation cross (0.5° × 0.5° visual angle) was presented for a random interval of 400, 500, or 600 ms on each trial. The search array consisted of six circles (2.3° × 2.3° visual angle) placed at equal intervals at an imaginary circle with a 5° radius around the fixation cross. Each circle was a different color and contained a white line. For half of the participants, one of the circles was either red or green on every trial, with the red or green circle being the target circle. For the other half of the participants, one of the circles was either blue or yellow on every trial, with the blue or yellow circle as the target circle. Participants were instructed to indicate with a keyboard response whether the white line inside the target circle was vertical or horizontal by pressing the appropriate key with their index finger. Participants had 800 ms to respond. After the search display, a feedback display showed the amount of money earned (if any) and the current total. If no response was given within the time frame, a 500-ms 1,000-Hz feedback tone informed participants that the trial timed out, and the feedback screen displayed “slow” along with the current total money. The color of the rewarded target (green, red, blue, or yellow) was counterbalanced across participants. For the rewarded target, 80% of trials were worth 12 cents and 20% were worth no money. For the unrewarded target, all trials were worth no money. Participants were informed that some trials were worth 12 cents and some trials were worth no money, without specifying any differences between the colors. Participants were informed that they would earn the amount associated with that trial (12 cents or 0 cents) if their responses were both accurate and fast enough. If they responded within the time and accurately, the amount earned was displayed after each trial. Training consisted of 360 trials across three blocks, and it was preceded by a practice block of 50 trials.

Fig. 1

Reward-training paradigm adopted from Anderson & colleagues (2011b). Participants were instructed to respond to the orientation of a line inside the target circle. For the rewarded target circle, 80% of trials were worth money ($0.12 for Study 1; $0.15 for Study 2) and 20% were worth no money. For the neutral circle, all trials were worth no money. The colors of the rewarded and unrewarded target circles were counterbalanced across participants

Reward-contingency awareness

Previous value-driven attentional bias studies have probed whether participants become aware of the reward contingency during the session, with mixed findings regarding whether awareness impacts attentional bias (Anderson, 2015; Bourgeois, Neveu, Bayle, & Vuilleumier, 2017; Sisk, Remington, & Jiang, 2019). As such, a probe of awareness of reward contingencies in the training paradigm was included (adapted from Anderson, 2015; please see the Appendix for the full probe). A binary variable of whether participants picked the one correct reward contingency from the six possible options was used to define aware and unaware groups.

Attention cuing task

The testing paradigm (see Fig. 2) was a cueing paradigm modified from Failing and Theeuwes (2014). Testing trials consisted of a fixation, a double-cue display, and a target display. The fixation was displayed for a randomly determined period of 400–600 ms. The cue display was presented for 200 ms and consisted of two differently colored circles (red, green, yellow, or blue; 3.4° diameter) with figure-eight placeholders inside the circles (2.3° × 1.1°). After 200 ms, an offset of two line segments in each of the figure-eight premasks revealed the target display. The target display consisted of the same circles as the cueing display, but with a letter revealed inside each circle (“S,” “P,” “H,” “E”; 2.3° × 1.1°). One of the two letters was a target letter (“S” or “P”) and the other was a distractor letter (“H” or “E”).

Fig. 2

a Testing paradigm adopted from Failing and Theeuwes (2014). A double cue was displayed for 200 ms with figure-eight premasks, then an offset would reveal target letters for 800 ms. Participants identified whether the letter “S” or the letter “P” was present. Letters “E” and “H” never served as the target letters. b Description of the four nonpredictive cue types if blue were the previously rewarded target color in training, yellow were the previously unrewarded target color in training, and the colors red and green never served as targets during the reward training. (Color figure online)

Participants were instructed to indicate with a keyboard response which target letter was present by pressing the appropriate key with their index finger. The target display remained on the screen for 800 ms or until a response was given. A black fixation cue turned briefly white if participants were either too slow or incorrect on a given trial, and participants were explicitly informed of this. For this phase, participants were explicitly informed that they would not receive money for their performance and that the circles were unrelated to the location of the target letter, emphasizing that the color of the circles was task irrelevant. Participants completed a 48-trial practice block prior to both training and testing. Testing consisted of 496 trials across five blocks. Trials were coded as one of four types: (1) neutral, in which neither color in the cue display was a target color in the training phase; (2) both, in which both colors in the cue display were target colors (one rewarded, the other unrewarded) in the training phase; (3) previously rewarded (PR), in which the cue display contained one neutral color circle and a circle that was the color of the previously rewarded target in the training phase; and (4) previously unrewarded (PU), in which the cue display contained one neutral color circle and a circle that was the color of the previously unrewarded target in the training phase (see Fig. 2).

Although targets were always equally likely to occur within either colored circle, we use the terms “valid” and “invalid” to describe the location of the target relative to the cue colors in the following way: on PR and PU trials, valid refers to trials in which the target occurred at the location of the color circle that had been a target color in the reward-training session, whereas invalid refers to trials in which the target occurred at the location of the neutral color circle. On “both” trials “valid” refers to trials in which the target appeared in the PR-color circle and “invalid” refers to trials in which the target appeared in the PU-color circle. There is no valid or invalid specified for neutral trials because both colors in those displays were exactly the same in terms of history and lack of reward.

Impulsivity measure

Participants completed the Barratt Impulsiveness Scale (BIS-11), a 30-item self-report that is one of the most widely used scales for impulsivity (Patton, Stanford, & Barratt, 1995). The BIS-11 has significant test–retest reliability (Stanford et al., 2009) and showed good internal consistency in this sample (α = .87).

Analyses

Attentional bias effects were measured using reaction time (RT). Any RTs below 100 ms were excluded from further analyses, as these responses are likely too fast to reflect accurate target discrimination. After exclusion for RT’s below 100 ms, more than 99% of the data remained. Only correct responses were used to calculate reaction time across both the reward training and attention cueing tasks. For any instances in which Mauchly’s test indicates that the assumption of sphericity should be rejected for the data when implementing an analysis of variance (ANOVA), Greenhouse–Geisser corrected degrees of freedom and probability values are reported.

To test for a linear effect of orienting in the cuing task, an ANOVA was conducted on RT using a within-subjects factor of trial type (invalid PR, neutral, valid PR). Follow-up ANOVAs were used to test paired differences for overall attentional bias (invalid PR minus valid PR), attentional benefit/capture (neutral minus valid PR), and attention cost/disengagement (invalid PR minus neutral). While the PR trial types are of primary interest, similar follow-up analyses will be conducted for the trial types of previously unrewarded (PU) and both PR and PU present. These follow-up analyses will determine whether a similar bias can be elicited from training alone (PU) and whether reward-driven bias is exhibited above and beyond training alone when both previously trained colors are present (“both” trials).

We hypothesized that value-driven attentional bias would be linked with individual differences in impulsivity. This was tested using a linear regression predicting self-reported BIS-11 scores from an attentional bias score (invalid − valid RT). The regression was conducted for PR trials, PU trials, and “both” trials. Examining this relationship across trial types allows for determination whether the relationship to impulsivity is reward specific or due to training. Additionally, to investigate if the relationship could be explained only by reward-biasing during the reward association training, a linear regression was conducted predicting self-reported BIS-11 scores from a reward-bias score in training (RT for trials with an unrewarded circle color as a target RT for trials with a rewarded circle color as a target).

Results

Reward training task

A paired t test was conducted on RTs of correct trials to test for a reward-driven training effect with reward (rewarded, unrewarded) as a within-subjects factor. As expected, results revealed participants were significantly faster for the rewarded stimulus (M = 562 ms, SD = 41 ms) relative to the unrewarded stimulus (M = 575 ms, SD = 40 ms), t(40) = 3.11, p < .01. Accuracy was good across conditions, with a trend toward better accuracy for the rewarded (M = 83.30 %, SD = 9.50 %) than for the unrewarded (M = 81.51%, SD = 9.25 %) stimuli, t(40) = 1.69, p = .10.

Attention cueing task

Attentional bias was measured in the cueing task, during which participants no longer received money for performance, and none of the colored circles were predictive of the target (letter “S” or “P”) identity or location. The trial types in the attention cuing paradigm were: neutral, in which both circles were untrained colors; previously rewarded (PR), in which one previously rewarded color was paired with one neutral color; previously unrewarded (PU), in which one previously unrewarded color was paired with one neutral color; and both, in which both circles were trained colors (one previously rewarded and one previously unrewarded). For all trial types, the target was equally likely to occur at either location (no colors were predictive of target location). Although there was no predictability between cue and target location, to be consistent with previous research, Valid trials will refer to when the target occurred at the location of the previously trained color, and Invalid trials refers to when the target occurred at the location opposite from the previously trained color. It was hypothesized that a linear orienting pattern would be exhibited, consistent with value-driven attentional bias, such that participants would be fastest on PR valid trials, slower on neutral trials, and slowest for PR invalid trials. These conditions also allow us to examine whether both capture (valid vs. neutral) and disengagement (invalid vs. neutral) of value-driven attention would be observed here. Accuracy was uniformly high across all conditions, and there was no evidence of a speed–accuracy trade-off in any of the analyses presented here. Accuracy data are presented in Supplemental Table S1.

Since the linear orienting effect was of primary interest, and we sought to replicate results from previous work that implemented the same cueing paradigm and showed evidence of both facilitated capture and impaired disengagement (Failing & Theeuwes, 2014), we first implemented a similar analytical approach to that of prior work, which analyzed each of these three trial types (PR, PU, both) separately. Note that a linear orienting effect cannot be examined with the inclusion of all three trial types into one model, because all are being compared with the same neutral trial type. For the PR trial types, a linear orienting effect was examined by implementing a repeated-measures ANOVA with the within-subjects factor of validity (PR valid, neutral, PR invalid). Results indicated a main effect of validity, F(2, 80) = 7.69, p < .01, η² = .16, which was linear in nature, F(1, 40) = 12.02, p < .01, η² = .23. Follow-up paired t tests were conducted to test the specific effects of facilitated capture (i.e., relatively faster responses for PR valid relative to neutral trials) and impaired disengagement (i.e., relatively slower responses for PR invalid relative to neutral trials) to the PR cues. Results demonstrated that participants were faster to PR valid trials (M = 503 ms, SD = 40 ms) relative to neutral trials (M = 509 ms, SD = 37 ms), t(40) = 2.89, p < .01, which is consistent with facilitated capture to the PR cue. The difference between PR invalid (M = 512 ms, SD = 41 ms) and neutral trials, indexing disengagement, was in the expected direction, but did not reach significance, p = .14.

To investigate whether training alone, without any reward history, produced attentional bias, an analysis was conducted on the PU trial type. It was expected that there would be no differences between PU valid, neutral, and PU invalid trials. A repeated-measures ANOVA indicated a trending effect of validity, F(2, 80) = 3.23, p = .05, η² = .08, which was linear in nature, F(1, 40) = 5.17, p = .03, η² = .12. Follow-up paired t tests did not provide evidence for significant capture, comparing PU valid trials (M = 505 ms, SD = 38 ms) to neutral trials (M = 509 ms, SD = 37 ms), p = .12, nor for significant disengagement, comparing PU invalid trials (M = 511 ms, SD = 39 ms) to the neutral trials, p = .35.

For the both-present trial type, it was expected that participants would bias attention to the PR cue when both the PR cue and PU cue were present, as had been shown in previous work with this cueing paradigm (Failing & Theeuwes, 2014). A similar repeated-measures ANOVA was conducted that tested a linear orienting effect within the both-present trials; for this “both” condition, valid is defined as when the target occurred at the PR location, and invalid refers to when the target occurred at the PU location. Results did not demonstrate an effect of validity, p = .61, indicating no difference between the neutral, Valid (M = 507 ms, SD = 41 ms) or invalid (M = 506 ms, SD = 42 ms) for the both-present trials.

Additional analyses of cuing task: Attentional bias differences across trial types

While the linear orienting effect (across valid, neutral, and invalid conditions) within each trial type was our primary interest, additionally, a test of whether the total attentional bias score (i.e., invalid RT − valid RT) differed by trial type was conducted. This test does not include the neutral trial type, and therefore it does not allow for an investigation of facilitated capture and impaired disengagement effects; however, it does allow for an examination of differences between the total attentional bias score across trial types. Of primary interest was examining differences in total attention bias between the PR trial type and the PU trial type. As such, a 2 × 2 repeated-measures ANOVA was conducted on RT with the within-subjects factors of trial type (PR trials, PU trials) and validity (valid, invalid). Results indicated no overall difference in RT between PR and PU trials, p = .92. There was a significant effect of validity, such that valid trials were faster than invalid trials, F(1, 40) = 16.94, p < .01, η² = .30, but the Validity × Trial Type interaction was not significant, p = .35. This was unexpected, because it was anticipated that reward history would have a greater effect on performance than training history alone, based on previous studies. We return to this finding below, in the post hoc analyses section.

Relation between reward-training and attention cuing

Previous work has demonstrated that individual differences in reward-biasing during training predicted value-directed attentional bias in testing (Failing & Theeuwes, 2014). In the current study, the paradigms used in training and testing differed in both task goals and stimulus configurations. Our results described above demonstrate that reward learning generalizes to value-driven attentional bias in testing; however, here we tested whether the degree of learning in training can predict the degree of attentional bias in testing. We tested this relationship using a correlational analysis, as has been used previously (Failing & Theeuwes, 2014). Training bias was calculated as the difference between RT for trials with the rewarded circle and RT for trials with the unrewarded circle (i.e., unrewarded trials RT − rewarded trials RT) in the training phase. Value-driven attentional bias in testing was calculated using the total attentional bias score (i.e., invalid RT − valid RT) separately for PR trials, PU trials, and “both” (both-trained present) trials. This analysis revealed a significant relationship between reward bias in training and value bias in attention for PR trials, r(39) = .45, p < .01, as well as a significant relationship between training and attention bias for Both present trials, r(39) = .32, p = .04. There was no significant relationship between training and attention bias on the PU trials, r(39) = .13, p = .43.

Relations with impulsivity

Regression analyses were conducted predicting impulsivity score (on the BIS-11) from total attentional bias (i.e., invalid RT − valid RT). Separate regressions were conducted using performance on (1) PR trials, (2) PU trials, and (3) both-present trials. Attentional bias on PR trials and on PU trials did not significantly explain variance in impulsivity, ps > .1. Attentional bias (to the PR cue) on both-present trials explained 19.4% of the variance in impulsivity, F(2, 38) = 5.82, p < .01, standardized β = .48. Regarding training, reward bias in the training phase did not explain variance in impulsivity, p = .73.

Post hoc analyses: Awareness

In using the current data to motivate the design of the second experiment, it is noteworthy that, overall, these results only partially replicated previous literature. Some previous work has found total attentional bias effects exclusively for stimuli with a reward-associated history, without evidence for total attentional bias effects to stimuli with similar training histories but no reward history (Anderson et al., 2011b; Failing & Theeuwes, 2014). Other work, however, has demonstrated attentional bias to previously trained stimuli without a history of reward; it should be noted that these paradigms have typically involved multiple sessions of training across many days (Kim & Anderson, 2019a; Kyllingsbaek, Schneider, & Bundesen, 2001; Shiffrin & Schneider, 1977). Given the somewhat inconsistent results, we further investigated the current data set to better understand what might be driving some of these effects.

One element that may influence results is whether participants were aware of the reward and stimulus pairings. Although we did not attempt to manipulate awareness between subjects, our postexperiment, self-report survey revealed a substantial number of subjects in each group of aware versus unaware. Based on responses to the awareness question, 17 participants were classified as aware of the reward contingencies during training (i.e., they identified the accurate reward contingency in the awareness probe) and 24 participants were classified as unaware of reward contingencies during training. This was a higher proportion of aware individuals than had been exhibited in studies that found no effect of awareness (Anderson, 2015; Anderson et al., 2013) and more similar to proportions reported in studies that found effects of awareness on attentional bias (Bourgeois et al., 2017; Sisk et al., 2019) . Because awareness was not a manipulated variable of interest, our data were not powered to investigate a three-way within-subjects and between-subjects interaction between validity, trial type, and awareness. However, the variability in awareness is one potential explanation for weaker effects in our data. In an effort to explore this possibility and better inform our design of Experiment 2, we elected to probe our data further, with the inclusion of awareness as a moderating variable.

Awareness and reward training task

To better characterize how our data may be interacting with awareness, given the relatively large portion of individuals aware of the reward association, we probed whether awareness had an effect during the reward association training task. A mixed-measures ANOVA was conducted to test for an Awareness × Reward interaction during this training task, with reward (rewarded, unrewarded) as a within-subjects factor and awareness (aware, unaware) as a post hoc between-subjects factor. In this analysis, there was again a significant within-subjects effect of reward on RT, F(1, 39) = 12.95, p < .01, η² = .28, which was now qualified by a significant Reward × Awareness interaction, F(1, 39) = 9.02, p < .01, η² = .219. Follow-up analyses of each awareness group separately revealed that aware participants were faster on rewarded compared with unrewarded trials, F(1, 16) = 14.24, p < .01, η² = .47, but there was no effect of reward in the unaware group, p = .43. There was no significant between-subjects main effect of awareness on RT, p = .71. Accuracy was also analyzed, in a similar mixed-measures ANOVA with the within-subjects factor of reward status and between-subjects factor of awareness status. Results indicated a trend for performance being more accurate for rewarded relative to unrewarded stimuli, F(1, 39) = 4.21, p = .05, and a trending Reward × Awareness interaction, F(1, 39) = 3.71, p = .06, which suggested any effect may be larger in the aware group (MRewarded = 0.86, SDRewarded = 0.06; MUnrewarded = 0.82, SDUnrewarded = 0.09) than the unaware group (MRewarded = 0.83, SDRewarded = 0.10; MUnrewarded = 0.81, SDUnrewarded = 0.09). There was no main effect of awareness status on overall accuracy, p = .46. Finally, to check whether aware and unaware participants differed on earning metrics during the reward training paradigm, an independent-samples t test was conducted, which confirmed that aware and unaware participants showed no significant differences in amount of money earned, p = .16.

Awareness and attention cueing task

To test for any effects of awareness during the cueing paradigm, a between-subjects factor of awareness was added to the original models. For the PR trials, a mixed-measures ANOVA with a within-subjects factor of validity (PR invalid, neutral, PR valid) and between-subjects factor of awareness confirmed an effect of validity, F(1.74, 67.93) = 9.74, p < .01, η² = .20, which was linear, as hypothesized, F(1, 39) = 15.76, p < .01, η² = .29. This effect was qualified by a validity by awareness interaction, F(1.74, 67.93) = 3.35, p = .04, η² = .08 (see Fig. 3a). There was no significant main effect of awareness on overall RT, p = .78. Due to the awareness by validity interaction, follow-up analyses were conducted separately for aware and unaware participants on performance for PR trials (Failing & Theeuwes, 2014). For aware individuals, this revealed a significant attentional capture effect (PR valid relative to neutral), F(1, 16) = 22.40, p < .01, η² = .58, but no significant disengagement effect (PR invalid relative to neutral), p = .17. Within the Unaware group, there was no evidence of attention capture, disengagement, or overall validity effects, all ps > .10. See Table 1 for RT’s across conditions.

Fig. 3

Performance on cueing task in study 1 for (a) PR trials and (b) PU trials. A significant orienting effect was found for PR trials in the aware group, and specifically characterized by a significant reward-driven capture effect. Error bars are within subjects 95% confidence intervals (Cousineau, 2005). * indicates significance at p < .05

Table 1 Reaction times in milliseconds for the cueing paradigm, separated by aware and unaware participants. Mean (SD)

Performance on PU trials was also examined. For PU trials, there was a trending effect of validity, F(2, 80) = 3.20, p = .05, η² = .08 that did not interact with awareness, p = .61 (see Fig. 3b; Table 1). There was also no main effect of awareness status on overall RT, p = .73. Because there was no interaction with awareness, trial-type effects were not statistically examined separately for Aware and Unaware subjects on PU trials.

Performance on “Both” trials (trials with both PR and PU colors present as cues) was also investigated to test whether this value bias was exhibited between previously trained cues. A similar mixed-measures ANOVA with factors of trial type and awareness was conducted for “both” trials. There was no evidence for an effect of validity (i.e., no difference between performance on valid, neutral, or invalid trials), p = .60, nor interaction with awareness, p = .76. There was also no significant difference in overall reaction time between aware and unaware subject, p = .78.

Discussion

This study was designed to test how value-driven attentional bias resulting from reward association history in a visual search task can generalize to attentional orienting in a cueing task, which has no history of reward and no prospect of reward receipt. Our results suggest that attentional bias driven by history of reward association is exhibited in a novel task context, as demonstrated by facilitated processing in locations cued by a stimulus with a reward association history. Our findings provide evidence that attentional bias can be observed even when training and testing differ in task goals and target stimuli, extending previous research that used testing paradigms that were similar or identical to the paradigms used to train the reward contingencies (Anderson, Laurent, & Yantis, 2011a; Failing & Theeuwes, 2014). Our results provide preliminary evidence that the attentional orienting exhibited to the previously rewarded stimulus cannot be explained solely by attentional sets adopted to complete a particular task. We additionally, however, find some evidence that the effect of reward history on attentional orienting in this cueing task may not differ from that of training history alone, which we aim to further investigate in the second experiment.

There remain multiple possibilities for the mechanism through which this value-driven attentional bias leads to attentional orienting in a novel task. One possibility is that pairing attentional selection of a stimulus with reward in training leads to value-driven changes in attentional priority (Failing & Theeuwes, 2018) that are exhibited even in untrained contexts. In this case, attentional priority of valued stimuli would be expected to be exhibited across any type of attentional selection, including spatial and nonspatial attention. Another possibility is that value-driven attentional bias is exhibited because history of reward association leads to automatic approach more broadly (Berridge & Robinson, 2016). If this were true, it would be expected that automatic approach to the valued stimulus would be exhibited across not just the orienting of spatial attention but other cognitive processes as well, such as motor control and decision-making. We investigate these possibilities in Experiment 2.

Additionally, Experiment 1 investigated attentional costs and benefits within value-driven attentional bias. We used a cueing task during the testing phase that has previously been used to examine reward-driven attentional bias in studies where the training and testing tasks were identical (Failing & Theeuwes, 2014). Here, we replicated the findings of attentional benefits and overall attention biases, and showed that value-driven attentional biasing does generalize to other tasks. However, our experiment did not replicate the previous finding that attentional costs are demonstrated when the previously rewarded cue is in the opposite location as the target (Failing & Theeuwes, 2014). Although we found a pattern of results in that direction, it was not significant in our experiment. It has been suggested that within value-driven attentional bias, attentional benefits may be stronger and more likely to be detected than attentional costs (Pool, Brosch, Delplanque, & Sander, 2016). Indeed, our findings reveal benefits but not costs. This may indicate that when training and testing contexts are identically matched, as with the previous research (Failing & Theeuwes, 2014), value-driven attentional bias effects are amplified relative to when a novel top-down attentional set is adopted due to differing task stimuli and goals.

In addition to the findings that reward history leads to attentional benefits, we find trending evidence that an overall attentional bias may be exhibited to previously trained but never rewarded stimuli. We do not, however, find evidence for attentional costs or benefits to that stimulus. This was a somewhat surprising finding, given that previous work has highlighted that attentional bias to stimulus with a reward history is due to the reward history alone without evidence for attentional bias to stimuli with similar training histories without a reward association (Anderson et al., 2011b; Failing & Theeuwes, 2014). While some work denotes that attentional bias to previously trained but unrewarded stimuli can be exhibited, typically these effects are detected after multiple sessions of training across many days (Kim & Anderson, 2019a; Kyllingsbaek et al., 2001; Shiffrin & Schneider, 1977). As such, while our first experiment provides some initial evidence that reward history can lead to attentional orienting in untrained tasks, we hope to replicate our results in a second experiment with increased power to detect effects.

One possibility that may have impacted our results is variation across individuals in whether they became incidentally aware of the reward contingencies (i.e., could identify which circle was worth money in the reward training phase). We examined reward-driven attentional bias separately in aware and unaware individuals as a post hoc analysis to better understand this possibility. While we cannot draw strong conclusions due to the post hoc nature of the analysis, we find a significant effect of reward history on attentional bias in Aware individuals though this effect does not reach significance in Unaware individuals, despite the latter group being larger. Most attentional bias to reward research has not reported interactions with awareness; in the few studies that have reported on awareness, participants were unaware of reward contingencies or there was not an effect of awareness on attentional bias to reward (Anderson, 2015; Anderson et al., 2013). It should be noted, though, that in these studies, relatively few (less than 20%) of participants can be classified as aware (Anderson, 2015; Anderson et al., 2013), and so they may be underpowered to detect moderating effects of awareness. In the current experiment, more than 40% of participants were aware of reward contingencies, and all reward-related findings in training and testing interacted with awareness. Other previous studies that had similar proportions of aware participants as ours (37%–55%) found that value-driven attentional bias was stronger in participants who were aware of the reward contingencies (Bourgeois et al., 2017; Sisk et al., 2019). While who becomes aware is an interesting individual difference that should be investigated in future research, the current study aims to characterize generalization of value-driven attentional bias and is not equipped to differentiate when in the study participants became aware of reward contingencies. To ensure awareness impacts all participants equally, and consequently increase power to detect more subtle effects of value-driven attentional bias (e.g., attentional costs), Experiment 2 will ensure all participants are explicitly aware of reward contingencies at the onset.

Finally, the current study aimed to investigate the link between value-driven attentional bias and impulsivity. Impulsivity has been linked with performance on visual tasks (Dickman & Meyer, 1988), as well as with reward sensitivity (Martin & Potts, 2004), and it is theorized that impulsivity reflects at least in part an inability to inhibit prepotent responses (Aichert et al., 2012). Our results provide some support for the idea that impulsive individuals may be more likely to perpetuate value-driven attentional biases even after the prospect of reward is removed. This finding, however, was specific to performance on “both-present” trials in the attention task; impulsivity was not linked to performance on any other kind of trial. Therefore, overall, the current results suggest that any effect of impulsivity on value-driven attentional biasing may not be a robust effect, or may be limited to only contexts in which the effects of previous exposure are equated. However, it is also possible that we did not observe associations between our task and self-reported impulsivity because our measure of impulsivity, while commonly used was limited. We assess this possibility in Experiment 2 by expanding the number of impulsivity measures used.

In summary, the results from the current experiment extend previous work on value-driven attentional bias by demonstrating that the effects of reward history on attentional orienting are exhibited even during a novel task with no history of or prospect of reward. These effects may be amplified in individuals who are aware of which stimulus was associated with reward, and future work should investigate the role of explicit awareness in either the development or maintenance of attentional bias to reward. Furthermore, although we demonstrated that the attentional bias to reward in an untrained cueing task is exhibited robustly as an attentional benefit (i.e., facilitation of response to targets in the location of a previously rewarded cue), we did not replicate previous results for attentional costs (i.e., slower responses when the target appears at the location opposite the previously rewarded cue). Experiment 2 was designed to (1) increase the ability to detect the effects of attentional orienting due to reward history in untrained tasks, (2) extend current findings to investigate how visual search training may lead to reward-associated biasing in tasks which rely primarily on other cognitive processes, such as inhibition and decision-making, and (3) further investigate the potential relation of impulsivity to value-driven biasing.

Experiment 2

The results from Experiment 1 provide initial evidence that reward association in one attention task can lead to attentional bias in a subsequent attention cueing task, despite the cueing paradigm having novel task goals, novel target stimuli, and explicitly no prospect for obtaining reward. While this provides an initial step demonstrating that bias induced by reward association can be exhibited in untrained tasks, further experimentation is required to bolster these findings and demonstrate their specificity to reward history. Additionally, it has yet to be tested how the reward association induced in this commonly implemented visual search paradigm might affect tasks largely unrelated to spatial attention, including motor inhibitory control and decision-making. While it is possible that effects on such tasks are exhibited in an attention-specific manner, it is also possible that high-value stimuli disrupt cognitive control processes broadly through increasing general “approach motivation” toward the previously rewarded stimulus. This latter mechanism is widely hypothesized to account for the way in which other stimuli with a history of reward (e.g., addiction cues, positive social stimuli) disrupt cognitive function (Berridge & Robinson, 2016; Robinson & Berridge, 2001). These two mechanisms would have different effects on the tasks we aim to investigate here.

Experiment 1 provided evidence that value-driven attentional orienting could be exhibited in a paradigm with novel task goals; Experiment 2 aims to bolster these initial findings and further investigate the effects of reward history on more dissimilar tasks, such as go/no-go and delay discounting tasks, which can better clarify the impacts of reward association training within a visual search task. Previous work has provided mixed evidence regarding the effects of reward history on motor inhibitory control. There is some evidence that reward association can in fact support better inhibitory control. For example, some studies have found that inhibitory control improves on trials with a stimulus signaling the prospect for reward during a stop-signal paradigm (Boehler, Hopf, Stoppel, & Krebs, 2012; Boehler, Schevernels, Hopf, Stoppel, & Krebs, 2014; Schevernels et al., 2015). Of note, however, is that for all of these paradigms, inhibitory control is measured while reward for successful performance is being conferred. It is less clear whether sustained improvements on inhibitory control, particularly when inhibition was never rewarded, might be evident. One possible mechanism through which sustained improvements on inhibitory control might occur is through the effects of increased attentional priority on inhibition, as research suggests that higher attentional priority stimuli facilitate inhibitory control (Chevalier, Chatham, & Munakata, 2014).

On the other hand, some work has demonstrated that reward history can interfere with inhibitory control. There is some evidence that reward-associated stimuli automatically elicit approach-oriented behavior (see Anderson, 2017, for review), and some theories have posited that by virtue of their rewarding properties, reward-associated stimuli can elicit approach behavior (Berridge & Robinson, 2016; Krieglmeyer et al., 2010; Robinson & Berridge, 2001). Studies that have used reward-associated colors as flankers in a modified flanker paradigm have found that the typical inhibition afforded to unrewarded stimuli is escaped by stimuli with a reward history (Anderson, Folk, Garrison, & Rogers, 2016). And the degree to which reward-associated stimuli escape inhibition has been linked with increased motor activity for responses linked with the reward-associated stimuli (Kim & Anderson, 2019c). Additionally, when button-press responses to a stimulus are rewarded, it is more difficult for healthy adults to inhibit motor responses to that stimulus when it is later used as a no-go stimulus in a go/no-go task (Davidow et al., 2019). This suggests that reward history may lead to an automatic motor excitation that could in fact disrupt motor inhibitory control. Experiment 2 will help to better elucidate the effect of reward history on motor inhibitory control by utilizing previously reward-associated stimuli as no-go targets in a go/no-go paradigm.

Additionally, Experiment 1 provided some support for a relationship between value-driven attentional bias and impulsivity, as measured by self-report on the BIS-11. One of the primary benefits of utilizing the BIS-11 is that it has been arguably the most widely used self-report measure of impulsivity across both research and clinical settings (Stanford et al., 2009), and it has previously been implemented in investigations of value-driven attention (Anderson et al., 2011b; Anderson, Kronemer, et al., 2016; Pearson et al., 2015). However, the BIS-11 measures impulsivity quite generally, not specifically impulsivity in the context of affect, which we might expect to be more related to the value-driven attention. In Experiment 2, we additionally measure impulsivity using the Urgency, Premeditation, Perseverance, Sensation Seeking, Positive Urgency, Impulsive Behavior Scale (UPPS-P), a scale that has been developed with the aim of measuring impulsivity in the context of affect (Lynam, Smith, Cyders, Fischer, & Whiteside, 2006). Given the positive valence of reward, it is possible that value-driven attention may be more closely linked with affective forms of impulsivity, which Experiment 2 will clarify. Additionally, Experiment 2 investigates the impact of reward history on impulsive choice responding in a delay discounting task. The delay discounting task has been developed as a measure of impulsive choice responding, with a willingness to forgo larger gains for more immediate rewards being indicative of greater levels of impulsivity (Reimers, Maylor, Stewart, & Chater, 2009). Previous work suggests that stimuli with a reward history can influence economic decision-making and risk-taking in decision-making (Anderson, 2017; Itthipuripat, Cha, Rangsipat, & Serences, 2015; San Martín, Appelbaum, Huettel, & Woldorff, 2016).

Experiment 2 builds upon initial findings in Experiment 1. Because results of Experiment 1 highlighted the potential impact of awareness of reward-contingencies, participants in Experiment 2 were uniformly made explicitly aware of which stimulus was to be rewarded in the training task, prior to beginning that task. By ensuring that all participants were explicitly aware of reward contingencies before training began, it was expected that the effects of value-driven attentional bias would be less variable than when individual differences in awareness arise throughout the experiment, thus increasing power to address the specific question at hand regarding the transfer of value-driven effects across tasks. After completing the reward-association task, participants completed the cueing task from Experiment 1, plus a delay discounting task, and a go/no-go task, all using the rewarded and unrewarded stimuli from the reward association training. Taken together, this experiment aimed to serve to further characterize how a history of reward association impacts performance on various tasks. First, it investigated whether value-driven increases in attentional priority are exhibited in an untrained attention task (i.e., the spatial cueing task). Second, it tested the effects of reward history on motor inhibition in the go/no-go task. And finally, it tested the relationships to impulsivity and the effects of reward history on impulsive choice responding in the delay discounting task.

Method

Participants

Participants were 45 healthy adults (ages 18–30 years, 60% female) with normal or corrected-to-normal eye-sight and no current diagnosis of depression, as confirmed by self-report, as well as no colorblindness, as confirmed using the Ishihara color vision test (Ishihara, 1973). Participants were recruited via flyers and a university LISTSERV. Monetary compensation was provided based on participant task performance (mean = $32.85; range: $24.00–$35.40). All tasks were programmed using OpenSesame (Mathôt et al., 2012), and stimuli were presented on a 22-inch color monitor with a refresh rate of 60 frames per second. All tasks were completed in a sound-attenuated booth. Participants were seated in a chair such that their eyes were approximately 75 cm from the screen. All procedures were approved by the UNC Chapel Hill Institutional Review Board.

Reward training and attention testing

The procedure and task described for reward-training in Experiment 1 were also used for Experiment 2, with the following differences: (1) in Experiment 2, participants were made explicitly aware of which target color circle would be associated with reward and which target color circle would never be associated with reward; (2) blue and yellow circles were used as the target circles (with the rewarded color counterbalanced across participants) in training for all participants in this experiment¹; (3) to account for a longer study visit and ensure that participants were earning the same rate of pay per hour, the reward amount was increased to 15 cents on the winning trials. The rate of reward remained the same, such that for the rewarded circle, 80% of trials were worth money and 20% were not, whereas no trials were worth money for the unrewarded circle. Initial training consisted of 360 trials across three blocks and was preceded by a practice block of 50 trials, as in Experiment 1. An additional block of training (120 trials) was completed prior to go/no-go task, and an additional block (120 trials) was completed prior to the delay discounting task to ensure that time since training did not impact effects. Order of completion of go/no-go and delay discounting task was counterbalanced across participants. The attention-cuing paradigm for Experiment 2 was identical to that used in Experiment 1.

Impulsivity

Participants completed the BIS-11, a 30-item self-report for assessing impulsivity (Patton et al., 1995); as in Experiment 1, it had good internal consistency in the current sample (α = .76). Another common measure of impulsivity is the Urgency, Premeditation, Perseverance, Sensation Seeking, Positive Urgency, Impulsive Behavior Scale (UPPS-P). The UPPS-P is a 59-item scale designed to assess impulsivity, including lack of planning, lack of perseverance, negative urgency, positive urgency, and sensation seeking (Lynam et al., 2006). The scale showed high internal consistency in this sample (α = .85).

Modified go/no-go task

Participants completed a go/no-go task for which the no-go stimuli consisted of the previously rewarded (PR) and previously unrewarded (PU) stimuli from the reward training task. Every trial began with a fixation screen presented for either 500, 1,000, or 1,500 ms, which was followed by a centrally presented circle for 500 ms (see Fig. 4). On 75% of trials, the color of the circle (white, cyan, green, pink, or orange) indicated that the participant should press the space bar (go trials). On the other 25% of trials, the color of the circle (yellow or blue) indicated that the participant should withhold a response (no-go trials). The color of the no-go trials corresponded to the PU and PR circles used in the attention tasks described previously. After a short practice block (50 trials), participants completed 360 trials of the task across three blocks. Differences in inhibitory control over PR and PU stimuli, as measured by false-alarm rate, is of primary interest in determining whether reward-driven differences in performance are exhibited in this task.

Fig. 4

Go/no-go paradigm. Every trial began with a fixation screen presented for either 500, 1,000, or 1,500 ms, which was followed by a centrally presented circle for 500 ms. On 75% of trials, the color of the circle indicated that the participant should press the space bar (go trials). On the other 25% of trials, the color of the circle (yellow or blue) indicated that the participant should withhold a response (no-go trials). Half of no-go trials were the previously rewarded stimulus, whereas the other half were the previously unrewarded stimulus

Delay discounting task

To measure impulsive choice responding, we employed a delayed discounting task (see Fig. 5), based on previously published work (Altamirano, Fields, D’Esposito, & Boettiger, 2011; Boettiger et al., 2007; Mitchell, Fields, D’Esposito, & Boettiger, 2005; Mitchell, Tavares, Fields, D’Esposito, & Boettiger, 2007; C. T. Smith & Boettiger, 2012). After a short practice block (eight trials), participants completed 82 trials across two blocks. There were four trial types: want, don’t want, sooner, and larger. The “want” trial type was of primary interest and consisted of 67% of trials. The remainder of the trial types were divided evenly between the control conditions (don’t want, sooner, and larger). The trials consisted of a fixation (500 ms), a cue indicating trial type (2,000 ms), a display with two options for participants to decide between (3,000 ms), and a fixation with varying intertrial intervals (2,500 ms, 4,000 ms, or 4,500 ms). The options displayed each had a given monetary amount and time. The options consisted of either $10 or $100 as the full amount at one of five future delays (1 week, 2 weeks, 1 month, 3 months, or 6 months) and a discounted amount (70%, 85%, 90%, or 95% of the full amount). The earlier option was always listed with the lesser amount. In most cases, “today” was the earlier option, but 3 months was the earlier alternative for some 6-month-delay trials. Participants were instructed to make their decisions as if they would really be receiving the money amounts indicated at the specified times and to indicate their response on the keyboard. For “want” trials, participants were instructed to select their preferred option, whereas less preferred options were selected for “don’t want” trials. Participants were instructed to indicate the sooner time for “sooner” trials and to indicate the larger monetary amount for “larger” trials. On 50% of trials, the cue and options were in blue font, and the remaining 50% of trials were in yellow font. These colors corresponded to PR and PU colors.

Fig. 5

Delay discounting task. There were four trial types: want, don’t want, sooner, and larger. The “want” trial type was of primary interest and consisted of 67% of trials. Participants were instructed to make their decisions as if they would really be receiving the money amounts indicated at the specified times and to indicate their response on the keyboard. For all trials, 50% were cued with the PR color and stimulus (a) whereas the other 50% were cued with the PU color and stimulus (b)

Temporal discounting was measured using the impulsive choice ratio (ICR), which was determined using the proportion of “sooner” trials selected. This measure has been used previously for delay discounting tasks, and has been commonly used with this particular task (Ainslie & Monterosso, 2003; Altamirano et al., 2011; Boettiger et al., 2007; Mitchell et al., 2005; Mitchell et al., 2007; C. T. Smith & Boettiger, 2012). One participant’s delay discounting task data was excluded due to misunderstanding the instructions. The primary difference of interest was whether the ICR differed for trials occurring with the PU stimulus as a cue versus trials with the PR stimulus as a cue.

Analyses

Similar to Experiment 1, a mixed-measures analysis of variance (ANOVA) was conducted on RT, with a within-subjects factor of trial type (invalid PR, neutral, valid PR) to test a linear effect of orienting. As in Experiment 1, only correct trials were analyzed, and all trials with RT’s below 100 ms were excluded from analysis. Because all subjects were made explicitly aware, awareness was not included as in effect of interest in analyses. As in Experiment 1, analyses were used to first test attentional bias to reward, and follow-up analyses were used to determine whether reward-driven bias is exhibited above and beyond training alone. As in Experiment 1, any RTs below 100 ms were excluded from further analyses, as these responses are likely too fast to reflect accurate target discrimination. After exclusion for RT’s below 100 ms, more than 99% of the data remained. Only correct responses were used to calculate reaction time. For any instances in which Mauchly’s test indicates that the assumption of sphericity should be rejected for the data when implementing an ANOVA, Greenhouse–Geisser corrected degrees of freedom and probability values are reported.

As in Experiment 1, linear regressions were conducted to test whether behavior biased by history of reward was linked with individual differences in impulsivity. In Experiment 2, however, there are measures of reward-biased behavior from three tasks: the cueing paradigm, the delay discounting task, and the go/no-go task.

Results

Reward training

A paired t test was conducted on RT’s of correct trials to test for a reward-driven training effect with reward (rewarded, unrewarded) as a within-subjects factor. As expected, results revealed participants were significantly faster for the rewarded stimulus (M = 528 ms, SD = 42 ms) than the unrewarded stimulus (M = 589 ms, SD = 40 ms) in the initial training, t(44) = 14.22, p < .01. For the retraining blocks that occurred immediately prior to the go/no-go task, participants were again faster on trials with the rewarded color, t(44) = 15.10, p <.01 (rewarded M = 508 ms, SD = 40 ms; unrewarded M = 586 ms, SD = 49 ms). The same pattern was seen for the retraining blocks that occurred just prior to the delay discounting task, t(44) = 14.69, p < .01; rewarded M = 508 ms, SD = 35 ms; unrewarded M = 579 ms, SD = 45 ms). Accuracy was high across conditions, with significantly better accuracy for the rewarded (M = 0.89, SD = 0.07) than the unrewarded (M = 0.77, SD = 0.13) stimuli, t(44) = 7.12, p < .01, in the initial training. This difference was also significant in the retraining blocks prior to the go/no-go task, t(44) = 7.24, p < .01 (rewarded M = 0.93, SD = 0.06; unrewarded M = 0.82, SD = 0.13), and prior to the delay discounting task, t(44) = 7.37, p < .01 (rewarded M = 0.81, SD = 0.13; unrewarded M = 0.93, SD = 0.06).

Attention cueing task

Consistent with value-driven attentional bias, and our results in Experiment 1, it was anticipated that participants would show a linear-orienting effect, with increasingly slower RTs when comparing valid PR trials with neutral baseline trials to invalid PR trials. To test this, a repeated-measures ANOVA was conducted on PR trials with the within-subjects factors of validity (valid, neutral, invalid). Results indicated there was an effect of validity on RTs, F(1.53, 67.12) = 11.11, p < .01, η² = .20, and the effect of validity was linear, following the expected pattern, F(1, 44) = 14.83, p < .01, η² = .25 (see Table 2 and Fig. 6a). Separate follow-up ANOVAs tested the specific effects of capture (PR valid relative to neutral) and disengagement (PR invalid relative to neutral). Results indicated a significant effect of capture, F(1, 44) = 13.76, p < .01, η² = .24, and a significant effect of disengagement, F(1, 44) = 54.11, p < .05, η² = .09.

Table 2 Reaction times in milliseconds for the cueing paradigm for Experiments 2. Mean (SD)

Fig. 6

Performance on cueing task in Experiment 2 for (a) PR trials, (b) PU trials, and (c) both-trained-present (PR & PU) trials. A significant orienting effect was found for PR trials, specifically characterized by both a significant disengagement effect and a significant capture effect. For both-trained-present (PR & PU) trials, there was an orienting effect found with a significant capture effect. Error bars are within subjects 95% confidence intervals (Cousineau, 2005). * indicates significance at p < .05

To examine whether the orienting effect was also exhibited to unrewarded but trained stimuli, we tested PU trials, which contained the color that was the unrewarded target color in the training phase. A similar mixed-measures ANOVA with the within-subjects factor of validity (valid, neutral, invalid) was implemented for PU trials. There was no effect of validity, p = .80, which does not support the presence of an attentional bias to PU cues (see Fig. 6b).

To test whether the value-biasing effect was exhibited when both the PR and the PU circles were presented at the same time, performance on “both” trials was analyzed. For this cue, a trial was coded as “valid” if the target appeared in the PR circle and considered “invalid” if the target appeared in the PU circle, as in Experiment 1. An ANOVA with the within-subjects factor of validity (valid, neutral, invalid) was conducted for both-trained-present trials. Results indicated there was an effect of validity, F(2, 88) = 9.68, p < .01, η² = .18. The effect of validity was linear, such that performance followed the expected pattern, F(1, 44) = 14.31, p < .01, η² = .25 (see Fig. 6c). Separate follow-up ANOVAs tested the specific effect of capture (valid relative to neutral) and disengagement (invalid relative to neutral). Results indicated a significant effect of capture, F(1, 44) = 12.81, p < .001, η² = .23, but not disengagement, p = .30. Accuracy was uniformly high across all conditions, and there was no evidence of a speed–accuracy trade-off in any of the analyses presented here. Accuracy data are presented in Supplemental Table S2.

Additional analyses of cuing task: Attentional bias differences across trial types

While the linear orienting effect within each trial type is of primary interest, additionally, a test of whether total attentional bias (i.e., the difference between valid and invalid trials) differed by trial type was conducted. As explained in Experiment 1, this test does not allow for an investigation of facilitated capture and impaired disengagement effects, because to conduct such an analysis all trials are compared to the same neutral trial type; however, it does allow for an examination of differences between total attentional bias effects across trial types. Of primary interest was examining differences between PR cues and PU cues. As such, a 2 × 2 repeated-measures ANOVA was conducted on RT, with the within-subjects factors of trial type (PR cue, PU cue) and validity (valid, invalid). Results indicated no overall difference between PR and PU trials, p = .27. There was a significant effect of validity, such that valid trials were faster than invalid trials, F(1, 44) = 7.62, p < .01, η² = .15. Furthermore, this was qualified by a significant validity by trial type interaction, F(1, 44) = 13.68, p < .01, η² = .24, suggesting that the validity effect was larger in the PR trials than the PU trials, in line with previous research.

Relation between reward-training and attention cuing

As in Experiment 1, we tested the relationship between reward-driven behavior during training and the value-driven effects on attentional bias in testing. Training bias was calculated as the difference between RT for trials with the rewarded circle and RT for trials with the unrewarded circle. Attentional bias was calculated using the total attentional bias score (invalid RT − valid RT) first for PR trials and then for “both” trials. Unlike in Experiment 1, our results here showed no relationship between reward bias in training and attention bias at testing for the PR trials, r(43) = .07, p = .65, or the “both” trials, r(43) = −.02, p = .87. Also unlike in Experiment 1, there was a relationship between participants showing less reward bias in training (i.e., relatively smaller values of rewarded RT − unrewarded RT) showing a larger attentional bias at testing for the PU trials r(43) = .31, p = .04. Given that we find no evidence of a relationship between biasing behavior toward reward in training with biasing behavior to the reward cue in testing, but we did in Experiment 1, we conducted a follow-up analysis (Lenhard & Lenhard, 2014) to test whether the strength of the correlations differed between Experiments 1 and 2. Results indicated that for the PR trials, there was a significant difference between the size of the relationships, Z = 1.85, p = .03. For the “both” trials, there was a trending difference between the size of the relationships, Z = 1.57, p = .06, and the difference reached significance for the PU trials, Z = 2.02, p = .02.

Relation with impulsivity

Separate linear regressions were conducted to test whether the degree to which behavior is biased by reward history on the current tasks predicted variability in impulsivity as measured by self-report on the BIS-11 and the UPPS. The four tasks were (1) PR trials on the cueing task, (2) “both-present” trials on the cueing task, (3) the go/no-go task, and (4) the delay discounting task. For the cueing task, the linear regression included total value-driven attentional bias (i.e., invalid RT − valid RT) as the predictor and self-reported impulsivity as the dependent variable. These regressions were performed separately for each of the trial types with a PR cue (PR and “both”). For the go/no-go task, reward biasing, defined as the difference between false-alarm rate for PR no-go trials and PU no-go trials, served as the predictor and self-reported impulsivity as the dependent variable. For the delay discounting task, the linear regression tested whether variability in self-reported impulsivity was predicted by the difference in ICR for PR cued trials and PU cued trials. No regression models significantly predicted variability in self-reported impulsivity on the BIS-11 (PR cueing p = .16; “both” cueing p = .61; go/no-go reward bias p = .31; delay discounting reward bias p = .76) or on the UPPS (PR cueing p = .29; “both” cueing p = .64; go/no-go reward bias p = .370; delay discounting reward bias p = .914). Additionally, to rule out effects due to reward-biasing in training, a regression was performed predicting impulsivity from reward biasing during the reward association training (rewarded RT − unrewarded RT), which indicated reward-biasing during training did not significantly predict impulsivity (BIS-11 p = .33; UPPS p = .58).

Go/no-go task

Overall, participants demonstrated good performance on go trials (accuracy M = .99, SD = .02; RT M = 418 ms, SD = 40 ms), suggesting strong engagement with the task. To test whether history of reward impacted inhibitory control, an ANOVA testing whether false-alarm rate differed between PR no-go stimuli and PU no-go stimuli in the go/no-go task. Results indicated that participants were better at inhibiting responses to PR no-go stimuli (accuracy M = .72, SD = .16) than PU no-go stimuli (accuracy M = .66, SD = .17), F(1, 44) = 6.40, p = .02, η² = .13, which provides new support that reward history can facilitate motor inhibition.

Delay discounting task

The impulsive choice ratio (ICR) was calculated as the proportion of “sooner” trials selected out of all trials (i.e., “sooner” and “later”). Participants provided responses quickly enough on 98% of trials. Whether cues associated with history of reward impacted impulsive choices was tested by an ANOVA with PR cues versus PU cues. There was no evidence for a difference in ICR between PR trials (M = .45, SD = .33) and PU trials (M = .45, SD = .33), p = .89. As a method of evaluating the strength of the evidence for the null hypothesis that there is no difference in ICR between cue types and the alternative hypothesis that there is an effect of cue type on ICR, the data were also examined by estimating a Bayes factor. This was implemented using Bayesian information criteria (Wagenmakers, 2007), which compares the fit of the data under the null hypothesis and the alternative hypothesis. An estimated Bayes factor (null/alternative) suggested that the data were 8.32:1 in favor of the null hypothesis, which provides moderate support for a model without an effect of reward history on choice. This analysis suggests a 95% chance that the true mean difference between the ICR for PR cues and PU cues falls between −0.03 and 0.04, overlapping the zero point of no difference between trial types.

A follow-up 2 × 2 repeated-measures ANOVA with the factors of trial type (PR vs, PU) and choice type (“sooner” vs. “later”) was conducted on RT, because it has been suggested that longer RTs for choices on the delay discounting task reflect more cognitive effort to override automatic choices (Basile & Toplak, 2015). Results did not show significant main effects of reward (p = .54) or choice type (p = .12) and no evidence for an interaction between these factors (p = .72).

Discussion

Across the experiments, we demonstrate that value-driven effects on cognition and behavior can be exhibited in untrained tasks with no history of reward, consistent with sustained reward association effects. We further clarify that effects of history of reward association can impact spatial attention in untrained tasks, as demonstrated by evidence for attentional orienting to stimuli with a history of reward association. We further find that the same training that impacts spatial orienting may have impacts on temporal attention, as demonstrated by facilitated inhibitory control in a go/no-go paradigm. We do not find evidence here for the effects of reward-associated cues on impulsive decision-making in a delay discounting task.

Across both experiments, we demonstrate that value-driven effects on attentional priority can impact tasks with differing targets and task goals. This indicates that value-driven attentional bias is not explained by reliance on a task-specific attentional set, and instead is exhibited in novel contexts regardless of top-down goals. Furthermore, Experiment 2 showed both attentional benefits at the location of the reward-associated stimulus and attentional costs when the reward-associated stimulus is not at a target location. The latter effect, attentional costs, tends to overall be weaker (Pool et al., 2016); indeed, this effect did not reach significance in Experiment 1, which had less power to detect this effect.

In addition, Experiment 2 aimed to address whether reward-associated stimuli have effects on motor inhibitory control. As described above, we demonstrated that history of reward-association led to attentional bias in the cueing paradigm. Additionally, we find the history of reward association leads to facilitation of motor inhibitory control in the go/no-go paradigm. Participants were better at withholding responses to previously rewarded stimuli compared to previously unrewarded stimuli. Other work has found that reward can facilitate control in a stop-signal paradigm, in which performance on the task is rewarded (Boehler et al., 2012; Boehler et al., 2014; Schevernels et al., 2015). In addition to reactive control processes, it is suggested that reward acts on attentional processing to support faster inhibition in the presence of a reward-associated cue (Schevernels et al., 2015). While other work has found that reward associated stimuli break through some inhibitory processes (Anderson et al., 2012; Kim & Anderson, 2019c; Wang et al., 2015), we find that reward history supports inhibitory control when following reward association training in a visual search task where inhibition is an explicit goal. One element of this go/no-go task that differs from the other tasks tested here (i.e., cueing and delay discounting) is that the circles are directly relevant to the task goals. Previous work has demonstrated that increased attentional priority facilitates the ability to stop responses to that stimulus (Chevalier et al., 2014), which is consistent with facilitated attentional processing to the previously rewarded stimulus in the go/no-go paradigm. However, future work should investigate whether facilitative effects on inhibitory control are also exhibited when the reward-associated stimuli are distractors rather than specifically task relevant. This effect also may be dependent on the type of reward association training implemented. Other work which has first rewarded motor responses to stimuli and subsequently implemented those stimuli as no-go targets in a go/no-go paradigm has found that motor inhibition was thwarted, rather than facilitated, by reward history (Davidow et al., 2019). Our results extend previous work to suggest that reward association conferred in a visual search task subsequently facilitates motor inhibition in an untrained and unrewarded task.

We find no evidence for a reward-associated impact on impulsive choice in the present study. Specifically, in the delayed discounting task there was no evidence that participants were more likely to select a smaller reward granted sooner compared with a larger reward granted later for a choice cued by a reward-associated color versus a color with no reward association. One other study has manipulated reward association and found impacts on spatial attention without evidence for impacts of the reward association on spatial decision-making (Bourgeois et al., 2018), which might suggest that evidence for reward effects on attention may not generalize to impacts on decision-making. However, our results may alternatively be in part due to the task design, in which the cue is presented for a rather short amount of time and therefore is expected to act upon faster, more implicit processes, whereas participants had 2 seconds to respond to the target stimuli, which allows for deeper processing. Future work could further investigate the role of reward history in decision-making under more restrictive time constraints.

Additionally, our results provide some suggestive evidence that awareness of reward contingencies during learning amplifies value-driven attentional bias. In Experiment 1, we show some evidence that value-driven attentional bias may be exhibited more strongly in individuals who were aware of the reward contingencies. In Experiment 2, when all participants were made explicitly aware of the reward contingencies prior to training, we find value-driven effects on both attentional benefits as well as attentional costs, which is generally a weaker effect and an effect that did not reach significance in Experiment 1. This is potentially important for understanding the mechanisms of learning and generalizability related to value-biasing. The link between awareness and learning has also been investigated in the literature on other forms of affective learning, and suggests that awareness can moderate the strength of affective learning. Previous work suggests that awareness is not always a necessary component for affective learning to occur (Knight, Waters, & Bandettini, 2009; C. N. Smith, Clark, Manns, & Squire, 2005; Wiens & Öhman, 2002), but affective learning may be expressed most strongly in the context of contingency awareness (Hamm & Weike, 2005; Lovibond, 2003, 2004; Purkis & Lipp, 2001; Tabbert et al., 2011; Tabbert, Stark, Kirsch, & Vaitl, 2006). However, it is unclear why robust value-driven attentional bias was observed in some previous studies with small proportions of individuals reporting awareness (Anderson, 2015; Anderson et al., 2013), whereas we only observed this bias in individuals who became aware of reward contingencies in the training session. One possibility is that in order for the expression of reward learning to be exhibited across disparate tasks, as is tested in our study, a more intense learning of the reward–cue association must have occurred.

Regarding how reward-biasing in training related to value-driven effects in testing after the prospect of reward had been removed, the results from the current experiments produced somewhat different results. Echoing previous results (Failing & Theeuwes, 2014; Yantis, Anderson, Wampler, & Laurent, 2012), Experiment 1 showed a moderate relationship between reward-bias in training and value-driven attentional bias in testing. In Experiment 2, however, reward-biasing in training did not predict the degree of value-driven attentional bias in testing. One possible explanation for this difference is that the mechanisms through which reward learning occurs may differ when explicit instruction is given. While both the subset of participants who learned to identify the reward contingencies (Experiment 1) and individuals explicitly told of reward contingencies (Experiment 2) were ultimately aware of the stimulus–reward association, how this awareness was established differed between the experiments.

Previous work has found that when explicit instruction about contingencies is provided, there is a decrease in neural activation in regions involved in sending reward feedback signals, such as the ventromedial prefrontal cortex and ventral striatum (Li, Delgado, & Phelps, 2011), which is related to increased recruitment of frontal control regions, such as the anterior cingulate cortex and dorsolateral prefrontal cortex (Li et al., 2011; Tabbert et al., 2011). It has been suggested that recruitment of frontal control regions in this type of task, regulate the degree to which a model-free or model-based learning approach is employed, with the latter resulting in dampened activation in the putamen (Lee, Shimojo, & O’Doherty, 2014). In Experiment 2, participants could employ this type of model-based learning because they are given explicit instructions as to which stimulus is associated with reward. As such, participants may have engaged in more top-down strategies during the training phase in Experiment 2 compared with Experiment 1.

Finally, we investigated how value-driven behavior across tasks might relate to impulsivity. Experiment 1 demonstrated a relationship between self-reported impulsivity and overall value-driven attentional bias, but this effect was not replicated in Experiment 2. The inconsistent relationship between impulsivity across these experiments mirrors the variable findings regarding the relationship between attentional bias to reward and impulsivity in previous work (Anderson et al., 2011b; Anderson, Kronemer, et al., 2016; Pearson et al., 2015). One possibility for this inconsistent effect is that impulsivity and value-driven attentional bias might share relationships to a third construct. Here, we demonstrate that it is unlikely explained by shared relationships to reward sensitivity, as demonstrated by our finding that the reward-biasing during training does not explain the relationship between impulsivity and value-driven attentional bias. Another possibility is that sustained value-driven attentional bias may reflect inflexible responding (compulsivity) more than it reflects acting without forethought (impulsivity). Impulsivity and compulsivity are both linked with one another, are both elevated in individuals with a history of substance use, and both involve frontostriatal neural circuits (Chamberlain, Stochl, Redden, & Grant, 2018; Robbins et al., 2012).

There are also some limitations of the study to consider. For example, while our study used a reward association paradigm that is common for examining attentional bias, this paradigm rewards attentional selection in the reward association phase in addition to conferring a stimulus–reward association. As such, our study is not equipped to disentangle whether the generalization of value-driven effects exhibited here are driven by the stimulus–reward association or the behavior–reward association conferred in the training paradigm, which future studies should investigate. Furthermore, while we took measures to reduce the effects of demand characteristics by providing neutral, consistent instructions, not conveying hypotheses, and not being in the experimental booth when participants completed the tasks, possible demand characteristic effects cannot be entirely ruled out. Previous work in a clinical population has provided some support that the effects of demand characteristics may be less evident when the experimenter is in a separate location from participants (Pinkham, Hopfinger, & Penn, 2012), as we have done in these experiments. We also believe that demand characteristics are less likely to explain our present results because participants show effects even when explicitly informed to ignore certain cues (e.g., on the cueing paradigm), suggesting that they may not be biasing behavior in such a way that they believe is consistent with experimenter desires. We also would expect that if results were driven by demand characteristics, these might be especially strong on tasks that utilize explicit processing, such as the Delay Discounting Task, which was the only task in which we did not find evidence of differential processing of the previously rewarded stimuli.

Another limitation of the present study is that while we showed that reward association did affect untrained cognitive control processes, it is unclear whether a potential intermediary step (e.g., working memory) was impacted. One plausible mechanism suggested by Rusz, Bijleveld, and Kompier (2019) is that attentional priority given to reward-associated stimuli in turn reduces capacity to carry out task-relevant processes by utilizing working memory resources. Indeed, ability to limit distraction by reward-associated stimuli has been associated with working memory capacity (Anderson et al., 2011b), and reward-associated stimuli are prioritized in working memory (Gong & Li, 2014; Klink, Jeurissen, Theeuwes, Denys, & Roelfsema, 2017). Future work should investigate the degree to which reward’s impact on working memory underlies transfer effects.

In summary, these experiments provide novel information characterizing how value-driven effects on attentional priority impact attentional orienting, inhibition, and impulsive choice in untrained and never-rewarded tasks. Our findings suggest that value-driven attentional bias effects are not explained by top-down attentional sets alone, as value-driven effects on attention are exhibited in novel tasks. Further, we find evidence that the changes in attentional priority induced through reward association might thwart cognitive control in some tasks (e.g., when serving as a distractor) while supporting cognitive control in others (e.g., when inhibiting motor responses). We find that increased attentional priority from history of reward association facilitates inhibitory control rather than disrupting it. Taken together, this demonstrates that when history of reward association modulates attention, this change in attentional priority is exhibited robustly across novel contexts.

Appendix A: Reward-contingency awareness probe modified from Anderson (2015)

Which option do you believe best describes the part of the experiment in which you were earning money (please choose only one):

• 1. The blue circle was generally worth more than the yellow circle regardless of what the line orientation was

• 2. The yellow circle was generally worth more than the blue circle regardless of what the line orientation was

• 3. The two circles were worth the same overall, but one color was worth more when it appeared with a horizontal line and the other was worth more when it appeared with a vertical line

• 4. Both color circles were generally worth more when presented with a vertical line

• 5. Both color circles were generally worth more when presented with a horizontal line

• 6. How much money I received was random and unrelated to the colors or lines