Intolerance of uncertainty and conditioned place preference in opioid addiction

Introduction

Drug addiction is thought to involve a disruption of reward processing such that the pursuit and use of a substance come to dominate an individual’s behavior, at the expense of other actions, despite the associated negative consequences (Wise & Bozarth, 1987; Clark & Robbins, 2002). A combination of positive and negative reinforcement processes may contribute to initial substance use, with prolonged exposure leading to changes in brain systems involved in motivation that mark the transition to addiction (Wise & Bozarth, 1987; Everitt & Robbins, 2005; Everitt et al., 2008). However, not all individuals exposed to drugs develop addiction, suggesting that some are more vulnerable than others. As one example, the personality trait of intolerance of uncertainty (IU) involves the tendency to perceive uncertain situations as aversive, and can result in avoidance and negative expectations about the consequences of such situations (Nelson et al., 2016). IU is higher in individuals with opioid use disorder (OUD), raising the possibility that it is a risk factor for this, and potentially other, substance abuse disorders (Carleton, Norton & Asmundson, 2007; Garami et al., 2017).

Individuals with higher IU appear to require more information before making decisions, presumably, to reduce uncertainty. For example, in one study, participants with higher IU sampled more marbles compared to those with lower IU before estimating the number of black vs. white marbles in a bag (Ladouceur et al., 1998). In addition, those with higher IU appear to prefer low-probability immediate rewards over high-probability but delayed rewards, suggesting that these individuals find waiting under uncertainty aversive, and are willing to take risks (e.g., choose a low-probability reward) as long as doing so allows them to avoid this aversive state (Luhmann, Ishida & Hajcak, 2011). Taken together, these results suggest that IU can bias decision-making under uncertain conditions, including those involving probabilistic rewards.

There is a large body of evidence that decision-making is altered in addiction. For example, individuals with substance abuse disorders tend to choose immediate small rewards over larger but delayed rewards, and also show a preference for immediate certain rewards over larger, but uncertain rewards (Clark & Robbins, 2002; MacKillop et al., 2011). One factor that could influence decision-making is reward sensitivity, which appears to be decreased in addiction, especially for non-drug rewards or cues that signal these rewards (Blum et al., 2000; Koob et al., 2004; Volkow et al., 2010; Luijten et al., 2017). This decrease, associated with a loss of motivation for non-drug rewards, could bias decision-making in favor of continuing drug use, which has become the dominant source of reinforcement. It may also contribute to relapse after a period of abstinence, which poses a major hurdle for the treatment of substance use disorders (Goudriaan et al., 2008).

Conditioned place preference (CPP) is a common paradigm that can be used to assess reward sensitivity in terms of preference for contexts previously paired with reward. This task is widely used in the study of addiction in animal models in order to understand the reinforcing effects of both drugs and other types of rewards, including food and access to a mate (Bardo & Bevins, 2000; Tzschentke, 2007). Several computer-based analogues have been developed for use in humans (Childs & De Wit, 2009; Molet, Billiet & Bardo, 2013; Astur, Carew & Deaton, 2014; Childs, Astur & De Wit, 2017). Most relevant to the current work is a study that examined whether IU was associated with CPP in a probabilistic task where college students could forage for reward (i.e., points in the task) in two virtual rooms, one of which was associated with more frequent reward (Radell et al., 2016). This study found that, when given a choice, participants with higher IU tended to return to the room previously associated with a higher chance of reward, while those with lower IU showed no such preference (Radell et al., 2016). There are multiple possible interpretations of this preference, including that it may have resulted from a tendency to choose the more certain choice, i.e., returning to a location known to contain reward in the past, or from an increased tendency to pursue reward (Radell et al., 2016).

While IU is predicted to have both cognitive and behavioral effects, most studies have focused on its cognitive aspects (Luhmann, Ishida & Hajcak, 2011). As discussed above, although research has started to investigate the relationship between IU and reward sensitivity, to our knowledge, none have done so in a population with OUD. Thus, the current study adopted the CPP task of Radell et al. (2016) to compare the potential behavioral effects of IU between individuals undergoing treatment for OUD and healthy controls. First, consistent with prior studies (Carleton, Norton & Asmundson, 2007; Garami et al., 2017), we hypothesized that patients would report higher IU than controls. Second, we hypothesized that if patients have reduced reward sensitivity to non-drug reward, they would show reduced preference for the context associated with reward relative to controls. Third, we expected IU to modulate behavior in both groups, i.e., within both the patient and control groups, high-IU individuals would have higher preference for the rewarded context than low-IU peers.

Materials and Methods

Procedure

For patients, testing took place in a quiet room at the RPAH Methadone Clinic within 30 min of receiving the daily drug dose. Controls were tested in a quiet room at the Western Sydney University or, as needed, in their homes. All participants completed the Intolerance of Uncertainty Scale (IUS) (Buhr & Dugas, 2002) and a computer-based CPP task adopted from a previous study (Radell et al., 2016), which was administered on a MacBook Pro. The CPP task, illustrated in Fig. 1, consisted of a tutorial, pre-training, training and a post-training phase. Participants controlled a cartoon avatar (a fox) and were instructed to help the fox collect as many golden eggs as possible; exact instructions are provided in the Appendix.

During the tutorial phase, the fox was placed in a lobby area with a single door. Participants were told that they could click on the door to enter a room (“tutorial room”). The tutorial room was visually distinct from the other rooms subsequently encountered in the task. The tutorial room contained eight chests located in a circular pattern and participants were required to click on the chests until they found two golden eggs. As a participant clicked on a chest, the fox moved to examine the chest, which opened and revealed whether there was an egg inside or not. During the tutorial, every chest contained an egg and therefore, the first two distinct chests the participants clicked completed this phase. The total number of eggs collected was always visible at the top of the screen but was reset at the conclusion of the tutorial phase, and therefore the two eggs collected found during the tutorial were not included in the final count.

Next, in the pre-training phase, the fox was returned to the lobby with the option of two doors (blue and brown) to choose from. The left or right placement of the doors was counterbalanced across participants. The doors led to two visually distinct rooms (blue and brown), which each contained eight chests to explore. During this phase, which lasted 4 min, participants were free to explore both of the rooms and click on the chests to search for eggs. Participants had a 5% chance of obtaining an egg from any chest, and once they had gained an egg, the chance to be rewarded again from the same chest decreased to 0 and then increased by 1% every 4 sec. Participants were not notified of these reward contingencies and therefore had to rely on trial and error. The time spent in each room, the total number of chests searched, and the total number of eggs collected in the task, were recorded.

Following the pre-training was a training phase. During the training phase, the room where participants had spent more time during the pre-training (i.e., the more-preferred room) was assigned to be “poor” while the less-preferred room was assigned to be ”rich.” In the “rich” room, each chest initially had a 0.8 probability of containing an egg; in the “poor” room, each chest initially had a 0.05 probability of containing an egg. Once an egg was collected from a chest, the probability of an egg from the same chest decreased to 0 and then increased by increments of 0.1 every 4 sec. Training consisted of two parts lasting 2 min each. In each part, the fox was placed in a lobby with one door unlocked and the other locked; the participant could then enter the unlocked room and spend the duration collecting eggs. The order in which rooms were unlocked (rich vs. poor in part 1 vs. part 2) was counterbalanced across participants. In the second part of training, the locks were switched, so that the participant could now spend the duration of part 2 collecting eggs in the other room. The order in which subjects searched chests, total number of chests searched and total score of eggs was again recorded.

The final phase was the post-training, which was identical to the pre-training. The fox was placed in the lobby with both rooms unlocked. Reward contingencies for this phase also followed the same pattern as the pre-training. In this phase, the first room that the participant entered was also recorded.

Results

Conditioned place preference task

Immediate preference

The first analysis examined whether there was an immediate preference to enter the previously-rich room at the start of the post-training phase. Since the control group was mostly female, while the patient group was mostly male, a Pearson chi-square test was used to confirm that there was no significant preference to enter the rich room first as a function of gender, χ²(1) = 0.519, p = 0.471, eliminating this as a possible confound. Next, a hierarchical three-way loglinear analysis was performed with factors of the first room entered in the post-training (rich vs. poor), IU (high vs. low) and group (control vs. patient). This produced a model that retained only the two-way interactions with a likelihood ratio of χ²(3) = 9.092, p = 0.028. The three-way interaction and main effects did not make a significant contribution to the model, with likelihood ratios of χ²(1) = 2.016, p = 0.156 and χ²(3) = 4.433, p = 0.218, respectively. The two-way interaction between the first room entered and group was significant, ${\mathrm{\chi }}_p^2\left(1\right)=4.964,p=0.026$, indicating that controls were significantly more likely than patients to enter the rich room first (Fig. 2A). Based on the odds ratio, the odds of a control entering the rich room first were 3.5 (1.12, 10.96) times higher than a patient entering this room. The two-way interaction between IU and group was also significant, ${\mathrm{\chi }}_p^2\left(1\right)=4.031,p=0.045$. The odds of a patient with high IU were 2.92 (1.01, 8.49) times higher than those of a control. This was not surprising since patients had higher IU than controls and so there were more patients in the high IU group. However, the two-way interaction between first room entered and IU was not significant, ${\mathrm{\chi }}_p^2\left(1\right)=0.085,p=0.771$.

Since the first room entered in the post-training may also be a function of which room was experienced during the second training phase, an additional hierarchical three-way loglinear analysis was conducted to examine this potential confound, with factors of the first room entered (rich vs. poor), the room experienced in the second training phase (rich vs. poor) and group (control vs. patient). This produced a model that retained only the two-way interactions, with a likelihood ratio of χ²(3) = 12.833, p = 0.005. The three-way interaction and main effects did not make a significant contribution to the model, with likelihood ratios of χ²(1) = 0.389, p = 0.533 and of χ²(3) = 5.658, p = 0.129, respectively. The two-way interaction between the first room entered and the last room experienced was significant, ${\mathrm{\chi }}_p^2\left(1\right)=7.776,p=0.005$. There were 4.14 (1.35, 12.66) higher odds of a participant entering the rich room first if they were in the poor room during the second training phase. The two-way interaction between the first room entered and group was once again significant, χ²(1) = 6.207, p = 0.013, confirming that controls were more likely to enter the rich room than patients. The two-way interaction between group and the last room experienced was not significant, χ²(1) = 1.325, p = 0.250, therefore, training order was successfully counterbalanced and the last room experienced cannot account for the increased tendency of controls to enter the rich room.

Overall preference

Next, to examine whether participants tended to spend more time in the rich room overall, a mixed-model analysis of variance (ANOVA) was performed, as described in the methods, on the percent time spent in the rich room during the pre-training and post-training phase of the task. There was a significant main effect of phase, F(1, 55) = 17.136, p < 0.001, ${\mathrm{\eta }}_p^2=0.238$, indicating that participants increased the amount of time spent in the rich room from pre-training to the post-training (Fig. 2B). There were, however, no other significant main effects or interactions.

Exploration

As described in the methods, a mixed-model ANOVA was performed on the total number of room entries to assess exploration, with a within-subjects factor of phase (pre-training vs. post-training) and between-subjects factors of group (control vs. patient) and IU (low vs. high). There was only a significant main effect of test, F(1, 55) = 15.988, p < 0.001, ${\mathrm{\eta }}_p^2=0.225$, indicating that number of room entries decreased from the pre-training to the post-training (Fig. 3A). There were no other significant main effects or interactions (all p > 0.17).

A mixed-model ANOVA was also performed on the total number of mouse clicks on chests with a within-subjects factor of room (rich vs. poor) and between-subjects factors of group (control vs. patient) and IU (low vs. high). There was a significant main effect of group $F\left(1,55\right)=7.853,p=0.007,{\mathrm{\eta }}_p^2=0.125$, and a group × IU interaction, $F\left(1,55\right)=5.019,p=0.029,{\mathrm{\eta }}_p^2=0.084$. There were no other significant main effects or interactions (all p > 0.22). As expected, the interaction indicated that controls with high IU searched more chests compared to high IU patients while low IU controls and patients searched a similar number of chests (Fig. 3B). Bonferroni-corrected independent-samples t-tests (alpha adjusted to 0.05/2 = 0.025) confirmed a significant difference in the total number of chests clicked by high IU controls compared to high IU patients, t(26) = 3.238, p = 0.003, r = 0.54. There was no difference between low IU controls and patients, t(29) = 0.440, p = 0.663.

Total score

Since high IU control participants searched significantly more chests, this raises the possibility that they also obtained a higher total score in the task (i.e., found more golden eggs). To confirm this, a between-subjects ANOVA was performed on the total score in the task computed as described in the methods, with factors of group (patient vs. control) and IU (high vs. low). There was a significant IU by group interaction, $F\left(1,55\right)=11.811,p=0.001,{\mathrm{\eta }}_p^2=0.177$, indicating that individuals with low IU obtained a similar total score regardless of group, while among those with high IU, controls found more eggs than patients (Fig. 4). Bonferroni-corrected independent-samples t-tests (alpha adjusted to 0.05/4 = 0.0125) found a significant difference between controls and patients with high IU, t(26) = 4.881, p < 0.001, r = 0.69, but not between controls and patients with low IU, t(29) = 0.100, p = 0.921. Among controls, those with high IU had a significantly higher score than those with low IU, t(25) = 2.891, p = 0.008, r = 0.501, but there was no difference in the total score obtained by patients with low and high IU, t(30) = 1.771, p = 0.087. There was also a significant main effect of group, $F\left(1,55\right)=10.856,p=0.002,{\mathrm{\eta }}_p^2=0.165$, with controls collecting more eggs than patients but, as shown by the interaction, this was driven by controls with high IU. Finally, the main effect of IU was not significant (p = 0.206). A similar analysis was performed by replacing group with gender to rule this out as a potential confound given that more patients were male, while more controls were female. The gender × IU interaction failed to reach significance, $F\left(1,55\right)=3.179,p=0.080,{\mathrm{\eta }}_p^2=0.055$. The main effects of gender and IU were also not significant (both p > 0.10).

Discussion

The results show that the training phase resulted in an immediate preference for the previously-rich room in controls but not in patients, who instead entered both rooms at approximately equal rates at the start of post-training (Fig. 2A). A possible explanation for this difference in preference is that chronic drug use in the OUD patient group may have altered reward thresholds and decreased reward sensitivity (Volkow et al., 2010). For example, chronic drug use downregulates dopamine receptors and dopamine production, providing a possible explanation for why addicted individuals have lower sensitivity to natural rewards, such as food, sex and water (Volkow et al., 2002). Thus, it is possible that the OUD patients had lower sensitivity or motivation for the rewards used in the current task. However, it is important to note that, across all participants, there was an increase in the time spent in the previously-rich room from pre-training to post-training, which suggests that both groups were sensitive to reward contingencies (Fig. 2B). Alternatively, since the poor room was always assigned to be the more-preferred room based on the pre-training phase, the tendency of patients to enter the poor room could also reflect preference to revisit a previously-preferred location, even if that location was not associated with reward.

Another possible explanation is that a different search strategy, such as foraging, was used by patients. Foraging involves choosing where and how to look for reward, including weighing a number of options associated with different reward contingencies (Platt & Huettel, 2008; Kolling et al., 2012). Typically, there is an element of risk or cost associated with leaving a location to search in a new location. This may include time and energy lost traveling to this location as well as having to assess the value of the possible rewards. Since there was little risk of time or energy loss in the current task, it is possible that patients may have more readily adopted a strategy of moving between rooms after obtaining reward (i.e., adopted a win-shift strategy). Drug addicts may spend most of their time foraging, especially under the control of stimuli that act as reinforcers (Belin et al., 2013). This is also consistent with an increase in reward seeking following expectancy violations, as previously reported in individuals with OUD (Myers et al., 2016). In the current task, this would have occurred during the post-training phase when the probability of reward decreased to pre-training levels. This behavior is not necessarily pathological, though it may not be optimal in situations where reward can only be found in a specific location.

However, in the current study, both patients and controls appeared to switch between rooms to a similar extent, and across participants, there was a decrease in this type of exploratory behavior from pre-training to post-training (Fig. 3A). This is likely the result of conditioning during the training phase, since both controls and patients showed an increase in the amount of time they spent in the rich room, from pre-training to post-training (Fig. 2B). Thus, the decrease in exploration is consistent with the observed increase in preference for the rich room. In the future, the task could be modified to examine preference when there is greater risk involved. For example, both reward and punishment learning could be assessed by introducing a chance to lose points when foraging in particular locations (e.g., high risk and high reward vs. low risk and low reward). This may affect preference and the strategy used by participants.

As expected, patients were also found to have greater IU than controls (Table 1), which is consistent with past research (Carleton, Norton & Asmundson, 2007; Nelson et al., 2016; Garami et al., 2017). However, neither of the two indicators of place preference (i.e., the first room entered and the room participants spent more time in during the post-training) was related to IU. This was contrary to our initial hypothesis that individuals, and in particular, controls with high IU should be more likely to enter the previously-rich room first during the post-training, as found in an earlier study using the same task (Radell et al., 2016). Instead, while more controls than patients did enter the rich room first, in the current study this was not related to IU. The prior study also found no increase in the time spent in the rich room from the pre-training to the post-training phase, again contrary to the current results where training increased the amount of time participants spent in the previously-rich room. Possible explanations for these discrepant findings include differences in the populations studied. The study of Radell et al. (2016) was conducted on college students who were considerably younger (mean age 20.7 years) and more highly educated than the controls in the current study. On the other hand, the patients in the current study had been subject to repetitive drug use over a number of years, and were currently on opioid maintenance, which may have impacted learning, motivation and decision-making.

Surprisingly, while IU was not related to preference, it was associated with overall performance in the task. Specifically, the results show that controls with high IU collected more eggs than both patients with high IU and controls with low IU (Fig. 4). This was largely due to searching more chests (Fig. 3B). In contrast, in a previous study with this task, there were no differences in total number of eggs collected by participants with high vs. low IU (Radell et al., 2016). There are a number of possible explanations for this tendency, including differences in motivation to acquire reward or in motor activity. It is also consistent with previous studies that have shown individuals with high IU require more information before making decisions under uncertain conditions (Ladouceur, Talbot & Dugas, 1997; Ladouceur et al., 1998). In this case, the decision may have been to continue searching or to give up or slow down, since it was not possible to leave the room during training. Thus, high IU controls may have continued to search in order to more clearly discern reward contingencies. If this was the case, however, similar results should have been obtained in high-IU patients as in high-IU controls. Instead, high-IU patients did not search as many chests and, therefore, did not collect as many eggs.

Although several studies have examined CPP in humans, to our knowledge, none have done so in a population undergoing treatment for OUD and using secondary reinforcers (e.g., points in a computer game). For example, in a study by Childs & De Wit (2009), humans received d-amphetamine or placebo in separate rooms, with the results indicating a preference for the drug-paired room (Childs & De Wit, 2009). In Molet, Billiet & Bardo (2013), a distinct virtual environment was paired with either pleasant music or static noise. Participants spent more time in the environment paired with pleasant music (Molet, Billiet & Bardo, 2013). Astur, Carew & Deaton (2014) also examined preference for two virtual rooms, one paired with chocolate, finding that participants spent more time in the chocolate-paired room, but only if they were food deprived (Astur, Carew & Deaton, 2014). Astur et al. (2016) assessed preference in a similar paradigm but with a secondary reinforcer (i.e., points) in young adults and found they spent more time in the points-paired room, and rated it as more enjoyable (Astur et al., 2016). Finally, Childs, Astur & De Wit (2017), assessed preference for a virtual room paired with a higher vs. lower monetary reward. The results showed a transient preference, both in terms of time spent in each room and subjective liking for the room with the higher reward, which was no longer apparent 24 hours later (Childs, Astur & De Wit, 2017). Similarly, in the current study, there was an increase in the time spent in a room with a higher probability of reward over one with a much lower probability, although this preference did not appear to differ between OUD patients and controls. Future studies could examine whether the type of reward used would modulate preference in addicted vs. non-addicted groups.

The current task differed from the typical non-human animal version of the CPP paradigm (for review, see Bardo & Bevins, 2000) in several ways. First, in the current task, it was not sufficient to simply be present in a particular room to obtain reinforcement, as in animal studies. Rather, participants were required to actively forage different locations for reward within the room. This may help increase the external validity of the task since reward seeking (e.g., for monetary, or other types of rewards) often involves active responding rather than simply going to and remaining at a particular location. Second, in animal tasks reward is usually delivered with 100% contingency in the rewarded room. In the current task, reward was probabilistic in that not every chest contained an egg. This was done to increase the task’s sensitivity to the potential effect of IU on behavior. Third, in animal studies, pre- and post-training tests are usually conducted in the absence of reward. In the current task, reward could be obtained in both the pre- and post-training phases, although the probability was low (5% for both the rich and poor rooms). This was to ensure participants remained motivated and to reduce frustration, although this opens the possibility that the overall preference for some participants was influenced by any rewards discovered. However, this effect is expected to be small given the relatively low chance of reward and limited time available during the post-training phase. Fourth, animal studies typically involve multiple conditioning sessions, spread over several days, with training and testing on separate days. The current study employed short, one-time 2 min training sessions, with a 4-min pre-training and post-training phase. Despite this, we still detected a significant increase in the time spent in the rich room from pre- to post-training (Fig. 2B) suggesting that training was successful, though it might be possible to amplify this effect by extending the duration of training.

Conclusions

In summary, while patients undergoing opioid maintenance treatment were confirmed to have higher IU than controls, no relationship was found between IU and preference for a room previously paired with a high probability of reward. However, IU did predict the total number of rewards obtained in the task—high IU controls outperformed low IU controls, as well as high and low IU patients. Controls, irrespective of IU, were more likely to enter the rich room than patients. This preference may reflect differences in motivation or learning about reward between the two groups, including changes in how contextual cues that predict secondary reinforcers are processed. It may also, at least in part, reflect acute opioid effects, since patients had received opioid treatment just prior to participating in the study. Future research should examine these possibilities further since understanding the factors that govern decision-making and learning processes in addicted individuals can contribute to developing or improving current treatment options, including the prevention of relapse.

Supplemental Information