Placebo and nocebo effects and operant pain-related avoidance learning

Supplemental Digital Content is Available in the Text.


Introduction
Placebo effects occur when treatment-related cues are presented with an inert medical treatment, resulting in an improved clinical state. Conversely, nocebo effects occur when these cues result in a worsened clinical state. 5 Research on placebo and nocebo mechanisms in pain has convincingly shown that positive or negative expectations can modulate self-reported pain intensity. 6,29,34 Furthermore, classical or Pavlovian conditioning-in which visual or somatosensory cues are paired with an outcome to elicit a conditioned response when presenting the cue in absence of the outcome-has been used to elicit pain modulation, both in combination with direct instruction of treatment expectations, 7,34 as well as without any co-occurring information, 21 suggesting a key role for learning in pain modulation.
Learning also is essential in the development and exacerbation of pain-related behavior. Previous research has demonstrated that after pairing a neutral cue with a painful stimulus, this initially neutral stimulus may start to elicit conditioned fear responses. 24,25 Furthermore, operant procedures-in which voluntary behavior is shaped based on its consequences-have been used to study avoidance behaviors, their role in the maintenance of fear of movement-related pain, 23 and their association with pain intensity and chronic pain. 35 However, little is known as to whether operant pain-related behaviors also modulate pain. 16,36 In the current study, we therefore adapted an existing operant learning task 23 to investigate the effect of operant pain-related behavior on placebo and nocebo effects. In this task, participants freely perform different reaching movements and can avoid pain by performing more effortful movements. Our adaptation consisted of manipulating the intensity of the pain stimulus (using a high-, medium-, or low-intensity stimulus) contingent upon movement effort. In a within-subject design, in 1 context (similar to the original task), participants could avoid the high-intensity pain stimulus by choosing movement trajectories that needed more effort, creating a high cost of avoidance. In the other context, movement trajectories remained the same, but the trajectory-pain contingencies were reversed, creating low cost of avoidance. During the test phase, all trajectories were paired with the same medium-intensity pain stimulus, analogous to classical conditioning studies investigating placebo and nocebo effects. As participants are still free to choose which movement to perform, the movements retain function as operant behaviors, but are no longer differentially reinforced.
We hypothesized that within each context, participants would (1) acquire lower pain expectations for trajectories paired with low vs medium vs high pain stimulation and (2) show a similar preference for performing trajectories paired with low vs medium vs high pain stimulation. Most importantly, we expected that during test, participants would (3) maintain these differential expectations and preferences when movements were no longer differentially reinforced, (4) show lower reported pain intensity for trajectories that were previously associated with low-intensity compared with medium-intensity stimuli (placebo analgesia), as well as higher reported pain intensity for trajectories that were previously associated with high-intensity compared with medium-intensity stimuli (nocebo hyperalgesia).

Participants
Sixty participants were recruited using social media and the online departmental experiment management system. Data of 2 other participants were excluded because of equipment failure. The final sample included 58 participants (age: 21 6 6; 38 female; 49 right handed).
Exclusion criteria were as follows: history of chronic pain, previous or current acute pain in the dominant hand, wrist, elbow, or shoulder, pregnancy, history of cardiovascular disease or respiratory disease, having a pacemaker or other implanted medical device, neurological diseases or psychiatric disorders, uncorrected hearing, or vision problems. This led to exclusion of 1 participant before the experiment. The experimental protocol was approved by the Social and Societal Ethics Committee of the KU Leuven (approval number: G-2015 12 420). Participants received course credit or a financial compensation of €8 for their participation.

HapticMaster
The HapticMaster (HM) is a 3 degrees of freedom, forcecontrolled robotic arm (Motekforce Link, Amsterdam, the Netherlands). The HM can be programmed to exert different forces depending on the direction in which it is moved. During the movement executed with the robotic arm, the position of the HM is recorded, enabling the use of trajectory information as a dependent variable, as well as using HM output as input for the administration of pain stimuli. The visual interface and experimental tasks were programmed in C/C11, using the Microsoft Integrated Development Environment (IDE) Visual Studio (Microsoft Corporation Redmond, WA) and the development platforms OpenGL for graphical support and HM API (Application Programming Interface), connecting to Affect 4.0 software for administration of the pain stimuli.

Painful stimulation
The pain stimulus was delivered by a commercial stimulator (DS5; Digitimer, Welwyn Garden City, United Kingdom). Each stimulus was a 100-Hz square wave pulse lasting for 500 ms. The stimulus was delivered using 2 SensorMedics electrodes (1-cm diameter, SensorMedics, Corp., Anaheim, CA), filled with K-Y gel (Johnson & Johnson, New Brunswick, NJ), attached to the triceps tendon of the dominant arm. The maximal intensity of the painful stimulus was set individually for each participant using a single staircase calibration. During calibration, participants rated each stimulus intensity on a labeled numeric rating scale (NRS) with anchors at 0 (you feel nothing) to 10 (maximal tolerable pain). The stimulus intensity was increased to tolerance levels, until participants rated the stimulus as "painful and demanding some effort to tolerate," corresponding to a rating of 8 on the NRS. Calibration resulted in a mean stimulus intensity of 3.71 mA (SD 5 1.81, range 5 1.3-10 mA), which was used as the high-intensity pain stimulus throughout the experiment. Two other pain stimulus intensities were used, which were also tailored for each participant: a lowintensity pain stimulus and a medium-intensity pain stimulus (respectively, 36% and 60% of the calibrated stimulus intensity).

Protocol
The experiment was conducted during a 1-hour laboratory session, consisting of a familiarization phase and 3 experimental phases. In each experimental phase, participants performed a robotic arm reaching task. On arrival to the laboratory, participants received written and oral information about the experiment and were informed that the experiment involved repeated painful electrocutaneous stimulations. Participants were free to withdraw participation at any time without any negative consequences. After providing informed consent, the electrodes for administration of the painful stimulation were attached.

Robotic arm reaching task
Participants sat in front of the HM, with the sensor of the HM positioned in front of them at shoulder height. The reaching task consisted of moving the sensor in a horizontal movement plane (0.35-m depth 3 1-m radius) with the dominant hand. Real-time visual feedback of their movement was provided on a 46-inch LCD screen (36PFL3208K/12; Koninklijke Philips N.V., Amsterdam, the Netherlands). A trial counter (the number of trials remaining) was displayed at the top of the screen. Furthermore, 3 trajectory arches were positioned in the middle of the movement plane, and a target arch was placed in the upper left corner of the movement plane. From the start position, the target arch could be reached through any of these arches resulting in 3 potential movement trajectories (T1-T3) (Fig. 1).
The HM was programmed such that the tractive force exerted by the participant varied with the increasing lateral displacement of the robotic arm sensor. This resulted in lower task effort for performing T1 (shortest trajectory), compared with T2 (intermediate trajectory), and T3 (longest trajectory). All participants performed movements in 2 different experimental contexts, indicated by the use of squared vs rounded arches (Fig. 1).
Visual cues further signaled the start of a trial (target arch and sensor representation changing color to green), and end of a trial (target arch color changing to red), see Figure 1, and were accompanied by a brief tone. At the end of the trial, participants released the sensor and the HM repositioned to its starting location. This was followed by an intertrial interval of 3 seconds during which the robotic arm remained fixed.

Familiarization
The experiment started with participants being familiarized with the HM device, the experimental contexts, and the 3 possible movement trajectories, as well as practice rating pain expectation and pain intensity using a 3-pedal foot switch. During the familiarization phase, participants performed 10 movements. Assignment of context was randomized for each trial. During familiarization, no painful stimulation was administered. Participants were encouraged to explore the different possible trajectories (T1-T3) and were exposed to the differences in tractive force. At the start of the familiarization phase, the trial counter was set at 10 and counted down when the target arch was reached.

Acquisition
The goal of the acquisition phase was to establish the different movement-pain contingencies in both contexts. Participants were free to choose which movement to perform to reach the target location. During movements, participants received electrocutaneous stimulation on each trial, when passing through one of the trajectory arches ( Fig. 1). In the high cost of avoidance context, T1 (shortest and easiest movement) was followed by the high-intensity stimulus, and T3 (longest and most effortful movement) was followed by the low-intensity stimulus. In the low cost of avoidance context, T1 was followed by the low-intensity pain stimulus, and T3 was followed by the high-intensity pain stimulus. In both contexts, T2 was followed by the mediumintensity pain stimulus (Fig. 1).
Participants performed 48 trials: 4 blocks of 6 trials in both contexts. The presentation order of the blocks was semirandomized, with the restriction that both contexts occurred randomly within each sequence of 2 blocks. Each block started with a rating of pain expectation for T1-T3, and participants rated the experienced pain intensity after each trial. After pain ratings, the trial counter decreased by 1 unit.

Voluntary movement test
The procedure during the voluntary movement test phase was similar to the acquisition phase (4 blocks of 6 trials in both contexts). However, different trajectories were no longer followed by pain stimuli of different intensities, but always followed by the medium-intensity stimulus. Participants again were able to choose which movement trajectory they wanted to perform to reach the target location. To minimize discontinuity between acquisition phase and the voluntary movement test phase, the trial counter did not reset in between phases and reached 0 at the end of the voluntary movement test phase.

Instructed movement test
In a final exploratory phase, participants could no longer choose which movement to perform. They were prompted to perform specific movement trajectories, by highlighting 1 trajectory arch at the start of the trial (Fig. 1). This phase consisted of 24 trials, presented in 4 blocks. Within each block, each trajectory was completed twice in each context, and presentation of context and trajectory cues was randomized. At the start of each block, participants provided pain expectations for the trajectories in both of the contexts. Similar to the voluntary movement test phase, each trajectory was followed by the medium-intensity stimulus and ended with ratings of pain intensity.

Ending the session
At the end of the instructed movement phase, participants were debriefed and were informed about the contingencies between the trajectories and pain stimuli in the different contexts and phases.

Postexperimental questionnaires
Trait questionnaires were filled out the next day using an online survey application (LimeSurvey; LimeSurvey GmbH, Hamburg, Germany), to limit spillover effects from the experiment. Participants filled out a set of trait questionnaires, including the Positive and Negative Affect Schedule (PANAS), 10,38 the Fear of Pain Questionnaire (FPQ), 22,30 and the Pain Catastrophizing Scale (PCS). 31,33 Inclusion of these questionnaires was chosen based on their associations with symptom reports and pain ratings. 8

Outcome measures
Pain expectations and pain intensity were assessed using the same NRS that was used during pain calibration, anchored at 0 (you feel nothing) to 10 (maximal tolerable pain). The scale also provides anchors for 1 (you feel something, but this is not painful-merely a sensation), 2 (the sensation starts being painful, but this is a very moderate pain), and 8 (painful and demanding some effort to tolerate). For expectation ratings, participants were asked to "Please rate the pain you expect when moving through the highlighted arch" (Fig. 1). For pain intensity ratings, participants were asked to "Please rate the pain you experienced during the previous movement." Ratings were given using a 3pedal foot switch (USB-3FS-2; Scythe, Tokyo, Japan), which was chosen to reduce interference with the arm reaching movements. Movement trajectory choice was detected during each trial, based on the lateral position of the HM at the moment of administration of the electrocutaneous stimulus.

Statistical analysis overview
To present similar analyses for the pain expectation, pain intensity, and choice data, we averaged pain expectations and pain intensity data and summed movement choices for each trajectory within a block. Although all participants performed the same number of trials for each block, the operant learning procedure resulted in individual participants performing a variable number of the different trajectories in each block (range for the different trajectories being [0-6]). As repeated-measures analysis of variance is unable to deal with this type of data, we chose to average trajectory data within the same block and analyze pain expectation and intensity ratings using multilevel analysis (linear mixed-model analysis). Movement choices were analyzed using generalized (Poisson) linear mixed models. We modelled a random intercept to account for within-subject clustering and fixed effects for all other factors. Significance threshold was set at P , 0.05, using Satterthwaite approximation for degrees of freedom, and applied Bonferroni corrections when exploring differences in estimated marginal means. Different analyses were performed separately for each of the different experimental phases (acquisition, voluntary movement test, and instructed movement test) and outcome variables (pain expectation, pain intensity, and movement choice), and were based on models that included fixed effects of context (high vs low cost), trajectory (T1-T3), and block (1-4) (We also performed secondary analyses that included [1] main effects and interaction effects of counterbalanced context assignment [high cost-squared vs high costrounded], and [2] analyses that were restricted to results of righthanded individuals [n 5 49], to account for potential effects of handedness on the reaching tasks. Neither of the alternative analyses changed the interpretation of the results). Confirmation of hypotheses was dependent on a significant context 3 trajectory interaction, which was explored further to test whether differences matched the contingencies that were presented during the acquisition phase. All analyses were performed using SPSS 24 (IBM, Corp., Armonk, NY).

Instructed movement test
Analysis of pain intensity ratings during the instructed movement test did not result in any significant main effects or interaction effects (all P-values .0.33).
Block level results suggested that these preferences were present early on during acquisition and did not change across blocks, see Figure S3 (available at http://links.lww.com/PR9/ A45), context 3 block 3 trajectory interaction (F(6, 1368) 5 1.243, P 5 0.281). There were no other significant main effects or interaction effects (all P-values .0.49).

Discussion
In this study, we investigated the acquisition of movement choices, pain expectations, and pain modulation in response to movement-pain contingencies. Participants performed an operant learning task in 2 contexts. In 1 context, avoidance responses were associated with high cost, while in the other, context avoidance was less costly. Similar to other placebo/nocebo classical conditioning procedures, participants were subjected to contingencies between specific actions and changes in pain intensity. In addition, similar to other operant learning procedures, participants were able to change actions in response to the experienced contingencies. Irrespective of context, the results revealed that pain expectations and movement choices can be readily acquired through operant learning procedures. These findings are in line with current models of pain-related disability, which posit that individuals learn to avoid situations that are expected to cause pain or harm, 35 as well as recent findings showing that operant learning tasks can be used to model painrelated fear and avoidance behavior. 23 Similar to other placebo/ nocebo experiments, acquired pain expectations do not fully match experienced pain intensities. 4 Furthermore, participants do not fully avoid painful movements, suggesting continued exploration of movement-pain contingencies. 18 During the test phase, participants reported different pain intensities across contexts in the absence of differential reinforcement, although this was limited to the most effortful movement (T3). Furthermore, within-context patterns of pain modulation only matched acquired expectations in the low cost context, in which we only observed a placebo effect, but no nocebo effect. Nevertheless, the resulting pain modulation was similar in magnitude to placebo effects, resulting from classical conditioning procedures without co-occurring instruction, 21 and shows that placebo pain modulation extends to information that has been acquired through an operant learning procedure. Other studies have shown pain modulation using different operant procedures, including positive or negative verbal reinforcement, 14,19,20 as well as monetary rewards and punishments, 11 but only a limited number of studies have shown that self-reported pain can be changed using pain as a reinforcer. Exceptions include studies by Becker et al., 1,2 who have shown that operant conditioning using temporary pain increases/decreases can be used to elicit habituation and sensitization of pain. Contrary to these studies, we did not reinforce or punish verbal pain behavior, but pain-related movements, and investigated the impact of these movements on pain intensity ratings. This is procedurally similar to a recent study by Jung et al.,15 in which operant learning (choice between 2 treatments of different efficacy) elicited placebo effects in a treatment context (compared with instructed no treatment trials), but no differences in pain reduction were observed between the placebo treatments.
Our study complements these findings by showing pain modulation in the context of instrumental avoidance behavior, thereby indicating that movements can acquire pain-modulating properties due to previous operant learning. We found that after being followed by pain reduction in an operant learning task, performing these movements elicited pain reduction. The voluntary nature of these movements differentiates our task from classical conditioning procedures that have previously been applied in placebo research. Although the mechanisms of operant learning procedures deserve further investigation, 9 these findings provide a procedural analog to real-life operant learning in the context of pain. The focus on voluntary movements is especially relevant for individuals suffering from pain, who may avoid movements that elicit pain or alter behavior to seek safety when experiencing pain. 32 Furthermore, recent findings show a feedback loop between expectations and subsequent (placebo or nocebo) pain effects, 13 suggesting a potential pathway for the development and maintenance of pain-related safety behaviors.
The large number of operant test trials enabled us to investigate the persistence of pain expectations, pain and movement choice modulations despite changed contingencies. Interestingly, pain expectations and pain intensity reports showed stronger persistence compared with movement choice, which quickly responded to a lack of reinforcement after acquisition. The quick change in movement choices is unexpected, as avoidance behaviors are quite resistant to extinction. 17,26 Furthermore, previous research showed much slower changes in avoidance behaviors during extinction of pain-related fear using a similar experimental setup. 23 Differences in the research design-the use of a between-participant (yoked) control group instead of within-participant controls-may partly explain these observed differences (see further discussion of limitation).
The fear-avoidance model of chronic pain proposes that persistence of expectations and pain modulation are maintained by continued avoidance behavior. 16,37 Our findings do not fit this proposed pathway and suggest that pain expectations and pain modulation can persist in the absence of avoidance behavior. This persistence is consistent with a lack of extinction of pain modulation and expectations observed in other placebo and nocebo studies 4,7 and could contribute to the experienced burden of acute or chronic pain. Furthermore, our findings suggest that exposure treatment for chronic pain-aimed at reducing fear of pain and pain-related avoidance behavior-may be hampered by persisting pain modulation during exposure to these feared behaviors.
During the instructed movement test phase, we no longer observed differential pain expectations or pain modulation across context, suggesting extinction of these responses. The differences between both test phases (voluntary vs instructed movements; blocked vs random presentation of context cues) hinders the interpretation of this finding.
Unexpectedly, and of interest, is that the cost of avoidance behavior moderated the magnitude of differences in expectations and pain modulation. During the voluntary movement test, we observed evidence for both placebo and nocebo expectations in the low cost context, whereas these effects were weaker in the high cost context. The pattern of expectations was matched with similar effects on pain modulation in the low cost context, but not for the high cost context. Context differences in choice behavior during acquisition suggest that movement trajectory choices not only depended on movement-pain contingencies but also reflected their relative cost. The resulting differences in experience with movement-pain contingencies may have led to a difference in the precision of pain expectations, which in turn may have led to differences in pain modulation. 3 However, these findings may also be the result of a confound induced by differences in the physical properties of the arm reaching movements. Alternatively, more difficult movements may have had different impact on endogenous pain modulation or may have distracted participants from pain. 27,28 These differences may have led participants to observe contingencies between the reaching movements and pain intensities differently. For example, participants may have extracted a rule, such as "more difficult movements lead to more pain." 39 Taken together, this suggests that in the low cost context, experiences during acquisition and voluntary movement test may have been more similar and consistent across different features, which has been suggested to increase the probability of a placebo effect. 12 Finally, differences in movement choices during the voluntary movement test could have contributed to the observed differences in expectations and pain modulation. In this phase, movement choices were similar for both contexts, with a stronger preference for less effortful (T1) compared with more effortful (T3) movements. These differences could result in different levels of exposure to the novel pain-movement contingencies, which would explain why for the most chosen movement (T1) pain intensity ratings did not differ across contexts, whereas pain intensity ratings still differed for the least chosen movement (T3).
By varying movement-pain contingencies across contexts in a within-subject design, we were able to control for physical properties of the arm reaching movements across context. However, our design holds a risk of carryover effects or even may have confused participants, who may have based their ratings on experimental cues (eg, the specific movement trajectory or associated force), without incorporating information about the experimental context. Moreover, the exposure to different outcomes for the same movement trajectory during acquisition may already have introduced participants to the idea that outcomes for a specific movement trajectory can differ or change throughout the experiment, which may explain some of our null results during the voluntary movement test phase.
A further limitation of this study is that the magnitude of pain modulation was relatively small. Although the observed magnitude is within range of studies using classical conditioning without explicit placebo/nocebo instruction, 21 it is uncertain whether these effects could contribute to meaningful differences in the daily life of individuals suffering from pain. A final limitation is that pain expectation was only assessed at the start of each block, whereas pain intensity and movement choices were measured on each trial. This limits drawing conclusions on changes in expectation that may have occurred within a block as well as make strong inferences about the interdependence of these different pain-related outcomes.
Despite these limitations, this study shows that while performing an operant learning task, participants learn to expect and avoid pain. During test phases, changes in reinforcement quickly changed pain-related behaviors. Changes in pain expectations were maintained after disconfirmation and co-occurred with pain modulation, suggesting difficulties in changing pain-related expectations and their lasting impact on pain.