Associations between aerobic exercise and dopamine-related reward-processing: Informing a model of human exercise engagement

Endurance or aerobic exercise has many physical and mental health benefits, but less is known about the specific impact that cardiovascular activity may have on dopamine-associated brain circuits involved in reward processing and mood regulation in humans. Understanding such effects will help to explain individual differences in both exercise uptake and maintenance. This study evaluated neural response to a classical taste-conditioning reward prediction error task with the use of functional magnetic resonance imaging, along with data on self- reported aerobic exercise among healthy young adult females ( N = 111). Results indicated positive associations between reported aerobic exercise and regional brain response that remained significant after multiple comparison correction for the right medial orbital frontal cortex response to unexpected sucrose receipt ( r = 0.315, p = .0008). The medial orbitofrontal cortex is implicated in reward and outcome value computation and the results suggest that aerobic exercise may strengthen this circuitry, or reciprocally, higher orbitofrontal cortical activity may reinforce exercise behavior. The findings aid in developing a model of how exercise engagement can modify reward-circuit function and could be used therapeutically in conditions associated with altered brain salience response.


Introduction
Exercise has many well-documented benefits, including offsetting a variety of chronic diseases (Pedersen & Saltin, 2015), improving mood and cognitive function (Basso & Suzuki, 2017), and in helping to manage mental health disorders (Smith & Merwin, 2021). Given that the long-term adoption of adaptive exercise behavior holds great potential for global public health (WHO, 2020), a deeper understanding of biobehavioral mechanisms that modulate and support exercise engagement is of great interest and utility.
Just over a decade ago, it was proposed that the means by which humans might be motivated to exercise was by way of the dopamine system (Knab & Lightfoot, 2010). Animal models support the idea that dopaminergic signaling regulates engagement in physical activity (Beeler & Burghardt, 2021;Foldi et al., 2017), but to date, we know much less about the impact of naturalistic aerobic exercise on brain activation, and how this potentially dopamine-mediated process might serve to reinforce human exercise behavior. A robust association between exercise engagement and activation in reward-related brain regions would suggest a population for whom engaging in exercise might be more motivating, and therefore easier to initiate and maintain. It is also possible that engaging in regular aerobic exercise brings about alterations in neural reward response that reflexively reinforce this behavior, which could explain why some individuals exercise in a maladaptive, excessive manner (Cunningham et al., 2016). Taken together, improved understanding of the neural mechanisms that may motivate and maintain exercise engagement holds potential for promoting the adoption of sustained and adaptive exercise behavior across both healthy and clinical samples.
Furthermore, targeted exercise could be important to manipulate or modulate dopamine-related neuronal activation as dopamine signaling helps to generate learning (Graybiel & Grafton, 2015) and promote approach behaviors (Wise, 2004). A majority of this dopamine-associated learning is thought to involve the mesolimbic pathway, implicating brain regions associated with goal-directed decision-making and the ability to maintain flexible responding based on the value of a given reward, or "reinforcer" (Gourley et al., 2016). Activation of this pathway provides the foundation for behavior initiation as well as reinforcing behavior once it has been initiated, and for generating conditioned responses (Schultz, 2016;Wise, 2004). We can explain these processes in part through examining reward prediction error (RPE) (Watabe-Uchida et al., 2017), a dopamine-associated signal that is generated when evaluating the difference between an expectation and an outcome (Schultz, 2016). The absolute value of the RPE represents the extent to which a deviation from what was expected was surprising, and it is conceptualized as a reflection of motivational salience (Fouragnan et al., 2017). Expectation and outcome can also be analyzed separately, providing information on brain response to unexpected receipt or omission of a stimulus. Whether exercise behavior is related primarily to receipt or omission brain response is not known.
In summary, there may be reciprocal effects between aerobic exercise and dopamine-related brain reward processing. To examine these possibilities, the current study conducted secondary data analysis to evaluate neural response to a classic RPE task with the use of functional magnetic resonance imaging (fMRI), along with data on self-reported aerobic exercise among healthy young adult females. In the original study that specifically sought to examine RPE among individuals with transdiagnostic eating disorders, brain salience response was inversely correlated with body mass index and binge-eating behavior, and positively correlated with ventral-striatal hypothalamic effective connectivity (Frank et al., 2021). Those results suggested that food restriction and overeating may alter brain circuitry in opposite directions and reinforce an individual's eating disorder behavior.
In the current study, we sought to examine the potential reciprocal nature of dopamine-related reward response and specifically exercise behavior, among healthy controls. Given prior work both in rodents (Beeler & Burghardt, 2021) as well as humans (Flack et al., 2021) that implicates increased activity in the dopamine system relative to physical activity, we hypothesized that we would find indication for greater salience response in the dopamine system relative to increased report of exercise. Identifying differences in neural activation between those who exercise more often compared to those who do not will help to develop a model of how exercise engagement can modify dopamine function and could be used therapeutically in conditions associated with altered brain salience response.

Participants and procedures
The current study comprises data from healthy young adult females (N = 111), drawn from a larger study (c.f., Frank et al., 2021). Participants were right-handed without history of head trauma, neurological disease, or other major medical illness; they were without history of any lifetime psychiatric disorder and were studied during the first 10 days of the menstrual cycle to reduce hormonal confounds. Psychiatric diagnoses were excluded using the Structured Clinical Interview for DSM-5 (doctoral-level interviewer) (First et al., 2015).
All procedural details are available elsewhere (Frank et al., 2021). In brief, all subjects participated in a classic sucrose taste-conditioning paradigm to evoke the dopamine-related RPE response. We asked participants to report their weekly minutes of endurance or aerobic exercise activities (Plowman & Smith, 2014). Those exercise behaviors had to be stable for at least three months. The activities were further defined as those that increase breathing and heart rate and are usually associated with sweating; examples were provided such as running, cycling, cardio exercises on devices such as elliptical or treadmill machines.
The Colorado Multiple Institutional Review Board approved the study. All participants provided written informed consent.

Functional magnetic resonance imaging (fMRI
Between 0700 and 0900 h, participants ate a provided breakfast (see Frank et al., 2021 for detail). FMRI of the brain was performed between 0800 and 0900 h (3 T GE Signa or Siemens Skyra 3 T scanner).

Taste reward task
The design was adapted from (O'Doherty et al., 2003). Participants learned to associate three unconditioned taste stimuli (US: 1 molar [M] sucrose solution [100 trials], no solution [100 trials], or artificial saliva [80 trials]) with paired conditioned visual stimuli (CS) during scanning (total task duration = 28 min). Each CS was probabilistically associated with its US such that 20% of sucrose and no solution CS trials were unexpectedly followed by no solution or sucrose US, respectively.

fMRI analysis
Image preprocessing and analysis were performed using SPM12 (http://www.fil.ion.ucl.ac.uk/spm/software/spm12/). Images were realigned to the first volume, normalized to the Montreal Neurological Institute template, and smoothed at 6 mm full-width-at-half-maximum Gaussian kernel. Subjects with head motion greater than one voxel were removed from the analysis. Data were preprocessed with slice-time correction and modeled with a hemodynamic response convolved function using the general linear model, including temporal and dispersion derivatives. A 128-second high-pass filter (removing lowfrequency BOLD signal fluctuations), motion parameters (as first-level analysis regressors), and SPM's FAST (pre-whitening attenuation of autocorrelation effects) were applied (Olszowy et al., 2019).

Prediction error analysis
Each participant's prediction error signal was modeled based on trial sequence and regressed with brain activation across all trials (DeGuzman et al., 2017;Frank et al., 2012;O'Doherty et al., 2003). The predicted value (V) at any time (t) within a trial is calculated as a linear product of weights (w i ) and the presence of a conditioned visual stimulus (CS) at time t, coded in a stimulus representation vector x i (t) where each stimulus x i is represented separately at each moment in time: Predicted stimulus value at time t is updated by comparing the predicted value at time t + 1 to that actually observed at time t, leading to the prediction error δ(t): where r(t) is the reward at time t. The parameter γ is a discount factor, which determines the extent to which rewards arriving sooner are more important than rewards that arrive later during the task, with γ = 0.99. The weights w i relate to how likely a particular unconditioned reward stimulus (US) follows the associated CS and are updated on each trial according to the correlation between prediction error and the stimulus representation: where α is a learning rate. A slow α = 0.2 was applied. Initial reward values were 1 for Sucrose Receipt and 0 for No Sucrose. Trial-to-trial prediction error was regressed with brain activation across all trials within each subject. The prediction error calculated for each trial was modeled as an absolute (reflecting degree of deviation of the outcome from the expectation) without separating positive or negative prediction error trials. Model prediction error values were then regressed against the fMRI data for each individual subject, to identify brain regions correlating with the model-predicted time series (O'Doherty et al., 2007).

Group-by-condition analysis
We developed first-level models to predict the response in each voxel as a function of each of five stimulus conditions: expected sucrose, unexpected sucrose, expected no-solution, unexpected no-solution, and expected artificial saliva. Three contrasts of interest were computed per subject: (1) unexpected sucrose receipt: trials with CS for no-solution followed by unexpected US sucrose contrasted against trials with CS for no-solution, followed by expected no-solution; (2) unexpected sucrose omission: trials with CS for sucrose solution followed by unexpected US no-solution contrasted against trials with CS for sucrose solution, followed by expected sucrose solution; (3) expected sucrose receipt: trials with CS for sucrose solution followed by expected US sucrose contrasted against trials with CS for artificial saliva solution followed by expected US artificial saliva.

Statistical analysis
Data were tested for normality (Shapiro-Wilk test) and ranked and normalized using the Rankit procedure if they were non-normally distributed (Soloman & Sawilowsky, 2009). Pearson correlations and partial correlations were used to test associations between behavior and brain activation and results were multiple comparisons, controlled using a False Discovery Rate (Benjamini & Hochberg, 1995). We tested partial correlations among multiple ROIs with exercise in a temporal difference (RPE) model, as well as in conditions specific to unexpected sucrose receipt (RSUU) and unexpected sucrose omission (RNOU). The original sample was evaluated using two scanners (see Frank et al., 2021); a scanner variable, BMI, and age were controlled for in the partial correlation analyses. We used a False Discovery Rate correction to adjust for multiple comparisons. SPSS 27 software was used for statistical analyses (IBM, Armonk, N.Y.).

Results
Participants (100% female) had mean age (SD) = 25.28 (5.05), mean BMI (SD) = 21.40 (SD = 1.63) and reported aerobic exercise an average of 159 min (SD = 164) per week. Controlling for scanner, age, and BMI, significant positive associations were evidenced between minutes of aerobic exercise and prediction error response (bilateral posterior insula, right medial OFC, left nucleus accumbens, left ventral striatum), activation to unexpected sucrose stimulus receipt (bilateral medial OFC, left nucleus accumbens, left ventral striatum) and activation to unexpected stimulus omission (right medial OFC). However, after adjustment for multiple comparisons, only the correlation with right medial OFC response to unexpected stimulus receipt remained significant (Table 1 and Fig. 1).

Discussion
The current study sought to identify potential reciprocal effects between exercise and dopamine-related brain reward processing. Amount of aerobic exercise was significantly positively correlated with right medial OFC response across all three reward conditions tested but remained significant after multiple comparison correction only for the unexpected stimulus receipt condition.
The current study suggests that engagement in aerobic exercise is associated with heightened motivational salience response in the right medial OFC during unexpected receipt of reward. Whether higher exercise drives higher brain response or whether greater medial orbitofrontal brain response facilitates higher engagement in exercise cannot be determined from this study. The right medial OFC is specifically associated with goal-directed decision making, it is implicated in reward and outcome value computation, and aids in regulating sensitivity to the value of a given outcome (Gourley et al., 2016). It is therefore possible that individuals who engage in more aerobic activity may be intrinsically more responsive to salient stimuli and especially stimulus receipt, or alternatively, engagement in aerobic exercise has modulated brain activity and dopamine signaling, which may then reflexively reinforce and functionally maintain the exercise behavior. These two possibilities may each be true, and be additionally related to unique individual-level factors (e.g., temperamental traits) that increase the likelihood that a regular and adaptive program of exercise is upheld (Laborde et al., 2020). Note: N = 111. ROI = region of interest; RPE = reward prediction error; RSUU = unexpected sucrose receipt; RNOU = unexpected sucrose omission. Animal studies suggest that antagonizing the dopamine system via dopamine D2/D3 receptor blockers reduces physical activity, implicating the dopamine circuitry (Hillebrand et al., 2005;Klenotich et al., 2015;Verhagen et al., 2009). A recent study in mice demonstrated that genetic knockdown of dopamine transporters increased wheel running for some, but not all animals, suggesting individual variability in this process (Beeler & Burghardt, 2021). In humans, propensity for physical activity is variable and heritable (de Geus et al., 2014;Flack et al., 2019;Herring et al., 2014;Klimentidis et al., 2018) and in addition to modulating the pleasure and reward system (Matta Mello Portugal et al., 2013), physical activity modulates major neurotransmitters (Matta Mello Portugal et al., 2013). Contrary to our hypothesis, we did not find the strongest aerobic exercise correlations with the RPE contrast, but rather for the unexpected receipt condition, suggesting that it is in particular the unexpected receipt or possibly the better than expected outcome condition brain response that is associated with aerobic exercise. This has not been described before to our knowledge. It is possible that exercise may in particular enhance the ability to value or enjoy stimuli or experiences, which could be important for intervening on psychiatric disorders. Altered brain salience response is characteristic of many psychiatric illnesses (e.g., depression; (Heshmati & Russo, 2015)), for which exercise has been proposed as generally effective in managing these disorders (Smith & Merwin, 2021). For example, some adults with generalized anxiety disorder demonstrate deficits in reinforcement-based decision-making and reduced RPE (White et al., 2017). For these individuals, determining if an exercise-induced improvement in motivational salience mediates response to standard psychotherapy treatment may serve to inform future treatment adaptations.

Limitations
Although longitudinal work with a more sophisticated analytic approach is needed to confirm the directionality of associations, our findings offer an important foundation for a model of understanding the therapeutic potential of endurance or aerobic exercise in intervening on or enhancing salience response. However, less vigorous activity has been implicated in improving depression (Morres et al., 2019), and the impact of other non-aerobic activity on brain salience response also warrants examination. Exercise was self-reported, and findings from the current study may reflect general activity level more so than effects of aerobic exercise itself. Further, some data on self-reported activity in healthy controls suggests that vigorous activity can be over-reported (Tomaz et al., 2016); future work might include objective measures of activity. Given possible confounds resulting from neuro-modulatory factors that are associated with anaerobic exercise (de Sousa et al., 2020), future work might include measurement of anaerobic exercise as well. Exercise has demonstrated association with the modulation of a variety of neurotransmitters (e.g., serotonin, norepinephrine) and neurohormones (e. g., brain derived neurotrophic factor) (Brellenthin et al., 2017;Heijnen et al., 2016;Szuhany et al., 2015). Therefore, while the RPE task is a measure of reward salience (Fouragnan et al., 2017), it is possible that results may reflect less specificity to the dopamine system, and instead, be a reflection of other biomarker activity in the studied ROIs. Of note, we focused our investigation on the study of healthy female controls; while this approach adds benefit in improving the generalization of our findings, our results do not specifically inform understanding across gender, or of addictive (Cook et al., 2014) or compulsive (Meyer et al., 2011) exercise.

Conclusions
In summary, our findings support the potential that aerobic exercise intervenes on reward-based processing such that there are reciprocal effects between exercise and possibly dopamine-and other neurotransmitter related brain activity in the medial OFC. While our study is cross-sectional, it lays a preliminary foundation in developing a model of adaptive exercise engagement and how it might modify reward response, and could be considered in intervening therapeutically in the OFC in psychiatric illnesses that include altered brain salience response.

Funding statement
The study was supported by National Institute of Mental Health grants MH096777 and MH103436 (Dr. Frank). Dr. Gorrell is supported by the National Institute of Mental Health (K23MH126201).