Figures
Abstract
Many of the decisions and actions in everyday life result from implicit learning processes. Important to psychopathology are, for example, implicit reward-seeking and punishment-avoidant learning processes. It is known that when specific actions get associated with a rewarding experience, such as positive emotions, that this will increase the likelihood that an organism will engage in similar actions in the future. Similarly, when actions get associated with punishing experiences, such as negative emotions, this may reduce the likelihood that the organism will engage in similar actions in the future. This study examines whether we can observe these implicit processes prospectively in the flow of daily life. If such processes take place then we expect that current behaviour can be predicted by how similar behaviour was experienced (in terms of positive and negative affect) at previous measurement moments. This was examined in a sample of 621 female individuals that had participated in an Experience Sampling data collection. Measures of affect and behaviour were collected at 10 semi-random moments of the day for 5 consecutive days. It was examined whether affective experience that was paired with certain behaviours (physical activity and social context) at previous measurements modified the likelihood to show similar behaviours at next measurement moments. Analyses were performed both at the level of observations (a time scale with units of ± 90 min) and at day level (a time scale with units of 24 h). As expected, we found that affect indeed moderated the extent to which previous behaviour predicted similar behaviour later in time, at both beep- and day-level. This study showed that it is feasible to track reward-seeking and punishment-avoidant behaviour prospectively in humans in the flow of daily life. This opens up a new toolbox to examine processes determining goal-oriented behaviour in relation to psychopathology in humans.
Citation: Wichers M, Kasanova Z, Bakker J, Thiery E, Derom C, Jacobs N, et al. (2015) From Affective Experience to Motivated Action: Tracking Reward-Seeking and Punishment-Avoidant Behaviour in Real-Life. PLoS ONE 10(6): e0129722. https://doi.org/10.1371/journal.pone.0129722
Academic Editor: Gabriel S. Dichter, UNC Chapel Hill, UNITED STATES
Received: November 7, 2014; Accepted: May 12, 2015; Published: June 18, 2015
Copyright: © 2015 Wichers et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Data Availability: All relevant data are available in a Supporting Information file.
Funding: This research was supported by the Fund for Scientific Research, Flanders and Twins, a Belgian nonprofit association for scientific research in multiple births (grant to the East Flanders Prospective Survey). The authors thank all twins for their cooperation. This research was additionally supported by an Aspasia grant (Dutch Organisation for Scientific Research; NWO) awarded to M. Wichers and by the Weijerhorst Foundation. Furthermore, M. Wichers was supported by the Brain Foundation of The Netherlands (fellowship grant nr 2012(1)-03).
Competing interests: Marieke Wichers and Jim van Os are PLOS ONE editorial board members. This does not alter the authors' adherence to PLOS ONE editorial policies and criteria, as detailed online in our guide for authors.
Introduction
Many of the decisions and actions in everyday life result from learning processes that are guided by interaction with environmental cues. The capacity to form and continuously update internal representations of stimulus-response associations allows one to select the most advantageous response from a repertoire, and to accurately predict its outcome [1–3]. These implicit associative learning processes are therefore critical mediators of daily life decisions and future course of actions. Consider the example of Peter and Christine who both join the athletics club, in Peter’s case with the goal of becoming a long-distance runner and in Christine’s to make new friends. Peter is tall, slim and fit which predisposes him to excel in endurance activities. He enjoys the training sessions, does not experience a lot of unpleasant physical sensations and improves quickly, soon becoming one of the best runners in the group. As a result, Peter’s repeated exposures to positive outcomes motivate him to keep training. Christine, on the other hand, who is a little shy, sensitive and less interested in running, welcomes the opportunity to join her new teammates in a party. She enjoys the party until one of the girls makes a critical remark about her clothes. She feels very awkward goes home early and skips some practices. When she is invited again to a similar party by her teammates, she declines, until eventually quitting the team. Although seemingly very different, both Peter’s and Christine’s behaviour was governed by two essential mechanisms of operant conditioning [3–6]. Positive reinforcement learning is the process through which new stimuli acquire motivational salience by virtue of being associated with positive emotions, thus becoming rewards. Rewards have, by definition, the potential to increase the likelihood that an organism will engage in actions previously associated with positive affective state [6], as demonstrated by Peter’s case. In this example Peter experiences reward which gets associated with his behaviour of being physically active. This may positively influence Peter’s motivation to repeat this behaviour in the future. Punishment learning, on the other hand, is necessary to avoid aversive outcomes [5,7]. Aversive stimuli decrease the frequency of behaviour linked to negative affective states [6], such as those in Christine’s situation. Christine experiences negative affect which gets paired with being in the above described social context. This may influences her social behaviour, i.e. her choice to avoid those situations in the future. In both examples, the learned associations between (un)pleasant affective experiences and the individual’s behaviour play an important role in motivating future choices to maintain or extinguish the pertinent behaviour. Associative learning processes, therefore, allow for a flexible adaptation of behaviour by guiding organisms in decisions of approach and avoidance of daily life situations, and ultimately prompt and maintain goal-oriented behaviour [6]. The fact that the neural circuits governing associative learning are present even in organisms with the most primitive neural networks [8] demonstrates the importance of this mechanism in survival and evolution.
Experimental studies have shown evidence for positive and negative experiences modifying behaviour in humans [5,9–13]. For example, positive self-talk and self-esteem can be increased by pairing images of a person’s own body with positive stimuli that signal social acceptance [11]. In addition, the unconscious tendency to consume alcohol has been found to decrease following pairing of the alcohol cue with an aversive outcome [9]. Importantly, inadequate reward and punishment learning may play a key role in many forms of deviant behaviour [3,5,14]. For example, whereas impaired punishment learning has been associated with increased risk-taking behaviour and gambling [15], enhanced punishment-avoidance is thought to be related to the acquisition of pain-related fear [12] and in the withdrawal from social situations. Similarly, abnormally high tendency for reward-seeking behaviour is linked with the risk for developing an addiction [9] and eating disorders [16], while suboptimal reward-driven behaviour is believed to be associated with anhedonia, avolition and depression [17,18].
Despite the importance of reward and punishment learning in human behaviour and their putative relevance to mental health, to date, incentive-driven responding has not been studied prospectively in the flow of daily life. This endeavour would provide an ecologically valid assessment of the extent to which affective experience motivates action, with the potential of uncovering real-life behavioural patterns linked to compromised mental health. Physical and social behaviour, as illustrated by the above described two examples, are important types of behaviour that are, in part, controlled by associative learning processes. Furthermore these behaviours have relevance to psychopathology and can be measured using momentary assessment techniques. In the current manuscript we will therefore focus on these two types of behaviour.
Momentary assessment techniques [19–27], enabling the researcher to capture the film rather than a snapshot of daily life reality [28–31], are ideally suited to examine the subtle temporal associations between affect and daily life behaviour over the course of the day(s). Therefore, we can use this method to detect patterns of reward-seeking and punishment-avoidant behaviour by examining whether rewarding or punishing affective states occurring in a certain daily life context, modify the likelihood of engaging in similar daily life contexts in the near future. If we can show that such rewarding or punishing affective experiences influence the likelihood that certain behaviour is repeated more or less often, respectively, then this would support the idea that we can detect the associative learning processes that motivate future behaviour.
The aim of this study is to examine whether it is possible to prospectively track the propagation of reward and punishment-driven behaviour in humans using momentary assessment methodology. For this purpose it was examined, using multilevel analyses, whether affective experience that occurred in the context of specific behaviours at previous measurements (i.e. at t-2 and t-1) modified the likelihood to show similar behaviour at next measurement moments (at time t). In order to obtain initial proof-of-principle, this study focuses on two frequently occurring daily life behaviours which are relevant to mental health states: social context and physical activity. To this end, data pertaining to a general population sample of 621 individuals who participated in an Experience Sampling Method (ESM) study were analysed.
Methods
Sample
The study sample consisted of 621 female subjects from the general population, who were part of twin pairs or were sisters of twin pairs living in Flanders (Belgium). Their age was between 18 and 46 years. Of the 621 subjects, 610 participated in the ESM study. Most participants were recruited from the East-Flanders Prospective Twin Survey, a population-based survey prospectively recording all multiple births in the province of East-Flanders since 1964 [32,33]. Selection criteria were being female and being over 18 years of age. Participants received a financial compensation for participation of 500 Belgium Francs (which is the equivalent of approximately 12,5 euros). Although subjects were twins, the current hypothesis did not require twin methodology. The project was approved by the Medical Ethics Committee of the K.U. Leuven under the number B3222010766. All participants gave written informed consent. Originally, the study was designed to assess stress-sensitivity in daily life; taking into consideration evidence for qualitative differences in the type of environmental stressors that are associated with depression in men and women [34,35], the study was designed to include females only. Mean age of the participants was 27 years (SD: 7.6 years, range (18–46). Sixty-five percent had a college or university degree, 33% completed secondary education and 2% had primary education only. The majority was currently employed (61.3% employed, 33.8% student, 2.2% unemployed, 2.3% homemaker and 0.4% sick leave).
Experience Sampling Method
The experience sampling method (ESM) [36] was used to conduct momentary assessments on 5 consecutive days. Subjects received a digital wristwatch and a set of ESM self-assessment forms collated in a booklet for each day. The wristwatch was programmed to emit a signal (“beep”) at an unpredictable moment in each of ten 90-minute time blocks between 7:30 and 22:30. The study uses a semi-random beep design in order to prevent anticipatory behaviour of participants, but also used the constraint that no two signals could occur within 15 minutes of each other [37]. Beep intervals did not overlap, thus ensuring that the first beep of the day was always earlier in time than the second and the second earlier than the third etc. The semi-random beep design was set up in a way that every moment of the day had an equal likelihood of being sampled.
Participants were aware that 10 beep signals would occur in a day between 7.30 and 22.30. They were instructed to fill out the ESM diary directly after each beep signal. The diary consisted of short questions on the current affect, behaviour and appraisals of the current situation (see below). All self-assessments were rated on 7-point Likert scales. Trained research assistants with ample experience in momentary assessment technology explained the ESM procedure to the participants during an initial briefing session, and a practice form was completed to confirm that subjects were able to understand the 7-point Likert scale. Subjects could call a telephone number in case they had questions or problems during the ESM sampling period. Subjects were instructed to complete their reports immediately after the beep, thus minimizing memory distortion, and to record the time at which they completed the form. The time at which subjects completed the report was compared to the actual time of the beep that signaled that it was time to fill out the report. In this way it was checked whether reports were filled out within 15 minutes. All reports not filled out within 15 min after the beep were excluded from the analysis, since previous work [22] has shown that reports completed after this interval are less reliable and consequently less valid. In addition, subjects with less than 17 valid reports (out of 50) were excluded from the analysis, as previous work has shown that measures of individuals with less than 30% of completed reports are less reliable [22]. Compliance to the ESM protocol was very high (96.4%) [38].
Measurements
Momentary affect.
Momentary affective states were assessed with 15 adjectives rated on 7-point likert scales. The choice of the ESM affect items was guided in part by the PANAS questionnaire [39] and in part by the results of previous ESM studies (selecting items with high loadings on NA and PA latent factors and sufficient within-person variability). A Factor analysis on the affect items identified two affect factors with eigenvalue >1. Ratings on the items ‘insecure’, ‘lonely’, ‘anxious’, ‘low’, ‘guilty’ and ‘suspicious’—weighted for factor loadings—were averaged to form NA (respective loadings were: 0.71, 0.60, 0.66, 0.68, 0.61, 0.64). The weighted average of ratings on ‘cheerful’, ‘content’, ‘energetic’, ‘enthusiastic’ formed PA (respective loadings were: 0.84, 0.71, 0.84, 0.83). The strongest cross-loading was -0.13 (loading of ‘content’ on NA factor). Five affect items did not load specifically on one of these factors. These were ‘tired’, ‘relaxed’, ‘irritated’, ‘sad’ and ‘calm’. Conform previous publications [40,41] that used this sample these five items were not used in the variables that measure NA and PA.
Appraisal of social context.
Appraisal of the social context was assessed by asking participants whether or not they were alone at the time of the beep. If not alone, they were asked whether they liked the company they were in at that moment. This was rated on a 7-point likert scale (from ‘not at all’ (1) to ‘very much’ (7)).
Physical activity.
ESM physical activity was assessed with a single item asking subjects how physically active they had been since the last beep, rated on a 7-point likert scale. During the ESM briefing, participants were instructed on how to score their level of physical activity. To this end, they were given an indication of the level of activity that corresponded to the score on the 7-point likert scale. Examples were provided as anchor points as follows: a score of ‘1’ corresponds to the level of physical activity of ‘resting’; ‘2’ corresponds to ‘sitting’; ‘3’ to ‘walking’; ‘4’ to ‘household chores such as vacuum cleaning’; ‘5’ to ‘biking’; ‘6’ to ‘playing tennis’ and ‘7’ to ‘running’. These anchor points were also provided in the ESM self-assessment forms that subjects completed at each beep. For instance, for each physical activity approximately equalling the level of physical activity associated with vacuum cleaning, participants rated a score of 4 on the 7-point likert scale.
Statistical analysis
ESM data have a hierarchical structure in which ESM observations (level 1) are clustered within participants (level 2). STATA 11.0 [42] was used to conduct multilevel regression analyses. Time-lagged multilevel analyses were used to examine reward- and punishment-based behaviour prospectively over time.
Reward-related behaviour is the result of an associative process in which positive affective experience is paired with a behaviour, and the frequency of this behaviour increases. Assuming that positive affect is rewarding, reward-driven behaviour was operationalized in ESM as the moderating effect of PA at (t-1) on the probability that behaviour at (t-1) was repeated at (t) (Fig 1). Punishment-related behaviour represents the opposite process in which the occurrence of the behaviour previously associated with negative affective experience decreases. Assuming that negative affect is punishing, punishment-related behaviour was thus operationalized in ESM as the moderating effect of NA at (t-1) on the probability that behaviour at (t-1) was repeated at (t) (Fig 1). Mathematically, the i-th momentary observation score of subject j is modelled as follows:
Behaviourij t = β0 + β1 affectij t-1 + β2 behaviourij t-1 + β3 affectij t-1*behaviourij t-1 ζ.j εij. Here, ζj represents the subject’s deviation from the overall mean (random intercept).
Reward-related behaviour is conceptualized as the daily life behaviour (such as being in pleasant company or engaging in physical activity) that increases in frequency at time t upon being associated with positive affect at time t-1. Punishment-related behaviour is conceptualized as the daily life behaviour the frequency of which decreases at time t upon being associated with negative affect at time t-1. Time lags t-1 and t-2 are examined in this study where t represents the time lag of one beep moment (± 90 minutes) or the time lag of one day.
In order to obtain a regression model that purely tests within-subject effects, the between-subject regression was subtracted from the original regression model, as follows: Behaviourij t behaviour.j t = β1 (affectij t-1 - affect.j t-1) + β2 (behaviourij t-1 − behaviour.j t-1) + β3 (affectij t-1*behaviourij t-1- affect.j t-1*behaviour.j t-1) +εij - ε.j. In other words, from each predictor in the regression model we subtracted the subject mean, leaving only pure within-subject effects. These regression analyses were performed using the XTREG command combined with the fixed effects (fe) option. Since the fe option is available only for multilevel analyses with no more than one extra level, clustering within twin pairs was accounted for by adding twin pair id as a covariate. The above example equations describe the model including one lag effect. The regressions performed in this study included 2 lags.
Generalisability over different time frames: In addition to beep-level (t duration = ± 90 min), reward-related and punishment-related behaviours were also explored at day-level (t duration = 1 day). For the day-level analyses, beep moment results of affect, behaviour and the interaction thereof were averaged per beep over the day. These analyses thus examined whether the average of the repeated (maximum 10) combinations of affect and behaviour at day (t-1) impacted on the average behaviour on beep moments at day (t).
Specificity of effects: Additionally, cross-context associations were added to examine the specificity of the effect. If time-lagged effects across the different behavioural contexts are equally strong as those within behavioural contexts this would be indicative for non-specific effects of affect impacting on future behaviour in general. If, on the other hand, within-context effects are much stronger than cross-context effects, then this supports the idea that we detected behaviour-specific processes of associative learning.
Results
Of the 610 subjects who participated in the ESM study, 31 were excluded because they had too few (less than 30%) beeps with valid responses. Of the 579 remaining subjects, 12 participants had been part of the first pilot data collection. Some of the PA items that were present in the final diary were not present in the pilot diary. As some of the PA items were missing in the pilot sample, the total PA scores could not be computed for these 12 participants. Another 4 participants had no intra-individual variations in or no consecutive measurement moments for physical activity, and were therefore excluded from the analysis. In the various analyses, the total number of beep-level measurements varied between 9,990 and 12,228. The total number of day-level measurements varied between 1,642 and 1,693. The data file and a short description of the variables are available in the supporting information files (see S1 Dataset and S1 Text). In Table 1 the correlations at observation-level are shown between NA, PA, (un)pleasantness of company and physical activity.
Reward-related behaviour
Table 2 shows the regression analyses results regarding temporal patterns of reward-related behaviour for beep and day-level. The beep-level analyses refer to multilevel time-lagged analyses in which time units are used from beep moment to beep moment (which are on average 90 minutes apart from each other). Day-level analyses refer to similar multilevel time-lagged analyses, but now using time units of 24 hours.
Beep-level.
Regarding the beep-level analyses, significant within-context interaction effects were found for both behaviours: engaging in physical activity and being in pleasant company. More specifically, PA moderated the effect of physical activity at beep (t-1) on physical activity at beep (t). The same was true for being in pleasant company. Significant effects were observed only for associations over one lag. No significant cross-context effects (e.g. from physical activity to pleasant company or vice versa) were found (see Table 2 and Fig 2). Thus, positive affect indeed moderated the likelihood of recurrence of active and social behaviour in the expected direction. Furthermore, the fact that no cross-context effects were observed makes it unlikely that observed effects represent non-specific effects of affect impacting on future behaviour in general.
The first two bars in each Fig. represent the analyses with positive affect at t-1 as the moderator of the time-lagged association between daily contexts and the second two bars represent the analyses with negative affect at t-1 as the moderator of these analyses. Each first bar is the within-context association (the same context is predictor at t-1 and outcome measure at t. This bar thus reflects the extent to which affect valence moderates the association between the analysed behaviour at time t-1 on the same behaviour at time t. The second bar refers to the cross-context association: the extent to which affect moderates the time-lagged association across daily contexts (for example: social context as the predictor and physical activity as the outcome). Error bars represent confidence intervals. The Fig.s show that, as hypothesized, positive and negative affect both significantly moderate the impact of behaviour at time t-1 on similar behaviour at time t, except for the analyses regarding physical activity at day-level (Fig 2d).
Day-level.
PA at day (t-1) moderated the effect of being in pleasant company at day t-1 on being in pleasant company at day (t) (one lag). This effect was not found for day (t-2) on day (t)(two lags). Thus, higher levels of PA in the context of pleasant company increased the level of pleasant company on the next day. For physical activity, no significant interaction effects were found for lag 1 and 2 (see Table 2; Fig 2). However, strongly negative main effects were observed regarding physical activity at day (t-1) and day (t-2) on day (t) (B = -0.28, p<0.001 and B = -0.24, p<0.001, respectively). This suggests a sinusoid pattern in which high levels of physical activity on one day are associated with less activity the next two days. Therefore, more lags might be necessary to reveal a potentially delayed effect of PA during physical activity and the occurrence of physical activity in the future. In order to examine such pattern, a post-hoc test was carried out of the effects of an additional lag. Corrected for effects at lag 1 and 2, lag 3 revealed a significant interaction effect where PA positively moderated the effect of physical activity at day (t-3) on physical activity at day (t) (B = 0.106, p = 0.003; Fig 3). No positive moderation effects of PA were found in cross-context analyses. In contrast, negative moderation was observed (see Table 2). Thus, also at day-level positive affect moderated the likelihood of recurrence of active and social behaviour in the expected direction.
A stronger association of physical activity with positive affect at day t-3, corrected for effects at day t-2 and day t-1, increases physical activity at day t. Similarly, a stronger association between physical activity with negative affect at day t-2, corrected for effects at day t-2 and day t-1, decreases physical activity at day t. No significant cross-context effects were found. Error bars represent confidence intervals. This Fig. shows that the hypothesized effects do show up in post-hoc analyses where we corrected not just for 2 but for three time lags (3 days).
Punishment-related behaviour
Beep-level.
NA at beep (t-1) negatively moderated the effect of unpleasant company at beep (t-1) on unpleasant company at beep (t). Thus, higher levels of NA decreased the effect of being in unpleasant company at beep (t-1) on being in unpleasant company at beep (t). No interaction effect of beep (t-2) was observed. A similar effect was found for physical activity in that NA at beep (t-1) negatively moderated the effect of physical activity at beep (t-1) on physical activity at beep (t). No significant negative moderation was observed for cross-context interaction effects (see Table 3 for effect sizes and p-values of the interaction effects). Thus, as expected, negative affect negatively moderated the likelihood of recurrence of active and social behaviour. Again, as expected, these effects were only present for within-context and not for cross-context analyses.
Day-level.
NA at day (t-1) negatively moderated the effect of being in unpleasant company at day (t-1) on being in unpleasant company at day (t). This interaction effect was not found for day (t-2). No significant negative moderation was observed for physical activity (see Table 3). However, similar to the analyses of reward-related behaviour, there were significant negative main effects of physical activity at day (t-1) on physical activity at day (t), possibly obscuring the associative patterns. Therefore, the effect of an extra day-lag was examined post-hoc. Results showed that NA negatively moderated the effect of physical activity at day (t-3) on physical activity at day (t) (B = -0.298, p = 0.008 (see Fig 3). No significant cross-context interaction effects were found (see Table 3). Thus, also at day-level negative affect moderated the likelihood of recurrence of active and social behaviour in the expected direction.
Discussion
The hypothesis that reward and punishment-related behaviour can be prospectively tracked using momentary assessment techniques was confirmed. Both at beep and day level, affect, experienced in relation to certain behavioural contexts, influenced the frequency with which similar behaviour was expressed in the future.
Social context
As expected, the experience of positive affect in pleasant social contexts increased the extent to which these contexts predicted the occurrence of similar social contexts in the near future at beep and day level (within-context), but did not increase the extent to which pleasant social contexts predicted the occurrence of other activities, such as physical activity (cross-context). At beep level, where measurements are on average 90 minutes apart, this finding could indicate two things. First, it could reflect a tendency to search for new enjoyable company directly following a previous rewarding social experience. Second, the finding may indicate that people tend to prolong the amount of time in a specific social context if it is associated with experience of positive affect. At day level, the latter interpretation is not very likely. Here, the repeated associations between positive affect and pleasant social contexts, derived from all measurement moments during the day together, predicted the frequency of being in similar pleasant company the next day. Day level findings, therefore, might reflect associative processes that involve a more complex integration of multiple response-outcome contingencies to effectively evaluate what future actions may be beneficial in the long-term. It is thought that higher-order cortical processing in the prefrontal and anterior cingulated cortex is involved in decision-making based on the synthesis and evaluation of a large number of previously internalized associations [43–46]. The beep-level effects in this study could be more indicative of short-term responses to immediate affective experience instead of being based on the integration of a long-term history of experiences.
Also, as predicted, negative affect associated with unpleasant social contexts decreased the likelihood of the occurrence of similar social contexts in the near future at both beep- and day-level. Similar to the reasoning above, the beep-level findings could mean either that people avoided new negative social situations after negative social experiences one or two hours earlier that day, or that people tended to shorten their contact with company in which they experienced negative affect. In conclusion, the valence of affective experience associated with social behaviour was found to modify the frequency of occurrence of this behaviour. This effect, however, was detected for one lag, but not for two lags. This means that the pertinent affective state was found to modify behaviour for up to 90 minutes at beep-level and for up to 1 day at day-level.
Physical activity
As hypothesized, significant within-context and non-significant cross-context effects were observed for physical activity at beep-level. That is, positive affective experience in the context of physical activity predicted increased frequency of occurrence of physical activity at the next measurement moment. Similarly, negative affective experience in the context of physical activity was associated with decreased propensity to engage in physical activity at the next measurement moment. Thus, both reward and punishment contingencies were found to modify behaviour at beep-level. Similar to social context, these effects were only found to hold at one time lag but not at two time lags. In contrast to the social context findings, however, no significant association between affective experience related to physical activity and frequency of future engagement in physical activity was detected at day level. The lack of significant findings at day level is likely caused by the strongly negative association between physical activity on similar activity the next day. Arguably, this pattern may be a demonstration of the natural day-to-day fluctuations in physical activity levels; people tend to be less active during the days following a physically more challenging day, reflecting a compensatory mechanism. It can be hypothesised that these day-to-day fluctuations in activity obscured the effects of the propagation of associative processes, since adding an additional third lag (Fig 3) revealed the expected within- and cross-context effects. It is thus important to find the appropriate level at which associative processes can be modelled for each type of behaviour studied. Behavioural modification regarding physical activity apparently requires examining patterns over multiple days to accurately track the intermittent nature of physical activity and rest.
Experimental paradigms
Previously, studies have shown evidence of reward and punishment-driven behaviour using experimental paradigms involving monetary rewards or losses. One experiment showed, using a monetary incentive delay task, that the ability to learn to avoid losses was associated with differences in insular sensitivity to anticipated losses and with self-reported levels of anxiety [47]. Another study showed that higher arousal responses at moments of a threat cue-signalling the delivery of a shock- were associated with increased learning of avoidance responses [10]. Also, reward sensitivity assessed experimentally [18,44,48–54] using monetary incentives was found to be modulated by stress, genetic predisposition and anhedonic states. To date, however, it is unknown how experimental findings based on secondary rewards and punishments extrapolate to real-life stimulus-response relationships, and how these manifest themselves in real-life situations, such as social contexts or personal activities. While experimental studies provide valuable insights into psychological and biological mechanisms of relevant traits, momentary assessment techniques may be able to expand on their findings and provide real-life validity to experimental associative processing models. Real life validation of experimental findings would connect existing knowledge on mechanisms of reward- and punishment-driven responding with concrete patterns of daily life sub-optimal goal-oriented behaviour that can be targeted for therapeutic modification. Therefore, future studies should combine experimental and momentary assessment techniques to examine how experimental findings translate to real life behavioural and affective patterns. To our knowledge, only one recent study [55] has combined experimental imaging and momentary assessment techniques. The authors showed that prefrontal stress-related dopamine activity is associated with real life reactivity to stress. The current momentary assessment study is the first to show that the propagation of incentive-based behaviour can be examined in the flow of daily life, paving the way for the first study combining experimental and momentary associative processing paradigms.
Clinical relevance
The findings of the current study showed that behaviour can be modified as a function of the affective experience associated with it, a pattern that was detectable at the micro-level with time lags of approximately 90 minutes, but also at day-level, with time lags of one day. It still needs to be established, though, whether these real-life associative patterns are indeed relevant to psychopathology, and whether their relevance can be modulated by time frame of analysis (beep or day) level. In other words, with respect to psychopathology, it remains to be determined whether it has potential to examine person-specific patterns of behavioural modification on a moment-to-moment or on a day-to-day basis.
However, should these prospectively measured behavioural patterns be involved in the development of psychopathology, then analysing momentary assessment data may represent a new tool to provide person-tailored clinically useful information [30]. Person-specific analyses may reveal person-tailored results regarding reward-seeking and punishment-avoidant behaviour in real life. Such person-specific insights into daily life learning mechanisms may be of direct clinical use as input for therapist-patient contacts [30,56–58].
Methodological issues
Effect sizes as reported in the current study are small. However, it is generally observed that in ESM studies effect sizes are smaller than effects found using traditional research designs since in ESM effects occur over very small periods of time. These effects represent averaged effects that happen each 90 minutes or each day, depending on the time frame. Cumulatively, these repeated effects seem influential enough to impact on mental health as has been shown in previous studies [59–61].
Second, it is unknown whether the appraisal of company reflects an individual’s active choice to be in the company of certain people, or the subjective interpretation of the social context. If the latter is true, then these findings may reflect an alteration in the appraisal of the company, rather than a change in behaviour. This would mean that the previous day’s PA experience during pleasant company would improve the appraisal of company during the next day. This alternative interpretation can mean two things: (i) it reflects non-specific effects of PA. However, since no cross-context effects of PA were found, this is not probable; (ii) it reflects reward-modulated interpretation of daily life context. Either form of reward-driven modification (at the behavioural level or at the level of interpretation of daily life contexts) is clinically relevant. The same line of reasoning can be applied to the analyses on aversive learning.
The data of the current study were collected from 1999 to 2002. No electronic ESM devices were available at that time. However, compliance to the research protocol was electronically monitored in a subgroup of the sample and was determined to be high (see for more details [38]).
The current study used ESM for 5 consecutive days. However, longer periods of sampling (2–3 weeks) are preferable, especially for the modelling at day-level. The development of electronic ESM devices which are less demanding for participants than the paper-and-pencil method, may facilitate extended sampling periods.
Another issue is the use of self-report only. ESM is one of the most accurate ways to measure affect. The experience of affect cannot be reliably estimated using objective measures. However, for physical activity, for example, it would be possible to use objective measures such as actigraphy. Since both self-report and objective measures contain a certain level of noise, the measure of physical activity could be improved by using a combination of both measures.
Finally, the sample included females only. Therefore, results do not necessarily extrapolate to male individuals.
Supporting Information
S1 Dataset. This file includes the variables that are used in the current manuscript.
https://doi.org/10.1371/journal.pone.0129722.s001
(XLS)
S1 Text. This file includes a short description of the variables included in the S1 dataset.
https://doi.org/10.1371/journal.pone.0129722.s002
(DOCX)
Author Contributions
Conceived and designed the experiments: MW ET CD NJ JvO. Performed the experiments: MW ET CD NJ. Analyzed the data: MW. Contributed reagents/materials/analysis tools: CD ET JvO. Wrote the paper: MW ZK JB.
References
- 1.
Kupfermann I, Kandel ER (1995) Learning and memory. In: Kandel ER SJ, Jessell TM, editor. Essentials of neural science and behaviour. Connecticut: Prentice Hall International.
- 2. de Wit S, Dickinson A (2009) Associative theories of goal-directed behaviour: a case for animal-human translational models. Psychol Res 73: 463–476. pmid:19350272
- 3.
Gleitman H (1995) Psychology. New York: Norton. pmid:12290759
- 4. De Houwer J, Thomas S, Baeyens F (2001) Associative learning of likes and dislikes: a review of 25 years of research on human evaluative conditioning. Psychol Bull 127: 853–869. pmid:11726074
- 5. Martin-Soelch C, Linthicum J, Ernst M (2007) Appetitive conditioning: neural bases and implications for psychopathology. Neurosci Biobehav Rev 31: 426–440. pmid:17210179
- 6. Cloninger CR, Gilligan SB (1987) Neurogenetic mechanisms of learning: a phylogenetic perspective. J Psychiatr Res 21: 457–472. pmid:3326938
- 7. Ernst M, Daniele T, Frantz K (2011) New perspectives on adolescent motivated behavior: attention and conditioning. Dev Cogn Neurosci 1: 377–389. pmid:21977221
- 8. Hawkins RD, Clark GA, Kandel ER (2006) Operant conditioning of gill withdrawal in Aplysia. J Neurosci 26: 2443–2448. pmid:16510722
- 9. Houben K, Schoenmakers TM, Wiers RW (2010) I didn't feel like drinking but I don't know why: the effects of evaluative conditioning on alcohol-related attitudes, craving and behavior. Addict Behav 35: 1161–1163. pmid:20810220
- 10. Delgado MR, Jou RL, Ledoux JE, Phelps EA (2009) Avoiding negative outcomes: tracking the mechanisms of avoidance learning in humans during fear conditioning. Front Behav Neurosci 3: 33. pmid:19847311
- 11. Martijn C, Vanderlinden M, Roefs A, Huijding J, Jansen A (2010) Increasing body satisfaction of body concerned women through evaluative conditioning using social stimuli. Health Psychol 29: 514–520. pmid:20836606
- 12. Meulders A, Vansteenwegen D, Vlaeyen JW (2011) The acquisition of fear of movement-related pain and associative learning: a novel pain-relevant human fear conditioning paradigm. Pain.
- 13. Gottfried JA, O'Doherty J, Dolan RJ (2003) Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science 301: 1104–1107. pmid:12934011
- 14. Kim BW, Kennedy DN, Lehar J, Lee MJ, Blood AJ, Lee S, et al. (2010) Recurrent, robust and scalable patterns underlie human approach and avoidance. PLoS One 5: e10613. pmid:20532247
- 15. Brunborg GS, Johnsen BH, Pallesen S, Molde H, Mentzoni RA, Myrseth H (2010) The relationship between aversive conditioning and risk-avoidance in gambling. J Gambl Stud 26: 545–559. pmid:20155305
- 16. Jansen A (1998) A learning model of binge eating: cue reactivity and cue exposure. Behav Res Ther 36: 257–272. pmid:9642846
- 17. Chase HW, Frank MJ, Michael A, Bullmore ET, Sahakian BJ, Robbins TW (2010) Approach and avoidance learning in patients with major depression and healthy controls: relation to anhedonia. Psychol Med 40: 433–440. pmid:19607754
- 18. Pizzagalli DA, Iosifescu D, Hallett LA, Ratner KG, Fava M (2008) Reduced hedonic capacity in major depressive disorder: evidence from a probabilistic reward task. J Psychiatr Res 43: 76–87. pmid:18433774
- 19. Csikszentmihalyi M, Larson R (1987) Validity and reliability of the Experience-Sampling Method. J Nerv Ment Dis 175: 526–536. pmid:3655778
- 20.
Larson R, Csikszentmihalyi M (1983) The experience sampling method. New directions for methodology of social and behavioral science: 41–56.
- 21. Shiffman S, Stone AA, Hufford MR (2008) Ecological momentary assessment. Annu Rev Clin Psychol 4: 1–32. pmid:18509902
- 22.
Delespaul P (1995) Assessing schizophrenia in daily life: The experience sampling method; Vries MWd, editor. Maastricht: University of Limburg.
- 23.
deVries M, Delespaul P, Dijkman-Caes C (1992) Consequences of depression for the experience of anxiety in daily life. In: deVries M, editor. The experience of psychopathology: investigating mental disorders in their natural settings. Cambridge: Cambridge University Press. pp. 141–147.
- 24. Mata J, Thompson RJ, Jaeggi SM, Buschkuehl M, Jonides J, Gotlib IH Walk on the bright side: Physical activity and affect in major depressive disorder. J Abnorm Psychol.
- 25. Wichers M, Peeters F, Rutten BP, Jacobs N, Derom C, Thiery E, et al. (2011) A time-lagged momentary assessment study on daily life physical activity and affect. Health Psychol.
- 26. Wichers M, Kenis G, Jacobs N, Myin-Germeys I, Schruers K, Mengelers R, et al. (2008) The psychology of psychiatric genetics: evidence that positive emotions in females moderate genetic sensitivity to social stress associated with the BDNF Val-sup-6-sup-6Met polymorphism. J Abnorm Psychol 117: 699–704. pmid:18729623
- 27. Collip D, Oorschot M, Thewissen V, Van Os J, Bentall R, Myin-Germeys I Social world interactions: how company connects to paranoia. Psychol Med: 1–11.
- 28. Trull TJ, Ebner-Priemer UW (2009) Using experience sampling methods/ecological momentary assessment (ESM/EMA) in clinical assessment and clinical research: introduction to the special section. Psychol Assess 21: 457–462. pmid:19947780
- 29. Myin-Germeys I, Oorschot M, Collip D, Lataster J, Delespaul P, van Os J (2009) Experience sampling research in psychopathology: opening the black box of daily life. Psychol Med 39: 1533–1547. pmid:19215626
- 30. Wichers M, Simons CJ, Kramer IM, Hartmann JA, Lothmann C, Myin-Germeys I, et al. (2011) Momentary assessment technology as a tool to help patients with depression help themselves. Acta Psychiatr Scand 124: 262–272. pmid:21838742
- 31. Wichers M, Hartmann JA, Kramer IM, Lothmann C, Peeters F, van Bemmel L, et al. (2011) Translating assessments of the film of daily life into person-tailored feedback interventions in depression. Acta Psychiatr Scand.
- 32. Derom C, Vlietinck R, Thiery E, Leroy F, Fryns JP, Derom R (2002) The East Flanders Prospective Twin Survey (EFPTS). Twin Res 5: 337–341. pmid:12537855
- 33. Derom CA, Vlietinck RF, Thiery EW, Leroy FO, Fryns JP, Derom RM (2006) The East Flanders Prospective Twin Survey (EFPTS). Twin Res Hum Genet 9: 733–738. pmid:17254399
- 34. Kendler KS, Gatz M, Gardner CO, Pedersen NL (2006) A Swedish national twin study of lifetime major depression. Am J Psychiatry 163: 109–114. pmid:16390897
- 35. Kendler KS, Thornton LM, Prescott CA (2001) Gender differences in the rates of exposure to stressful life events and sensitivity to their depressogenic effects. Am J Psychiatry 158: 587–593. pmid:11282693
- 36. Myin-Germeys I, Oorschot M, Collip D, Lataster J, Delespaul P, van Os J (2009) Experience sampling research in psychopathology: opening the black box of daily life. Psychol Med: 1–15. pmid:19335938
- 37.
Delespaul P, Reis HT, deVries M (1995) Empirical studies in ESM reliability and validity. D: ESM time and RIR event sampling in social interactions. In: deVries M, editor. Assessing schizophrenia in daily life. Maastricht: Ipser.
- 38. Jacobs N, Nicolson NA, Derom C, Delespaul P, van Os J, Myin-Germeys I (2005) Electronic monitoring of salivary cortisol sampling compliance in daily life. Life Sci 76: 2431–2443. pmid:15763075
- 39. Watson D, Clark LA, Tellegen A (1988) Development and validation of brief measures of positive and negative affect: the PANAS scales. J Pers Soc Psychol 54: 1063–1070. pmid:3397865
- 40. Wichers M, Peeters F, Rutten BP, Jacobs N, Derom C, Thiery E, et al. (2012) A time-lagged momentary assessment study on daily life physical activity and affect. Health psychology: official journal of the Division of Health Psychology, American Psychological Association 31: 135–144.
- 41. Wichers M, Aguilera M, Kenis G, Krabbendam L, Myin-Germeys I, Jacobs N, et al. (2008) The catechol-O-methyl transferase Val158Met polymorphism and experience of reward in the flow of daily life. Neuropsychopharmacology 33: 3030–3036. pmid:17687265
- 42.
StataCorp. (2009) Stata Statistical Software. College Station, TX.
- 43. Seo H, Lee D (2008) Cortical mechanisms for reinforcement learning in competitive games. Philos Trans R Soc Lond B Biol Sci 363: 3845–3857. pmid:18829430
- 44. Santesso DL, Dillon DG, Birk JL, Holmes AJ, Goetz E, Bogdan R, et al. (2008) Individual differences in reinforcement learning: behavioral, electrophysiological, and neuroimaging correlates. Neuroimage 42: 807–816. pmid:18595740
- 45. Knutson B, Bhanji JP, Cooney RE, Atlas LY, Gotlib IH (2008) Neural responses to monetary incentives in major depression. Biol Psychiatry 63: 686–692. pmid:17916330
- 46. Croxson PL, Walton ME, O'Reilly JX, Behrens TE, Rushworth MF (2009) Effort-based cost-benefit valuation and the human brain. J Neurosci 29: 4531–4541. pmid:19357278
- 47. Samanez-Larkin GR, Hollon NG, Carstensen LL, Knutson B (2008) Individual differences in insular sensitivity during loss anticipation predict avoidance learning. Psychol Sci 19: 320–323. pmid:18399882
- 48. Pizzagalli DA, Bogdan R, Ratner KG, Jahn AL (2007) Increased perceived stress is associated with blunted hedonic capacity: potential implications for depression research. Behav Res Ther 45: 2742–2753. pmid:17854766
- 49. Dillon DG, Deveney CM, Pizzagalli DA (2011) From Basic Processes to Real-World Problems: How Research on Emotion and Emotion Regulation Can Inform Understanding of Psychopathology, and Vice Versa. Emot Rev 3: 74–82. pmid:21584224
- 50. Jocham G, Neumann J, Klein TA, Danielmeier C, Ullsperger M (2009) Adaptive coding of action values in the human rostral cingulate zone. J Neurosci 29: 7489–7496. pmid:19515916
- 51. Schonberg T, O'Doherty JP, Joel D, Inzelberg R, Segev Y, Daw ND (2010) Selective impairment of prediction error signaling in human dorsolateral but not ventral striatum in Parkinson's disease patients: evidence from a model-based fMRI study. Neuroimage 49: 772–781. pmid:19682583
- 52. Bogdan R, Santesso DL, Fagerness J, Perlis RH, Pizzagalli DA (2011) Corticotropin-releasing hormone receptor type 1 (CRHR1) genetic variation and stress interact to influence reward learning. J Neurosci 31: 13246–13254. pmid:21917807
- 53. Forbes EE, Dahl RE, Almeida JR, Ferrell RE, Nimgaonkar VL, Mansour H, et al. (2012) PER2 rs2304672 polymorphism moderates circadian-relevant reward circuitry activity in adolescents. Biol Psychiatry 71: 451–457. pmid:22137505
- 54. Olino TM, McMakin DL, Morgan JK, Silk JS, Birmaher B, Axelson DA, et al. (2014) Reduced reward anticipation in youth at high-risk for unipolar depression: a preliminary study. Dev Cogn Neurosci 8: 55–64. pmid:24369885
- 55.
Hernaus D, Lataster J, Collip D, Ceccarini J, Booij L, van Os J, et al. (submitted) In-vivo prefrontal dopamine release during stress moderates psychotic experiences in response to daily life stress in psychosis-prone individuals, but not in healthy controls.
- 56. Bonsall MB, Wallace-Hadrill SM, Geddes JR, Goodwin GM, Holmes EA (2011) Nonlinear time-series approaches in characterizing mood stability and mood instability in bipolar disorder. Proc Biol Sci.
- 57. Oorschot M, Lataster T, Thewissen V, Wichers M, Myin-Germeys I (2011) Mobile Diagnostics for Schizophrenia: a data-driven momentary approach. Schizophrenia Bulletin
- 58. Rosmalen JGM, Wenting AMG, Roest AM, de Jonge P, Bos EH (in press) Revealing Causal Heterogeneity Using Time Series Analysis of Ambulatory Assessments: Application to the Association Between Depression and Physical Activity Following Myocardial Infarction. Psychosomatic Medicine.
- 59. Wichers M, Geschwind N, Jacobs N, Kenis G, Peeters F, Derom C, et al. (2009) Transition from stress sensitivity to a depressive state: longitudinal twin study. The British journal of psychiatry: the journal of mental science 195: 498–503. pmid:19949197
- 60. Kramer I, Simons CJ, Wigman JT, Collip D, Jacobs N, Derom C, et al. (2013) Time-Lagged Moment-to-Moment Interplay Between Negative Affect and Paranoia: New Insights in the Affective Pathway to Psychosis. Schizophrenia bulletin.
- 61. Wichers M (2014) The dynamic nature of depression: a new micro-level perspective of mental disorders that meets current challenges. Psychological Medicine 44: 1349–1360. pmid:23942140