Skip to main content
SpringerLink
Log in

Evaluating the utility of administering a reaction time task in an ecological momentary assessment study

  • Original Investigation
  • Published:
Psychopharmacology Aims and scope Submit manuscript

Abstract

Rationale

Cognitive processes underlying drug use have typically been assessed in laboratory settings. More detailed and ecologically valid data may be possible if assessments were conducted in an ecological momentary assessment (EMA) setting.

Objectives

We evaluated the feasibility and utility of administering a reaction time task on a hand-held computer (personal digital assistant, PDA) in an EMA setting.

Materials and methods

Twenty-two smokers and 22 non-smokers carried around the PDA for 1 week as they went about their daily lives. They were beeped at random times four times per day (random assessments, RAs). Participants were also instructed to press an “anxiety assessment” (AA) button on the PDA whenever they felt suddenly anxious. At each assessment (RA, AA), participants responded to items assessing subjective, pharmacological, and contextual variables, and subsequently completed a Stroop task (classic-Stroop, emotional-Stroop, or smoking-Stroop task).

Results

Participants responded to 81.2% of RAs, completed assessments in an average of 4.44 min, reported no interruptions on the majority of assessments (62.4%), and produced data with adequate reliability. Using generalized estimating equation (GEE) analyses, age was associated with the classic-Stroop effect, state anxiety was associated with the emotional-Stroop effect, and Fagerstrom Test for Nicotine Dependence scores were associated with the smoking-Stroop effect.

Conclusions

The study provided evidence for the feasibility and utility of the approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. Additionally, researchers have proposed that modified cognitive tasks, including Stroop tasks, can be used as interventions that target attention (Mathews and MacLeod 2002). EMA settings may ultimately prove to be useful in delivering attentional retraining interventions.

  2. We estimated internal reliability using a split-half approach. Using a random number generator in SAS (ranuni), we split the trial-level dataset into two halves, computed means for each half, computed a Pearson’s correlation between the two means (over all assessments), and applied the Spearman–Brown formula to derive the split-half reliability coefficient. Each random number generates different splits, which yield somewhat differing estimates. We input ten different random numbers and took the mean of the ten estimates. We used the same approach for each measure.

  3. Key findings were observed if compound symmetry or independence correlation structures were used.

  4. Blocking significantly moderated a number of associations. For example, blocking significantly moderated the effect of smoking status on the classic-Stroop effect (PE = 60.6, SE = 17.6, p < 0.001). The effect of smoking status was significant in the blocked condition (PE = 67.5, SE = 21.8, p < 0.001) but not in the mixed (PE = 8.87, SE = 16.9, p > 0.5). These effects are not discussed further in this paper.

  5. Although the internal reliability of the Stroop effects were in the moderate range, difference scores tend to be less reliable, particularly, when the two original scores are highly correlated (Nunnally and Bernstein 1994), as was the case in this study.

  6. Mean RTs in the current study appeared somewhat slower than those in previous laboratory studies, employing manual responses (e.g., mean RT on neutral trials = 884 ms in the current study vs 641 ms in Waters et al. 2003). This may be because of the influence of interruptions in the current study (reported interruptions was associated with mean RT). However, differences in response methods (e.g., stylus vs finger responses) or study populations could also account for differences in RT.

References

  • Ballesteros S, Reales JM, Manga D (2000) Effects of type of design (blocked vs. randomized) on Stroop and emotional Stroop tasks. Psicothema 12(Suppl 2):60–63

    Google Scholar 

  • Carpenter KM, Schreiber E, Church S, McDowell D (2006) Drug Stroop performance: relationships with primary substance of use and treatment outcome in a drug-dependent outpatient sample. Addict Behav 31:174–181

    Article  PubMed  Google Scholar 

  • Cox WM, Hogan LM, Kristian MR, Race JH (2002) Alcohol attentional bias as a predictor of alcohol abusers’ treatment outcome. Drug Alcohol Depend 68:237–243

    Article  PubMed  Google Scholar 

  • Cox WM, Fadardi JS, Pothos EM (2006) The addiction-Stroop test: theoretical considerations and procedural recommendations. Psychol Bull 132:443–476

    Article  PubMed  Google Scholar 

  • Cox WM, Pothos EM, Hosier SG (2007) Cognitive-motivational predictors of excessive drinkers’ success in changing. Psychopharmacology 192:499–510

    Article  PubMed  CAS  Google Scholar 

  • Edwards MS, Burt JS, Lipp OV (2006) Selective processing of masked and unmasked verbal threat material in anxiety: influence of an immediate acute stressor. Cogn Emot 20:812–835

    Article  Google Scholar 

  • Egloff B, Hoff M (2001) Interactive effects of state anxiety and trait anxiety on emotional Stroop interference. Pers Individ Differ 31:875–882

    Article  Google Scholar 

  • Field M, Rush M, Cole J, Goudie AJ (2007) The smoking Stroop and delay discounting in smokers: effects of environmental smoking cues. J Psychopharmacol 21:603–610

    Article  PubMed  Google Scholar 

  • Franken IHA, Kroon LY, Wiers RW, Jansen A (2000) Selective cognitive processing of drug cues in heroin dependence. J Psychopharmacol 14:395–400

    Article  PubMed  CAS  Google Scholar 

  • Franken IHA, Hendriks VM, Stam CJ, Van den Brink W (2004) A role for dopamine in the processing of drug cues in heroin dependent patients. Eur Neuropsychopharmacol 14:503–508

    Article  PubMed  CAS  Google Scholar 

  • Heatherton TF, Kozlowski LT, Frecker RC, Fagerstrom KO (1991) The Fagerstrom test for nicotine dependence: a revision of the Fagerstrom tolerance questionnaire. Br J Addict 86:1119–1127

    Article  PubMed  CAS  Google Scholar 

  • Hester R, Dixon V, Garavan H (2006) A consistent attentional bias for drug-related material in active cocaine users across word and picture versions of the emotional Stroop task. Drug Alcohol Depend 81:251–257

    Article  PubMed  CAS  Google Scholar 

  • Jarvik ME, Henningfield JE (1993) Pharmacological adjuncts for the treatment of nicotine dependence. In: Orleans CT, Slade J (eds) Nicotine dependence: principles and management. New York, NY: Oxford University Press, pp. 245–261

  • Kenemans JL, Wieleman JST, Zeegers M, Verbaten MN (1999) Caffeine and Stroop interference. Pharmacol Biochem Behav 63:589–598

    Article  PubMed  CAS  Google Scholar 

  • Kindt M, Bierman D, Brosschot JF (1996) Stroop versus Stroop: comparison of a card format and a single-trial format of the standard color-word Stroop task and the emotional Stroop task. Pers Individ Differ 21:653–661

    Article  Google Scholar 

  • Kucera H, Francis WN (1967) Computational analysis of present day American English. Brown University Press, Providence, RI

    Google Scholar 

  • MacLeod CM (1992) The Stroop task: the “gold standard” of attentional measures. J Exper Psychol Gen 121:12–14

    Article  Google Scholar 

  • MacLeod C, Hagan R (1992) Individual differences in the selective processing of threatening information, and emotional responses to a stressful life event. Behav Res Ther 30:151–161

    Article  PubMed  CAS  Google Scholar 

  • MacLeod CM, MacDonald PA (2000) Inter-dimensional interference in the Stroop effect: uncovering the cognitive and neural anatomy of attention. Trends Cogn Sci 4:383–391

    Article  PubMed  Google Scholar 

  • MacLeod C, Rutherford EM (1992) Anxiety and selective processing of emotional information: mediating roles of awareness, trait and state variables, and personal relevance of stimulus materials. Behav Res Ther 30:479–491

    Article  PubMed  CAS  Google Scholar 

  • Marissen MAE, Franken IHA, Waters AJ, Blanken P, van den Brink W, Hendriks VM (2006) Attentional bias predicts heroin relapse following treatment. Addiction 101:1306–1312

    Article  PubMed  Google Scholar 

  • Mathews A, MacLeod C (2002) Induced processing biases have causal effects on anxiety. Cogn Emot 16:331–354

    Article  Google Scholar 

  • Melara RD, Mounts JRW (1993) Selective attention to Stroop dimensions: effects of baseline discriminability, response mode, and practice. Mem Cogn 21:627–645

    CAS  Google Scholar 

  • Mogg K, Bradley BP (1998) A cognitive-motivational analysis of anxiety. Behav Res Ther 36:809–848

    Article  PubMed  CAS  Google Scholar 

  • Mogg K, Mathews A, Bird C, Macgregor-Morris R (1990) Effects of stress and anxiety on the processing of threat stimuli. J Pers Soc Psychol 59:1230–1237

    Article  PubMed  CAS  Google Scholar 

  • Moskowitz DS, Young SN (2006) Ecological momentary assessment: what it is and why it is a method of the future in clinical psychopharmacology. J Psychiatry Neurosci 3:13–20

    Google Scholar 

  • Munafo M, Mogg K, Roberts S, Bradley BP, Murphy M (2003) Selective processing of smoking-related cues in current smokers, ex-smokers and never-smokers on the modified Stroop task. J Psychopharmacol 17:310–316

    Article  PubMed  Google Scholar 

  • Nunnally JC, Bernstein IH (1994) Psychometric theory, 3rd edn. McGraw Hill, New York

    Google Scholar 

  • Parrott AW (1991) Performance tests in human psychopharmacology (1): test reliability and standardization. Hum Psychopharmacol Clin Exper 6:1–9

    Article  Google Scholar 

  • Provost SC, Woodward R (1991) Effects nicotine gum on repeated administration of the Stroop test. Psychopharmacology 104:536–540

    Article  PubMed  CAS  Google Scholar 

  • Richards A, French CC, Johnson W, Naparstek J, Williams J (1992) Effects of emotion manipulation and anxiety on performance on an emotional Stroop task. Br J Clin Psychol 83:479–491

    Google Scholar 

  • Robinson TE, Berridge KC (1993) The neural basis of craving: an incentive-sensitization theory of addiction. Brain Res Rev 18:247–291

    Article  PubMed  CAS  Google Scholar 

  • Robinson TE, Berridge KC (2000) The psychology and neurobiology and addiction: An incentive-sensitization view. Addiction 95:S91–S117

    Article  PubMed  Google Scholar 

  • Rutherford EM, MacLeod C, Campbell LW (2004) Negative selectivity effects and emotional selectivity effects in anxiety: differential attentional correlates of state and trait variables. Cogn Emot 18:711–720

    Article  Google Scholar 

  • Sayette MA, Martin CS, Perrott MA, Wertz JM, Hufford MR (2001) A test of the appraisal–disruption model of alcohol and stress. J Stud Alcohol 62:247–256

    PubMed  CAS  Google Scholar 

  • Shiffman S, Waters AJ (2004) Negative affect and smoking lapses: a prospective analysis. J Consult Clin Psychol 7:192–201

    Google Scholar 

  • Shiffman S, Paty JA, Gnys M, Elash C, Kassel JD (1995) Nicotine withdrawal in chippers and regular smokers—subjective and cognitive effects. Health Psychol 14:301–309

    Article  PubMed  CAS  Google Scholar 

  • Shiffman S, Johnston JA, Khayrallah M, Elash CA, Gwaltney CJ, Paty JA, Gnys M, Evoniuk G, De Veaugh-Geiss (2000) The effect of bupropion on nicotine craving and withdrawal. Psychopharmacology 148:33–40

    Article  PubMed  CAS  Google Scholar 

  • Shiffman S, Gwaltney CJ, Balabanis MH, Liu KS, Paty JA, Kassel JD, Hickcox M, Gnys M (2002) Immediate antecedents of cigarette smoking: an analysis from ecological momentary assessment. J Abnorm Psychology 111:531–54

    Article  Google Scholar 

  • Shiffman S, Patten C, Gwaltney CJ, Paty JA, Gnys M, Kassel J, Hickcox M, Waters AJ, Balabanis MH (2006) Natural history of nicotine withdrawal. Addiction 101:1822–1832

    Article  PubMed  Google Scholar 

  • Shiffman S, Balabanis MH, Gwaltney CJ, Paty JA, Gnys M, Kassel JD, Hickcox M, Paton SM (2007) Prediction of lapse from associations between smoking and situational antecedents assessed by ecological momentary assessment. Drug Alcohol Depend 91:159–168 (doi:10.1016/j.drugalcdep.2007.05.017)

    Google Scholar 

  • Spielberger CD, Gorsuch RL, Lushene R, Vagg PR, Jacobs GA (1983) Manual for the state-trait anxiety inventory. Consulting Psychologists, Palo Alto, CA

    Google Scholar 

  • Stone AA, Shiffman S (2002) Capturing momentary, self-report data: a proposal for reporting guidelines. Annals Behav Med 24:236–243

    Article  Google Scholar 

  • Stone AA, Shiffman S, Atienza A, Nebeling L (2007) The science of real time data capture. Oxford University Press, Oxford

    Google Scholar 

  • Stroop JR (1935) Studies of interference in serial verbal reactions. J Exp Psychol 18:643–661

    Article  Google Scholar 

  • Troyer AK, Leach L, Strauss E (2006) Aging and response inhibition: normative data for the Victoria Stroop test. Aging Neuropsychol Cogn 13:20–35

    Article  Google Scholar 

  • Waters AJ, Sayette MA (2005) Implicit cognition and tobacco addiction. In: Wiers RW, Stacy AW (eds) Handbook on implicit cognition and addiction. Sage, Thousand Oaks, CA, pp 309–338

    Google Scholar 

  • Waters AJ, Shiffman S, Sayette MA, Paty JA, Gwaltney CG, Balabanis MH (2003) Attentional bias predicts outcome in smoking cessation. Health Psychol 22:378–387

    Article  PubMed  Google Scholar 

  • Wertz J, Sayette MA (2001) Effects of smoking opportunity on attentional bias in smokers. Psychol Addict Behav 15:268–271

    Article  PubMed  CAS  Google Scholar 

  • West R (1999) Age differences in lapses of intention in the Stroop task. J Gerontol 54B:P34–P43

    Google Scholar 

  • Williams JGW, Mathews A, MacLeod C (1996) The emotional Stroop task and psychopathology. Psychol Bull 120:3–24

    Article  PubMed  CAS  Google Scholar 

  • Zeger SL, Liang KY, Albert PS (1988) Models for longitudinal data: a generalized estimating equation approach. Biometrics 44:1049–1060

    Article  PubMed  CAS  Google Scholar 

Download references

Acknowledgment

This work was supported by institutional research funds of the UT M. D. Anderson Cancer Center and was conducted while the first author was in the Department of Behavioral Science, UT M. D. Anderson Cancer Center. We thank Dr. Tenko Raykov, Ph.D., for providing statistical consultation in the development of the manuscript. We thank David Wetter for the assistance in item selection, Elizabeth Miller for the assistance in data collection, and Samuel Riley for the administrative assistance. The reported experiment complied with the laws of the USA.

Author information

Authors and Affiliations

  1. Department of Medical and Clinical Psychology, Uniformed Services University of the Health Sciences, 4301 Jones Bridge Road, Bethesda, MD, 20814, USA

    Andrew J. Waters

  2. Department of Biostatistics, The University of Texas M. D. Anderson Cancer Center, P.O. Box 30149, Unit 1330, Houston, TX, 77230, USA

    Yisheng Li

Authors
  1. Andrew J. Waters
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yisheng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andrew J. Waters.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Waters, A.J., Li, Y. Evaluating the utility of administering a reaction time task in an ecological momentary assessment study. Psychopharmacology 197, 25–35 (2008). https://doi.org/10.1007/s00213-007-1006-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00213-007-1006-6

Keywords

Search

Navigation