Elsevier

Brain and Cognition

Volume 81, Issue 1, February 2013, Pages 82-94
Brain and Cognition

Reliability and plasticity of response inhibition and interference control

https://doi.org/10.1016/j.bandc.2012.09.010Get rights and content

Abstract

This study investigated the internal reliability, temporal stability and plasticity of commonly used measures of inhibition-related functions. Stop-signal, go/no-go, antisaccade, Simon, Eriksen flanker, Stroop and Continuous Performance tasks were administered twice to 23 healthy participants over a period of approximately 11 weeks in order to assess test–retest correlations, internal consistency (Cronbach’s alpha), and systematic between as well as within session performance changes. Most of the inhibition-related measures showed good test–retest reliabilities and internal consistencies, with the exception of the stop-signal reaction time measure, which showed poor reliability. Generally no systematic performance changes were observed across the two assessments with the exception of four variables of the Eriksen flanker, Simon and Stroop task which showed reduced variability of reaction time and an improvement in the response time for incongruent trials at second assessment. Predominantly stable performance within one test session was shown for most measures. Overall, these results are informative for studies with designs requiring temporally stable parameters e.g. genetic or longitudinal treatment studies.

Highlights

► We examine test-retest-reliability, internal consistency and plasticity of tasks measuring inhibition-related functions. ► Most measures yield good test-retest-reliability and internal consistencies. ► No systematic within- and between-session changes are present. ► Therefore these measures are able to represent inhibition-related traits. ► Results are relevant for pathophysiological, genetic and pharmacological research.

Introduction

A hallmark feature of cognitive control is the ability to inhibit automatic or inappropriate reactions and to resolve response conflict stemming from irrelevant or distracting stimuli or stimulus features (Barkley, 1997, Miller and Cohen, 2001). While the term inhibition has different meanings in different contexts of psychology and neuroscience (Aron et al., 2007), inhibition is often defined as the ability to cancel a prepotent response or to suppress an action which is inappropriate, irrelevant or no longer required (Friedman and Miyake, 2004, Nigg, 2000, Schachar et al., 2000). Different paradigms have been used to measure different forms of inhibition, including motor response inhibition, reflex inhibition and interference inhibition.

Prominent experimental paradigms of motor response inhibition include the stop-signal (Logan, Cowan, & Davis, 1984) and the go/no-go task (Meere, Stemerdink, & Gunning, 1995). In the go/no-go task a response has to be either executed or not depending on the go or no-go-signal and the task thus measures selective motor response inhibition. In the stop-signal task a motor response has to be withheld when a stop-signal appears unpredictably a short time after the go-signal. The stop task thus measures a later stage of inhibition, where the go response is already triggered by the go-signal and on its way of execution, but needs to be retracted (Rubia et al., 2001). A different variant is the antisaccade task that measures the inhibition of a prepotent, automatic saccadic response (Hallett, 1978). A more cognitive form of inhibition is interference inhibition which measures the ability to resolve response conflict due to irrelevant but incompatible and therefore interfering stimulus features that, if not inhibited, will lead to an erroneous response, such as the Simon (Simon & Rudell, 1967), Eriksen flanker (Eriksen & Eriksen, 1974), and Stroop (Stroop, 1935) tasks. The Continuous Performance Test (CPT) is more typically considered a measure of sustained attention, where targets have to be detected among non-targets, but also contains an inhibitory component, viz. the suppression of a response to irrelevant targets (Cornblatt et al., 1988, Rosvold et al., 1956, Soreni et al., 2009).

While these tasks differ with respect to their specific cognitive demands and may be grouped according to different criteria (Dillon and Pizzagalli, 2007, Friedman and Miyake, 2004, Harnishfeger, 1995, Hommel, 2011, Nigg, 2000) they all share the requirement to inhibit the processing of a prepotent, bottom-up generated, inappropriate response. Given that the inhibition of prepotent responses is thought to be mediated by prefrontal brain areas (Aron et al., 2007, Rieger et al., 2003, Rubia et al., 2003) these measures are often discussed within the framework of prefrontally mediated cognitive control (Miller & Cohen, 2001). Following Friedman and Miyake (2004) we refer to these measures as “inhibition-related functions”.

Inhibition-related functions have been found to be significantly heritable (Friedman et al., 2008, Groot et al., 2004, Heiser et al., 2006, Schachar et al., 2011). It has been shown using a variety of imaging methods and using several of the above mentioned paradigms that the prefrontal cortex, most prominently the inferior prefrontal cortex and the supplementary motor area (SMA), as well as subcortical striato-thalamic projection targets represent a neural correlate common to such paradigms (Aron et al., 2003, Aron et al., 2004, Aron et al., 2007, Eagle et al., 2008, Ettinger et al., 2008, Picton et al., 2007, Rieger et al., 2003, Rubia et al., 2007, Rubia et al., 2003).

Of importance to the clinical domain, deficits in inhibition-related functions represent a prevalent dysfunction in a number of psychiatric disorders including schizophrenia (Henik and Salo, 2004, Hepp et al., 1996, Lipszyc and Schachar, 2010), attention-deficit/hyperactivity disorder (ADHD) (Boonstra, Kooij, Oosterlaan, Sergeant, & Buitelaar, 2010) and obsessive-compulsive disorder (OCD) (Chamberlain, Fineberg, Blackwell, Robbins, & Sahakian, 2006). The study of inhibition-related functions in the clinical context is of importance for at least three reasons, namely to increase our knowledge concerning the (1) neurocognitive pathophysiology, (2) genetics and (3) pharmacological mechanisms of the psychiatric disorders mentioned above.

First, given the proximity of inhibition-related cognitive tasks to the neural level (Siebner, Callicott, Sommer, & Mattay, 2009) these have been used profitably to assess the cognitive profile and associated brain functional alterations in different psychiatric and neurological patient groups (Aron and Poldrack, 2005, Aron et al., 2003, Gauggel et al., 2004, Hutton and Ettinger, 2006). Second, on the basis of the observed heritability of inhibition-related functions (Gagne and Saudino, 2010, Kuntsi et al., 2006) it has been investigated to what extent the deficits observed in patient groups represent endophenotypes, or markers of genetic risk. For example, some studies have shown that not only patients with schizophrenia, OCD or ADHD but also their clinically unaffected biological first-degree relatives show inhibition-related deficits (Calkins et al., 2008, Chamberlain et al., 2007, Curtis et al., 2001, Ettinger et al., 2004, Ettinger et al., 2006, Schachar et al., 2005, Slaats-Willemse et al., 2003, Thaker et al., 2000) supporting the notion that such deficits share common genetic background and represent an endophenotype for these psychiatric conditions. Third, pharmacological treatment studies of psychiatric patients (Klein et al., 2002, Meere et al., 1999, Scheres et al., 2003) as well as single-dose administration studies in healthy volunteers (Barrett et al., 2004, Chamberlain et al., 2009, Costa et al., 2012, Ettinger et al., 2009, Nandam et al., 2011) have applied inhibition-related tasks as treatment targets in repeated measures designs in order to better understand the mechanisms of action of compounds such as nicotine, methylphenidate, atomoxetine, or antipsychotics at the neurocognitive level.

Overall, a large body of evidence has thus accumulated on inhibition-related (dys)-function across different neuropsychiatric conditions for these three purposes (for review, see Aron and Poldrack, 2005, Hutton and Ettinger, 2006, Lipszyc and Schachar, 2010, Westerhausen et al., 2011). Importantly, all three approaches require precise and valid measurement using reliable paradigms. Reliability is a key psychometric criterion of any psychological test (Anastasi & Urbina, 1997) and a necessary prerequisite for its validity. Reliability is most frequently measured in terms of temporal stability (or test–retest reliability) and internal consistency (e.g. Cronbach’s alpha). The issue of temporal stability is particularly important for treatment studies involving repeated assessments (Becker, 2007) and is also an explicit validity criterion of an endophenotype (Gottesman & Gould, 2003).

Therefore, our main aim was to provide a comprehensive assessment of the temporal stability and internal consistency of inhibition-related paradigms that are commonly used in the clinical and cognitive neuroscience literature. Specifically, we focussed on the go/no-go, stop-signal, antisaccade, Eriksen flanker, Simon, Stroop and CPT paradigms and studied healthy volunteers in order to obtain a clear assessment of reliability in the absence of possibly interfering symptom or treatment effects observed in clinical populations. Previous studies of individual tasks have generally reported moderate to high reliabilities (Cornblatt et al., 1988, Ettinger et al., 2003, Kindlon et al., 1995, Klein and Berg, 2001, Kuntsi et al., 2005, Logan, 1994, Smyrnis, 2008, Soreni et al., 2009, Zabel et al., 2009). Our aim was to estimate the stability of a number of widely used tasks in one study in order to allow comparison of reliability estimates across tasks.

It is known from psychometrics that the reliability of a test is positively correlated with the number of items in the test and that the reliability will increase if the scale is extended with items that do not reduce the average inter-item correlation (Carmines and Zeller, 1979, Spearman, 1910). Therefore, we explored whether such a relationship would also be observed in the paradigms under investigation here by calculating test–retest reliability coefficients not only for the total number of trials in each task but also for subscores consisting of the first 25%, 50%, and 75% of the trials to assess the adequacy of test length.

An issue related to the temporal stability of cognitive control concerns the plasticity of such functions. Effects of repeated exposure are often observed on the level of cognitive performance (Dahlin, Nyberg, Backman, & Neely, 2008); these typically represent improvements due to practice or increased task familiarity and strategy use. Importantly, such between-session changes represent a potential confound in the study of pharmacological influences on performance (Ettinger et al., 2003, Klein et al., 2002). Therefore, in this study we also aimed to characterise the extent to which performance improves from test to retest over intervals often used in clinical treatment studies, viz. approximately 11 weeks.

Importantly, plasticity effects may be observed not only from one session to the next but also within a single testing session. Such plasticity may represent learning due to rapid adaptation to the task demands or acquisition of task relevant strategies (Kelly et al., 2006, Verbruggen and Logan, 2009) as well as deterioration in performance e.g. due to fatigue. Therefore, the extent to which performance varies, improves or deteriorates over several trials within the same session was also explored in this study by studying systematic within-session changes. Finally, given the importance of intra-individual variability in performance particularly in the clinical context (see e.g. Kaiser et al., 2008, Klein et al., 2006, Kuntsi et al., 2010, Sergeant, 2005, Vaurio et al., 2009) the intra-individual standard deviation and the coefficient of variation, ICV, of performance measures were also investigated (Bellgrove et al., 2004, Klein et al., 2006, Nandam et al., 2011).

Section snippets

Participants

The present study was undertaken in a sample of healthy, non-medicated, unrelated adults. Volunteers were recruited using local newspaper advertisements, flyers and mailing lists. Ex ante participants were screened by telephone for basic inclusion criteria. At the first assessment they provided demographic and psychometric information. Inclusion criteria were: Age 18–55 years, good German language skills, no history of neurological complications, no current medical condition, and no current

Descriptive statistics

Twenty-three participants completed the study (age mean = 23.70, SD = 3.02, range 20–31; 69.6% females; years of education mean = 15.57, SD = 1.58; estimated verbal IQ mean = 114.78, SD = 9.80). The average test–retest interval was 77.65 days (SD = 25.97, range 28–105). Analysis of age effects showed that a number of variables on the Eriksen flanker, go/no-go and Simon tasks correlated significantly with age (all p < 0.01): In the Eriksen flanker task at baseline the variables mean RT (r = 0.59), SD of RT (r = 

Discussion

Reliability is a key psychometric criterion for any psychological test. In this study we assessed commonly used inhibition-related paradigms by estimating test–retest reliability as a measure of temporal stability and internal consistency. To explore the relationship between test length and reliability we calculated test–retest reliability coefficients for subscores of differing lengths. Finally we investigated between-session as well as within-session changes in performance to provide an

Limitations

A first limitation of our study concerns the generalizability of the findings. In order to avoid interference of age effects, clinical symptoms or treatment effects, we examined a healthy, young sample with specific tasks. This of course limits the extent to which these findings generalise to other (e.g. older, less educated or clinical) populations and other variants of inhibition-related tasks. Therefore, the reliability measures observed in this study should be seen as sample representative

Conclusions

In summary, most of the assessed measures of inhibition-related functions displayed good test–retest-reliability and good internal consistencies combined with a lack of systematic within- and between-session changes in healthy individuals. Therefore, these measures may represent inhibition-related traits which are important for clinical research and may be used profitably in pathophysiological, genetic and pharmacological studies. Future research with variation of task parameters, sample

Acknowledgment

This work was supported by the Deutsche Forschungsgemeinschaft (ET 31/2-1).

References (116)

  • D.G. Dillon et al.

    Inhibition of action, thought, and emotion: A selective neurobiological review

    Applied and Preventive Psychology

    (2007)
  • E.E. Emeric et al.

    Influence of history on saccade countermanding performance in humans and macaque monkeys

    Vision Research

    (2007)
  • U. Ettinger et al.

    Smooth pursuit and antisaccade eye movements in siblings discordant for schizophrenia

    Journal of Psychiatric Research

    (2004)
  • U. Ettinger et al.

    Effects of acute nicotine on brain function in healthy smokers and non-smokers: Estimation of inter-individual response heterogeneity

    Neuroimage

    (2009)
  • P.E. Hallett

    Primary and secondary saccades to goals defined by instructions

    Vision Research

    (1978)
  • H.H. Hepp et al.

    The Stroop effect in schizophrenic patients

    Schizophrenia Research

    (1996)
  • B. Hommel

    The Simon effect as tool and heuristic

    Acta Psychologica

    (2011)
  • S. Kaiser et al.

    Intra-individual reaction time variability in schizophrenia, depression and borderline personality disorder

    Brain and Cognition

    (2008)
  • C. Klein et al.

    Instrumental and test–retest reliability of saccadic measures

    Biological Psychology

    (2005)
  • C. Klein et al.

    Intra-subject variability in attention-deficit hyperactivity disorder

    Biological Psychiatry

    (2006)
  • A. Miyake et al.

    The unity and diversity of executive functions and their contributions to complex “frontal lobe” tasks: A latent variable analysis

    Cognitive Psychology

    (2000)
  • L.S. Nandam et al.

    Methylphenidate but not atomoxetine or citalopram modulates inhibitory control and response time variability

    Biological Psychiatry

    (2011)
  • B.B. Quednow et al.

    Attenuation of the prepulse inhibition of the acoustic startle response within and between sessions

    Biological Psychology

    (2006)
  • K. Rubia et al.

    Mapping motor inhibition: Conjunctive brain activations across different versions of go/no-go and stop tasks

    Neuroimage

    (2001)
  • K. Rubia et al.

    Right inferior prefrontal cortex mediates response inhibition while mesial prefrontal cortex is responsible for error detection

    Neuroimage

    (2003)
  • C.W.N. Saville et al.

    On the stability of instability: Optimising the reliability of intra-subject variability of reaction times

    Personality and Individual Differences

    (2011)
  • Aichert, D.S., Wöstmann, N. M., Costa, A., Macare, C., Wenig, J., Möller, H. J., Rubia, K., Ettinger, U. (in press)....
  • A. Anastasi et al.

    Psychological testing

    (1997)
  • A.R. Aron et al.

    Converging evidence for a fronto-basal-ganglia network for inhibitory control of action and cognition

    Journal of Neuroscience

    (2007)
  • A.R. Aron et al.

    Stop-signal inhibition disrupted by damage to right inferior frontal gyrus in humans

    Nature Neuroscience

    (2003)
  • J. Asendorpf

    Psychologie der Persönlichkeit

    (1999)
  • D.M. Barch et al.

    CNTRICS final task selection: Executive control

    Schizophrenia Bulletin

    (2009)
  • R.A. Barkley

    Behavioral inhibition, sustained attention, and executive functions: Constructing a unifying theory of ADHD

    Psychological Bulletin

    (1997)
  • S.L. Barrett et al.

    Effects of amisulpride, risperidone and chlorpromazine on auditory and visual latent inhibition, prepulse inhibition, executive function and eye movements in healthy volunteers

    Journal of Psychopharmacology

    (2004)
  • J.J. Bartko

    Measurement and reliability: Statistical thinking considerations

    Schizophrenia Bulletin

    (1991)
  • Bäumler, G. (1985). Farbe-Wort-Interferenztest (FWIT) nach J.R. Stroop. Göttingen:...
  • R.E. Becker

    Lessons from Darwin: 21(st) century designs for clinical trials

    Current Alzheimer Research

    (2007)
  • N. Becser et al.

    Reliability of cephalic thermal thresholds in healthy subjects

    Cephalalgia

    (1998)
  • A.M. Boonstra et al.

    To act or not to act, that’s the problem: Primarily inhibition difficulties in adult ADHD

    Neuropsychology

    (2010)
  • A. Buchner et al.

    Further evidence for systematic reliability differences between explicit and implicit memory tests

    Quarterly Journal of Experimental Psychology, Section A: Human Experimental Psychology

    (2003)
  • E.G. Carmines et al.

    Reliability and validity assessment

    (1979)
  • J.C. Caruso

    A comparison of the reliabilities of four types of difference scores for five cognitive assessment batteries

    European Journal of Psychological Assessment

    (2004)
  • Chamberlain, S. R., Hampshire, A., Muller, U., Rubia, K., Del Campo, N., Craig, K., Sahakian, B. J. (2009). Atomoxetine...
  • S.R. Chamberlain et al.

    Motor inhibition and cognitive flexibility in obsessive-compulsive disorder and trichotillomania

    American Journal of Psychiatry

    (2006)
  • S.R. Chamberlain et al.

    Impaired cognitive flexibility and motor inhibition in unaffected first-degree relatives of patients with obsessive-compulsive disorder

    American Journal of Psychiatry

    (2007)
  • Cook, D. A. & Beckman, T. J. (2006). Current concepts in validity and reliability for psychometric instruments: Theory...
  • A. Costa et al.

    Methylphenidate effects on neural activity during response inhibition in healthy humans

    Cerebral Cortex

    (2012)
  • L.J. Cronbach

    Coefficient alpha and the internal structure of tests

    Psychometrika

    (1951)
  • C.E. Curtis et al.

    Saccadic disinhibition in schizophrenia patients and their first-degree biological relatives. A parametric study of the effects of increasing inhibitory load

    Experimental Brain Research

    (2001)
  • E. Dahlin et al.

    Plasticity of executive functioning in young and older adults: Immediate training gains, transfer, and long-term maintenance

    Psychology and Aging

    (2008)
  • Cited by (158)

    • Neurobehavioral maturation of motor response inhibition in adolescence – A narrative review

      2022, Neuroscience and Biobehavioral Reviews
      Citation Excerpt :

      Dedicated studies exploring reliability of executive tasks in adult samples have found variable reliabilities for AST measures. In a test-retest study of 23 adults over an interval of 23–105 days, antisaccade latency and accuracy showed excellent internal consistency at baseline (Cronbach’s α = 0.92 and 0.94, respectively) and retest (Cronbach’s α = 0.97 and 0.94, respectively; Wöstmann et al., 2013). Test-retest reliability in the same sample showed excellent reliability for AS latency and accuracy (ICC = 0.88 and 0.92, respectively).

    View all citing articles on Scopus
    View full text