Assessment of subtypes of cognitive control

The dual mechanisms of control framework (Braver, 2012) divides cognitive control into two distinct, reciprocally activated modes: proactive and reactive control, each important in enacting certain goal-directed behaviors. The AX-continuous performance task (AX-CPT; Carter et al., 1998; Cohen, Barch, Carter, & Servan-Schreiber, 1999; J. D. Cohen et al., 1997; Servan-Schreiber, Cohen, & Steingard, 1996) and dot pattern expectancy (DPX; Henderson et al., 2012; MacDonald et al., 2005) are commonly used cue–probe cognitive tasks in which variation in cue and probe expectancy are used to assess the impact of (cue derived) context on proactive (preparatory) and reactive cognitive control. The two tasks are structurally identical, differing only in their use of letter versus dot pattern stimuli, and slight variations in cue–probe pair frequency (i.e., 70% vs. 68.75% target pairs). We have recently suggested that different timing-related parameters may induce different processes for control (Janowich & Cavanagh, under revision), and that seemingly trivial idiosyncrasies between studies may threaten external validity. Considering how timing and temporal prediction are fundamental features of human neurocognition (Buhusi & Meck, 2005; Paton & Buonomano, 2018), we aim to assess if task timing-related parameters modulate the use of proactive versus reactive control across the representative literature.

In AX-CPT and DPX, delay and trial count parameters vary widely and are often given scarce or no discussion. Does knowing the length of the cue–probe delay increase use of proactive control, and is proactive control more strongly instantiated ahead of a known short delay? Further, does increased repetition of a task set over time strengthen one’s preference for exerting proactive control? As cognitive control comes at the cost of valuable cognitive resources (Shenhav, Botvinick, & Cohen, 2013), we hypothesize that people might utilize distinct control processes to handle goals with different timelines or temporal expectations, and that the development of habitual response patterns over many trials is likely to moderate preparatory processes. This meta-analysis exploits the variation in the expectancy literature to advance our understanding of timing and repetition effects on cognitive control instantiation, as well as facilitating discussion on interpretation of the heterogeneous results in AX-CPT and DPX studies.

The experimental tasks

An example of AX-CPT/DPX task flow and parameters is depicted in Fig. 1. In this task paradigm, a probe stimulus (X or Y) is presented following a paired cue stimulus (A or B) in target and nontarget combinations. In a two-alternative-forced choice manner (2AFC), participants are instructed to respond to both cue and probe stimuli. The target AX sequence dictates a common target response set; whereas all other cue–probe pairs require an alternative response set. Because 70% of trials are composed of AX cue–probe target pairs, and AY, BX, and BY cue–probe nontarget pairs are much more rare (10% trials of each), a strong expectancy (e.g., habit) is generated to respond according to the AX rule (Servan-Schreiber et al., 1996).

Fig. 1
figure 1

Example of AX-CPT/DPX task design. In a–c, typical AX-CPT task designs with known (a), jittered (b), or unknown (c) cue–probe delays are depicted. A probe stimulus (X or Y) is presented following a paired cue stimulus (A or B) in target and nontarget combinations. In a two-alternative-forced choice manner, participants are instructed to respond to both cue and probe stimuli with left or right trigger buttons on a joystick or computer keyboard. In the target AX sequence, X probes following A cues demand a right trigger press; all other cues and probes are to be responded to with the left trigger; 70% of trials are composed of AX cue–probe target pairs, entailing a left–right cue–probe response sequence, and AY, BX, and BY cue–probe nontarget pairs are much more rare (10% trials of each). Habitual responses are expected for AX sequences, whereas AY cue–probe pairs demand reactive control in responding to Y. B cues are expected to elicit proactive control, as the upcoming probe response can be fully prepared. a The delay between cue and probe stimuli is fully known, remaining at 1,000 ms for 250 consecutive trials. b The delay between cue and probe stimuli is jittered (randomly) at around 3,000 ms (±500 ms). c The delay between cue and probe is randomly chosen each trial to be either 1,000 ms or 3,000 ms. D. DPX cue and probe stimuli corresponding to AX-CPT cues and probes. In DPX, 68.75% of trials are AX, 12.5% AY, 12.5% BX, and 6.25% BY

This expectancy version of the AX tasks was developed out of an earlier line of continuous performance test (CPT) work in the 1950s (Rosvold, Mirsky, Sarason, Bransome, & Beck, 1956) in order to study the effects of expectancy and context on cognitive control (J. Cohen & Servan-Schreiber, 1992; Servan-Schreiber et al., 1996). In the original continuous performance test, participants would detect target events in a series of stimuli (e.g. “Respond to X” or “Respond to X only when it follows A”). Persons with schizophrenia showed impaired performance on this task, and these deficits were exacerbated in versions of the task that depended on maintenance of task context (“. . . . only when it follows”; J. Cohen & Servan-Schreiber, 1992). Through computational models of performance in the continuous performance task and other attention-demanding tasks, it was shown that the internal representation of context information is critical for successful task performance, and researchers hypothesized that this may be the key functional deficit underlying behavioral impairments in people with schizophrenia (J. Cohen & Servan-Schreiber, 1992). As such, the expectancy AX-CPT was designed to specifically elicit deficits in context processing (Servan-Schreiber et al., 1996).

Common A and rare B cues introduce different contexts, with distinct rules to follow for the forthcoming common X or rare Y probe stimuli. AY and BX sequences thus require the use of distinct types of cognitive control and are most commonly used as dependent variables of interest in AX-CPT and DPX tasks. AY pairs require reactive cognitive control to overcome the prepotent AX response. Accordingly, errors on the Y trial are thought to result from greater use of proactive control (e.g., the typical AX response is overprepared). Conversely, BX pairs require proactive cognitive control to maintain the rare B cue rule over the cue–probe delay period, so that the common X probe can elicit the correct, rare, BX response. Poor performance on BX trials is associated with failures in proactive control. The Behavioral Shift Index (Braver, Paxton, Locke, & Barch, 2009) serves as a composite measure of AY and BX error rates or reaction times ((AY − BX) / (AY + BX)), to quantify the balance between proactive and reactive control styles within an individual. Given that the inclusion of AY and BX means and standard deviations in most manuscripts facilitates the calculation of standardized mean differences, and that AY-BX error rate and reaction time indices capture complementary differences in exertion of proactive and reactive control, we use AY-BX differences as outcome measures of proactive versus reactive control in this meta regression.

As described above, the DPX differs from AX-CPT in stimulus type, using dots instead of letters. Although prior work has found some differences in factors explaining performance of the two tasks (MacDonald et al., 2005), here we collapse across AX-CPT and DPX paradigms in order to gain statistical power and make broader conclusions about the impact of task structural parameters (vs. task stimuli).

Delay knowledge in the AX-CPT and DPX literature

The majority of AX-CPT and DPX experiments use a known cue–probe delay length, consisting of either a single delay throughout the experiment, or delays varying by block. This makes it easy to develop a task rhythm and anticipate the timing of the upcoming probe stimulus. However, delay length is not always known. Some studies have jittered the cue–probe delay length within a small interval, adding some unpredictability to probe onset timing. In contrast, other studies have interspersed short and long delays within experimental blocks, such that the delay length for each trial could not be anticipated. Here, we formally investigate differences between small, largely imperceptible interval variation due to jitter (<500 ms) and large “unknown” delay variations that may more meaningfully interact with time estimation.

Because the use of known versus unknown delays changes the structure and prediction demands of the task, we hypothesized that studies with different delay lengths would alter peoples’ use of proactive versus reactive control. First, we hypothesized that full knowledge of the upcoming delay would significantly bias participants toward the use of proactive control, as they would be able to prepare to respond at the appropriate time. In contrast, we expected that studies with a jittered delay would bias participants toward exerting reactive control, and that this effect would be exacerbated by a completely unpredictable upcoming delay.

Delay length in AX-CPT and DPX literature

Throughout the AX-CPT and DPX literature, the delay length between an informative cue and a test probe (cue–probe delay, or CPD) has varied widely, and is most often considered an incidental parameter and given no or scarce discussion. This is theoretically important, as information in the phonological loop of working memory is thought to decay in about 2 seconds, unless actively refreshed by some rehearsal process (Baddeley, Thomson, & Buchanan, 1975). If cue rule information is maintained differently over short versus long delays, variation in this parameter may assess distinct cognitive processes. Indeed, our recent work suggests there are reliable differences in brain activation to the rare B cue that solely depend on delay length (Janowich & Cavanagh, under review).

In several AX-CPT and DPX studies, manipulation of the cue–probe delay has been used to assess context maintenance aspects of cognitive control, as measured by BX performance (Barch et al., 2009; MacDonald et al., 2005). Context maintenance refers to an internal representation of information (e.g., task goals), held in mind in order to mediate an appropriate behavioral response (J. Cohen & Servan-Schreiber, 1992). By quantifying whether proactive/reactive control behavior differs based on delay parameters, we can begin to understand whether context maintenance is utilized similarly/universally in all delay contexts, or is subject to timing demands.

In addition to the effects of cue–probe delay on context maintenance, this meta-analysis addresses how cue–probe delay may also alter goal-switching control upon encountering a rare AY cue–probe sequence. The current metaregression offers a distinct and important contribution to the literature, in that the focus specifically on the AX-CPT and DPX tasks enables us to bring to light how delay conditions may alter both goal-switching control (AY) and context maintenance (BX).

In our healthy young adult meta-analysis sample, we hypothesized that short cue–probe delay lengths would bias participants toward (over-) exerting proactive control, such that the immediacy of the upcoming probe would require use of a strong prepotent stimulus–response preference. Conversely, long cue–probe delay lengths may shift participants toward reactive control, as it might be too cognitively taxing to undergo many seconds worth of active rehearsal.

Some may question whether the intertrial interval (ITI) is (also) important in shaping the interaction between proactive and reactive control; therefore, we have included ITI as a moderator in our analyses although we have no specific hypotheses about this parameter.

Trial set count in AX-CPT and DPX literature

AX-CPT and DPX tasks are premised upon the exertion of control over rare cue and probe stimuli, yet the number of trials of repeated behavior (over which habits to respond are developed and strengthened) varies widely. Trial set counts are defined in this manuscript as the number of trials performed on a distinct task set. We hypothesized that studies with a greater number of trials of repeated behavior will cultivate stronger predispositions to respond to the common (vs. rare) stimulus–response rule, and thus bias toward the use of proactive control.

Standard versus distractor AX-CPT and DPX comparison

Recently, many investigators have modified the AX-CPT and DPX paradigms to include mid-delay distractors (Braver et al., 2001; Fröber & Dreisbach, 2016; Gómez-Ariza, Martín, & Morales, 2017; Maraver, Bajo, & Gomez-Ariza, 2016; Morales, Gómez-Ariza, & Bajo, 2013). This modification may be useful in increasing the difficulty of maintaining cue stimuli over the delay and preventing “ceiling” performance in healthy young adults. However, the ramifications of mid-delay distractors on proactive versus reactive control usage has yet to be reviewed. We hypothesized that mid-delay distractors would generally increase the use of reactive control, as the distractors would make it more difficult to maintain the cue and prepare a response. As contending with distractors would occupy considerable cognitive resources, we did not anticipate that control metrics would be moderated by trial set count or delay length.

Young versus slightly older versus older adult comparison

Age has been known to be associated with performance in AX-CPT and DPX tasks, with older (elderly) adults demonstrating decrements in proactive control and increases in reactive control (slowed BX performance; Braver et al., 2001; Paxton, Barch, Racine, & Braver, 2008) and more accurate (Braver, Satpute, Rush, Racine, & Barch, 2005) and faster AY performance (Paxton et al., 2008) relative to healthy young adults. We only identified three AX-CPT studies (cited immediately above) conducted with older adults, and as such report only basic summary comparisons between age groups. Because of the very small number of studies, we are underpowered to analyze the effects of moderator variables on older adult performance. As this review focuses on the influence of task parameters on normative task performance, we do not focus further on studies run in older adult samples.

Many studies include slightly older, healthy adults (ages 30–45 years), typically matched to participants with schizophrenia. The potential difference in performance between these slightly older, healthy adults versus college-aged students has not been addressed. This is important because it is unclear whether this age-related change in proactive versus reactive control occurs in middle adulthood, and whether it interacts with delay-related factors mediating control. We hypothesized that younger adults would show stronger proactive control compared to slightly older, healthy adults.

Schizophrenia subgroup comparison

AX-CPT and DPX tasks have been used to quantify abnormalities of proactive and reactive control in special populations, particularly aging and participants with schizophrenia. These special populations are characterized by disproportionate difficulty on BX (context maintenance) trials (Barch et al., 2009; J. D. Cohen et al., 1999), suggesting poorer proactive control. However, with common variation in task parameters, it is difficult to ascertain the underlying cognitive processes responsible for these deficits. We hypothesized that the population of people with Schizophrenia would show a bias toward reactive control (as has been reported widely in the literature), and that this bias toward reactive control would be strengthened with increasing delay length due to increased difficulty on BX context maintenance trials.

The current investigation

In this metaregression, we aimed to test the following three a priori hypotheses: (1) delay knowledge, (2) delay length, and (3) trial set count moderate the use of proactive versus reactive control. We also tested the effects of mid-delay distractors and ITI parameters on control, although we had no specific hypotheses about these parameters. To understand how control varies in different experimental populations, we investigated if prior findings of reduced proactive control in elderly adults extend to slightly older adults (ages 30–45 years), and if findings of reduced proactive control in persons with schizophrenia are dependent on these parameter differences between studies. Finally, we detail descriptive patterns across the literature and methodologies, as we have noticed that EEG studies tend to use different parameters than behavioral or fMRI studies. Implications for this parameter difference between modalities are discussed further.

Method

A series of metaregressions (Berkey, 1995; Van Houwelingen, Arends, & Stijnen, 2002) were conducted to describe the effects of delay knowledge, cue–probe delay length, and trial set length on AX-CPT and DPX (here forward, expectancy task) measures of proactive versus reactive control. All analyses were conducted using the metafor package (Viechtbauer, 2010) written for R (Version 3.2.2; http://www.R-project.org).

Study identification, screening, and inclusion

Study selection was structured according to the Meta-Analysis Reporting Standards (APA Publications and Communications Board Working Group on Journal Article Reporting Standards, 2008). A full outline of study selection procedures is depicted in Fig. 2. ScienceDirect and PubMed databases were queried using the keywords (“AX-CPT,” “DPX,” and “cognitive”), to gather an initial sample of English-language literature in which the AX-CPT paradigm was used (through September 2017). This yielded 309 abstracts. Peer-reviewed research studies with novel data using AX-CPT and DPX were assessed further; all review papers or reanalyses of prior published data were excluded.

Fig. 2
figure 2

Meta-analysis data selection. Flow chart detailing selection of manuscripts (k) and data-points (dp) to be included in meta-analyses. For manuscripts with multiple experiments, participant subgroups, and/or delay lengths, distinct data points were established. Colored ovals indicate final selection for the primary (purple, blue, and green) and subgroup analyses (yellow, orange, and pink). The bottom section shows the variables assessed as moderators for our outcome control indices (AY-BX error rate and RT)

Further discussion on study selection will differentiate between manuscript selection (“k”) and data-point selection (“dp”), which distinguishes each data set obtained with a distinct delay length, both between experiments within a manuscript, as well as between delay lengths within an experiment. For studies utilizing multiple cue–probe delay lengths and reporting distinct probe behavioral measures, each cue–probe delay length was used as a separate data point. Studies selected for inclusion are accessible in Table 1 (full raw data are available in additional Table l).

Table 1 Selected studies. Table listing all studies and data points included in the meta-analysis, along with the subgroups in which each was categorized and notes regarding which study conditions were selected for inclusion

Study selection: Healthy young adults and schizophrenia patients

Inclusion of manuscripts required AX-CPT or DPX behavioral data from human samples consisting of healthy young adults (ages 18–45 years). For manuscripts also using patient groups, multiple retests, or an experimental intervention, data points were extracted exclusively for the healthy young adults in the control/baseline condition. Data from participants with schizophrenia (k = 7, dp = 11) were included for a later subgroup analysis. Owing to the small selection of studies assessing persons with schizophrenia, the sample of studies includes patients with and without medication (noted in Table 1), and with varying disease duration lengths. One manuscript separated data by patient medication status; each medication group is included as a separate data point.

Study selection: Expectancy paradigm

To ensure comparison across similar expectancy paradigms, studies were included only if they used standard cue/probe proportions (70% AX, 10% each of AY, BX, and BY), or AX proportions within a negligible margin of 70% (±10%). We included 18 AX-CPT data points (from k = 9 studies) deviating slightly from the 70% AX standard (deviant mean = 70.40%, mean deviation from 70% = 5.38%, AX range = 60%–79%). Inclusion required standard AX-CPT or DPX stimuli (intact letters or dots), and a two-alternative-forced-choice response format. Studies in which distractors were presented during the delay (k = 6, dp = 7) were also not included in the primary analyses, but are included in later subanalyses.

Study selection: Age

The expectancy literature consists primarily of studies conducted in college-aged students (k = 31, dp = 46, mean age = 22.2 ± 2.14 SD, range: 19.4–26.0), but also includes many studies using slightly older, healthy adults typically matched to persons with schizophrenia (k = 5, dp = 10, mean age = 37.8, range: 31.6–43.6). As the college-aged and slightly older adults included in expectancy studies appear to be two distinct populations (with statistically different ages, t = −13.007, df = 10.843, p < .001), we conducted our main analyses on the majority group of studies with mean participant age less than 30 years. Later subgroup analyses examined studies with a mean participant age greater than 30 years.

Expectancy studies meeting our standard criteria only include three studies of older (elderly) adults (k = 3, dp = 6, mean age = 72.27). As the goal of our review was to understand the role of task parameters in commonly studied populations, we did not find it appropriate to include this small population in our analyses, nor were we sufficiently powered to conduct moderator analyses on older (elderly) adult data. However, to better situate our contrast between young and slightly older adults, we conduct post hoc comparisons of accuracy and reaction time between older (elderly) adults, young adults, and slightly older adults.

Study selection: Available data

For inclusion in the meta-analysis, studies were required to include information sufficiently describing experimental parameters, including cue–probe delay length, intertrial interval, and trial set count. When multiple delays were included within a study, we needed to know whether delay lengths were separated by block or mixed unpredictably by trial, and see behavioral results parsed by delay length and delay knowledge. For inclusion in accuracy and/or reaction time analyses, studies were required to include the relevant means and standard deviation for probe types “AY” and “BX”. When only standard errors of the mean were available, we computed standard deviation from the SEM and study sample size.

Study selection: Missing data

In the case of any study with missing data, the corresponding author was contacted by email and asked to furnish the additional data. Nineteen authors were contacted on behalf of 24 manuscripts. From this, authors of eight of the manuscripts provided us with the necessary additional data. In cases in which data were not furnished, but graphs of behavioral data were available, we computed precise estimations of behavioral means and standard deviations using ruler functions in Adobe Illustrator.

Study selection: Summary

Based on these criteria, 25 studies, consisting of 45 data points and 1,367 unique healthy young adult participants were included in the primary meta-analyses. Mid-delay distractor analyses included six manuscripts and seven data points. Subgroup analyses for schizophrenia patients included seven manuscripts and 11 data points. Slightly older adult analyses included five manuscripts and 10 data points.

Outcome measures

Error rate and reaction time data means and standard deviations for AY and BX probe stimuli were compiled. When manuscripts reported only standard error of the mean, standard deviation was computed as SD = SEM / sqrt(n). AY and BX cue–probe combinations have been established as markers of proactive and reactive control, and their relationship has been used to assess ratios of proactive versus reactive control, with higher (AY − BX / AY + BX) scores indicating greater proactive control and lower scores indicating greater reactive control (Braver, 2012; Braver et al., 2009).

Separate outcome measures of effect size for error rate and reaction time were created with Cohen’s d_av (Cumming, 2012; Lakens, 2013). Because the correlations between pairs of (AY and BX) observations (r) were not available, standardized mean differences (Borenstein, 2009; Borenstein, Hedges, Higgins, & Rothstein, 2009) were calculated using a formula designed for independent groups, with standard deviation computed as the within-groups standard deviation pooled across groups

The standardized mean difference (effect sizes) were computed by dividing the mean AY − BX difference by the within-groups standard deviation for AY and BX, pooled across groups:

$$ \frac{\left(\mathrm{mean}\ \mathrm{AY}\hbox{-} \mathrm{mean}\ \mathrm{BX}\right)/}{\left(\mathrm{sqrt}\ \left(\left(\mathrm{N}\hbox{-} 1\right)\ast \mathrm{AY}\ {\mathrm{stdev}}^{\wedge }2+\left(\mathrm{N}\hbox{-} 1\right)\ast \mathrm{BX}\ {\mathrm{stdev}}^{\wedge }2\right)\right)/\left.\left.\left({2}^{\ast}\mathrm{N}\hbox{-} 2\right)\right)\right)}. $$

Variances were calculated separately for error rate and reaction time, using a between-subjects formula:

$$ \left(\left(2\ast \mathrm{N}/{\mathrm{N}}^{\wedge }2\right)+\left(\left({\left[\kern0.2em \mathrm{error}\kern0.34em \mathrm{rate}\kern0.34em \mathrm{OR}\kern0.34em \mathrm{reaction}\kern0.34em \mathrm{time}\_\mathrm{av}\right]}^{\wedge }2\right)/\left({4}^{\ast}\mathrm{N}\right)\right)\right) $$

Confidence intervals were estimated at 95% to assess the likelihood of a given study’s results of containing the true population mean.

Methods: Metaregression procedures

In all analyses, more positive effect sizes indicate greater use of proactive control, whereas more negative effect sizes indicate greater use of reactive control. With our composite measures of AY-BX performance, we cannot precisely distinguish increased proactive control from decreased reactive control, but we consider the general proportional shifts in use of proactive versus reactive control on a continuous spectrum.

Baseline metaregressions

Initially, we established a baseline summary of expectancy task performance in healthy young adults using a fixed-effects model. The fixed-effects model enables only conditional inference about the existing literature (Hedges & Vevea, 1998), but is important in guiding interpretation of existing studies in light of any effect of delay or trial parameters on performance.

Following the fixed-effect model, we conducted a baseline random-effects metaregression. A random-effects meta-analysis model was used to allow for true variance in proactive/reactive behavior between studies, in addition to sampling variance (Riley, Higgins, & Deeks, 2011). Random-effects analyses more conservatively accounts for the variance between studies’ methods and sample characteristics by treating each study’s variance as purely random (Viechtbauer, 2010). As such, the random-effects model can be used to make unconditional inference about similar studies outside of the meta-analysis sample. The baseline random-effects model established the level of variance between studies, without any moderators taken into account.

For all random-effects metaregressions, we used the restricted maximum likelihood estimation (REML) method, and computed unbiased estimates of the sampling variances (vtype = “UB”). Knapp and Hartung adjustments (Knapp & Hartung, 2003) to the standard Wald-type tests were always applied (test = “knha”). The Knapp and Hartung adjustment helps to better control for the Type I error rate in mixed-effect metaregressions.

Moderators: Simple main effects

We then ran a series of univariate random-effects meta-regressions to understand the simple main effects of delay knowledge, delay length, and trial set count (separately) on accuracy and reaction time Behavioral Shift Index composites. First, we conducted a set of random-effects metaregressions assessing the moderating effect of delay knowledge on proactive/reactive accuracy and reaction time. Next, to understand the effect of delay length on expectancy task performance (both accuracy and reaction time measures), we applied a mixed-effects model with delay length as the continuous moderator hypothesized to account for variability in the true effects (Viechtbauer, 2010). A mixed-effects model assesses the effect of the moderator (delay length) at the study level, while also assuming random variance between studies, and computes the amount of variance accounted for by this moderator. Although we did not hypothesize that intertrial interval would alter task performance, we added intertrial interval as an additional moderator, addressing concerns that intertrial interval, or its interaction with cue–probe delay, might account for variation in task performance. We then ran a set of mixed-effects metaregressions with trial set count as the moderator variable.

Moderators: Interactions

After quantifying the simple moderating effects of delay knowledge, delay length, and trial set count separately, we conducted univariate random-effects metaregressions to understand their interactions (Delay Knowledge × Delay Length, Delay Knowledge × Trial Set Count, and Delay Length × Trial Set Count). All interaction analyses included random effects for both the individual data point and the delay knowledge subgroup.

Subgroup analyses

Finally, we ran a similar series of metaregressions for our subgroups of interest: persons with schizophrenia, slightly older adults, and studies with mid-delay distractors. Procedures were repeated as described above for the main study sample, but did not include delay knowledge analyses, as all subgroup studies included a known delay.

Results

All results are for the primary analyses on healthy young adults in standard AX-CPT and DPX paradigms, unless explicitly stated otherwise. Forest plots were generated to summarize between-study variation (Lewis & Clarke, 2001) in accuracy (Fig. 3) and reaction time (Fig. 4) metrics of proactive versus reactive control.

Fig. 3
figure 3

Forest plot of proactive/reactive control error rate difference. Forest plot ordered by subgroup, delay knowledge, and trial set count. Cue–probe delay (CPD) (ms) and intertrial interval (ITI) (ms) are also included for reference. Scores reflect the standardized mean difference of AY-BX error rate and 95% confidence interval (CI), with more negative scores indicating greater reactive control and more positive scores indicating greater proactive control. Triangles on the CI bars indicate CIs that exceed the plotting range of standardized mean differences. Colored diamonds show the random effects model summary scores for each subgroup, and the black diamond at the base shows the overall random effects model summary for all studies combined. (Color figure online)

Fig. 4
figure 4

Forest plot of proactive/reactive control reaction time difference. Forest plot ordered by subgroup, delay knowledge, and trial set count. Cue–probe delay (CPD) (ms) and intertrial interval (ITI) (ms) are also included for reference. Scores reflect the standardized mean difference of AY-BX reaction time and 95% confidence interval (CI), with more negative scores indicating greater reactive control and more positive scores indicating greater proactive control. Triangles on the CI bars indicate CIs that exceed the plotting range of standardized mean differences. Colored diamonds show the random effects model summary scores for each subgroup, and the black diamond at the base shows the overall random effects model summary for all studies combined. (Color figure online)

Delay and trial parameters by behavior/imaging modality

We first ran a set of one-way ANOVAs on all studies in our meta-analysis to understand whether delay length or trial set count differed between studies of different imaging modalities (behavior vs. EEG vs. fMRI). We found that AX-CPT and DPX delay lengths differ between imaging modalities, F(2, 70) = 6.472, p = .003: EEG studies use significantly shorter cue-probe delays (n = 12, mean = 1.86 s) than behavioral studies (n = 46, 3.08 s; EEG vs. BEH t = −3.645, p < .001, Cohen’s d = −.843) or fMRI studies (dp = 15, mean = 4.44 s; EEG vs. fMRI t = −4.146, p < .001, Cohen’s d = −1.496). In addition, cue–probe delay length was negatively correlated with trial set count, F(1, 67) = 7.282, p = .009, R2 = .084, and trial set counts were significantly different by modality, F(2, 66) = 34.77, p < .001, being larger in EEG studies relative to both behavioral (EEG vs. BEH t = 4.803, p < .001, Cohen’s d = 2.391) and fMRI (EEG vs. fMRI t = 5.108, p < .001, Cohen’s d = 2.169) studies. The outcomes of meta-analytic findings reported below should be considered in light of these systematic variations between different modalities, particularly as threats to external validity.

Baseline variation in accuracy and reaction time metrics

We first tested for meaningful between-study variation in both accuracy and reaction time indices of control. In a fixed-effects univariate metaregression, we observed significant variance in the accuracy outcome measure, Q(df = 44) = 300.442, p < .001, z = 11.591. We also observed significant variance in the reaction time outcome measure, Q(df = 43) = 400.614, p < .001, z = 25.260.

In a random-effects univariate metaregression, we observed significant variance in the accuracy outcome measure, Q(df = 44) = 300.442, p < .001, t = 5.355, tau2 = .325, SE = 0.084, I2 = 86.61%, H2 = 7.47. We also observed significant variance in the reaction time outcome measure, Q(df = 43) = 400.614, p < .001, t = 10.213, tau2 = .461, SE = 0.116, I2 = 89.51%, H2 = 9.53.

Differences in AX-CPT versus DPX paradigms

We conducted univariate random-effects meta-regressions to test the effect of stimulus type: AX-CPT letters versus DPX dots as a categorical moderator. In healthy young adults, (AX-CPT dp = 41; DPX dots dp = 4) there was no significant effect of paradigm on accuracy (p = .469) or reaction time (p = .266). In slightly older adults (AX-CPT dp = 5; DPX dp = 5), there was no significant effect of paradigm on accuracy (p = .530). Only two DPX data points (and four AX-CPT data points) in slightly older adults included reaction time data, so we were underpowered to detect potential paradigm-evoked differences in reaction time in slightly older adults (p = .051).

Main effects: Delay knowledge

Univariate random-effects meta-analyses for accuracy and reaction time were conducted with delay knowledge as a categorical moderator (known vs. jittered vs. unknown). Overall, delay knowledge did not account for a significant portion of variance in accuracy (R2 = 8.55%), F(1, 42) = 2.159, p = .128. The difference in accuracy for studies with unknown versus known delays was significant, F(1, 42) = 4.255, p = .045, but accuracy in studies with unknown versus jittered delays did not differ, F(1, 42) = 1.832, p = .183, nor did studies with known versus jittered delays, F(1, 42) = .000, p = .984.

Overall, delay knowledge did account for a significant portion of variance in reaction time (R2 = 19.43%), F(1, 41) = 4.993, p = .011. Reaction time differed significantly for studies with unknown versus known delays, F(1, 41) = 9.811, p = .003, but there was no difference in reaction time for studies with unknown versus jittered delays, F(1, 41) = 1.942, p = .171, nor for studies with known versus jittered delays, F(1, 41) = .697, p = .409. In summary, delay knowledge explained significant variance in RT, with known delay driving relatively increased RT indices of proactive control.

Main effects: Cue–probe delay length and intertrial interval

We conducted univariate random-effects metaregressions for accuracy and reaction time, with cue–probe delay length and intertrial interval (ITI) as continuous moderators. Delay length was not a significant moderator of accuracy, F(1, 41) = .049, p = .827, nor was ITI, F(1, 41) = .108, p = .744. The delay–ITI interaction for accuracy was also not significant, F(1, 41) = .245, p = .623.

Delay length was not a significant moderator of reaction time, F(1, 40) = .205, p = .653, nor was ITI, F(1, 40) = .027, p = .871. The delay–ITI interaction for reaction time was also not significant, F(1, 40) = .375, p = .544. In summary, contrary to our hypothesis, delay length did not explain meaningful variance in accuracy or RT relevant to proactive versus reactive control. In addition, ITI also had no effect on control metrics.

Main effects: Trial set length

We conducted univariate random-effects metaregressions for accuracy and reaction time, with trial set length as a continuous moderator. Trial set length was a significant moderator of accuracy (R2 = 5.71%), F(1, 41) = 4.562, p = .039, such that increased trial set count led to accuracy index measures of greater proactive control.

Trial set length was a significant and robust moderator of reaction time, F(1, 40) = 10.967, p = .002, accounting for 21.89% of variance (R2 = 21.89%), such that increased trial count led to RT index measures of greater proactive control. In summary, both accuracy and RT measures of proactive versus reactive control were altered by trial set length, with increased trial set length associated with greater proactive control.

Interactions: Delay known by delay length and intertrial interval

In a series of univariate mixed-effects metaregressions, we assessed whether there was an interaction between delay knowledge and delay length or ITI in moderating accuracy or reaction time. We found no significant interaction of delay knowledge (known vs. unknown) and delay length on accuracy, F(1, 40) = 1.035, p = .315. The interaction of delay knowledge (known vs. unknown) and ITI also did not have a significant moderating effect on accuracy, F(1, 39) = 1.070, p = .307.

The interaction of delay knowledge (known vs. unknown) and delay length did not significantly moderate reaction time, F(1, 39) = .106, p = .746. However, the interaction of delay knowledge (known vs. unknown) and ITI was a significant moderator of reaction time, F(1, 38) = 5.285, p = .027. Overall, the interaction of delay knowledge and ITI accounted for a significant amount of reaction time variance (R2 = 33.68%), F(1, 38) = 4.054, p = .005. In summary, the interaction of ITI length and delay knowledge was a significant moderator of the RT index of proactive versus reactive control, with longer ITIs associated with less proactive control, but the effect was only present for known delays.

Interactions: Delay known by trial set count

In a set of univariate mixed-effects metaregressions, we assessed whether there was an interaction between delay knowledge (factor) and trial set count (as a continuous variable) in moderating accuracy or reaction time. We observed a significant and robust interaction of delay knowledge and trial count on moderating accuracy, F(1, 37) = 4.350, p = .003; these variables accounted for 38.58% of accuracy variance. Following up this significant interaction, the interaction of known versus unknown delay studies with trial set count was strongly significant, F(1, 37) = 12.373, p = .001. There was no interaction involving jittered versus known studies, F(1, 37) = .292, p = .592, nor jittered versus unknown studies, F(1, 37) = .353, p = .556.

The interaction of delay knowledge and trial set count was a significant and robust moderator of reaction time, F(1, 36) = 5.412, p < .001, accounting for 42.28% of variance. Following up this significant interaction, we found that the interaction of known versus unknown delay studies with trial set count was significant, F(1, 36) = 4.586, p = .039, whereas the interactions with jittered versus known, F(1, 36) = .038, p = .846, and jittered versus unknown, F(1, 36) = .750, p = .392, studies were not significant. In summary, the interaction of delay knowledge and trial set count was a robust and significant predictor of control metrics for accuracy and reaction time, with known delay studies of high trial count associated with the highest rates of proactive control.

Interactions: Trial set count by delay length and intertrial interval

A series of univariate mixed-effects metaregressions were run to understand whether there was an interaction between trial set count and delay length or ITI on accuracy or reaction time. The interaction between trial set count and delay length did not moderate accuracy, F(1, 39) = .000, p = .995, nor did the interaction between trial set count and ITI, F(1, 39) = .046, p = .831.

Trial set count and delay length did not show a significant interaction for reaction time, F(1, 38) = .310, p = .581, nor did trial count and ITI, F(1, 38) = .121, p = .730. In summary, neither trial count nor ITI interacted with trial set count to moderate accuracy or RT control indices.

Subgroup: Mid-delay distractors

Healthy young adult accuracy in standard expectancy paradigms did not differ from that in paradigms with mid-delay distractors (dp = 7), F(1, 69) = .122, p = .728, but reaction time was marginally different, F(1, 61) = 3.548, p = .064. All studies with distractor paradigms were run with fully known delay lengths, so delay knowledge is not included in any analyses for this subgroup. Accuracy was not moderated by delay length, F(1, 5) = .056, p = .823, nor ITI, F(1, 5) = .733, p = .431, nor trial set count, F(1, 5) = .002, p = .964. Reaction time was not moderated by delay length, F(1, 5) = .453, p = .531, nor ITI, F(1, 5) = .731, p = .431, nor trial set count, F(1, 5) = 1.131, p = .3360. In summary, paradigms with mid-delay distractors did not show significant control biases, relative to standard paradigms. Distractor paradigm control metrics were not modified by delay length nor ITI nor trial set count.

Subgroup: Healthy, slightly older adults

Healthy, slightly older adults (mean age >30; k = 5, dp = 10, mean age = 37.8 years, range: 31.6–43.6) differed significantly from healthy young adults (mean age < 30; k = 31, dp = 46, mean age = 22.2 ± 2.14 SD, range: 19.4–26.0) in accuracy, F(1, 69) = 7.392, p = .008, but not reaction time, F(1, 61) = .388, p = .536, indices of control. All studies with slightly older adults were run with fully known delay lengths, so delay knowledge was not included in any analyses for this subgroup. We used univariate metaregressions to assess the effects of delay length, ITI, and trial set count in slightly older adults (dp = 10). Delay length did not moderate accuracy, F(1, 8) = 1.345, p = .280, nor did ITI, F(1, 8) = .444, p = .524. Trial set count conferred a marginally significantly effect on accuracy accounting for 24.80% of variance, F(1, 8) = 4.319, p = .071. Increasing trial set count was associated with a trend toward decreased accuracy index of proactive control, which is the opposite direction from the trial set effects in healthy young adults. This effect of trial set count between younger and slightly older adults was marginally significant, F(1, 47) = 3.246, p = .078. Reaction time was not moderated by delay length, F(1, 4) = .664, p = .461, nor ITI, F(1, 4) = 1.550, p = .281, nor trial set count, F(1, 4) = 4.543, p = .100.

In post hoc analyses, older (elderly) adult accuracy and reaction time was compared with that of slightly older and young adults. Accuracy did not differ between slightly older adults and older (elderly) adults, F(1, 58) = .298, p = .587, whereas reaction time metrics of control did differ between slightly older and older (elderly) adults, F(1, 53) = 7.715, p = .008, with older (elderly) adults showing greater reactive control. As expected, both accuracy, F(1, 58) = 7.334, p = .009, and reaction time, F(1, 53) = 8.773, p = .005, differed between older (elderly) adults and young adults.

In summary, slightly older adults showed accuracy performance that was similar to that in older (elderly) adults and significantly less proactive than that in young adults. Conversely, slightly older adult reaction time metrics were similar to that in younger adults, and more proactive than those shown in older (elderly) adults. Slightly older adults also showed a marginally significant effect of trial set length on accuracy. Interestingly, increasing trial set count tended to decrease proactive control, which was an opposite pattern from that in young adults. This effect was marginally different between groups, where more trials led to a greater effect size differentiation between healthy young and slightly older adult participants.

Subgroup: Schizophrenia

Studies in persons with schizophrenia included four studies sampling young adults with schizophrenia (k = 4, mean age = 22.0 years), six studies sampling slightly older adults with schizophrenia (dp = 6, mean age = 37.7), and one study with unreported sample age. When compared to their age-matched controls, young adults with schizophrenia did not differ in accuracy (dp = 7), F(1, 5) = 1.620, p = .259, nor reaction time (dp = 7), F(1, 5) = 1.786, p = .239, from healthy young adults. In contrast, slightly older adults with schizophrenia showed significantly different (more reactive) accuracy than their age-matched healthy (slightly older) adults (dp = 12), F(1, 10) = 12.744, p = .005. Reaction time metrics did not differ between slightly older adults with schizophrenia and healthy slightly older adults (dp = 7), F(1, 5) = 1.350, p = .298.

All data points with these samples were run with fully known delay lengths, so delay knowledge was not included in any analyses for this subgroup. We used univariate metaregressions to assess the effects of delay length, ITI, and trial set count in participants with schizophrenia. We collapsed across age for moderator analyses due to the small number of studies in each age range. Accuracy was not moderated by delay length, F(1, 9) = .011, p = .920, but ITI showed a marginally significant effect, F(1, 9) = 4.721, p = .058, R2 = 21.39%. Trial set count was a very strong moderator of accuracy, F(1, 7) = 25.969, p = .001, R2 = 100.00%, such that increasing trial set count was associated with increased proactive control. This effect of trial set count on accuracy was similar to that found in healthy young adults, F(1, 46) = 2.233, p = .142. Reaction time was not moderated by delay length, F(1, 6) = .778, p = .412, nor ITI, F(1, 6) = 1.035, p = .348, nor trial set count, F(1, 4) = 2.825, p = .168.

In summary, slightly older adults with schizophrenia showed more reactive accuracy performance compared with healthy, slightly older adults, but there were no differences in performance between young adults with schizophrenia and their healthy young adult controls. Collapsing across age, trial set count was the only moderator to bias performance in schizophrenic patients, enhancing proactive control accuracy indices in a similar manner as in healthy young adults.

Discussion

In this series of metaregressions, we quantified the moderating influence of several experimental parameters that vary throughout the AX-CPT and DPX literature. In healthy young adults, we found that delay knowledge and trial set count, but not delay length or ITI, were significant moderators of behavior. Delay knowledge increased the reaction time index of proactive control, and comparison of known versus unknown delay type revealed differences in reaction time as well as accuracy, such that known delays were associated with increased indices of proactive control. Trial set count moderated both accuracy and reaction time, with increasing trial count associated with increased proactive control. Finally, the interaction of trial count and delay type conferred significant additional predictive benefits for accuracy and reaction time, such that the effects of trial set count were stronger in studies with a known delay.

Importantly, we observed that delay parameters and trial set count differs between imaging modalities, such that EEG studies use significantly shorter cue–probe delays and have higher trial set counts than behavioral or fMRI studies. Although the choices of delay length may be incidental to the need for a longer delay time in fMRI and practical benefits to shortened trial length, these systematic differences in delay length render comparison across AX-CPT and DPX studies problematic. Even though we do not find that delay length moderates AY-BX behavioral metrics of control, delay length may still be an important variable in studies examining neural correlates of control. Further, EEG-measured neural correlates of control may not be directly generalizable to those observed during fMRI due to different cognitive processes evoked by larger versus smaller trial set counts.

Beyond highlighting the methodological importance of parameter selection in continuous performance tasks, these meta-analytic findings help us understand more generally how cognitive control might work. We observed that knowledge of delay duration biases performance toward proactive control, suggesting that the ability to plan to execute a task at a precise time increases the amount or robustness of preparation. Alternatively, the lack of temporal knowledge might bias toward a “choice” to not activate proactive preparation systems, saving valuable cognitive resources. As AX-CPT and DPX tasks are commonly used to study working memory performance, it is important to consider that distinct working memory processes might be elicited when different proactive/reactive strategies are utilized.

As trial count increases, both accuracy and reaction time metrics of proactive control increase, suggesting that preparatory control processes become more automatic or habitual as they are repeatedly executed. Intriguingly, this effect becomes much stronger (explaining ~40% of variance in performance) when delay length is known (vs. unknown). This finding suggests that it is not just the repetition of a process that habituates control, but even more so the rhythmic, temporally predictable repetition of that process. In support of the importance of rhythmic predictability for habituation of control, we found that studies with middelay distractors did not differ significantly from standard expectancy studies, but failed to show moderating effects of delay knowledge or trial set count (as observed in standard studies). Whether this rhythmic predictability also facilitates different mechanisms for proactive context maintenance or reactive inhibition is a pressing question for future work.

In contrast to the robust increase in proactive control with trial count observed in healthy young adults, slightly older adults do not show this effect, and in fact greater trial set count here is associated with an opposite trend toward greater reactive control. Importantly, slightly older adults showed reactive accuracy performance similar to that in older (elderly) adults, and significantly less proactive than that in young adults. However, slightly older adult reaction time metrics were similar to that in younger adults, and more proactive than those shown in older (elderly) adults. These findings are important because slightly older adults are typically compared with participants with schizophrenia without addressing potential changes in control preferences in from healthy young adulthood to healthy middle age. More aging studies are needed to test how proactive and reactive changes in slightly older adulthood facilitate this shift from proactive to reactive accuracy performance.

In studies of persons with schizophrenia, we observed an interesting distinction between young adults with schizophrenia and slightly older adults with schizophrenia. Young adults with schizophrenia showed similar control ratios compared with their age-matched controls, whereas slightly older adults with schizophrenia showed more reactive accuracy than their age-matched controls. This may suggest that over time, the disease limits the efficacy of proactive control systems or biases toward reactive control processes. However, these age-based findings are based on analysis of only four (young) and six (slightly older) schizophrenia data points, and should be interpreted with caution. Collapsing across age, there is a strong effect of trial count in persons with schizophrenia, with greater trial repetition associated with greater proactive control. This effect of trial count, similar to that observed in healthy young adults, is interesting because it suggests that the context maintenance deficits (failures in rare cue BX trials) long observed in this population could be altered in part by extended rhythmic task repetition.

The lack of significant influence of either cue–probe delay length or ITI was surprising, and contrary to our hypotheses. One possible explanation is that although specific timing intervals do not alter the ratio of proactive versus reactive control (with delay knowledge already instantiating proactive control), timing demands may vary the instantiation and type of proactive control. Supporting this hypothesis, an EEG experiment examining AX-CPT and DPX instantiation at different cue–probe delay intervals does show distinct neural signatures during the cue–probe delay based on delay length (Janowich & Cavanagh, under review). It is possible that manifest behavioral indicators are too crude to reveal subtle delay-induced changes on the relative influence of difference control systems. A prior meta-analysis (Lee & Park, 2005) surveyed the relative impact of increasing delay length on overall working memory performance in persons with schizophrenia versus healthy controls, and also found no significant relationship.

Limitations and future directions

Our study focused on understanding the effects of delay knowledge, delay length, and trial set length in the AX-CPT and DPX literature. There are several limitations to this meta analysis, as well as many potential confounding factors that should be considered in its interpretation. First, although we limited our selection of studies to those with standard ~70% AX proportions, we included studies within a 10% range of the standard. We were underpowered to detect changes as a result of slightly varying expectancy, but this factor may play a role in explain some residual between-study variance. The expectancy studies included in the metaregression sample did vary in several aspects that are beyond the scope of this paper, but may be influential, including behavioral/imaging modality, overall task session length, response time-out speed, or cultural differences in the populations from which study samples were gathered.

In our meta-analysis, we collapsed across AX-CPT and DPX studies, which varied only in stimulus type (letters vs. dots). Only one prior study has directly compared these paradigms (with otherwise identical parameters) in the same sample of healthy young adults, and found similar behavioral performance, as well as general engagement of the same brain networks (Lopez-Garcia et al., 2015). However, in slightly older adults, a large-scale study (n = 131) did show a general decrease in performance for DPX relative to AX-CPT (Strauss et al., 2014). A post hoc test of our meta-analysis data showed that there were no significant differences in accuracy nor reaction time control metrics based on use of AX-CPT versus DPX paradigms, but future work may be needed to understand how differences in paradigm could alter other aspects of control processing.

Finally, our study used the standardized mean differences of AY-BX for accuracy and RT as our outcome measures. Although we discuss results in terms of changes toward proactive or reactive control, composite measures of AY-BX performance cannot fully disentangle whether a composite shift toward proactive control is due specifically to enhancement of proactive control (improvement on BX trials), a weakening of reactive control (worsening performance on AY trials), or a combination of both. Detailed statistical analysis of specific AY and BX differences is beyond the scope and data available for this meta-analysis. However, we observed trends in healthy young adults showing increasing AY errors and in persons with schizophrenia showing decreasing BX errors with increasing trials, allowing us to speculate that healthy young adults exhibit a relative weakening of reactive control with increasing trial count, whereas persons with schizophrenia exhibit a strengthening of proactive control.

A major limitation to the calculation of our AY-BX outcome measures is that individual correlations between AY and BX were not available from the literature. As such, we were forced to rely on between-subject formulas to calculate effect size and variance. If more complete data were to come available, a follow-up analysis should be conducted, estimating r (the correlation between AY and BX) from related studies, and performing a sensitivity using a range of plausible correlations (Borenstein et al., 2009).

Although a AY-BX subtraction measure is similar to the commonly used Behavioral Shift Index (Braver et al., 2009), which has been useful in parsing proactive versus reactive control, other task performance measures (i.e.; BX-AX) may better capture important aspects of cognition. Different cue–probe pairs in the AX-CPT and DPX paradigms have been shown to reflect distinct aspects of cognitive processing. In a large-scale confirmatory factor analysis, context processing was strongly correlated with BX cue–probe performance, and this relationship showed convergent validity across AX-CPT and DPX tasks (MacDonald et al., 2005). Context processing shared significant variance with both intellectual functioning and working memory. AY trials, in contrast, loaded onto the preparatory factor (and shared more variance with preparatory factor in DPX than in AX-CPT). Preparatory factor shared significant variance with working memory, but not intellectual functioning. Overall, behavioral response to AY-BX probes does seem to capture a convergence of context processing and preparation, but other outcome measures should be considered in future analyses.

Future studies should advance these meta-analytic findings by methodically assessing the parameters tested in this study. For instance, a future study could compare participants’ performance on an expectancy task with known delay and unknown delay blocks, to directly understand the varied processes evoked by delay knowledge. In addition, an large-scale experiment could be run on Amazon Mechanical Turk, testing trial set counts ranging from 50 to 500 (in intervals of 25), to understand the exact nature of the relationship between trial set count and control. Finally, neuroimaging studies (and meta-analyses) could be conducted to investigate neural differences based on delay knowledge, trial count, as well as parameters not explicitly associated with behavior, like delay length.

Conclusions

The present series of meta-regressions revealed significant and robust effects of delay knowledge and trial set count on error rate and reaction time metrics of proactive versus reactive control. In healthy young adults, studies with full knowledge of upcoming delay length shifted both accuracy and reaction time measures toward an increased use of proactive control, relative to studies in which the upcoming delay was unknown. Increasing trial set count also increased the use of proactive control in both healthy young adults and persons with schizophrenia, whereas it increased the use of reactive control in healthy slightly older adults. These results demonstrate that delay knowledge and trial set count are critical parameters in expectancy studies, guiding distinct cognitive control behaviors reflected in both error rate and reaction time measures. Researchers using the AX-CPT or DPX paradigms should no longer consider delay knowledge or trial set count as incidental parameters, and should select these parameters intentionally in accordance with the control type(s) of experimental interest. More broadly, this meta-regression advances our knowledge of cognitive control instantiation, providing strong evidence that cognitive control becomes more reactive when timing demands are not known, and more proactive when timing demands are known. Further, our finding that healthy young adults (and persons with schizophrenia) shift toward proactive control with increasing repetitions of a task set gives quantitative evidence that proactive systems are preferentially activated by increasingly regular patterns of expectancy.