The Autism Biomarkers Consortium for Clinical Trials: Initial Evaluation of a Battery of Candidate EEG Biomarkers

Neural biomarkers are objective markers that may serve as an intermediate phenotype related to core features (e.g., social attention) or neurobiological functioning (e.g., excitatory/inhibitory balance) in autism spectrum disorder (ASD). Scalp electrophysiology (EEG) is a noninvasive technique for recording brain activity and is well suited for biomarker development in neurodevelopmental conditions (1–4). EEG does not require participant responses and can be collected over brief recording periods. Applicable across age and developmental levels, EEG has been widely used in ASD, with established best practices (5, 6). However, reproducibility is a pervasive problem in ASD neuroimaging research, and it is not known whether inconsistent findings reflect true population heterogeneity, methodological or acquisition variability, or poor psychometric qualities. The present study focused on addressing these limitations of previous research by using a large sample with a constrained age range, prespecifying derived variables and directional hypotheses, and evaluating basic psychometric properties to rule in or out EEG metrics potentially suitable as biomarkers in future clinical trials.

The Autism Biomarkers Consortium for Clinical Trials (ABC-CT) (7) selected four EEG assays with strong prior evidence of group discrimination or relation to phenotype. First, face processing is a fundamental social ability that is affected in ASD and is putatively related to early social communicative symptoms (8). The N170 is a face-sensitive event-related potential (ERP) (9) evident by age 4 (10), with developmental precursors in infancy (11, 12). The latency of the N170 to upright faces is delayed in individuals with ASD, although variability exits within reports (13, 14). Second, biological motion perception is critical to social understanding (15). Biological motion perception is indexed by the P1, N2, and P3 ERP components, and attenuated responses have been observed in ASD (16). Third, visual evoked potentials (VEPs) provide information about the integrity of visual pathways and index biological mechanisms related to ASD. Children with ASD and associated genetic syndromes show reduced-amplitude VEPs (17–19). Fourth, resting-state EEG characterizes atypicalities in intrinsic neural function in ASD, manifested as spectral power increased in low- and high-frequency bands and decreased in midrange activity (20). Psychometric evaluation of these assays focused on 1) valid acquisition and signal, 2) construct performance, and 3) 6-week stability. We also sought to replicate discrimination between ASD and typically developing groups and phenotypic correlations within the ASD group.

Methods

ABC-CT Protocol

Details on the ABC-CT protocol have been published previously (7, 21), and details on the full protocol, the participants, and the EEG protocol are provided in the online supplement. The study included measures of social function and a battery of EEG and eye-tracking tasks administered across five sites during 2-day visits spanning three time points; the analyses here focus on time 1 (T1) and time 2 (T2 [T1 + 6 weeks]). A central institutional review board ensured that informed consent or assent was obtained from guardians or participants after they had been given an explanation of the study procedures and an opportunity to ask questions.

Participant Characteristics

Across five sites, we included 280 children with ASD (mean age, 8.6 years [SD=1.6, range=6.0–11.5]; mean full-scale IQ, 98.6 [SD=18.1, range=60–150]) who met gold-standard diagnostic criteria and 119 typically developing children (mean age, 8.5 years [SD=1.6, range=6.0–11.5]; mean full-scale IQ, 115.1 [SD=12.6, range=80–150]) without ASD or psychiatric conditions. Full diagnostic and inclusion criteria as well as demographic characteristics are presented in the online supplement and in Table S1 for the total sample and for the subsamples that contributed valid data to each assay. Age did not differ significantly between the ASD and typically developing groups, nor based on inclusion in each assay (F values <1.0, p values >0.20).

EEG Acquisition Standardization

All procedures were manualized (21). Each experiment started with a welcome screen, followed by experiment-specific information, behavioral directions, and then a start screen. Behavioral assistants provided supplemental verbal directions and behavioral supports responsive to the participant, as per guidelines. During acquisition, the experimenter coded the participant’s behavior for attention and compliance.

EEG equipment.

All sites had an EGI 128-channel acquisition system, from the Net Amps 300 series (three sites) or 400 series (two sites), 128-electrode EGI HydroCel Geodesic Sensor Nets, Logitech Z320 computer speakers, a Cedrus StimTracker (for visual presentation timing), and a 23-inch monitor. A standard acquisition setup was implemented: a 1,000-Hz sampling rate, a 0.1- to 200-Hz filter, EGI MFF file format, onset recording of amplifier and impedance calibrations, and a postacquisition 0.1-Hz digital high-pass filter. EPrime, version 2.0, was used for experimental control.

EEG protocol.

The four EEG assays were administered on the second day of each time point: resting state, followed by a faces task, followed by a counterbalanced biological motion perception (BM) task or VEP. Quality review and signal transformation were performed centrally (21) (see the online supplement for details on the methods).

Resting-state assay.

The resting-state experiment included three blocks of two 30-second videos, for a total of 180 seconds. Video stimuli consisted of abstract moving images subtending 8×6 degrees. The primary outcome variable was the slope of the EEG power spectrum (2–50 Hz), calculated by fitting a least-squares linear regression of log-transformed power as a function of frequency (Figure 1A). Slope indexes cortical excitation-to-inhibition ratio and long-range temporal correlations (22, 23). Secondary variables were alpha (6–12.99 Hz) and gamma (35–54.99 Hz) power. (Results for the delta, theta, and beta bands are provided in the online supplement.) Power was averaged across the entire scalp.

Faces assay.

The faces task included 216 trials, acquired in six blocks of 36. Each trial consisted of a fixation crosshair (500–650 ms), stimulus (500 ms), and blank screen (500–650 ms). Three female neutral faces, presented upright and inverted, and three upright houses were presented, for 72 trials per stimulus condition (24). We focused on the right posterior-temporal region created by averaging across the channels (electrodes 89, 90, 91, 95, 96) for each segment and within stimulus type. Peak amplitude and latency for the P100 (60–200 ms), reflecting attention allocation, and the N170, reflecting early-stage facial processing (120–400 ms), were identified using an automated algorithm, visually inspected for accuracy and removed if invalid (details are provided in the online supplement). As shown in Figure 1D, all variables were to upright faces, with the primary variable N170 latency; the secondary variable was the P100 latency.

VEP assay.

VEP stimuli consisted of two black-and-white checkerboards that reversed phase, presented in four blocks of 50 500-ms trials, for a total of 200 trials. Analysis focused on the midline occipital region (average of electrodes 70, 74, 75, 82, 83). All peaks were visually inspected for accuracy; peaks that were deemed invalid were removed from analyses (see the online supplement). As shown in Figure 1G, the primary dependent variable was the P100 amplitude (VEP P100 amplitude, 60–200 ms), and the secondary variable was the N1 amplitude (VEP N1 amplitude, 20–120 ms), reflecting activity in the primary visual cortex and dorsal-medial occipital cortex, respectively (25).

Biological motion perception assay.

The BM assay included 112 trials, in four blocks of 28, with each trial starting with a fixation crosshair (1,025–1,200 ms) followed (randomly) by 1,000 ms of a dynamic biological walking motion or a scrambled point-light display. We used the same region as in the faces task (the right posterior temporal region). We focused on the biological motion specificity effect (BMS; the difference between biological minus scrambled motion response), with the primary variable the P3 average amplitude (P3 amplitude, 330–500 ms) and the secondary variable the N200 peak amplitude (N200 amplitude, 120–400 ms). In addition to the N200 and P300 (Figure 1J), we also abstracted the P100 (80–180 ms), as in the faces task and VEP. Peaks that were deemed invalid were removed from analyses (see the online supplement).

Analytic Plan

We conducted five sets of prespecified analyses to address the psychometric properties of these potential biomarkers and their relation to ASD diagnosis and behaviors (21). The results for T1 are described here, with expanded results provided in the online supplement; results for T2 are provided in the online supplement to demonstrate replication of effects.

Acquisition.

We present rates of acquisition by group below, and additional details are provided in Table S2 in the online supplement. We used chi-square tests to compare rates within each diagnostic group across the five sites; for BM and VEP, we also examined the group-by-order interaction (counterbalanced to the third or fourth position in the battery). We used analyses of covariance (ANCOVAs) to check for differences in characteristics between participants whose data were included and those whose data were excluded (see Table S3 in the online supplement). We then used Pearson correlations to examine the relationship between the number of trials included and child characteristics (see Table S4 in the online supplement).

Construct performance.

Construct performance was established in the typically developing group using experiment-specific evaluation for the faces task and the BM task. Construct performance was related to directional stimuli contrasts (upright faces vs. upright houses, biological motion vs. scrambled motion, respectively) using repeated-measures analysis of variance (ANOVA).

Six-week stability.

We assessed short-term stability of individual biomarkers in both groups from T1 to T2 using intraclass correlation via mixed models with absolute agreement. In Table 1 and in Table S5 in the online supplement, we provide values for subgroups within the ASD group: split by age (8.5 years) and IQ (full-scale IQ of 75). The data supplement includes an expanded version of Table 1 with 95% confidence intervals for all intraclass correlations reported. A priori acceptability criteria defined intraclass correlations ≥0.5 as moderate and ≥0.75 as high (26).

Group discrimination.

We first ran an independent-samples ANOVA with group as the between-subject variable, and we report partial eta-squared (η_p²) for effect size; if variances differed significantly between groups, we report Welch ANOVAs with omega-squared (ω²) for reporting effect size. We also include ANCOVAs with child age, number of trials, sex, and full-scale IQ as covariates. For BM and VEP, the contrast also included order and the group-by-order interaction.

Phenotypic correlations.

We assessed associations between biomarkers and child characteristics in the ASD group. Analyses are reported using Pearson’s correlations; we also verified biomarker-to-phenotype relations using partial correlations controlling for age and number of trials.

Results

Resting Assay

Acquisition.

Valid resting-state data were provided by 86% of the ASD sample and 92% of the typically developing sample, and inclusion rates did not differ significantly by site. Children with ASD who provided valid data had greater nonverbal and full-scale IQ scores and better social and communication skills than those who did not provide valid data (see Table S3 in the online supplement).

The ASD group provided fewer valid data trials than the typically developing group (Table 2). Among children in the ASD group, more valid trials were provided by those who were older, had higher IQ scores, had better communicative functioning, and had fewer autism behaviors (see Table S4 in the online supplement).

Six-week stability.

Intraclass correlations for the typically developing and ASD groups are presented in Table 1. Values were generally moderate for slope (Figure 1C) and alpha power, and low for gamma power.

Group discrimination.

When covariates were included, slope (Figure 1A,B), alpha power, and gamma power did not differ by group (Table 2). Exploratory analyses of the other frequency bands (delta, theta, and beta) did not show group differences (see Table S6 in the online supplement).

Phenotype correlations.

As shown in Table 3, a more negative (steeper) slope and lower gamma power were related to better face memory. Lower alpha power and lower gamma power were associated with higher IQ. Only alpha and full-scale IQ remained significant after covarying for age and number of trials.

Faces Assay

Acquisition.

Valid data for the faces task were provided by 77% of the ASD sample and 97% of the typically developing sample. Sites differed at T1 as a result of very high rates of acquisition at one site (91%, compared with 64%–78% in the others). Children with ASD for whom data were available were more likely to be older, have higher verbal, nonverbal, and full-scale IQ, have fewer autism symptoms, and have better social and communication skills than those who did not provide data (see Table S3 in the online supplement).

Compared to the typically developing group, the ASD group had fewer trials for upright faces (Table 2). Among children in the ASD sample, more data were provided by those who were older, had higher IQ scores, and had fewer autism repetitive behaviors and pragmatic problems (see Table S4 in the online supplement).

Construct performance.

Construct performance was established by comparing the ERP response to upright faces versus upright houses (Figure 1D). As expected, the typically developing group exhibited a more negative N170 amplitude and faster latency response to faces compared to houses (F values >6.97, p values <0.01).

Six-week stability.

As shown in Table 1, all values were moderate to high in the typically developing and ASD groups for the N170 latency (Figure 1F) and P100 latency, as well as the amplitudes of these components (see Table S5 in the online supplement).

Group discrimination.

As hypothesized, the ASD group demonstrated significantly slower upright faces N170 latency (Figure 1E) and upright faces P100 latency compared to the typically developing group (Table 2). This pattern was consistent when covariates were included.

Phenotype correlations.

As shown in Table 3, faster N170 latency was related to better face memory (r=−0.141, p<0.05) (see also Figure 1F). The P1 latency was uncorrelated with any prespecified variables.

VEP Assay

Acquisition.

Valid data were provided by 85% of the ASD sample and 96% of the typically developing sample; rates of acquisition did not differ significantly by site or experiment order (see Table S2 in the online supplement). Children with ASD who provided data had higher IQ and better social and communication skills on average than those who did not provide data (see Table S3 in the online supplement).

Compared to the typically developing group, the ASD group had fewer data trials (“VEP Trials” Table 2). In the ASD group, children who were older, had higher IQ scores, and had fewer repetitive behaviors and pragmatic problems provided more valid trials (see Table S4 in the online supplement).

Six-week stability.

As shown in Table 1, intraclass correlations over 6 weeks were moderate to high (Figure 1I) except for the low value found for the N1 in children with ASD who had a full-scale IQ ≤75.

Group discrimination.

In contrast to our predictions, the P100 (Figure 1H) and N1 amplitudes did not differ between groups (Table 2). In adjusted analyses, the group-by-order interaction was significant (F=3.97, df=1, 343, p=0.047; η_p²=0.011) (see Table S7 in the online supplement). Within groups, the P100 was larger for typically developing children for whom the experiment was in third position in the battery compared with fourth position (F=4.05, df=1, 108, p=0.05; η_p²=0.036), but there was no difference by order within the ASD group (F=0.52, df=1, 231, p=0.47; η_p²=0.002).

There were no significant differences in the VEP N1 amplitude response by group.

Phenotype correlations.

A larger P100 was related to better face memory (Table 3). There were no behavioral correlates for the N1.

Biological Motion Assay

Acquisition.

Valid data for the BM assay were provided by 67% of the ASD sample and 88% of the typically developing sample (see Table S2 in the online supplement). Given the low rate of valid data availability in the ASD group, the findings for this group are presented in the online supplement.

Construct performance.

The BMS effect was not observed in the typically developing group. No significant stimulus effects were detected at the N200 amplitude (F=0.08, df=1, 103, p=0.77), and the P3 amplitude was greater to scrambled motion than to biological motion (F=4.79, df=1, 103, p=0.03; η_p²=0.044) (Figure 1J).

Six-week stability.

All BMS intraclass correlation values were low (Table 1; see also Figure 1L as an example).

Group discrimination and phenotype correlations.

There were no group differences or phenotypic relations for the BMS at any component (Figure 1K; see also Tables S8 and S9 in the online supplement).

Discussion

Acquisition Rates

We prespecified that successful acquisition across sites and across key demographic and clinical factors should be >70% in the ASD groups in order to be viable in future clinical trials. Values were acceptable for the resting-state, faces, and VEP assays across diagnostic groups but fell below this threshold for the BM assay in the ASD group. The primary data loss was at the stage of signal validation (number of attended trials; presence of target component), rather than at participant acquisition (ability to wear the net). Our fixed protocol demanded sustained attention (faces task, BM task), which could be addressed through use of briefer assays (e.g., fewer experimental trials) or alternate implementation of the biomarker battery (e.g., removing experiments or implementing them over multiple days).

Children with ASD who were older, had higher IQ, and had greater social and communication skills provided more valid EEG data. Similarly, children with ASD who were older, had higher IQ, and had fewer repetitive behaviors had more valid trials. In contrast, in the typically developing group, only age was related to provision of data. These results suggest a potential complication for future clinical trials, with reduction in potential enrollment or restricted range of child characteristics if EEG is required for inclusion. Further, treatments may alter skills associated with acquisition (e.g., increasing attention), and thus EEG improvement might result from target engagement or also from changes in signal (and lower artifact) available for analysis. Future efforts will need to consider how to disentangle child ability, amount of data, EEG signal-to-noise ratio, and treatment effects.

Notably, we fully manualized our process for site and individual recording quality control, and we used manualized criteria to review all biomarker values for validity. While these procedures are laborious, they reduce the likelihood of introducing bias into the process of acquisition and data derivation. We were also able to implement these steps with post-baccalaureate staff in a manner that is comparable to the training and reliability standards of diagnostic and intelligence assessments. Given that EEG signals directly assess perception and attention, ignoring acquisition processes may reduce the objectiveness of the measurement.

Construct Performance

Our resting-state, faces, and VEP assays evoked the predicted components and EEG component contrasts based on the literature. Overall, our results indicated quantitative variability in the EEG responses between groups. In contrast, the BM assay did not demonstrate construct validity by specified metrics, suggesting that it may not be viable for future clinical trials.

Six-Week Stability

Our protocol focused on stability across 6 weeks to parallel a short-term clinical trial (27). We note that published research from the ABC-CT feasibility study (28) indicates that slope intraclass correlations (ICCs) were higher over a shorter time frame (6 days, ICC=0.699) in comparison with those reported here. Increased reliability over a shorter period likely reflects less age-related or treatment changes.

The majority of biomarkers assessed showed moderate stability that was similar in the ASD and typically developing groups and similar to social-affective clinical measures (Autism Impact Measure social reciprocity subscale, ICC=0.68; Pervasive Developmental Disorder Behavior Inventory social approach behaviors subscale, ICC=0.78; Faja et al., unpublished data), social-cognitive measures (NEPSY-II memory for faces subtest, ICC=0.63), and eye-tracking measures (ICC=0.23–0.87 by variable; [29]) across the same time period. We also note that the sample was clinically stable (Clinical Global Impressions severity scale score, ICC=0.957; Faja et al., unpublished data).

Values were also similar within assay (upright faces P100 latency, N170 latency) and for the same components abstracted across different assays (P100 amplitude). The BMS may have performed poorly because it is a difference score, as the response to biological motion itself showed moderate stability (see Table S4 in the online supplement).

Group Discrimination

All prespecified biomarkers demonstrated group discrimination in smaller samples in previous research. In the present study, P1 and N170 latency to upright faces (at T1 and T2) exhibited predicted latency delays in children with ASD at both T1 and T2. Resting slope differed between groups (at T1 and T2); controlling for covariates resulted in significance at T2 but not at T1. VEP responses did not differ between groups. Across all markers, lack of or inconsistency in group differences suggests limited viability for our biomarkers for diagnostic use.

Phenotype Correlations

Several of our biomarkers demonstrated statistically significant relationships with intellectual function in children with ASD. EEG power across all bands was related to IQ (30). Multiple biomarkers (Table 3) were related to face memory, potentially reflecting reliance on multiple neural systems, including basic visual attention, memory, and face perception (31). Standardized cognitive and language tasks, the face memory task, and the EEG stimuli require attention and inhibition, and were all acquired in the lab in temporal proximity. Our measures of autism symptoms, in contrast, were parent report and reflect a broader survey of behaviors over extended periods of time.

Our a priori hypotheses regarding relationships of EEG variables to social function were not confirmed. Exploratory analyses indicated that greater P100 amplitude to upright faces was associated with greater social ability (Vineland Adaptive Behavior Scales, 3rd ed.) and reduced social-communicative symptoms (Social Responsiveness Scale, 2nd ed.). We did not observe this relationship for the P100 amplitude elicited by VEP or the BM task, suggesting that it may be specific to early-stage face processing.

Biomarker Viability

In terms of meeting basic acquisition and psychometric standards, three assays merit further investigation: the resting-state, faces, and VEP assays. Based on poor acquisition, lack of construct performance, and lack of phenotype correlations, the BM task does not demonstrate potential utility for clinical trials in this population as acquired.

The eventual use of the biomarker is of critical importance, as the impact of these performance metrics may differ based on use. We note that overlap in distributions among typically developing and ASD children suggest that none of these assays are likely to be viable diagnostic biomarkers for ASD. If used for an inclusion or stratification marker, higher intraclass correlations may be critical, as the biomarker needs to be stable enough to consistently identify the same individual (pretreatment). A shorter retest period would be needed to eliminate potential developmental effects or clinical change that could influence our 6-week measure. For a surrogate endpoint, it may be more important that the marker track clinical change or the trial team may use the test-retest value to adjust the target treatment change value. While further validation of these markers needs to occur in the contexts of use (32), Loth has also noted (33) the need for markers that might predict placebo response.

Several caveats apply to the study. First, our results may not generalize to other ages and neuroimaging methods. Second, as our study was focused on reproducibility, the reported results focus on a subset of potential analyses to be conducted on data-rich EEG signals. We actively share our protocols, experiments, and data to enable the broader scientific community to partner with us in these exploratory investigations. Lastly, the translation of EEG measures to preclinical models will also be critical to understanding the mechanisms of treatment response; within our battery, two of the tasks are broadly relevant, while the face and BM tasks are likely only applicable to primate models.

In summary, our results provide support for the viability of well-standardized EEG acquisition in school-age children with ASD, and experiments that assess face processing, basic visual processing, and resting-state brain activity demonstrate desirable measurement properties for exploration as biomarkers that may be of use in clinical trials.