Validation of the Emotiv EPOC EEG system for research quality auditory event-related potentials in children

Background. Previous work has demonstrated that a commercial gaming electroencephalography (EEG) system, Emotiv EPOC, can be adjusted to provide valid auditory event-related potentials (ERPs) in adults that are comparable to ERPs recorded by a research-grade EEG system, Neuroscan. The aim of the current study was to determine if the same was true for children. Method. An adapted Emotiv EPOC system and Neuroscan system were used to make simultaneous EEG recordings in nineteen 6- to 12-year-old children under “passive” and “active” listening conditions. In the passive condition, children were instructed to watch a silent DVD and ignore 566 standard (1,000 Hz) and 100 deviant (1,200 Hz) tones. In the active condition, they listened to the same stimuli, and were asked to count the number of ‘high’ (i.e., deviant) tones. Results. Intraclass correlations (ICCs) indicated that the ERP morphology recorded with the two systems was very similar for the P1, N1, P2, N2, and P3 ERP peaks (r = .82 to .95) in both passive and active conditions, and less so, though still strong, for mismatch negativity ERP component (MMN; r = .67 to .74). There were few differences between peak amplitude and latency estimates for the two systems. Conclusions. An adapted EPOC EEG system can be used to index children’s late auditory ERP peaks (i.e., P1, N1, P2, N2, P3) and their MMN ERP component.


INTRODUCTION
An auditory event-related potential (ERP) is the average pattern of electrical activity generated by neurons in response to a particular auditory event. Auditory ERPs can be measured without a listener's overt attention. Such "passive" auditory ERPs are a useful means of investigating the role of auditory processing in people who find it difficult to pay attention to stimuli, to make decisions about stimuli, or plan overt responses to stimuli. Thus, passive auditory ERPs have proved useful for investigating auditory processing in attention deficit hyperactivity disorder (ADHD;Taylor et al., 1997), schizophrenia (Todd, Michie & Jablensky, 2003); autism (McPartland et al., 2004); developmental dyslexia (McArthur, Atkinson & Ellis, 2009); and specific language impairment (Whitehouse, Barry & Bishop, 2008).
A limitation of passive auditory ERPs is that they are typically measured using research-grade equipment housed in a laboratory. Such settings can be intimidating for many people, particularly children and adults with cognitive disorders. Fortunately, recent research has shown that a commercial "gaming" electroencephalography (EEG) system, called "EPOC" by Emotiv (www.emotiv.com), can be adapted to produce valid ERPs. Badcock et al. (2013) examined auditory ERPs in "passive" (standard and deviant tones are ignored) and "active" (deviant tones are counted) listening conditions in adults, using an adapted EPOC system and a research-grade Neuroscan system. They found high reliability for the "late auditory ERP" peaks (i.e., P1, N1, P2, N2, and P3) but not for the "mismatch negativity" component (MMN; see Näätänen et al., 2004). The EPOC system has also been successfully used to measure the auditory P3 response (Debener et al., 2012;De Vos, Gandras & Debener, 2014) and the visual P3 response (Duvinage et al., 2013;De Vos et al., 2014). Considered together, the outcomes of these seminal studies suggest that the EPOC system can be adapted to record valid auditory P1, N1, P2, N2, and P3 ERP peaks in adults.
To our knowledge, no study has yet tested if an adapted EPOC system can produce valid auditory ERPs in children. This cannot be inferred from previous validation studies done with adults because children have (1) different ERPs to adults due to cortical and cognitive immaturity (Ponton et al., 2002;Coch, Sanders & Neville, 2005;Mahajan & McArthur, 2012;Mahajan & McArthur, 2013); (2) "noisier" ERPs than adults (Coch & Gullick, 2012); and (3) more difficultly keeping still during long test sessions than adults, so their EEG (and ERP) responses may be contaminated to a greater degree by electrical noise generated by movement. The aim of the current study was to test the validity of children's passive and active auditory ERPs measured via an adapted EPOC system. In line with an analogous adult study (i.e., Badcock et al., 2013) we predicted that the adapted EPOC system would produce valid ERPs for the highly reliable late auditory ERP peaks (P1, N1, P2, P2, P3) but invalid ERPs for the less reliable MMN component.

MATERIALS AND METHODS
The Macquarie University Human Research Ethics Committee approved the methods used in this study (approval number: 5201200658).

Participants
Participants were twenty-one children (11 females, 10 males) aged between 6 and 12 years (M = 9.23, SD = 1.80). Parents or guardians of the children provided written informed consent for their child's participation (see Supplemental Information for information-consent form), and children were reimbursed $15 for their time. Participants were required to have normal hearing and vision, and no history of epilepsy. One child was excluded from the study due to a reported hearing loss; another child was excluded because the EPOC event-markers failed to record; and a third was excluded due to a condition coding error. Therefore, the final sample included 18 children. Based on our previous research demonstrating a large effect size (0.7, Badcock et al., 2013), G*Power (Faul et al., 2007;Faul et al., 2009) estimated that a sample size of 11 would provide adequate power for this experiment (0.8).

Stimuli
Presentation (Version 16; Neurobehavioural Systems) was used to deliver tones in passive and active conditions (see below) at a volume that was comfortable for each participant (note: the volume remained fixed across conditions). Each condition consisted of 566, 175-ms, 1000-Hz standard tones (10-ms rise and fall time; 85% of trials) and 100, 175-ms, 1200-Hz deviant tones (10-ms rise and fall time; 15% of trials). Deviant tones were presented after 3 to 35 (randomly allocated) standard tones. The stimulus onset asynchrony was jittered between 900 and 1100 ms to minimize EEG activity related to anticipatory processes. Tones were presented binaurally via Phillips SHS4700/37 ear clip headphones (Phillips, Amsterdam, Netherlands) fixed to the EPOC headset.
In the passive condition, participants were instructed to watch a silent movie and ignore the tones presented through the headphones. In the active condition, participants were instructed to count the "high" tones whilst watching the silent movie. Participants were asked, and reminded where necessary, to stay as still as possible. Each condition lasted approximately 13 min, separated by a short-break.

Neuroscan system
The research-grade EEG system (Neuroscan Version 4.3; widely accepted for psychophysiological research) used an EEG electrode cap (EasyCap, Herrsching, Germany) fitted with 14 Ag-AgCl electrodes located at F3, F7, FC4, FT7, T7, P7, P8, T8, FT8, FC4, F8, F4, M1 (online reference), and M2. Electrodes placed above and below the left eye measured vertical eye movements ("VEOG"), and electrodes placed on the outer side of each eye measured horizontal eye movements ("HEOG"). Please note that the M2 (right mastoid), VEOG, and HEOG electrodes were set up as per standard procedures even though these electrodes were not used in the analysis as EPOC does not provide equivalent measurements. The ground electrode was positioned between FPz and Fz.
Neuroscan was recorded at 1000 Hz. Triggers were inserted into the EEG to indicate the onset of each stimulus. These triggers were generated by Presentation, and were inserted into the EEG via a parallel port.

Adapted EPOC system
The EPOC system used a wireless headset with flexible plastic arms that held gold-plated sensors against the head at 16 sites that aligned with the research EEG headset: AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8, AF4, M1 and M2. M1 acted as a ground reference point for measuring the voltage of the other sensors. M2 acted as a feed-forward reference point for reducing electrical interference from external sources. The remaining signals were high-pass filtered with a 0.16 Hz cut-off, pre-amplified and low-pass filtered at an 83 Hz cut-off. The analogue signals were then digitised at 2048 Hz, filtered using a 5th-order sinc notch filter (50-60 Hz), and low-pass filtered before being down-sampled to 128 Hz (specifications taken from the gaming EEG system web forum). The effective bandwidth was 0.16-43 Hz. EPOC was recorded at 128 Hz.
The EPOC system was adapted to accurately time-lock the EEG signal to the onset of each stimulus by marking the EEG signal with an electrical pulse triggered by a wireless transmission system (Thie, 2013). The system consisted of transmitter and receiver units that were linked using infrared (IR) light. The transmitter unit was attached to the audio output of the stimulus presentation computer. The receiver unit was mounted in close proximity to the participant (i.e., taped to their shoulder or resting on a table) with its output wires attached to two of the EEG electrodes (O1 and O2). These electrodes were attached directly to the Driven Right Leg (DRL) through wires and 4700-ohm resistors that mimicked a perfect connection with the scalp. The transmitter unit was made up of a microcontroller board (Arduino Uno) and an interface board. This board amplified the audio stimuli and fed it to the Arduino's analogue input. The receiver waited for a number from the transmitter to trigger a 100-ms-wide pulse. There was a 19-ms delay (accounted for the trigger processing) between the onset of the stimulus and the onset of the marker pulse due to the buffering of the audio signal in order to determine its frequency and the transmission of the 8-bit number.

Procedure
Neuroscan was setup first and adjusted until sensor impedance was below 5 kOhms. The EPOC headset was fitted over the EasyCap (for a detailed description, see Badcock et al., 2013). This allowed for simultaneous measurements of EEG by the Neuroscan and EPOC systems (see Fig. 1; for electrode locations see Fig. 2). EPOC electrode connectivity was tested using the TestBench software. Sensors were adjusted until connectivity reached the "green" level, which represented impendences less than 220 kOhms (measured using a resistor between an electrode and the DRL, M2 in the current setup). The total setup time was approximately 55 min.

Offline EEG processing
Both Neuroscan and EPOC EEG recordings were processed in the same way using EEGLAB version 11.0.4.3b (Delorme & Makeig, 2004). Large artefacts in each EEG were first excluded by eye. The Neuroscan EEG data were then downsampled to 128 Hz in order to match the sampling rate of the EPOC system. The EEG data were then bandpass filtered from 0.1 to 30 Hz, separated into epochs that started −102 ms before the onset of each tone and ended 500 ms after the onset of each tone, and baseline corrected between −102 and 0 ms. Any epochs with an amplitude in excess of ±150 µV were excluded.
Ocular artefact removal was attempted using Independent Components Analysis in EEGLAB (note: channels capturing the eye-movements for Neuroscan were not included in this process to maintain equivalent processing between the systems). This process did not identify eye-blink related components in any of the datasets. Therefore, eye-blinks were either not consistent or strong enough to meaningfully affect the data.
In order to best compare the two systems, only those epochs accepted for both EEG systems (i.e., shared epochs) were included in the analysis (we thank Phillip Ruhnau for this suggestion). For each child, the shared epochs were averaged together to produce late auditory ERP waveforms that comprised P1, N1, P2, and N2 peaks for the standard and deviant tones, in the passive and active conditions. Shared epochs to standard and deviant tones in the passive condition were averaged separately and then subtracted (i.e., the ERP to standard tones was subtracted from the ERP to deviant tones) to produce a mismatch negativity (MMN) waveform.

Analysis
In line with the previous EPOC validation study done with adults, the analysis focused on data from frontal sites in the left and right hemispheres: F3 and F4 for Neuroscan, and AF3 and AF4 for EPOC.
The ERPs produced by the two systems were compared in three ways: (1) total number of accepted epochs were used to compare the quality of the Neuroscan and EPOC EEG data, (2) intraclass correlations (ICCs, for details see McArthur & Bishop, 2004;Bishop et al., 2007) were used to index the similarity of Neuroscan and EPOC waveforms (between −102 to 500 ms), and (3) peak amplitude and latency measures were used to compare the size and timing of each ERP peak or component. The number of epochs and peak comparison data sets were tested for normality (Shapiro-Wilk) and equal variance (F test). Singleand paired-sample t-tests and Wilcoxon-signed ranks were used to evaluate the statistical reliability between EEGs systems comparisons, and Cohen's d was used to evaluate the magnitude of the effects. We used a criteria of p < .05 unless otherwise specified.
Regarding the size and timing of ERP peaks and components, peak amplitude and latency measures were initially calculated using an automated procedure that identified the point of maximum amplitude (positive or negative) within appropriate time intervals or "search windows." These intervals were determined by visual inspection of the relevant grand mean ERP waveforms (as described by Hoormann et al., 1998), and were as follows: 50 to 140 ms (P1); 70 to 140 (N1); 140 to 200 ms (P2); 260 to 400 ms (N2); 260 to 400 ms (P3); 140 to 260 (MMN). We then checked the validity of each peak measure for each child by visually inspecting individual waveforms. This revealed that the N1 and P2 peaks were missing in 11 to 14 (60 to 77%) children across all conditions, which is characteristic of children's auditory ERPs (Ponton et al., 2000;Mahajan & McArthur, 2012;Mahajan & McArthur, 2013). Invalid data points for missing peaks were deleted from the dataset. A further 15% of the measures produced by the automated peak detection were invalid, identifying an end-point of the range greater in magnitude that the true peak. Invalid end-point measures were corrected manually to ensure all peak amplitude and latency measures for all children were valid.

Number of accepted epochs
The distributions for the number of accepted epochs were negatively skewed, thus, Wilcoxon Signed Rank Tests were used to compare the data recorded by the two systems. The median number of accepted epochs, inter-quartile range, and Wilcoxon signed ranks statistics are presented in Table 1. There were statistically fewer acceptable epochs for EPOC than Neuroscan in all conditions. Nevertheless, the number of accepted epochs for both the EPOC and Neuroscan systems was more than adequate for waveform generation for all participants.

P1, N1, P2, and N2
The mean of the group ERP waveforms produced by the Neuroscan and EPOC systems to the standard and deviant tones in the passive and active conditions are displayed in Fig. 3 (see Figs. S1 and S2 for the auditory ERPs of individual children). The ICCs between late auditory ERP waveforms generated by the two systems to standard and deviant tones in the passive and active conditions are presented in Table 2. The range of ICCs for the standard tones was 0.91 to 0.95 and for the deviant tones was 0.82 to 0.86. All of these distributions were negatively skewed; therefore, statistical differences to zero were assessed using single-sample Wilcoxon signed ranks, all of which were significant: all Z = 4.48, p < .001. These results indicate a strong correspondence between the measurements made with the two systems.

P3
The mean of the group late auditory ERP waveforms produced by the Neuroscan and EPOC systems to the deviant tones in the active condition are displayed in Fig. 3 (see Fig. S2 for the auditory ERPs of individual children). The corresponding ICC values are shown in Table 2. The ICCs for F3/AF3 and F4/AF4 were 0.85 and 0.83 respectively, and the negatively skewed distributions were significantly different to zero: both single-sample Wilcoxon signed ranks, Z = 4.48, p < .001. These results indicate a strong correspondence between the measurements made with the two systems.

MMN
The mean of the group MMN ERP waveforms produced by the Neuroscan and EPOC systems are presented in Fig. 4

Peak amplitude and latency
The descriptive statistics for P1, N1, P2, N2, P3 and MMN peak amplitude and latency measures produced by the Neuroscan and EPOC systems for standard and deviant tones in the passive and active conditions at F3/AF4 and F4/AF4 are reported in Tables 3-5. Peak comparisons between the two systems were conducted using paired-samples t-tests and Wilcoxon singed rank tests, depending upon the normality of the data as indicated in the tables. Due to multiple comparisons, statistical tests with p-values less than .01 will be highlighted (p < .05 and .001 are also indicated in the tables).

P1, N1, P2, and N2
For the P1, N1, P2, and N2 late auditory ERP peaks, there were 18 comparisons that differed statistically between the two systems. Two of the differences reflected reduced N2 amplitude in the EPOC system: differences of .99 and .93 µ Vs in the passive and active conditions respectively, both small in magnitude (d = 0.29 and .30). Sixteen of the differences reflected a delay in the latency of the peaks measured by the EPOC system and 9 of these were evident to the standard tone. The average delay for these comparisons was 8..88 ms (SD = 2.45) and effect sizes were small to large (d = 0.29 to 1.09).

P3
The differences in P3 amplitude between the systems were non-significant and small (d < 0.05). The P3 produced by the EPOC system at F3/AF3 was significantly later than that produced by the Neuroscan system by 15 ms (see Table 5). Cohen's d effects sizes were moderate (d = 044 and 0.45).

MMN
The differences in MMN amplitude and latency between the two systems were nonsignificant and small (d < 0.03).

DISCUSSION
The aim of the current study was to assess the validity of the Emotiv EPOC gaming EEG system as an auditory ERP measurement tool in children. To this end, we simultaneously measured ERPs using a research-grade Neuroscan system and the EPOC system in children aged between 6 and 12 years. Children were presented with standard and deviant tones in both passive (ignore tones) and active (count high tones) listening conditions. There are three key findings. First, whilst both EEG systems recorded a high proportion of accepted epochs, fewer were acceptable for EPOC. This was also found by Badcock et al. (2013) when they tested adults. Fewer acceptable epochs with EPOC may stem from reduced stability of EPOC's saline-soaked cotton sensors resting on the scalp, relative to the gel used with Neuroscan, which effectively glues to sensor the scalp with gel. Having said this, EPOC recorded adequate numbers of acceptable epochs to produce reliability later auditory ERPs.
Second, the systems produced similar late auditory ERP (ICCs: 0.82-0.95) and MMN waveforms (ICCs: 0.67-0.74). These ICCs are higher than those previously reported by Badcock et al. (2013) who found that ICCs in adults for the late auditory ERPs ranged from 0.57 to 0.80, and that the ICC for the MMN was 0.44. Apart from differences in populations studied (i.e., children versus adults), there are three major differences between the current and the previous study that may explain the different ICCs. First, the current study was conducted within a shielded room, whereas the previous study was not. It is conceivable that the shielded room resulted in cleaner EEG recordings. Second, the analysis for the current study used only overlapping epochs that were accepted by both EEG systems. However, the previous study rejected fewer epochs, which would have resulted in a high proportion of overlapping epochs. Thus, this seems an unlikely explanation of the higher ICCs in the current study. Third, the current study used a wireless triggering system for the EPOC system, while the previous study used a wired system. In the previous study, we noted that participant movement with the wired system caused interference to the EEG signal. However, fewer epochs were rejected in the previous study so this too seems an unlikely explanation for the higher ICCs in the current study. It therefore seems most likely that the shielded room produced the higher ICCs in the current study compared to the previous study done with adults by Badcock et al.
Third, there were only a few differences between the peak amplitude and latency measures produced by the EPOC and Neuroscan systems, which mostly related to delayed latencies for the EPOC system (i.e., an average delay was 8.1 ms (SD = 5.92)). This represents a single sample at 128 Hz. Since this delay was small, and occurred in a minority of comparisons, we do not believe it significantly compromises the use of the EPOC system as a measure of auditory P1, N1, P2, N2, or P3 ERPs in children.
Overall, the findings of the present study paired with Badcock et al. (2013) suggest that EPOC compares well with Neuroscan for investigating late auditory ERPs in children. This opens up new opportunities for conducting ERP studies with children with or without cognitive impairments who find the laboratory settings associated with traditional research-grade EEG systems threatening or uncomfortable. It also paves the way for large-scale studies of the development of typical and atypical ERPs since it allows the measurement of children's ERPs in settings such as schools, childcare centres, hospitals, and private clinical practices.