Toward a minimally invasive brain–computer interface using a single subdural channel: A visual speller study
Highlights
► Visual motion stimuli evoke cortical high gamma ECoG response. ► ECoG high gamma feature elevates BCI performance of visual speller. ► Spatial consistency is found between ECoG and fMRI response to visual motion. ► Location of subdural BCI electrodes can be determined by fMRI before surgery. ► Feasibility of minimally invasive BCI with single subdural electrode is demonstrated.
Introduction
The past decade has seen promising development in brain–computer interfaces (BCIs) aiming to help severely motor disabled patients to interact with the external world by directly interpreting brain activity. While the majority of BCIs rely on brain activity recorded with non-invasive electroencephalograph (EEG) technology (Daly and Wolpaw, 2008, Lebedev and Nicolelis, 2006), interest has increasingly been drawn toward the use of invasive electrocorticographic (ECoG) signals for BCI applications (Lebedev and Nicolelis, 2006, Leuthardt et al., 2004, Schalk and Leuthardt, 2011).
Compared to EEG, ECoG has two major advantages for implementing advanced BCIs. First, the spatial resolution of ECoG is much higher than that of EEG. ECoG records signals originating from brain tissues directly beneath the electrode surface (surface area of 1–10 mm2) with little influence from adjacent tissues (Bullock et al., 1995, Nunez and Srinivasan, 2006), whereas EEG's spatial resolution is at a multi-centimeter scale due to volume conduction of currents through tissues of the head. Such a fine spatial resolution makes ECoG ideal for imaging the cortical dynamics of sensory and cognitive functions that originate from relatively small brain regions (Liu et al., 2009, Mesgarani and Chang, 2012, Vinjamuri et al., 2011). Second, low voltage, high frequency brain activity that is barely detectable in scalp EEG is readily observed in invasive ECoG recordings (Leuthardt et al., 2004). In the past few years, ECoG BCIs have achieved promising results using brain signals extracted from the motor cortex (Kubanek et al., 2009, Miller et al., 2010, Vinjamuri et al., 2011), language-related brain regions (Leuthardt et al., 2011, Pei et al., 2011a), as well as auditory and visual cortex (Brunner et al., 2010, Wilson et al., 2006).
When using ECoG or other invasive recording techniques, it is preferable to minimize the cortical area used for acquiring sufficient neural information for BCI applications. To date, ECoG BCI studies have been carried out with epilepsy patients who underwent open skull surgery and implantation of large electrode arrays for clinical purposes. However, surgery of this type is both unnecessary and risky for the potential BCI users (i.e. patients with amyotrophic lateral sclerosis). Instead, electrodes for BCI purposes can be inserted through a burr hole into the brain, thus implementing a ‘minimally invasive’ BCI, given that sufficient information can be extracted within a limited cortical area.
The recently developed visual motion BCI paradigm using motion onset visual evoked potentials (mVEPs) may serve as a candidate for building minimally invasive BCIs (Guo et al., 2008, Jin et al., 2012). The mVEP speller utilizes the modulation effect of mVEPs by overt attention (i.e., eye gaze) for a spelling application: the attended visual motion stimuli elicit a more negative peak around 200 ms post-stimulus (N200) over parietal-occipital areas than the unattended stimuli, and BCI user intent is classified based on this difference. BCI based on mVEP can be considered an extension of the widely studied P300 BCI (Donchin et al., 2000, Farwell and Donchin, 1988), which utilizes an attention-related positive event-related potential (ERP) peak around 300 ms (P300) over the parietal cortex elicited by the attended visual flash stimuli. Based on invasiveness, mVEP BCI is a better candidate than P300 BCIs, as visual motion processing (reflected by mVEPs in EEG) is believed to be highly focused in a brain region known as the human middle temporal (MT) complex (DeYoe et al., 1996, Huk et al., 2002, Zeki et al., 1991). Compared with other types of BCIs for motor and speech decoding (Kubanek et al., 2009, Leuthardt et al., 2004, Leuthardt et al., 2011), relatively less information is needed for the operation of the mVEP type. By taking advantage of the visual speller design (Farwell and Donchin, 1988, Hong et al., 2009), a 36-choice spelling application can be realized on the basis of a series of binary decisions on whether the stimulus presented at certain time point is attended or not. Moreover, mVEP BCI may further benefit from ECoG recordings with broader frequency band response. Specifically, BCI classification may be facilitated by including the high gamma responses as a new feature, since visual motion processing is reflected in both low frequency mVEP responses (Matsumoto et al., 2004) and high frequency (50–120 Hz) power increase (Rauschecker et al., 2011). In the ECoG-based visual motion BCI, we predicted that stronger high gamma response would be elicited by the attended visual motion stimuli than unattended stimuli.
In this study, aiming for a minimally invasive BCI system, we explored the possibility of implementing an ECoG-based visual motion BCI using only one subdural channel. Five epilepsy patients with ECoG electrodes placed over the parietal-temporal-occipital regions were recruited. To achieve minimal invasiveness, one ‘optimal’ electrode per patient was selected according to ECoG responses ideal for BCI control. After the selection of the optimal electrode, ECoG signals in both the low frequency range (i.e., the traditional mVEP) and high gamma range were extracted as features for classification. Prior to the electrode implantation surgery, fMRI scans were made for two patients to localize brain regions specialized for visual motion processing. We compared the spatial locations of the optimal electrodes for BCI classification with the functional regions pre-operatively mapped with fMRI to evaluate whether fMRI can guide electrode implantation for BCI purposes. Our results demonstrate that a minimally invasive visual motion BCI can be implemented using non-invasive fMRI measurements to determine the electrode locations pre-operatively.
Section snippets
Participants
Participants were five patients (see Table 1 for additional information) with intractable epilepsy. Intracranial ECoG electrode grids were temporarily placed in their brain to localize seizure foci prior to surgical resection. The patients all had normal or corrected-to-normal vision. Written informed consent was obtained from each patient before participation. This study was approved by the Institutional Review Board at both Tsinghua University and the affiliated Yuquan Hospital.
Each patient
Modulation of intracranial visual motion responses by attention
A typical time–frequency response to the overtly attended visual motion stimuli is illustrated in Fig. 2a. The ECoG responses were comprised of both a low frequency ERP and a power increase at the high gamma frequency range. Statistical analyses showed that the power increase was significant over the 60–140 Hz frequency range (p < 0.05, FDR corrected). Compared to the unattended stimuli, the attended ECoG response at this chosen electrode showed a negative deflection of the ERP amplitude around 200
Discussion
In this study, we investigated the possibility of implementing a minimally invasive BCI through a novel mVEP BCI paradigm. The visual motion stimuli evoked ERP responses over broad brain regions, whereas the elicited high gamma power increases were restricted to the visual motion processing areas. Classification accuracy between the BCI target and non-target trials was enhanced by using high gamma power as an additional feature. Furthermore, the ECoG electrodes with significant high gamma
Acknowledgments
This work was supported by the National Natural Science Foundation of China under grants #61071003 and #90820304, National Program on Key Basic Research Projects of China (2011CB933204), and China Postdoctoral Science Foundation. The authors would like to thank Wei Wu for his help and comments on statistical methods; Rui Li, Hua Guo, Le He, and Yandong Zhu from Center for Biomedical Imaging Research of Tsinghua University for their assistance during MRI data collection; and Ryan Morrill for
References (81)
- et al.
Single-trial analysis and classification of ERP components–a tutorial
Neuroimage
(2011) - et al.
EEG coherence has structure in the millimeter domain: subdural and hippocampal recordings from epileptic patients
Electroencephalogr. Clin. Neurophysiol.
(1995) - et al.
High-frequency gamma oscillations and human brain mapping with electrocorticography
Prog. Brain Res.
(2006) - et al.
Cortical surface-based analysis. I. Segmentation and surface reconstruction
Neuroimage
(1999) - et al.
Brain–computer interfaces in neurological rehabilitation
Lancet Neurol.
(2008) - et al.
An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest
Neuroimage
(2006) - et al.
Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials
Electroencephalogr. Clin. Neurophysiol.
(1988) - et al.
Cortical surface-based analysis. II: Inflation, flattening, and a surface-based coordinate system
Neuroimage
(1999) - et al.
Transformations towards the normal distribution of broad band spectral parameters of the EEG
Electroencephalogr. Clin. Neurophysiol.
(1982) - et al.
Thresholding of statistical maps in functional neuroimaging using the false discovery rate
Neuroimage
(2002)
Visual half-field development in children: detection of motion-defined forms
Vision Res.
The development of hemispheric asymmetry in human motion VEPs
Vision Res.
N200-speller using motion-onset visual response
Clin. Neurophysiol.
Human gamma-frequency oscillations associated with attention and memory
Trends Neurosci.
A combined brain–computer interface based on P300 potentials and motion-onset visual evoked potentials
J. Neurosci. Methods
Motion-onset VEPs: characteristics, methods, and diagnostic use
Vision Res.
Brain-machine interfaces: past, present and future
Trends Neurosci.
Coupling of neural activity and fMRI-BOLD in the motion area MT
Magn. Reson. Imaging
Timing, timing, timing: fast decoding of object information from intracranial field potentials in human visual cortex
Neuron
Subregions of human MT complex revealed by comparative MEG and direct electrocorticographic recordings
Clin. Neurophysiol.
Spatiotemporal dynamics of electrocorticographic high gamma activity during overt and covert word repetition
Neuroimage
Electroencephalographic (EEG)-based communication: EEG control versus system performance in humans
Neurosci. Lett.
How distributed is visual category information in human occipito-temporal cortex? An fMRI study
Neuron
A novel P300-based brain–computer interface stimulus presentation paradigm: moving beyond rows and columns
Clin. Neurophysiol.
Multi-modal volume registration by maximization of mutual information
Med. Image Anal.
The Psychophysics Toolbox
Spat. Vis.
Does the 'P300' speller depend on eye gaze?
J. Neural Eng.
Rapid Communication with a "P300" Matrix Speller Using Electrocorticographic Signals (ECoG)
Front. Neurosci.
LIBSVM: a library for support vector machines
Categorical speech representation in human superior temporal gyrus
Nat. Neurosci.
Cortical spatio-temporal dynamics underlying phonological target detection in humans
J. Cogn. Neurosci.
Long-term asynchronous decoding of arm motion using electrocorticographic signals in monkeys
Front. Neuroeng.
Control of goal-directed and stimulus-driven attention in the brain
Nat. Rev. Neurosci.
Functional mapping of human sensorimotor cortex with electrocorticographic spectral analysis. II. Event-related synchronization in the gamma band
Brain
Functional mapping of human sensorimotor cortex with electrocorticographic spectral analysis. I. Alpha and beta event-related desynchronization
Brain
Neural mechanisms of selective visual attention
Annu. Rev. Neurosci.
Mapping striate and extrastriate visual areas in human cerebral cortex
Proc. Natl. Acad. Sci. U.S.A.
The mental prosthesis: assessing the speed of a P300-based brain–computer interface
IEEE Trans. Rehabil. Eng.
Dynamic predictions: oscillations and synchrony in top-down processing
Nat. Rev. Neurosci.
Nonuniform high-gamma (60-500 Hz) power changes dissociate cognitive task and anatomy in human cortex
J. Neurosci.
Cited by (42)
Implementation of artificial intelligence and machine learning-based methods in brain–computer interaction
2023, Computers in Biology and MedicineIntracranial brain-computer interface spelling using localized visual motion response
2022, NeuroImageCitation Excerpt :The attended visual motion stimuli elicit a more negative peak approximately 200 ms poststimulus (N200) over parietal-occipital areas than the unattended stimuli, and user intent can be decoded using this neural signature. In intracortical EEG, the visual motion response exhibits large SNR and broad bandwidth (Gaglianese et al., 2017; Zhang et al., 2013), which was not shown in previous EEG studies (Kuba and Kubová, 1992; Mackay and Rietveld, 1968). Although MT area differs from person to person (Huang et al., 2019), there is still a spatial consistency between fMRI and EEG recorded visual motion response (Bucher et al., 2006; Gaglianese et al., 2017).
Speech frequency-following response in human auditory cortex is more than a simple tracking
2021, NeuroImageCitation Excerpt :1) A presurgical high-resolution T1-weighted structural MRI scan and a post-implantation CT scan were acquired for each subject. The CT images were registered to the presurgical MRI images with statistical parametric mapping (SPM) implementations of the mutual information-based transform algorithm (Wells et al., 1996; Zhang et al., 2013) in FreeSurfer (Fig. 1A). ( 2) For each bipolar iEEG channel, the best channel position was located between the two corresponding electrode positions, and its location on the surface of the cortex was identified as the vertex nearest to the contact site in the 3D-volume space.
iEEG: Dura-lining electrodes
2020, Handbook of Clinical NeurologyCitation Excerpt :iEEG signals can be recorded in several ways, with electrodes on the brain surface, lining the dura (electrocorticography, ECoG), which is the focus of this chapter, or with electrodes that penetrate the brain (stereo EEG, sEEG). Implanting ECoG electrodes to control a BCI has sometimes been considered “minimally invasive,” as it would only require a small burr hole to implant a few electrodes on the brain surface without penetrating the brain tissue (Leuthardt et al., 2009; Zhang et al., 2013). Most of what has been learned about ECoG signals is from recordings in patients who suffer from intractable epilepsy.
Correlation-based channel selection and regularized feature optimization for MI-based BCI
2019, Neural NetworksCitation Excerpt :Brain–computer interfaces (BCIs) provide an alternative communication channel between the brain and the outside world (Krepki, Blankertz, Curio, & Muller, 2007; Wolpaw, Birbaumer, McFarland, Pfurtscheller, & Vaughan, 2002; Zhang et al., 2013).
Neural decoding by invasive electrocorticography
2019, Intelligent Biomechatronics in Neurorehabilitation
- 1
Authors equally contributed to this paper.