Elsevier

NeuroImage

Volume 71, 1 May 2013, Pages 30-41
NeuroImage

Toward a minimally invasive brain–computer interface using a single subdural channel: A visual speller study

https://doi.org/10.1016/j.neuroimage.2012.12.069Get rights and content

Abstract

Electrocorticography (ECoG) has attracted increasing interest for implementing advanced brain–computer interfaces (BCIs) in the past decade. However, real-life application of ECoG BCI demands mitigation of its invasive nature by minimizing both the size of the involved brain regions and the number of implanted electrodes. In this study, we employed a recently proposed BCI paradigm that utilizes the attentional modulation of visual motion response. With ECoG data collected from five epilepsy patients, power increase of the high gamma (60–140 Hz) frequency range was found to be associated with the overtly attended moving visual stimuli in the parietal-temporal-occipital junction and the occipital cortex. Event-related potentials (ERPs) were elicited as well but with broader cortical distribution. We achieved significantly higher BCI classification accuracy by employing both high gamma and ERP responses from a single ECoG electrode than by using ERP responses only (84.22 ± 5.54% vs. 75.48 ± 4.18%, p < 0.005, paired t-test, 3-trial averaging, binary results of attended vs. unattended). More importantly, the high gamma responses were located within brain regions specialized in visual motion processing as mapped by fMRI, suggesting the spatial location for electrode implantation can be determined prior to surgery using non-invasive imaging. Our findings demonstrate the feasibility of implementing a minimally invasive ECoG BCI.

Highlights

► Visual motion stimuli evoke cortical high gamma ECoG response. ► ECoG high gamma feature elevates BCI performance of visual speller. ► Spatial consistency is found between ECoG and fMRI response to visual motion. ► Location of subdural BCI electrodes can be determined by fMRI before surgery. ► Feasibility of minimally invasive BCI with single subdural electrode is demonstrated.

Introduction

The past decade has seen promising development in brain–computer interfaces (BCIs) aiming to help severely motor disabled patients to interact with the external world by directly interpreting brain activity. While the majority of BCIs rely on brain activity recorded with non-invasive electroencephalograph (EEG) technology (Daly and Wolpaw, 2008, Lebedev and Nicolelis, 2006), interest has increasingly been drawn toward the use of invasive electrocorticographic (ECoG) signals for BCI applications (Lebedev and Nicolelis, 2006, Leuthardt et al., 2004, Schalk and Leuthardt, 2011).

Compared to EEG, ECoG has two major advantages for implementing advanced BCIs. First, the spatial resolution of ECoG is much higher than that of EEG. ECoG records signals originating from brain tissues directly beneath the electrode surface (surface area of 1–10 mm2) with little influence from adjacent tissues (Bullock et al., 1995, Nunez and Srinivasan, 2006), whereas EEG's spatial resolution is at a multi-centimeter scale due to volume conduction of currents through tissues of the head. Such a fine spatial resolution makes ECoG ideal for imaging the cortical dynamics of sensory and cognitive functions that originate from relatively small brain regions (Liu et al., 2009, Mesgarani and Chang, 2012, Vinjamuri et al., 2011). Second, low voltage, high frequency brain activity that is barely detectable in scalp EEG is readily observed in invasive ECoG recordings (Leuthardt et al., 2004). In the past few years, ECoG BCIs have achieved promising results using brain signals extracted from the motor cortex (Kubanek et al., 2009, Miller et al., 2010, Vinjamuri et al., 2011), language-related brain regions (Leuthardt et al., 2011, Pei et al., 2011a), as well as auditory and visual cortex (Brunner et al., 2010, Wilson et al., 2006).

When using ECoG or other invasive recording techniques, it is preferable to minimize the cortical area used for acquiring sufficient neural information for BCI applications. To date, ECoG BCI studies have been carried out with epilepsy patients who underwent open skull surgery and implantation of large electrode arrays for clinical purposes. However, surgery of this type is both unnecessary and risky for the potential BCI users (i.e. patients with amyotrophic lateral sclerosis). Instead, electrodes for BCI purposes can be inserted through a burr hole into the brain, thus implementing a ‘minimally invasive’ BCI, given that sufficient information can be extracted within a limited cortical area.

The recently developed visual motion BCI paradigm using motion onset visual evoked potentials (mVEPs) may serve as a candidate for building minimally invasive BCIs (Guo et al., 2008, Jin et al., 2012). The mVEP speller utilizes the modulation effect of mVEPs by overt attention (i.e., eye gaze) for a spelling application: the attended visual motion stimuli elicit a more negative peak around 200 ms post-stimulus (N200) over parietal-occipital areas than the unattended stimuli, and BCI user intent is classified based on this difference. BCI based on mVEP can be considered an extension of the widely studied P300 BCI (Donchin et al., 2000, Farwell and Donchin, 1988), which utilizes an attention-related positive event-related potential (ERP) peak around 300 ms (P300) over the parietal cortex elicited by the attended visual flash stimuli. Based on invasiveness, mVEP BCI is a better candidate than P300 BCIs, as visual motion processing (reflected by mVEPs in EEG) is believed to be highly focused in a brain region known as the human middle temporal (MT) complex (DeYoe et al., 1996, Huk et al., 2002, Zeki et al., 1991). Compared with other types of BCIs for motor and speech decoding (Kubanek et al., 2009, Leuthardt et al., 2004, Leuthardt et al., 2011), relatively less information is needed for the operation of the mVEP type. By taking advantage of the visual speller design (Farwell and Donchin, 1988, Hong et al., 2009), a 36-choice spelling application can be realized on the basis of a series of binary decisions on whether the stimulus presented at certain time point is attended or not. Moreover, mVEP BCI may further benefit from ECoG recordings with broader frequency band response. Specifically, BCI classification may be facilitated by including the high gamma responses as a new feature, since visual motion processing is reflected in both low frequency mVEP responses (Matsumoto et al., 2004) and high frequency (50–120 Hz) power increase (Rauschecker et al., 2011). In the ECoG-based visual motion BCI, we predicted that stronger high gamma response would be elicited by the attended visual motion stimuli than unattended stimuli.

In this study, aiming for a minimally invasive BCI system, we explored the possibility of implementing an ECoG-based visual motion BCI using only one subdural channel. Five epilepsy patients with ECoG electrodes placed over the parietal-temporal-occipital regions were recruited. To achieve minimal invasiveness, one ‘optimal’ electrode per patient was selected according to ECoG responses ideal for BCI control. After the selection of the optimal electrode, ECoG signals in both the low frequency range (i.e., the traditional mVEP) and high gamma range were extracted as features for classification. Prior to the electrode implantation surgery, fMRI scans were made for two patients to localize brain regions specialized for visual motion processing. We compared the spatial locations of the optimal electrodes for BCI classification with the functional regions pre-operatively mapped with fMRI to evaluate whether fMRI can guide electrode implantation for BCI purposes. Our results demonstrate that a minimally invasive visual motion BCI can be implemented using non-invasive fMRI measurements to determine the electrode locations pre-operatively.

Section snippets

Participants

Participants were five patients (see Table 1 for additional information) with intractable epilepsy. Intracranial ECoG electrode grids were temporarily placed in their brain to localize seizure foci prior to surgical resection. The patients all had normal or corrected-to-normal vision. Written informed consent was obtained from each patient before participation. This study was approved by the Institutional Review Board at both Tsinghua University and the affiliated Yuquan Hospital.

Each patient

Modulation of intracranial visual motion responses by attention

A typical time–frequency response to the overtly attended visual motion stimuli is illustrated in Fig. 2a. The ECoG responses were comprised of both a low frequency ERP and a power increase at the high gamma frequency range. Statistical analyses showed that the power increase was significant over the 60–140 Hz frequency range (p < 0.05, FDR corrected). Compared to the unattended stimuli, the attended ECoG response at this chosen electrode showed a negative deflection of the ERP amplitude around 200

Discussion

In this study, we investigated the possibility of implementing a minimally invasive BCI through a novel mVEP BCI paradigm. The visual motion stimuli evoked ERP responses over broad brain regions, whereas the elicited high gamma power increases were restricted to the visual motion processing areas. Classification accuracy between the BCI target and non-target trials was enhanced by using high gamma power as an additional feature. Furthermore, the ECoG electrodes with significant high gamma

Acknowledgments

This work was supported by the National Natural Science Foundation of China under grants #61071003 and #90820304, National Program on Key Basic Research Projects of China (2011CB933204), and China Postdoctoral Science Foundation. The authors would like to thank Wei Wu for his help and comments on statistical methods; Rui Li, Hua Guo, Le He, and Yandong Zhu from Center for Biomedical Imaging Research of Tsinghua University for their assistance during MRI data collection; and Ryan Morrill for

References (81)

  • M.A. Hollants-Gilhuijs et al.

    Visual half-field development in children: detection of motion-defined forms

    Vision Res.

    (1998)
  • M.A. Hollants-Gilhuijs et al.

    The development of hemispheric asymmetry in human motion VEPs

    Vision Res.

    (2000)
  • B. Hong et al.

    N200-speller using motion-onset visual response

    Clin. Neurophysiol.

    (2009)
  • O. Jensen et al.

    Human gamma-frequency oscillations associated with attention and memory

    Trends Neurosci.

    (2007)
  • J. Jin et al.

    A combined brain–computer interface based on P300 potentials and motion-onset visual evoked potentials

    J. Neurosci. Methods

    (2012)
  • M. Kuba et al.

    Motion-onset VEPs: characteristics, methods, and diagnostic use

    Vision Res.

    (2007)
  • M.A. Lebedev et al.

    Brain-machine interfaces: past, present and future

    Trends Neurosci.

    (2006)
  • M.T. Lippert et al.

    Coupling of neural activity and fMRI-BOLD in the motion area MT

    Magn. Reson. Imaging

    (2010)
  • H. Liu et al.

    Timing, timing, timing: fast decoding of object information from intracranial field potentials in human visual cortex

    Neuron

    (2009)
  • R. Matsumoto et al.

    Subregions of human MT complex revealed by comparative MEG and direct electrocorticographic recordings

    Clin. Neurophysiol.

    (2004)
  • X. Pei et al.

    Spatiotemporal dynamics of electrocorticographic high gamma activity during overt and covert word repetition

    Neuroimage

    (2011)
  • H. Sheikh et al.

    Electroencephalographic (EEG)-based communication: EEG control versus system performance in humans

    Neurosci. Lett.

    (2003)
  • M. Spiridon et al.

    How distributed is visual category information in human occipito-temporal cortex? An fMRI study

    Neuron

    (2002)
  • G. Townsend et al.

    A novel P300-based brain–computer interface stimulus presentation paradigm: moving beyond rows and columns

    Clin. Neurophysiol.

    (2010)
  • W.R. Wells et al.

    Multi-modal volume registration by maximization of mutual information

    Med. Image Anal.

    (1996)
  • D.H. Brainard

    The Psychophysics Toolbox

    Spat. Vis.

    (1997)
  • P. Brunner et al.

    Does the 'P300' speller depend on eye gaze?

    J. Neural Eng.

    (2010)
  • P. Brunner et al.

    Rapid Communication with a "P300" Matrix Speller Using Electrocorticographic Signals (ECoG)

    Front. Neurosci.

    (2011)
  • C. Chang et al.

    LIBSVM: a library for support vector machines

  • E.F. Chang et al.

    Categorical speech representation in human superior temporal gyrus

    Nat. Neurosci.

    (2010)
  • E.F. Chang et al.

    Cortical spatio-temporal dynamics underlying phonological target detection in humans

    J. Cogn. Neurosci.

    (2011)
  • Z.C. Chao et al.

    Long-term asynchronous decoding of arm motion using electrocorticographic signals in monkeys

    Front. Neuroeng.

    (2010)
  • M. Corbetta et al.

    Control of goal-directed and stimulus-driven attention in the brain

    Nat. Rev. Neurosci.

    (2002)
  • N.E. Crone et al.

    Functional mapping of human sensorimotor cortex with electrocorticographic spectral analysis. II. Event-related synchronization in the gamma band

    Brain

    (1998)
  • N.E. Crone et al.

    Functional mapping of human sensorimotor cortex with electrocorticographic spectral analysis. I. Alpha and beta event-related desynchronization

    Brain

    (1998)
  • R. Desimone et al.

    Neural mechanisms of selective visual attention

    Annu. Rev. Neurosci.

    (1995)
  • E.A. DeYoe et al.

    Mapping striate and extrastriate visual areas in human cerebral cortex

    Proc. Natl. Acad. Sci. U.S.A.

    (1996)
  • E. Donchin et al.

    The mental prosthesis: assessing the speed of a P300-based brain–computer interface

    IEEE Trans. Rehabil. Eng.

    (2000)
  • A.K. Engel et al.

    Dynamic predictions: oscillations and synchrony in top-down processing

    Nat. Rev. Neurosci.

    (2001)
  • C.M. Gaona et al.

    Nonuniform high-gamma (60-500 Hz) power changes dissociate cognitive task and anatomy in human cortex

    J. Neurosci.

    (2011)
  • Cited by (42)

    • Intracranial brain-computer interface spelling using localized visual motion response

      2022, NeuroImage
      Citation Excerpt :

      The attended visual motion stimuli elicit a more negative peak approximately 200 ms poststimulus (N200) over parietal-occipital areas than the unattended stimuli, and user intent can be decoded using this neural signature. In intracortical EEG, the visual motion response exhibits large SNR and broad bandwidth (Gaglianese et al., 2017; Zhang et al., 2013), which was not shown in previous EEG studies (Kuba and Kubová, 1992; Mackay and Rietveld, 1968). Although MT area differs from person to person (Huang et al., 2019), there is still a spatial consistency between fMRI and EEG recorded visual motion response (Bucher et al., 2006; Gaglianese et al., 2017).

    • Speech frequency-following response in human auditory cortex is more than a simple tracking

      2021, NeuroImage
      Citation Excerpt :

      1) A presurgical high-resolution T1-weighted structural MRI scan and a post-implantation CT scan were acquired for each subject. The CT images were registered to the presurgical MRI images with statistical parametric mapping (SPM) implementations of the mutual information-based transform algorithm (Wells et al., 1996; Zhang et al., 2013) in FreeSurfer (Fig. 1A). ( 2) For each bipolar iEEG channel, the best channel position was located between the two corresponding electrode positions, and its location on the surface of the cortex was identified as the vertex nearest to the contact site in the 3D-volume space.

    • iEEG: Dura-lining electrodes

      2020, Handbook of Clinical Neurology
      Citation Excerpt :

      iEEG signals can be recorded in several ways, with electrodes on the brain surface, lining the dura (electrocorticography, ECoG), which is the focus of this chapter, or with electrodes that penetrate the brain (stereo EEG, sEEG). Implanting ECoG electrodes to control a BCI has sometimes been considered “minimally invasive,” as it would only require a small burr hole to implant a few electrodes on the brain surface without penetrating the brain tissue (Leuthardt et al., 2009; Zhang et al., 2013). Most of what has been learned about ECoG signals is from recordings in patients who suffer from intractable epilepsy.

    • Correlation-based channel selection and regularized feature optimization for MI-based BCI

      2019, Neural Networks
      Citation Excerpt :

      Brain–computer interfaces (BCIs) provide an alternative communication channel between the brain and the outside world (Krepki, Blankertz, Curio, & Muller, 2007; Wolpaw, Birbaumer, McFarland, Pfurtscheller, & Vaughan, 2002; Zhang et al., 2013).

    • Neural decoding by invasive electrocorticography

      2019, Intelligent Biomechatronics in Neurorehabilitation
    View all citing articles on Scopus
    1

    Authors equally contributed to this paper.

    View full text