Computational Neuroscience
Feasibility of approaches combining sensor and source features in brain–computer interface

https://doi.org/10.1016/j.jneumeth.2011.11.002Get rights and content

Abstract

Brain–computer interface (BCI) provides a new channel for communication between brain and computers through brain signals. Cost-effective EEG provides good temporal resolution, but its spatial resolution is poor and sensor information is blurred by inherent noise. To overcome these issues, spatial filtering and feature extraction techniques have been developed. Source imaging, transformation of sensor signals into the source space through source localizer, has gained attention as a new approach for BCI. It has been reported that the source imaging yields some improvement of BCI performance. However, there exists no thorough investigation on how source imaging information overlaps with, and is complementary to, sensor information. Information (visible information) from the source space may overlap as well as be exclusive to information from the sensor space is hypothesized. Therefore, we can extract more information from the sensor and source spaces if our hypothesis is true, thereby contributing to more accurate BCI systems. In this work, features from each space (sensor or source), and two strategies combining sensor and source features are assessed. The information distribution among the sensor, source, and combined spaces is discussed through a Venn diagram for 18 motor imagery datasets. Additional 5 motor imagery datasets from the BCI Competition III site were examined. The results showed that the addition of source information yielded about 3.8% classification improvement for 18 motor imagery datasets and showed an average accuracy of 75.56% for BCI Competition data. Our proposed approach is promising, and improved performance may be possible with better head model.

Highlights

► We proposed combined approaches between sensor and source features in brain–computer interface. ► Combining approach of sensor and source features improved about 3.8% in classification accuracy than sensor features alone. ► Source features were supplementary to sensor features. ► Amount of invisible information in sensor space was significant in the low performance datasets.

Introduction

The brain–computer interface (BCI) has received much attention as one of the most fascinating applications in the bio-signal processing community because of the feasibility that BCI may provide bedridden patients with control of certain objects, providing them with a better quality of life. A BCI system generally consists of two phases, as described in Fig. 1: a calibration phase and a feedback phase. The collection of a training dataset from bio-signal measurement systems such as EEG (electroencephalography), MEG (magnetoencephalography), and other brain imaging systems are normally required in the calibration phase. Sensors (electrodes or SQUIDs) outside the head measure brain signals originating from numerous active brain sources representing dynamic brain activity. These signals contain essential clues on neural information and complex processing mechanisms. Some informative features are extracted from these measured signals and used to construct an online classifier. In the feedback phase, the user generates brain signals according to the user's intention and then an online classifier tries to decrypt the user's intention in real-time. The decrypted information is then transformed into instructions that are sent to a computer or machine in order to control it accordingly.

For better development of a BCI system, two ingredients are essential. One involves developing an experimental paradigm that is as easily applicable to a BCI system as possible. During an experimental procedure, brain signals are collected and classified. In the two-class paradigm for a BCI system, left or right hand movement imagery has been widely used (Guger, 2011). Pfurtscheller and Lopes da Silva (1999) reported that relative power changes [event related (de)synchronization (ERD/ERS)] occur in a specific brain area when the subject imagines a hand movement without real muscle activity. If a subject imagines limb movement, then contra-lateral ERD and ipsilateral ERS arise in the motor cortex of the brain (Pfurtscheller et al., 1997, Pfurtscheller and Lopes da Silva, 1999). This is one of the most popular features of BCI. The other ingredient in a BCI system involves the extraction of features from different conditioned brain signals. These features should be as distinct and informative as possible because they could account for conditioned information.

Many BCI investigators have reported that their systems and methods resulted in good performance and are applicable for clinical or entertainment use such as rehabilitation, gaming, the virtual-environment, and so on (Blankertz et al., 2006, McFarland et al., 2010, Müller et al., 2008, Lotte et al., 2010, Sellers et al., 2006, Velliste et al., 2008, Wolpaw and McFarland, 2004). EEG is favorable in BCI development because it provides information on rapidly moving brain dynamics, is convenient due to ease of mobility, and is cost effective (Ahn et al., 2010, Guger, 2011, McFarland et al., 2010). However, EEG is heavily contaminated by noise originating from inside and outside the head, leading to a low signal-to-noise ratio (SNR). In addition, it shows very low spatial resolution due to the signal from infinitely many sources in the brain being blurred by volume conduction. This makes it difficult to discriminate between signal and noise, so it is too hard to extract reasonably good features usable by BCI systems. To deal with this issue, many techniques have been proposed to enhance the SNR of electrode signals, including spatial filtering methods such as Laplacian and common spatial patterns (CSP) (Ramoser et al., 2000, Blankertz et al., 2008b). Some researchers in the BCI field have focused on feature extraction methods like the autoregressive parameter (Curran et al., 2004), connectivity feature (Wei et al., 2007), or other variants of CSP (Blankertz et al., 2008a, Lemm et al., 2005, Dornhege et al., 2006). A comparative study of such CSP-related approaches for EEG and MEG has been reported (Kang et al., 2009).

As an effort to seek additional or better information, source imaging has recently been applied. Some researchers have reported that source imaging could outperform the use of sensor space alone (Ahn et al., 2010, Besserve et al., 2008, Besserve et al., 2011, Lotte et al., 2009, Grosse-Wentrup et al., 2006, Grosse-Wentrup et al., 2009, Yuan et al., 2008, Noirhomme et al., 2008, Congedo et al., 2006, Kamousi et al., 2005, Qin et al., 2004). Source imaging involves reconstructing the source activity in the given target region within the brain, so it may localize the active source region and give a source signal pattern explaining the physiological information. With their own acquired data or BCI Competition data, relevant studies show the superiority of source features over sensor features, regardless of the source imaging techniques and head models in use. Kamousi et al. (2005) localized the source activation location through a dipole fitting method to classify left or right motor imagery. Congedo et al. (2006) applied sLORETA along with CSP to a single BCI Competition dataset (self-paced finger tapping experiment) and showed its improvement. For the same dataset, Noirhomme et al. (2008) proposed various features in sensor or source spaces, and compared the performance across various feature types and prior information incorporated into the inverse solver. They used the spherical head model with 400 predefined voxels near the motor cortex, and reported comparable accuracy to the competition's winner when simple priori information and power features of reconstructed source activity were used.

Besserve et al. (2011) recently proposed a source imaging approach using a realistic head model for an online BCI system. In the test with motor or non-motor experimental datasets from 5 subjects, they showed that source features were superior to sensor features in terms of the information transfer rate (ITR), and the redundancy between the power features and coherence features was smaller in the source space. This result may be due to some fraction of noise being filtered out by source imaging, and thus extracting features from the source signal may be beneficial in a BCI system. The majority of reports on this topic show the following:

  • Sensor information naturally contains all obtainable (invisible or visible) information.

  • The visibility of information in the sensor space may be limited due to noise contamination.

  • Source features are reconstructed from sensor information, thus they originate from sensor information.

  • Source features may include the most informative sensor features together with additional complementary information. This insight is supportive in that the accuracy of source features outperforms that of sensor features alone.

From this understanding, we can infer that:

Source imaging may reveal some invisible information in the sensor space into visible information. Thus, the visible information distribution between the sensor and source spaces may differ and there may exist some exclusive visible information for each space.

If this inference is true, then a reasonable combined method of sensor and source features may yield a synergistic effect, leading to better performance. The detailed investigation on the above inference and information distribution between the sensor and source features has been lacking. This hypothetical perspective is depicted with a simple illustration in Fig. 2.

To observe such a perspective, we investigated the information distribution and the overlapping space between the sensor and source feature spaces. We introduced the feature type as the averaged power features over the temporal window because this measure is widely acceptable in most BCI research with source imaging. How features are discriminated is quantified as the classification accuracy estimated by cross-validation. In the present work, adaptive beamforming was introduced as a source localizer. Beamforming yields relatively reasonable localization results (Sekihara et al., 2005), and any kind of voxel set in the brain can be customized for the source space. In this study, any head models or source imaging methods are applicable; we used a spherical head model and beamforming technique. The classification accuracies of the sensor or source features for the motor imagery datasets were assessed and two approaches combining information from both the sensor and source spaces in BCI were proposed. One approach is to simply expand the feature dimension by concatenating both feature vectors, and the other is to use multiple classifiers. In total, four methods – sensor features alone, source features alone, and two combination approaches – were compared to assess the feasibility of the combined approaches. A total of 23 datasets (18 datasets collected on our system and 5 datasets from the BCI Competition III web site) are used in this work. To the best of our knowledge, this is the largest number of datasets used in a BCI study with source imaging to date.

This paper is organized as follows. The detailed experimental paradigm is explained, and then a brief overview of the beamforming technique is provided. Feature selection and BCI classification performance are also discussed in the following section. In Section 3, the statistical results and their interpretation are presented with the statistical p-value, and an in-depth investigation of the visible information distribution in the sensor and source spaces is discussed. The mathematical validation of our proposed combination approach and an additional assessment with the BCI Competition III datasets are presented in Section 4. Finally, the findings are summarized briefly and the conclusion is provided.

Section snippets

Experimental data acquisition

Nine healthy subjects (3 females and 6 males) participated in the experimental EEG data collection. All subjects were right-handed, and their mean age was 26.3 years old with a standard deviation of 5.4 years. The EEG data were acquired under the following experimental paradigm. We instructed the subjects to fix their gaze on the center of the screen at all times during the experimental procedure. For the first 2 s, a fixation cross appeared, after which a left or right arrow (direction) was

Statistical p-value and accuracy

The spatial distribution of the p-values from the Student's t-tests may show which area (where the channels are located) is more significant in discriminating the two different conditions. For each of the two spaces (sensor and source), the spatial p-value distribution from a unpaired Student's t-test between two conditions is depicted in Fig. 6, which describes the p-value distribution of the sensor and source spaces for dataset E2. To see the comparative difference between spatial

Exclusive information between sensor and source spaces

As we discussed in the previous section, the hit-trial distributions (Fig. 9) showed the different trial classifications between sensor and source spaces. Interestingly, both sensor and source spaces could contain more or less visible exclusive information. We understand that exclusive information in the source space may come from denoising effect of source imaging; source imaging enables to filter out some level of noise and projects sensor information onto source space, thus some information

Conclusion

EEG-based BCI is a popular research area because of its convenience and good temporal resolution. However, EEG has too low of an SNR to extract a discriminative feature for practical BCI. Thus, source imaging has been proposed as a better feature for BCI. In the present work, we investigated the combination of features from the sensor and source spaces and found that complementary information may exist in the two. From the investigation using 18 hand motor imagery datasets and BCI Competition

Acknowledgements

This work was supported by the NIPA (NIPA-2011-C1090-1131-0006), the NRF grant (NRF-2010-0006135), and the BioImaging Research Center at GIST.

References (37)

  • BlankertzB. et al.

    Invariant common spatial patterns: alleviating nonstationarities in brain–computer interfacing

    Adv Neural Inf Process Syst

    (2008)
  • BlankertzB. et al.

    Optimizing spatial filters for robust EEG single-trial analysis

    IEEE Signal Process Mag

    (2008)
  • CongedoM. et al.

    Classification of movement intention by spatially filtered electromagnetic inverse solutions

    Phys Med Biol

    (2006)
  • CurranE. et al.

    Cognitive tasks for driving a brain–computer interfacing system: a pilot study

    IEEE Trans Neural Syst Rehabil Eng

    (2004)
  • DornhegeG. et al.

    Combined optimization of spatial and temporal filters for improving brain–computer interfacing

    IEEE Trans Biomed Eng

    (2006)
  • Grosse-WentrupM. et al.

    Adaptive spatial filters with predefined region of interest for EEG based brain–computer-interfaces

  • Grosse-WentrupM. et al.

    Beamforming in noninvasive brain–computer interfaces

    IEEE Trans Biomed Eng

    (2009)
  • GugerC

    Recent advances in brain–computer interface systems

  • Cited by (0)

    View full text