Computational NeuroscienceFeasibility of approaches combining sensor and source features in brain–computer interface
Highlights
► We proposed combined approaches between sensor and source features in brain–computer interface. ► Combining approach of sensor and source features improved about 3.8% in classification accuracy than sensor features alone. ► Source features were supplementary to sensor features. ► Amount of invisible information in sensor space was significant in the low performance datasets.
Introduction
The brain–computer interface (BCI) has received much attention as one of the most fascinating applications in the bio-signal processing community because of the feasibility that BCI may provide bedridden patients with control of certain objects, providing them with a better quality of life. A BCI system generally consists of two phases, as described in Fig. 1: a calibration phase and a feedback phase. The collection of a training dataset from bio-signal measurement systems such as EEG (electroencephalography), MEG (magnetoencephalography), and other brain imaging systems are normally required in the calibration phase. Sensors (electrodes or SQUIDs) outside the head measure brain signals originating from numerous active brain sources representing dynamic brain activity. These signals contain essential clues on neural information and complex processing mechanisms. Some informative features are extracted from these measured signals and used to construct an online classifier. In the feedback phase, the user generates brain signals according to the user's intention and then an online classifier tries to decrypt the user's intention in real-time. The decrypted information is then transformed into instructions that are sent to a computer or machine in order to control it accordingly.
For better development of a BCI system, two ingredients are essential. One involves developing an experimental paradigm that is as easily applicable to a BCI system as possible. During an experimental procedure, brain signals are collected and classified. In the two-class paradigm for a BCI system, left or right hand movement imagery has been widely used (Guger, 2011). Pfurtscheller and Lopes da Silva (1999) reported that relative power changes [event related (de)synchronization (ERD/ERS)] occur in a specific brain area when the subject imagines a hand movement without real muscle activity. If a subject imagines limb movement, then contra-lateral ERD and ipsilateral ERS arise in the motor cortex of the brain (Pfurtscheller et al., 1997, Pfurtscheller and Lopes da Silva, 1999). This is one of the most popular features of BCI. The other ingredient in a BCI system involves the extraction of features from different conditioned brain signals. These features should be as distinct and informative as possible because they could account for conditioned information.
Many BCI investigators have reported that their systems and methods resulted in good performance and are applicable for clinical or entertainment use such as rehabilitation, gaming, the virtual-environment, and so on (Blankertz et al., 2006, McFarland et al., 2010, Müller et al., 2008, Lotte et al., 2010, Sellers et al., 2006, Velliste et al., 2008, Wolpaw and McFarland, 2004). EEG is favorable in BCI development because it provides information on rapidly moving brain dynamics, is convenient due to ease of mobility, and is cost effective (Ahn et al., 2010, Guger, 2011, McFarland et al., 2010). However, EEG is heavily contaminated by noise originating from inside and outside the head, leading to a low signal-to-noise ratio (SNR). In addition, it shows very low spatial resolution due to the signal from infinitely many sources in the brain being blurred by volume conduction. This makes it difficult to discriminate between signal and noise, so it is too hard to extract reasonably good features usable by BCI systems. To deal with this issue, many techniques have been proposed to enhance the SNR of electrode signals, including spatial filtering methods such as Laplacian and common spatial patterns (CSP) (Ramoser et al., 2000, Blankertz et al., 2008b). Some researchers in the BCI field have focused on feature extraction methods like the autoregressive parameter (Curran et al., 2004), connectivity feature (Wei et al., 2007), or other variants of CSP (Blankertz et al., 2008a, Lemm et al., 2005, Dornhege et al., 2006). A comparative study of such CSP-related approaches for EEG and MEG has been reported (Kang et al., 2009).
As an effort to seek additional or better information, source imaging has recently been applied. Some researchers have reported that source imaging could outperform the use of sensor space alone (Ahn et al., 2010, Besserve et al., 2008, Besserve et al., 2011, Lotte et al., 2009, Grosse-Wentrup et al., 2006, Grosse-Wentrup et al., 2009, Yuan et al., 2008, Noirhomme et al., 2008, Congedo et al., 2006, Kamousi et al., 2005, Qin et al., 2004). Source imaging involves reconstructing the source activity in the given target region within the brain, so it may localize the active source region and give a source signal pattern explaining the physiological information. With their own acquired data or BCI Competition data, relevant studies show the superiority of source features over sensor features, regardless of the source imaging techniques and head models in use. Kamousi et al. (2005) localized the source activation location through a dipole fitting method to classify left or right motor imagery. Congedo et al. (2006) applied sLORETA along with CSP to a single BCI Competition dataset (self-paced finger tapping experiment) and showed its improvement. For the same dataset, Noirhomme et al. (2008) proposed various features in sensor or source spaces, and compared the performance across various feature types and prior information incorporated into the inverse solver. They used the spherical head model with 400 predefined voxels near the motor cortex, and reported comparable accuracy to the competition's winner when simple priori information and power features of reconstructed source activity were used.
Besserve et al. (2011) recently proposed a source imaging approach using a realistic head model for an online BCI system. In the test with motor or non-motor experimental datasets from 5 subjects, they showed that source features were superior to sensor features in terms of the information transfer rate (ITR), and the redundancy between the power features and coherence features was smaller in the source space. This result may be due to some fraction of noise being filtered out by source imaging, and thus extracting features from the source signal may be beneficial in a BCI system. The majority of reports on this topic show the following:
- •
Sensor information naturally contains all obtainable (invisible or visible) information.
- •
The visibility of information in the sensor space may be limited due to noise contamination.
- •
Source features are reconstructed from sensor information, thus they originate from sensor information.
- •
Source features may include the most informative sensor features together with additional complementary information. This insight is supportive in that the accuracy of source features outperforms that of sensor features alone.
From this understanding, we can infer that:
“Source imaging may reveal some invisible information in the sensor space into visible information. Thus, the visible information distribution between the sensor and source spaces may differ and there may exist some exclusive visible information for each space.”
If this inference is true, then a reasonable combined method of sensor and source features may yield a synergistic effect, leading to better performance. The detailed investigation on the above inference and information distribution between the sensor and source features has been lacking. This hypothetical perspective is depicted with a simple illustration in Fig. 2.
To observe such a perspective, we investigated the information distribution and the overlapping space between the sensor and source feature spaces. We introduced the feature type as the averaged power features over the temporal window because this measure is widely acceptable in most BCI research with source imaging. How features are discriminated is quantified as the classification accuracy estimated by cross-validation. In the present work, adaptive beamforming was introduced as a source localizer. Beamforming yields relatively reasonable localization results (Sekihara et al., 2005), and any kind of voxel set in the brain can be customized for the source space. In this study, any head models or source imaging methods are applicable; we used a spherical head model and beamforming technique. The classification accuracies of the sensor or source features for the motor imagery datasets were assessed and two approaches combining information from both the sensor and source spaces in BCI were proposed. One approach is to simply expand the feature dimension by concatenating both feature vectors, and the other is to use multiple classifiers. In total, four methods – sensor features alone, source features alone, and two combination approaches – were compared to assess the feasibility of the combined approaches. A total of 23 datasets (18 datasets collected on our system and 5 datasets from the BCI Competition III web site) are used in this work. To the best of our knowledge, this is the largest number of datasets used in a BCI study with source imaging to date.
This paper is organized as follows. The detailed experimental paradigm is explained, and then a brief overview of the beamforming technique is provided. Feature selection and BCI classification performance are also discussed in the following section. In Section 3, the statistical results and their interpretation are presented with the statistical p-value, and an in-depth investigation of the visible information distribution in the sensor and source spaces is discussed. The mathematical validation of our proposed combination approach and an additional assessment with the BCI Competition III datasets are presented in Section 4. Finally, the findings are summarized briefly and the conclusion is provided.
Section snippets
Experimental data acquisition
Nine healthy subjects (3 females and 6 males) participated in the experimental EEG data collection. All subjects were right-handed, and their mean age was 26.3 years old with a standard deviation of 5.4 years. The EEG data were acquired under the following experimental paradigm. We instructed the subjects to fix their gaze on the center of the screen at all times during the experimental procedure. For the first 2 s, a fixation cross appeared, after which a left or right arrow (direction) was
Statistical p-value and accuracy
The spatial distribution of the p-values from the Student's t-tests may show which area (where the channels are located) is more significant in discriminating the two different conditions. For each of the two spaces (sensor and source), the spatial p-value distribution from a unpaired Student's t-test between two conditions is depicted in Fig. 6, which describes the p-value distribution of the sensor and source spaces for dataset E2. To see the comparative difference between spatial
Exclusive information between sensor and source spaces
As we discussed in the previous section, the hit-trial distributions (Fig. 9) showed the different trial classifications between sensor and source spaces. Interestingly, both sensor and source spaces could contain more or less visible exclusive information. We understand that exclusive information in the source space may come from denoising effect of source imaging; source imaging enables to filter out some level of noise and projects sensor information onto source space, thus some information
Conclusion
EEG-based BCI is a popular research area because of its convenience and good temporal resolution. However, EEG has too low of an SNR to extract a discriminative feature for practical BCI. Thus, source imaging has been proposed as a better feature for BCI. In the present work, we investigated the combination of features from the sensor and source spaces and found that complementary information may exist in the two. From the investigation using 18 hand motor imagery datasets and BCI Competition
Acknowledgements
This work was supported by the NIPA (NIPA-2011-C1090-1131-0006), the NRF grant (NRF-2010-0006135), and the BioImaging Research Center at GIST.
References (37)
- et al.
Improving quantification of functional networks with EEG inverse problem: evidence from a decoding point of view
NeuroImage
(2011) - et al.
Neurophysiological predictor of SMR-based BCI performance
NeuroImage
(2010) - et al.
Machine learning for real-time single-trial EEG-analysis: from brain–computer interfacing to mental state monitoring
J Neurosci Methods
(2008) - et al.
EEG-based discrimination between imagination of right and left hand movement
Electroen Clin Neurophysiol
(1997) - et al.
Event-related EEG/MEG synchronization and desynchronization: basic principles
Clin Neurophysiol
(1999) - et al.
Localization bias and spatial resolution of adaptive and non-adaptive spatial filters for MEG source reconstruction
NeuroImage
(2005) - et al.
A p300 event-related potential brain–computer interface (BCI): the effects of matrix size and inter stimulus interval on performance
Biol Psychol
(2006) - et al.
Source space based brain computer interface
- et al.
Non-invasive classification of cortical activities for brain computer interface: a variable selection approach
- et al.
The Berlin brain–computer interface presents the novel mental typewriter hex-o-spell