Long Chen
ABSTRACT
Visual imagery (VI) has become a research hotspot with the development of brain-computer Interface (BCI). But it's very difficult to decode the electroencephalogram (EEG) during the VI task because of the lack of effective classification features. However, the classification of emotion-based EEG now can achieve a very high accuracy. We proposed a new BCI paradigm in which emotions would be integrated into VI to improve the performance. To verify the feasibility of our paradigm, we designed a two-session EEG acquisition experiment. We designed three kinds of emotions and three kinds of semantic correspondence i.e., negative animal, positive human and neutral thing. Sixteen subjects were asked to observe and imagine three pictures selected by subjects in visual observation (VO) and VI session, respectively. Then, we used power spectral density (PSD), common spatial pattern (CSP) and Riemann tangent space three features with SVM to analysis the EEG during three statements. The classification results show that the Riemann features obtain the highest average accuracies which are 84.36% and 67.36% for VO and VI, respectively. All the three types of features can exceed chance level on all the subjects. The results prove that our VI integrating emotion is a promising paradigm.
- María A. Cervera, Surjo R. Soekadar, Junichi Ushiba, José del R. Millán, Meigen Liu, Niels Birbaumer, and Gangadhar Garipelli. 2018. Brain-computer interfaces for post-stroke motor rehabilitation: a meta-analysis. Ann. Clin. Transl. Neurol. 5, 5 (May 2018), 651–663. https://doi.org/10.1002/acn3.544Google Scholar
Cross Ref
- F Lotte, L Bougrain, A Cichocki, M Clerc, M Congedo, A Rakotomamonjy, and F Yger. 2018. A review of classification algorithms for EEG-based brain–computer interfaces: a 10-year update. J. Neural Eng. 15, 3 (June 2018), 031005. https://doi.org/10.1088/1741-2552/aab2f2Google ScholarCross Ref
- B. J. Edelman, J. Meng, D. Suma, C. Zurn, E. Nagarajan, B. S. Baxter, C. C. Cline, and B. He. 2019. Noninvasive neuroimaging enhances continuous neural tracking for robotic device control. Sci. Robot. 4, 31 (June 2019), eaaw6844. https://doi.org/10.1126/scirobotics.aaw6844Google Scholar
- Reza Abiri, Soheil Borhani, Eric W Sellers, Yang Jiang, and Xiaopeng Zhao. 2019. A comprehensive review of EEG-based brain–computer interface paradigms. J. Neural Eng. 16, 1 (February 2019), 011001. https://doi.org/10.1088/1741-2552/aaf12eGoogle ScholarCross Ref
- Minpeng Xu, Jin Han, Yijun Wang, Tzyy-Ping Jung, and Dong Ming. 2020. Implementing Over 100 Command Codes for a High-Speed Hybrid Brain-Computer Interface Using Concurrent P300 and SSVEP Features. IEEE Trans. Biomed. Eng. 67, 11 (November 2020), 3073–3082. https://doi.org/10.1109/TBME.2020.2975614Google ScholarCross Ref
- Xiaolin Xiao, Minpeng Xu, Jing Jin, Yijun Wang, Tzyy-Ping Jung, and Dong Ming. 2020. Discriminative Canonical Pattern Matching for Single-Trial Classification of ERP Components. IEEE Trans. Biomed. Eng. 67, 8 (August 2020), 2266–2275. https://doi.org/10.1109/TBME.2019.2958641Google ScholarCross Ref
- Jing Jin, Ruocheng Xiao, Ian Daly, Yangyang Miao, Xingyu Wang, and Andrzej Cichocki. 2020. Internal Feature Selection Method of CSP Based on L1-Norm and Dempster-Shafer Theory. IEEE Trans. Neural Netw. Learn. Syst. (August 2020), 1–12. https://doi.org/10.1109/TNNLS.2020.3015505Google Scholar
- Markus Knauff, Jan Kassubek, Thomas Mulack, and Mark W. Greenlee. 2000. Cortical activation evoked by visual mental imagery as measured by fMRI: NeuroReport 11, 18 (December 2000), 3957–3962. https://doi.org/10.1097/00001756-200012180-00011Google ScholarCross Ref
- Nataliya Kosmyna, Jussi T. Lindgren, and Anatole Lécuyer. 2018. Attending to Visual Stimuli versus Performing Visual Imagery as a Control Strategy for EEG-based Brain-Computer Interfaces. Sci. Rep. 8, 1 (December 2018), 13222. https://doi.org/10.1038/s41598-018-31472-9Google Scholar
- Milan Rybář, Riccardo Poli, and Ian Daly. 2021. Decoding of semantic categories of imagined concepts of animals and tools in fNIRS. J. Neural Eng. 18, 4 (August 2021), 046035. https://doi.org/10.1088/1741-2552/abf2e5Google ScholarCross Ref
- Bertrand Thirion, Edouard Duchesnay, Edward Hubbard, Jessica Dubois, Jean-Baptiste Poline, Denis Lebihan, and Stanislas Dehaene. 2006. Inverse retinotopy: Inferring the visual content of images from brain activation patterns. NeuroImage 33, 4 (December 2006), 1104–1116. https://doi.org/10.1016/j.neuroimage.2006.06.062Google ScholarCross Ref
- Max A van den Boom, Mariska J Vansteensel, Melissa I Koppeschaar, Matthijs A H Raemaekers, and Nick F Ramsey. 2019. Towards an intuitive communication-BCI: decoding visually imagined characters from the early visual cortex using high-field fMRI. Biomed. Phys. Eng. Express 5, 5 (August 2019), 055001. https://doi.org/10.1088/2057-1976/ab302cGoogle ScholarCross Ref
- Seo-Hyun Lee, Minji Lee, and Seong-Whan Lee. 2020. Neural Decoding of Imagined Speech and Visual Imagery as Intuitive Paradigms for BCI Communication. IEEE Trans. Neural Syst. Rehabil. Eng. (December 2020), 1–1. https://doi.org/10.1109/TNSRE.2020.3040289Google ScholarCross Ref
- Wei-Long Zheng, Jia-Yi Zhu, and Bao-Liang Lu. 2019. Identifying Stable Patterns over Time for Emotion Recognition from EEG. IEEE Trans. Affect. Comput. 10, 3 (July 2019), 417–429. https://doi.org/10.1109/TAFFC.2017.2712143Google ScholarDigital Library
- Huiping Jiang, Zequn Wang, Rui Jiao, and Shan Jiang. 2020. Picture-induced EEG Signal Classification Based on CVC Emotion Recognition System. Comput. Mater. Contin. 65, 2 (2020), 1453–1465. https://doi.org/10.32604/cmc.2020.011793Google Scholar
- Merve Dogruyol Basar, Adil Deniz Duru, and Aydin Akan. 2020. Emotional state detection based on common spatial patterns of EEG. Signal Image Video Process. 14, 3 (April 2020), 473–481. https://doi.org/10.1007/s11760-019-01580-8Google ScholarCross Ref
- Aurelien Appriou, Andrzej Cichocki, and Fabien Lotte. 2020. Modern Machine-Learning Algorithms: For Classifying Cognitive and Affective States From Electroencephalography Signals. IEEE Syst. Man Cybern. Mag. 6, 3 (July 2020), 29–38. https://doi.org/10.1109/MSMC.2020.2968638Google ScholarCross Ref
- Mengmeng Yan, Zhao Lv, Wenhui Sun, and Ning Bi. 2020. An improved common spatial pattern combined with channel-selection strategy for electroencephalography-based emotion recognition. Med. Eng. Phys. 83, (September 2020), 130–141. https://doi.org/10.1016/j.medengphy.2020.05.006Google ScholarCross Ref
- Cédric Simar, Ana-Maria Cebolla, Gaëlle Chartier, Mathieu Petieau, Gianluca Bontempi, Alain Berthoz, and Guy Cheron. 2020. Hyperscanning EEG and Classification Based on Riemannian Geometry for Festive and Violent Mental State Discrimination. Front. Neurosci. 14, (December 2020), 588357. https://doi.org/10.3389/fnins.2020.588357Google ScholarCross Ref
- Yixin Wang, Shuang Qiu, Xuelin Ma, and Huiguang He. 2021. A prototype-based SPD matrix network for domain adaptation EEG emotion recognition. Pattern Recognit. 110, (February 2021), 107626. https://doi.org/10.1016/j.patcog.2020.107626Google Scholar
- Ren Li, Jared S. Johansen, Hamad Ahmed, Thomas V. Ilyevsky, Ronnie B. Wilbur, Hari M. Bharadwaj, and Jeffrey Mark Siskind. 2021. The Perils and Pitfalls of Block Design for EEG Classification Experiments. IEEE Trans. Pattern Anal. Mach. Intell. 43, 1 (January 2021), 316–333. https://doi.org/10.1109/TPAMI.2020.2973153Google ScholarDigital Library
- Lang, P. J., Bradley, M. M., & Cuthbert, B. N. (2008). International Affective Picture System (IAPS): Affective ratings of pictures and instruction manual (Rep. No. A-8). Gainesville, FL: University of Florida.Google Scholar
- A. Barachant, S. Bonnet, M. Congedo, and C. Jutten. 2012. Multiclass Brain–Computer Interface Classification by Riemannian Geometry. IEEE Trans. Biomed. Eng. 59, 4 (April 2012), 920–928. https://doi.org/10.1109/TBME.2011.2172210Google ScholarCross Ref
- Alexandre Barachant, Stéphane Bonnet, Marco Congedo, and Christian Jutten. 2013. Classification of covariance matrices using a Riemannian-based kernel for BCI applications. Neurocomputing 112, (July 2013), 172–178. https://doi.org/10.1016/j.neucom.2012.12.039Google ScholarDigital Library
- Xinjiang Lu and Yunxu Bai. 2020. Robust Least-Squares Support Vector Machine Using Probabilistic Inference. IEEE Trans. Cybern. (October 2020), 1–9. https://doi.org/10.1109/TCYB.2020.3026680Google ScholarCross Ref
- Rasmus Bro and Age K. Smilde. 2014. Principal component analysis. Anal Methods 6, 9 (2014), 2812–2831. https://doi.org/10.1039/C3AY41907JGoogle ScholarCross Ref
- A. D. Bigirimana, N. Siddique, and D. Coyle. 2020. Emotion-Inducing Imagery Versus Motor Imagery for a Brain-Computer Interface. IEEE Trans. Neural Syst. Rehabil. Eng. 28, 4 (April 2020), 850–859. https://doi.org/10.1109/TNSRE.2020.2978951Google ScholarCross Ref
- Daniela Iacoviello, Andrea Petracca, Matteo Spezialetti, and Giuseppe Placidi. 2015. A real-time classification algorithm for EEG-based BCI driven by self-induced emotions. Comput. Methods Programs Biomed. 122, 3 (December 2015), 293–303. https://doi.org/10.1016/j.cmpb.2015.08.011Google ScholarDigital Library
Recommendations
Brain-computer interface research at Katholieke Universiteit Leuven
ISABEL '11: Proceedings of the 4th International Symposium on Applied Sciences in Biomedical and Communication TechnologiesWe present an overview of our Brain-computer interface (BCI) research, invasive as well as non-invasive, during the past four years. The invasive BCIs are based on local field-and action potentials recorded with microelectrode arrays implanted in the ...
Quantification of event related brain patterns for the motor imagery tasks using inter-trial variance technique
AbstractQuantification of event-related (de) synchronization (ERD/ERS) patterns is a challenging task in the field of motor imagery (MI)-based brain–computer interface (BCI). Accurately determining the optimal time and frequency band for localizing the ...
A hybrid brain-computer interface for smart home control
HCII'11: Proceedings of the 14th international conference on Human-computer interaction: interaction techniques and environments - Volume Part IIBrain-computer interfaces (BCI) provide a new communication channel between the human brain and a computer without using any muscle activities. Applications of BCI systems comprise communication, restoration of movements or environmental control. Within ...
Comments