Abstract
Automatic emotion recognition is important in human-computer interaction (HCI). Although extensive electroencephalography (EEG)-based emotion recognition research has been conducted in recent years, effectively identifying the correlation between EEG signals and emotions remains a challenge. In this study, a new method that combines a novel pre-processing technique with a 3D convolutional neural network (3DCNN)-based classifier is proposed. After the data undergo preprocessing, 3DCNN is used to extract temporal and spatial features from the 2D-map EEG feature sequences. The features are then fed to a fully connected network to obtain binary or multi-category results. Extensive experiments are conducted on the DEAP dataset, and results show that the proposed method surpasses other state-of-the-art methods. The process of selecting the hyper-parameters of 3DCNN is also investigated by comparing three models. Source codes used in this study are available on https://github.com/heibaipei/V-3DCNN.
This work was supported by the Key Project of National Key R &D Project (No. 2017YFC1703303); Natural Science Foundation of Fujian Province of China (No. 2019J01846, No. 2018J01555, No. 2017J01773); External Cooperation Project of Fujian Province, China (No. 2019I0001); Science and Technology Guiding Project of Fujian Province, China (2019Y0046).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alarcao, F.: Emotions recognition using EEG signals: a survey. IEEE Trans. Affect. Comput. 10(3), 374–393 (2017)
Ang, K., Yang Chin, Z., Zhang, H., Guan, C.: Filter bank common spatial pattern (FBCSP) in brain-computer interface. In: Proceedings of the International Joint Conference on Neural Networks, pp. 2390–2397 (2008). https://doi.org/10.1109/IJCNN.2008.4634130
Chanel, G., Rebetez, C., Betrancourt, M., Pun, T.: Emotion assessment from physiological signals for adaptation of game difficulty. Syst. Man Cybern. 41(6), 1052–1063 (2011)
Ieracitano, C., Mammone, N., Bramanti, A., Hussain, A., Morabito, F.C.: A convolutional neural network approach for classification of dementia stages based on 2D-spectral representation of EEG recordings. Neurocomputing 323, 96–107 (2019)
Jie, X., Cao, R., Li, L.: Emotion recognition based on the sample entropy of EEG. Bio-Med. Mater. Eng. 24(1), 1185 (2014)
Kim, S., Kang, H.: An analysis of smartphone overuse recognition in terms of emotions using brainwaves and deep learning. Neurocomputing 275, 1393–1406 (2018)
Koelstra, S., et al.: DEAP: a database for emotion analysis; using physiological signals. IEEE Trans. Affect. Comput. 3(1), 18–31 (2012)
Lee, Y.Y., Hsieh, S.: Classifying different emotional states by means of EEG-based functional connectivity patterns. PLoS ONE 9(4), e95415 (2014)
Lin, M., Chen, Q., Yan, S.: Network in network. In: International Conference on Learning Representations (2014)
Liu, W., Zheng, W.-L., Lu, B.-L.: Emotion recognition using multimodal deep learning. In: Hirose, A., Ozawa, S., Doya, K., Ikeda, K., Lee, M., Liu, D. (eds.) ICONIP 2016. LNCS, vol. 9948, pp. 521–529. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46672-9_58
Luo, T.J., Zhou, C.L., Chao, F.: Exploring spatial-frequency-sequential relationships for motor imagery classification with recurrent neural network. BMC Bioinform. 19(1), 344 (2018). https://doi.org/10.1186/s12859-018-2365-1
Mao, X., Li, Z.: Implementing emotion-based user-aware e-learning. In: Human Factors in Computing Systems, pp. 3787–3792 (2009)
Schirrmeister, R.T., et al.: Deep learning with convolutional neural networks for EEG decoding and visualization. Hum. Brain Mapp. 38(11), 5391–5420 (2017). https://doi.org/10.1002/hbm.23730, https://onlinelibrary.wiley.com/doi/abs/10.1002/hbm.23730
Szegedy, C., et al.: Going deeper with convolutions. In: Computer Vision and Pattern Recognition, pp. 1–9 (2015)
Tang, H., Liu, W., Zheng, W.L., Lu, B.L.: Multimodal emotion recognition using deep neural networks. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, E.S.M. (eds.) Neural Information Processing, pp. 811–819. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-70093-9_86
Wagh, K.P., Vasanth, K.: Electroencephalograph (EEG) based emotion recognition system: a review. In: Saini, H.S., Singh, R.K., Patel, V.M., Santhi, K., Ranganayakulu, S.V. (eds.) Innovations in Electronics and Communication Engineering. LNNS, vol. 33, pp. 37–59. Springer, Singapore (2019). https://doi.org/10.1007/978-981-10-8204-7_5
Li, X., Song, D., Zhang, P., Yu, G., Hou, Y., Hu, B.: Emotion recognition from multi-channel EEG data through convolutional recurrent neural network. In: 2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), vol. 1, pp. 352–359, December 2016. https://doi.org/10.1109/BIBM.2016.7822545
Yang, Y., Wu, Q., Qiu, M., Wang, Y., Chen, X.: Emotion recognition from multi-channel EEG through parallel convolutional recurrent neural network. In: 2018 International Joint Conference on Neural Networks (IJCNN), vol. 1, pp. 1–7, July 2018. https://doi.org/10.1109/IJCNN.2018.8489331
Zhang, D., Yao, L., Zhang, X., Wang, S., Chen, W., Boots, R.: EEG-based intention recognition from spatio-temporal representations via cascade and parallel convolutional recurrent neural networks, p. 1. Arxiv (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, Y., Wu, Q., Ruan, Q. (2022). EEG Emotion Classification Using 2D-3DCNN. In: Memmi, G., Yang, B., Kong, L., Zhang, T., Qiu, M. (eds) Knowledge Science, Engineering and Management. KSEM 2022. Lecture Notes in Computer Science(), vol 13369. Springer, Cham. https://doi.org/10.1007/978-3-031-10986-7_52
Download citation
DOI: https://doi.org/10.1007/978-3-031-10986-7_52
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-10985-0
Online ISBN: 978-3-031-10986-7
eBook Packages: Computer ScienceComputer Science (R0)