Skip to main content
SpringerLink
Log in

Intelligent optimal feature selection-based hybrid variational autoencoder and block recurrent transformer network for accurate emotion recognition model using EEG signals

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

In the context of emotion recognition, Artificial Intelligence technology has demonstrated several functions in people's lives. Computing research is now focused on Electroencephalogram (EEG) signals to identify emotional states. The connection and interaction between multichannel EEG signals give important information about emotional states. However, most existing emotion identification techniques perform poorly in practical applications by preventing their advancement. The main objective of this paper is to design an efficient model for emotion recognition based on deep learning technology by EEG signals. The proposed model for emotion recognition collects the EEG signals from the standard benchmark datasets. Then, the signal decomposition is performed using the Tunable Q-factor Wavelet Transform with the collected EEG signals. The decomposed signals are taken for the optimal feature selection phase, where the significant features of the emotion are selected through the hybrid optimization algorithm named Aquila Fireworks Optimization Algorithm (AFOA). Finally, the EEG emotion classification is performed using Hybrid Variational Autoencoder and Block Recurrent Transformer Network. The tuning of the parameter is made through the same AFOA to improve the efficiency of classification. The experimental analysis is conducted to analyze the efficiency of the recommended emotion recognition model with the comparison over the traditional techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability

The data underlying this article are available in EEG Brainwave Dataset: Feeling Emotions, at https://www.kaggle.com/datasets/birdy654/eeg-brainwave-dataset-feeling-emotions: Access Date: 2023–01-19 and Emotion-Recognition-from-DEAP, at https://github.com/hi-akshat/Emotion-Recogniton-from-EEG-Signals: Access Date: 2023–07-19.

References

  1. Shojaeilangari, S., Yau, W.-Y., Nandakumar, K., Li, J., Teoh, E.K.: Robust representation and recognition of facial emotions using extreme sparse learning. IEEE Trans. Image Process. 24(7), 2140–2152 (2015)

    Article  ADS  MathSciNet  PubMed  Google Scholar 

  2. Wu, C.-H., Liang, W.-B.: Emotion recognition of affective speech based on multiple classifiers using acoustic-prosodic information and semantic labels. IEEE Trans. Affect. Comput. 2(1), 10–21 (2011)

    Article  Google Scholar 

  3. Yoon, W.-J., Park, K.-S.: Building robust emotion recognition system on heterogeneous speech databases. IEEE Trans. Consum. Electron. 57(2), 747–750 (2011)

    Article  Google Scholar 

  4. Zhang, C., Xue, L.: Autoencoder With emotion embedding for speech emotion recognition. IEEE Access 9, 51231–51241 (2021)

    Article  Google Scholar 

  5. Yan, J., Zheng, W., Xu, Q., Lu, G., Li, H., Wang, B.: Sparse Kernel reduced-rank regression for bimodal emotion recognition from facial expression and speech. IEEE Trans. Multimedia 18(7), 1319–1329 (2016)

    Article  Google Scholar 

  6. Huang, Y., Xiao, J., Tian, K., Wu, A., Zhang, G.: Research on robustness of emotion recognition under environmental noise conditions. IEEE Access 7, 142009–142021 (2019)

    Article  Google Scholar 

  7. Zhang, H., Xu, M.: Weakly supervised emotion intensity prediction for recognition of emotions in images. IEEE Trans. Multimedia 23, 2033–2044 (2021)

    Article  ADS  Google Scholar 

  8. Zhang, T., Zheng, W., Cui, Z., Zong, Y., Li, Y.: Spatial–temporal recurrent neural network for emotion recognition. IEEE Trans. Cybernet. 49(3), 839–847 (2019)

    Article  Google Scholar 

  9. Han, J., Zhang, Z., Ren, Z., Schuller, B.: EmoBed: strengthening monomodal emotion recognition via training with crossmodal emotion embeddings. IEEE Trans. Affect. Comput. 12(3), 553–564 (2021)

    Article  Google Scholar 

  10. Cimtay, Y., Ekmekcioglu, E., Caglar-Ozhan, S.: Cross-subject multimodal emotion recognition based on hybrid fusion. IEEE Access 8, 168865–168878 (2020)

    Article  Google Scholar 

  11. Yoon, Y.C.: Can we exploit all datasets? Multimodal emotion recognition using cross-modal translation. IEEE Access 10, 64516–64524 (2022)

    Article  Google Scholar 

  12. Li, P., et al.: EEG based emotion recognition by combining functional connectivity network and local activations. IEEE Trans. Biomed. Eng. 66(10), 2869–2881 (2019)

    Article  ADS  PubMed  Google Scholar 

  13. Guo, J., et al.: Dominant and complementary emotion recognition from still images of faces. IEEE Access 6, 26391–26403 (2018)

    Article  Google Scholar 

  14. Li, Y., Zheng, W., Zong, Y., Cui, Z., Zhang, T., Zhou, X.: A bi-hemisphere domain adversarial neural network model for EEG emotion recognition. IEEE Trans. Affect. Comput. 12(2), 494–504 (2021)

    Article  CAS  Google Scholar 

  15. Ayari, N., Abdelkawy, H., Chibani, A., Amirat, Y.: Hybrid model-based emotion contextual recognition for cognitive assistance services. IEEE Trans. Cybernet. 52(5), 3567–3576 (2022)

    Article  CAS  Google Scholar 

  16. Li, J., Qiu, S., Shen, Y.-Y., Liu, C.-L., He, H.: multisource transfer learning for cross-subject EEG emotion recognition. IEEE Trans. Cybernet. 50(7), 3281–3293 (2020)

    Google Scholar 

  17. Chen, L., Li, M., Su, W., Wu, M., Hirota, K., Pedrycz, W.: Adaptive feature selection-based AdaBoost-KNN with direct optimization for dynamic emotion recognition in human-robot interaction. IEEE Trans. Emerg. Top. Comput. Intell. 5(2), 205–213 (2021)

    Article  Google Scholar 

  18. Chao, H., Dong, L.: Emotion recognition using three-dimensional feature and convolutional neural network from multichannel EEG signals. IEEE Sens. J. 21(2), 2024–2034 (2021)

    Article  ADS  Google Scholar 

  19. Zhang, H.: Expression-EEG based collaborative multimodal emotion recognition using deep autoencoder. IEEE Access 8, 164130–164143 (2020)

    Article  Google Scholar 

  20. Majid Mehmood, R., Du, R., Lee, H.J.: Optimal feature selection and deep learning ensembles method for emotion recognition from human brain EEG sensors. IEEE Access 5, 14797–14806 (2017)

    Article  Google Scholar 

  21. Song, T., Zheng, W., Song, P., Cui, Z.: EEG emotion recognition using dynamical graph convolutional neural networks. IEEE Trans. Affect. Comput. 11(3), 532–541 (2020)

    Article  Google Scholar 

  22. Maheshwari, D., Ghosh, S.K., Tripathy, R.K., Sharma, M., RajendraAcharya, U.: Automated accurate emotion recognition system using rhythm-specific deep convolutional neural network technique with multi-channel EEG signals. Comput. Biol. Med. 134, 104428 (2021)

    Article  PubMed  Google Scholar 

  23. Zhong, M., Yang, Q., Liu, Y., Zhen, B., Zhao, F., Xie, B.: EEG emotion recognition based on TQWT-features and hybrid convolutional recurrent neural network. Biomed. Signal Process. Control 79(Part 2), 104211 (2023)

    Article  Google Scholar 

  24. Cheng, W.X., Gao, R., Suganthan, P.N., Yuen, K.F.: EEG-based emotion recognition using random Convolutional Neural Networks. Eng. Appl. Arti. Intell. 116, 105349 (2022)

    Article  Google Scholar 

  25. Bagherzadeh, S., Maghooli, K., Shalbaf, A., Maghsoudi, A.: Emotion recognition using effective connectivity and pre-trained convolutional neural networks in EEG signals. Cognitive Neurodyn. 16, 1087–1106 (2022)

    Article  Google Scholar 

  26. Abualigah, L., Yousri, D., Elaziz, M.A., Ewees, A.A., Al-qaness, M.A.A., Gandomi, A.H.: Aquila optimizer: a novel meta-heuristic optimization algorithm. Comput. Ind. Eng. 157, 107250 (2021)

    Article  Google Scholar 

  27. Tan, Y., Zhu, Y.: Fireworks algorithm for optimization. Adv. Swarm Intell. First Int. Conf. 1, 355–364 (2010)

    Google Scholar 

  28. Hutchins, D., Schlag, I., Wu, Y., Dyer, E., Neyshabur, B.: Block-recurrent transformers. In: 36th Conference on Neural Information Processing Systems (NeurIPS 2022) (2022)

  29. Zhang, Y., Chi, A., Mirjalili, S.: Enhanced Jaya algorithm: a simple but efficient optimization method for constrained engineering design problems. Knowl.Based Syst. 233, 107555 (2021)

    Article  Google Scholar 

  30. Abdollahzadeh, B., Gharehchopogh, F.S., Mirjalili, S.: Artificial gorilla troops optimizer: a new nature-inspired metaheuristic algorithm for global optimization problems. Int. J. Intell. Syst. 36, 434 (2021)

    Article  Google Scholar 

  31. Liu, D., Liu, G.: A Transformer-based variational autoencoder for sentence generation. In: Conference: 2019 International Joint Conference on Neural Networks (IJCNN), July 2019 (2019)

  32. Foggia, P., Greco, A., Saggese, A., Vento, M.: Multi-task learning on the edge for effective gender, age, ethnicity and emotion recognition. Eng. Appl. Artif. Intell. 118, 105651 (2023)

    Article  Google Scholar 

  33. Mukhiddinov, M., Djuraev, O., Akhmedov, F., Mukhamadiyev, A., Cho, J.: Masked face emotion recognition based on facial landmarks and deep learning approaches for visually impaired people. Sensors 23(3), 1080 (2023)

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  34. Sams, A.S., Zahra, A.: Multimodal music emotion recognition in Indonesian songs based on CNN-LSTM, XLNet transformers. Bull. Electrical Eng. Inform. 12(1), 355–364 (2023)

    Article  Google Scholar 

  35. Bilal Er, M., Çiğ, H., Aydilek, İB.: A new approach to recognition of human emotions using brain signals and music stimuli. Appl. Acoust. 175, 107840 (2021)

    Article  Google Scholar 

  36. Krishna, A.H., Sri, A.B., Priyanka, K.Y.V.S., Taran, S., Bajaj, V.: Emotion classification using EEG signals based on tunable-Q wavelet transform. IET Sci. Measure. Technol. 13(3), 375–380 (2019)

    Article  Google Scholar 

  37. Issa, S., Peng, Q., You, X.: Emotion classification using EEG brain signals and the broad learning system. IEEE Trans. Syst. Man Cybernet. Syst. 51(12), 7382–7391 (2021)

    Article  Google Scholar 

  38. Issa, H., Peng, Q., Issa, S., You, X., Peng, R., Wang, J.: Person-independent emotion and gender prediction (EGP) system using EEG signals. Int. Arab J. Inf. Technol. 19, 4 (2022)

    Google Scholar 

Download references

Funding

This research did not receive any specific funding.

Author information

Authors and Affiliations

  1. Electrical Electronics and Communication Engineering (EECE), GITAM University, Patancheru Mandal, Hyderabad, Telangana, 502329, India

    C. H. Narsimha Reddy & K. Manjunathachari

  2. Information Science and Engineering, Atria Institute of Technology, Hebbal, Bengaluru, 560024, India

    Shanthi Mahesh

Authors
  1. C. H. Narsimha Reddy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shanthi Mahesh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. K. Manjunathachari
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors have made substantial contributions to conception and design, revising the manuscript, and the final approval of the version to be published. Also, all authors agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Corresponding author

Correspondence to C. H. Narsimha Reddy.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Ethical approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Reddy, C.H.N., Mahesh, S. & Manjunathachari, K. Intelligent optimal feature selection-based hybrid variational autoencoder and block recurrent transformer network for accurate emotion recognition model using EEG signals. SIViP 18, 1027–1039 (2024). https://doi.org/10.1007/s11760-023-02702-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-023-02702-z

Keywords

Search

Navigation