Skip to main content
SpringerLink
Log in

Analysis of brain areas in emotion recognition from EEG signals with deep learning methods

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Emotion recognition technology is widely employed in areas such as brain-computer interfaces, healthcare, security, e-commerce, education, and entertainment. This technology serves to enhance and manage the interaction between humans and machines. The challenges in this field arise from the fact that emotions are abstract, fluctuate based on internal and external factors, and differ greatly among individuals. Recently, research leveraging electroencephalography signals for more dependable and accurate emotion analysis has been gaining traction. This article presents an emotion analysis conducted using EEG signals and identifies the brain areas that were found to be most impactful. Three different data sets were used in the study and the performances of these data sets in emotion recognition were determined according to the areas of the brain. The research was composed of four stages. In the first stage, EEG data were obtained from DEAP, GAMEEMO and SEED-V data sets. In the second stage, preprocessing was carried out and EEG channels were decomposed according to segments. Later, OSW method was applied to increase the data size in datasets and DWT and statistical methods were employed for feature extraction. In the third stage, the deep learning model was defined, and the CNN architecture was applied. In the last stage, classification was performed, and accuracy values were calculated according to the brain areas of each data set. At the end of the study, 72.42%, 95.30% and 82.06% accuracy values were obtained for DEAP, GAMEEMO and SEED-V data sets, respectively. In the results of the analysis made according to the brain areas, it was determined that the results of the channels in the left part of the brain and the channels in the frontal region of the brain were more successful and effective.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Data availability

The datasets analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Ding Y, Robinson N, Zeng Q, Chen D, Phyo wai AA, Lee TS, Guan C (2020) TSception: A deep learning framework for emotion detection using EEG. International Joint Conference on Neural Networks (IJCNN), 19-24 July, Glasgow, UK. pp 1–7. https://doi.org/10.1109/IJCNN48605.2020.9206750

  2. Unguren E (2016) The Effect of Neuroanatomical and Neurochemical Structure of the Brain on Personality and Behavior. Int J Alanya Fac Manag 7(1):193–219

    Google Scholar 

  3. Tatum WO, Husain AM, Benbadis SR, Kaplan PW (2008) Handbook of EEG INTERPRETATION. Demos Medical Publishing, United States of America

  4. Sanei S, Chambers JA (2007) EEG Signal Processing. John Wiley & Sons Ltd., Cardiff University, UK

  5. Lajnef T, Chaibi S, Ruby P, Aguera PE, Eichenlaub JB, Samet M, Kachouri A, Jerbi K (2015) Learning machines and sleeping brains: Automatic sleep stage classification using decision-tree multi-class support vector machines. J Neurosci Methods 250:94–105

    Article  PubMed  Google Scholar 

  6. Junxiu L, Guopei W, Yuling L, Senhui Q, Su Y, Wei L, Yifei B (2020) EEG-based emotion classification using a deep neural network and sparse autoencoder. Front Syst Neurosci 14:43. https://doi.org/10.3389/fnsys.2020.00043

  7. Seal A, Reddy PPN, Chaithanya P, Meghana A, Jahnavi K, Krejcar O, Hudak R (2020) An EEG Database and Its Initial Benchmark Emotion Classification Performance. Comput Mathe Methods Med 2020, Article ID 8303465, 14

  8. Varol O (2010) Raw EEG data classification and applications using SVM, BSc Thesis, Istanbul Technical University Electrical-Electronics Engineering Faculty

  9. Huang Z, Wang M (2021) A review of electroencephalogram signal processing methods for brain-controlled robots. Cogn Robot 1:111–124, ISSN 2667–2413

  10. Klem GH, Luders HO, Jasper HH, Elger C (1999) The ten-twenty electrode system of the International Federation. The International Federation of Clinical Neurophysiology. Electroencephalogr Clin Neurophysiol Suppl 52:3–6

    CAS  PubMed  Google Scholar 

  11. Seeck M, Koessler L, Bast T, Leijten F, Michel C, Baumgartner C, He B, Beniczky S (2017) The standardized EEG electrode array of the IFCN. Clin Neurophysiol 128(10):2070–2077

    Article  PubMed  Google Scholar 

  12. Tan A (2021) Emotion-Mind Relationship as a Source of Information. Atlas J Soc Sci 1(6):21–45

    Google Scholar 

  13. Ekman P (1992) An argument for basic emotions. Cogn Emot 6:169–200

    Article  Google Scholar 

  14. Russell JA (2003) Core affect and the psychological construction of emotion. Psychol Rev 110:145–172

    Article  PubMed  Google Scholar 

  15. Abgeena A, Garg S (2022) A novel convolution bi-directional gated recurrent unit neural network for emotion recognition in multichannel electroencephalogram signals. Technol Health Care 31(4):1215–1234. https://doi.org/10.3233/THC-220458

  16. Tigga NP, Garg S (2022) Efficacy of novel attention-based gated recurrent units transformer for depression detection using electroencephalogram signals. Health Inf Sci Syst 11(1):1. https://doi.org/10.1007/s13755-022-00205-8

  17. Pappalettera C, Miraglia F, Cotelli M, Rossini PM, Vecchio F (2022) Analysis of complexity in the EEG activity of Parkinson’s disease patients by means of approximate entropy. GeroScience 44:1599–1607

    Article  PubMed  PubMed Central  Google Scholar 

  18. Ahmad I, Wang X, Zhu M, Wang C, Pi Y et al (2022) EEG-based epileptic seizure detection via machine/deep learning approaches: A systematic review. Comput Intell Neurosci. https://doi.org/10.1155/2022/6486570

  19. Alhagry S, Fahmy AA, El-Khoribi RA (2017) Emotion Recognition based on EEG using LSTM Recurrent Neural Network. Int J Adv Comput Sci Applic 8(10)

  20. Asghar MA, Khan MJ, Khan F, Amin Y, Rizwan M, Rahman M, Badnava S, Mirjavadi SS (2019) EEG-based multi-modal emotion recognition using bag of deep features: An optimal feature selection approach. Sensors 19(23). https://doi.org/10.3390/s19235218

  21. Luo Y, Xie J, Qin Y, Wu G, Lui J, Jiang F, Cao Y, Ding X (2020) EEG-Based Emotion Classification Using Spiking Neural Networks. IEEE Access 8:46007–46016

    Article  Google Scholar 

  22. Liu J, Wu G, Luo Y, Qiu S, Yang S, Wei L, Yifei B (2020) EEG-Based Emotion Classification Using a Deep Neural Network and Sparse Autoencoder. Front SystNeurosci 14, ISSN 1662-5137

  23. Alakus TB, Turkoglu I (2020) Emotion recognition with deep learning using GAMEEMO data set. Electron Lett 56:1364–1367

    Article  ADS  Google Scholar 

  24. Abd A, Baykara M (2021) Feature extraction approach based on statistical methods and wavelet packet decomposition for emotion recognition using EEG Signals. International Conference on INnovations in Intelligent SysTems and Applications (INISTA), Kocaeli, Turkey, pp 1–7. https://doi.org/10.1109/INISTA52262.2021.9548406

  25. Akay M, Tuncer T (2021) Automatıc EEG emotion recognıtıon method based on multı-level wavelet transform and local bınary patterns. Int J  Innov Eng Appl 5(2):75–80. https://doi.org/10.46460/ijiea.904838

  26. Aslan M (2021) CNN based efficient approach for emotion recognition. J King Saud Univ - Comput Inf Sci 34(9):7335–7346. https://doi.org/10.1016/j.jksuci.2021.08.021

  27. Liu W, Qiu J-L, Zheng W-L, Lu B-L (2022) Comparing Recognition Performance and Robustness of Multimodal Deep Learning Models for Multimodal Emotion Recognition. IEEE Trans Cogn Dev Syst 14(2):715–729

    Article  Google Scholar 

  28. Koelstra S, Muehl C, Soleymani M, Lee J-S, Yazdani A, Ebrahimi T, Pun T, Nijholt A, Patras I (2012) DEAP: A Database for Emotion Analysis; Using Physiological Signals. IEEE Trans Affect Comput 3(1):18-31

  29. Alakus TB, Gonen M, Turkoglu I (2020) Database for an emotion recognition system based on EEG signals and various computer games – GAMEEMO. Biomed Signal Process Control 60, ISSN 1746-8094

  30. Garg S, Patro RK, Behera S, Tigga NP, Ranjita P (2021) An overlapping sliding window and combined features based emotion recognition system for EEG signals. Appl Comput Inf. https://doi.org/10.1108/ACI-05-2021-0130

    Article  Google Scholar 

  31. Edla DR, Ansari MF, Chaudhary N, Dodia S (2018) Classification of Facial Expressions from EEG signals using Wavelet Packet Transform and SVM for Wheelchair Control Operations. Procedia Comput Sci 132:1467–1476, ISSN 1877–0509

  32. Cheong LC, Sudirman R, Hussin SS (2015) Feature extraction Of EGG signal using wavelet transform for autism classification. ARPN J Eng Appl Sci 10(19):8533–8540

  33. Kumar N, Alam K, Siddiqi AH (2017) Wavelet transform for classification of EEG signal using SVM and ANN. Biomed Pharmacol J 10(4):2061–2069. https://doi.org/10.13005/bpj/1328

  34. Bird JJ, Faria DR, Manso LJ, Ayrosa PPS, Ekart A (2021) A study on CNN image classification of EEG signals represented in 2D and 3D. J Neural Eng 18(2). https://doi.org/10.1088/1741-2552/abda0c

  35. Roy SK, Krishna G, Dubey SR, Chaudhuri BB (2019) HybridSN: exploring 3-D–2-D CNN feature hierarchy for hyperspectral image classification. IEEE Geosci Remote Sens Lett 17:277–281

    Article  ADS  Google Scholar 

  36. Mattioli F, Porcaro C, Baldassarre G (2022) A 1D CNN for high accuracy classification and transfer learning in motor imagery EEG-based brain-computer interface. J Neural Eng 18(6). https://doi.org/10.1088/1741-2552/ac4430

  37. Jana GC, Sharma R, Agrawal A (2020) A 1D-CNN-spectrogram based approach for seizure detection from EEG signal. Procedia Comput Sci 167:403–412

    Article  Google Scholar 

  38. Tang W, Long G, Liu L, Zhou T, Jiang J, Blumenstein M (2019) Rethinking 1D-CNN for time series classification: a stronger baseline. ArXiv abs/2002.10061. https://doi.org/10.48550/arXiv.2002.10061

  39. Kiranyaz S, İnce T, Abdeljaber O, Avcı O, Gabbouj M (2019) 1-D convolutional neural networks for signal processing applications. Proceedings in International Conference on Acoustics, Speech, and Signal Processing, 12 – 17 May, Brighton, UK

  40. Farhad Z, Retno W (2021) Emotion classification using 1D-CNN and RNN based on DEAP Dataset. 10th International Conference on Natural Language Processing (NLP 2021) pp 363–378. https://doi.org/10.5121/csit.2021.112328

  41. Mattioli F, Porcaro C, Baldassarre G (2021) A 1D CNN for high accuracy classification and transfer learning in motor imagery EEG-based brain-computer interface. J Neural Eng 18(6). https://doi.org/10.1088/1741-2552/ac4430

  42. Prevedello LM, Halabi SS, Shih G, Wu CC, Kohli, MD, Chokshi FH, Erickson BJ, Kalpathy-Cramer J, Andriole KP, Flanders AE (2019) Challenges related to artificial intelligence research in medical imaging and the importance of image analysis competitions. Radiol Artif Intell 1(1):e180031. https://doi.org/10.1148/ryai.2019180031

  43. Kazeminia S, Baur C, Kuijper A, Ginneken B, Navab N, Albarqouni S, Mukhopadhyay A (2020) GANs for medical image analysis. Artif Intell Med 109:101938. https://doi.org/10.1016/j.artmed.2020.101938

  44. Roccetti M, Delnevo G, Casini L, Mirri S (2021) An alternative approach to dimension reduction for pareto distributed data: a case study. J Big Data 8(39). https://doi.org/10.1186/s40537-021-00428-8

Download references

Acknowledgements

This work was supported by a grant given to the performance project by the Scientific Research Projects Management Unit of Fırat University, Elazig, Turkey [grant number: TEKF.23.30]. The authors thank the Fırat University Scientific Research Projects Execution Unit, which provided financial support during their studies.

Author information

Authors and Affiliations

  1. Faculty of Engineering, Department of Software Engineering, Karadeniz Technical University, Trabzon, Türkiye

    Musa Aslan

  2. Faculty of Technology, Department of Software Engineering, Fırat University, Elazığ, Türkiye

    Muhammet Baykara

  3. Faculty of Engineering, Department of Software Engineering, Kırklareli University, Kırklareli, Türkiye

    Talha Burak Alakuş

Authors
  1. Musa Aslan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Muhammet Baykara
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Talha Burak Alakuş
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Musa Aslan – methodology, formal analysis, writing, validation. Muhammet Baykara – supervision, review, editing. Talha Burak Alakuş – writing, editing, original draft, review, validation.

Corresponding author

Correspondence to Talha Burak Alakuş.

Ethics declarations

Competing interests

The authors declare no conflict of interest.

Ethical and informed consent for data used

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix 1

figure a

Appendix 2

figure bfigure b

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aslan, M., Baykara, M. & Alakuş, T.B. Analysis of brain areas in emotion recognition from EEG signals with deep learning methods. Multimed Tools Appl 83, 32423–32452 (2024). https://doi.org/10.1007/s11042-023-16696-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-16696-w

Keywords

Search

Navigation