Abstract
In this paper, we discuss the use of facial expression in tandem with feature extraction and neural network to recognize distinct facial emotions like happy, sad, angry, surprised, and neutral. We look at the limitations of existing emotion identification algorithms using convolutional neural networks on CK+ and modified FER13 datasets and achieved decent prediction. Expression recognition utilizing a brain activity system is more difficult and time-consuming than facial emotion recognition using CNN. Alongside with recognizing the user's emotion, the paper also aims to analyze and generate an emotion log file comprising all of the emotions recorded with a time stamp. We have achieved 97.87% accuracy on FER13 dataset, and for CK+ dataset in testing, we got 94.93% of accuracy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Lasri I et al (2019) Facial emotion recognition of students using convolutional neural network. In: 2019 3rd international conference on intelligent computing in data sciences (ICDS), https://doi.org/10.1109/ICDS47004.2019.8942386
Zhao X et al (2016) Peak-piloted deep network for facial expression recognition. Lecture notes in computer science (including Subseries lecture notes in artificial intelligence and lecture notes in bioinformatics). vol 9906 LNCS. pp 425–442, https://doi.org/10.1007/978-3-319-46475-6_27
Zahara L et al (2020) The facial emotion recognition (FER-2013) dataset for prediction system of micro-expressions face using the convolutional neural network (CNN) algorithm based Raspberry Pi. In: 2020 5th international conference on informatics and computing (ICIC), 2020. https://doi.org/10.1109/ICIC50835.2020.9288560
Zafar B et al (2018) A novel discriminating and relative global spatial image representation with applications in CBIR. Appl Sci 8(11):1–23. https://doi.org/10.3390/app8112242
Ali N et al (2018). A hybrid geometric spatial image representation for scene classification. https://doi.org/10.1371/journal.pone.0203339
Ali N et al (2016). A novel image retrieval based on visual words integration of sift and surf. https://doi.org/10.1371/journal.pone.0157428
Ekman P, Friesen WV (1971) Constants across cultures in the face and emotion. J Pers Soc Psychol 17(2):124–129. https://doi.org/10.1037/H0030377
Sajid M et al (2019). The impact of asymmetric left and asymmetric right face images on accurate age estimation. https://doi.org/10.1155/2019/8041413
Sajid M et al (2018). Data augmentation-assisted makeup-invariant face recognition. https://doi.org/10.1155/2018/2850632
Ratyal N, Taj I, Bajwa U, Sajid M (2018) Pose and expression invariant alignment based multi-view 3D face recognition. KSII Trans Internet Inf Syst 12(10):4903–4929. https://doi.org/10.3837/tiis.2018.10.016
Oktavia NY, Wibawa AD, Pane ES, Purnomo MH (2019) Human Emotion Classification Based on EEG Signals Using Naïve Bayes Method. In: Proceedings 2019 international seminar on application for technology of information and communication Industry 4.0 retrospect, prospect and challenges (iSemantic), 2019. pp 319–324 https://doi.org/10.1109/ISEMANTIC.2019.8884224
Ayvaz U, Gürüler H, Devrim MO (2017) Use of facial emotion recognition in e-learning systems. Information Technologies and Learning Tools 60(4):95. https://doi.org/10.33407/itlt.v60i4.1743
Tang C, Xu P, Luo Z, Zhao G, Zou T (2015) Automatic facial expression analysis of students in teaching environments lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics), vol 9428, pp 439–447 https://doi.org/10.1007/978-3-319-25417-3_52
Zadeh MMT, Imani M, Majidi B (2019) Fast facial emotion recognition using convolutional neural networks and gabor filters. In: 2019 IEEE 5th conference on knowledge based engineering and innovation KBEI 2019, pp 577–581, https://doi.org/10.1109/KBEI.2019.8734943
Ghaffar F (2020) Facial emotions recognition using convolutional neural net, pp 7–12 https://arxiv.org/ftp/arxiv/papers/2001/2001.01456.pdf
Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition, vol 1. https://doi.org/10.1109/CVPR.2001.990517
Tabora V (2021) Face detection using opencv with haar cascade classifiers | by Vincent Tabora becoming human: artificial intelligence magazine. https://becominghuman.ai/face-detection-using-opencv-with-haar-cascade-classifiers-941dbb25177. Accessed 1 Sep 2021
Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L-C (2018) MobileNetV2: inverted residuals and linear bottlenecks. in 2018 IEEE conference on computer vision and pattern recognition, pp 4510–4520. https://doi.org/10.1109/CVPR.2018.00474
Sambare M (2021) FER-2013 | Kaggle https://www.kaggle.com/msambare/fer2013?select=test. Accessed 1 Sept 2021
Lucey P, Cohn JF, Kanade T, Saragih J, Ambadar Z, Matthews I (2010) The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion- specified expression. In 2010 IEEE computer society conference on computer vision and pattern recognition-workshops, pp 94–101. https://doi.org/10.1109/CVPRW.2010.5543262
Shawon A (2021) CKPLUS | Kaggle https://www.kaggle.com/shawon10/ckplus
Mehendale N (2020) Facial emotion recognition using convolutional neural networks (FERC). SN Appl Sci 2(3):1–8. https://doi.org/10.1007/s42452-020-2234-1
Jung H, Lee S, Yim J, Park S, Kim J (2015) Joint fine-tuning in deep neural networks for facial expression recognition. In: Proceedings of the IEEE international conference on computer vision, vol 2015 Inter, pp 2983–2991, https://doi.org/10.1109/ICCV.2015.341
Zhang K, Huang Y, Du Y, Wang L (2017) Facial expression recognition based on deep evolutional spatial-temporal networks. IEEE Trans Image Process 26(9):4193–4203. https://doi.org/10.1109/TIP.2017.2689999
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Harsha, B.K., Shruthi, M.L.J., Indumathi, G. (2022). Convolutional Neural Network-Based Contemporaneous Human Facial Expression Identification. In: Mahajan, V., Chowdhury, A., Padhy, N.P., Lezama, F. (eds) Sustainable Technology and Advanced Computing in Electrical Engineering . Lecture Notes in Electrical Engineering, vol 939. Springer, Singapore. https://doi.org/10.1007/978-981-19-4364-5_28
Download citation
DOI: https://doi.org/10.1007/978-981-19-4364-5_28
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-4363-8
Online ISBN: 978-981-19-4364-5
eBook Packages: Computer ScienceComputer Science (R0)