Skip to main content

Advertisement

SpringerLink
Log in

A robust feature extraction with optimized DBN-SMO for facial expression recognition

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Facial expression is the most common technique is used to convey the expressions of human beings. Due to different ethnicity and age, faces differ from one individual to another so that an automatic facial expression analysis and recognition is a difficult operation. To solve this difficulty, this paper proposes a robust feature extraction with optimized DBN-SMO for facial expression recognition (FER). Initially, the pre-processing stage is performed then texture descriptors of Local Phase Quantization (LPQ), Weber Local Descriptor (WLD) and Local Binary Pattern (LBP) are used to extract the features. Moreover, Discrete Cosine Transform (DCT) features are extracted to enhance the recognition rate and reduce the computational cost. After that, the Principal component analysis (PCA) is used for dimension reduction. Finally, a deep belief network (DBN) with Spider monkey optimization (SMO) algorithm is used to classify basic expressions for FER. Here, SMO algorithm is used to optimize bias factors and initial connection weights that control the efficiency of the DBN. The proposed work is performed in the MATLAB environment. Experiments performed on Karolinska Directed Emotional Faces (KDEF), Man-Machine Interaction (MMI), Cohn Kanade (CK+), Extended Denver Intensity of Spontaneous Facial Actions (DISFA+) and Oulu-Chinese Academy of Science Institute of Automation (Oulu-CASIA) datasets and it provides a classification accuracy of 97.93%, 95.42%, 97.58%, 95.76%, and 92.38% respectively, this is higher than other current procedures for seven-class emotion.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Ali K and Hughes CE (2019) Facial expression recognition using disentangled adversarial learning. arXiv preprint arXiv:1909.13135

  2. Alphonse SA, Dharma D (2018) Novel directional patterns and a generalized supervised dimension reduction system (GSDRS) for facial emotion recognition. Multimed Tools Appl 77(8):9455–9488

    Article  Google Scholar 

  3. Alrjebi MM, Liu W, Li L (2018) Face recognition against illuminations using two directional multi-level threshold-LBP and DCT. Multimed Tools Appl:1–21

  4. Chen J, Lv Y, Xu R, Xu C (2019) Automatic social signal analysis: facial expression recognition using difference convolution neural network. Journal of Parallel and Distributed Computing 131:97–102

    Article  Google Scholar 

  5. Dabhi MK, Pancholi BK (2016) Face detection system based on viola-jones algorithm. International Journal of Science and Research (IJSR) 5(4):62–64

    Article  Google Scholar 

  6. Eng KS, Ali H, Cheah AY, Chong YF (2019) Facial expression recognition in JAFFE and KDEF datasets using histogram of oriented gradients and support vector machine. In IOP conference series: materials science and engineering. IOP Publishing 705(1):012031

    Google Scholar 

  7. Fan X, Tjahjadi T (2017) A dynamic framework based on local Zernike moment and motion history image for facial expression recognition. Pattern Recogn 64:399–406

    Article  Google Scholar 

  8. Fang Y, Chang L (2015) Multi-instance feature learning based on sparse representation for facial expression recognition. In: International conference on multimedia modeling. Springer, Cham, pp 224–233

    Chapter  Google Scholar 

  9. Georgescu M-I, Ionescu RT, Popescu M (2019) Local learning with deep and handcrafted features for facial expression recognition. IEEE Access 7:64827–64836

    Article  Google Scholar 

  10. Ijjina EP and Mohan CK (2014) Facial expression recognition using Kinect depth sensor and convolutional neural networks. In machine learning and applications (ICMLA), 2014 13th international conference on IEEE, 392-396

  11. Jain DK, Zhang Z and Huang K (2017) Multi angle optimal pattern-based deep learning for automatic facial expression recognition. Pattern Recogn Lett

  12. Ji Y, Hu Y, Yang Y, Shen F, Shen HT (2019) Cross-domain facial expression recognition via an intra-category common feature and inter-category distinction feature fusion network. Neurocomputing 333:231–239

    Article  Google Scholar 

  13. Jiang R, Ho ATS, Cheheb I, Al-Maadeed N, Al-Maadeed S, Bouridane A (2017) Emotion recognition from scrambled facial images via many graph embedding. Pattern Recogn 67:245–251

    Article  Google Scholar 

  14. Jung H, Lee S, Yim J, Park S and Kim J (2015) Joint fine-tuning in deep neural networks for facial expression recognition, in the IEEE international conference on computer vision (ICCV), IEEE

  15. Khan SA, Hussain A, Usman M (2018) Reliable facial expression recognition for multi-scale images using weber local binary image based cosine transform features. Multimed Tools Appl 77(1):1133–1165

    Article  Google Scholar 

  16. Kumar S, Kumari R, Sharma VK (2015) Fitness based position update in spider monkey optimization algorithm. Procedia Computer Science 62:442–449

    Article  Google Scholar 

  17. Li M, Xu H, Huang X, Song Z, Liu X, Li X (2018) Facial expression recognition with identity and emotion joint learning. IEEE Trans Affect Comput:1–1

  18. Liang D, Liang H, Yu Z, Zhang Y (2020) Deep convolutional BiLSTM fusion network for facial expression recognition. Vis Comput 36(3):499–508

    Article  Google Scholar 

  19. Liu Y, Zeng J, Shan S and Zheng Z (2018) Multi-Channel pose-aware convolution neural networks for multi-view facial expression recognition, in Automatic Face & Gesture Recognition (FG 2018), 2018 13th IEEE international conference on IEEE, 458-465

  20. Lopes AT, de Aguiar E, De Souza AF, Oliveira-Santos T (2017) Facial expression recognition with convolutional neural networks: coping with few data and the training sample order. Pattern Recogn 61:610–628

    Article  Google Scholar 

  21. Luo Y, Wu C, Zhang Y (2013) Facial expression recognition based on fusion feature of PCA and LBP with SVM. Optik - International Journal for Light and Electron Optics 124(17):2767–2770

    Article  Google Scholar 

  22. Maheswari VU, Raju SV and Reddy KS (2019) Local directional weighted threshold patterns (LDWTP) for facial expression recognition. In 2019 fifth international conference on image information processing (ICIIP), IEEE, 167-170

  23. Mandal M, Verma M, Mathur S, Vipparthi SK, Murala S, Kumar DK (2019) Regional adaptive affinitive patterns (RADAP) with logical operators for facial expression recognition. IET Image Process 13(5):850–861

    Article  Google Scholar 

  24. Mavadati M, Sanger P and Mahoor MH (2016) Extended disfa dataset: investigating posed and spontaneous facial expressions. In proceedings of the IEEE conference on computer vision and pattern recognition workshops, 1-8

  25. Muhammad G, Hussain M, Alenezy F, Bebis G, Mirza AM and Aboalsamh H (2012) Race recognition from face images using weber local descriptor. In systems, signals and image processing (IWSSIP), 2012 19th international conference on IEEE, 421-424

  26. Munir A, Hussain A, Khan SA, Nadeem M and Arshid S (2018) Illumination-invariant facial expression recognition using selected merged binary patterns for real world images, Optik-International Journal for Light and Electron Optics, Illumination invariant facial expression recognition using selected merged binary patterns for real world images

  27. Naik S and Jagannath R (2018) GCV-based regularized extreme learning machine for facial expression recognition, advances in intelligent systems and computing, 129-138

  28. Ryu B, Rivera AR, Kim J, Chae O (2017) Local directional ternary pattern for facial expression recognition. IEEE Trans Image Process 26(12):6006–6018

    Article  MathSciNet  Google Scholar 

  29. Sajjad M, Shah A, Jan Z, Shah SI, Baik SW and Mehmood I (2017) Facial appearance and texture feature-based robust facial expression recognition framework for sentiment knowledge discovery, Cluster Computing, 1-19

  30. Sajjad M, Zahir S, Ullah A, Akhtar Z and Muhammad K (2019) Human behavior understanding in big multimedia data using CNN based facial expression recognition. Mobile networks and applications, 1-11

  31. Sarode N, Bhatia S (2010) Facial expression recognition. International Journal on computer science and Engineering 2(5):1552–1557

    Google Scholar 

  32. Shah N and Priyanka (2018) An Improved Framework for Human Face Recognition, Advances in Intelligent Systems and Computing, 175–180

  33. Sharma R, Patterh MS (2015) A new pose-invariant face recognition system using PCA and ANFIS. Optik-International Journal for Light and Electron Optics 126(23):3483–3487

    Article  Google Scholar 

  34. Sun Z, Hu Z-P, Chiong R, Wang M and Zhao S (2018) An adaptive weighted fusion model with two subspaces for facial expression recognition, Signal, Image, and Video Processing, 1–9

  35. Sun Z, Hu Z-p, Wang M, Zhao S-H (2019) Dictionary learning feature space via sparse representation classification for facial expression recognition. Artif Intell Rev 51(1):1–18

    Article  Google Scholar 

  36. Tripathi A, Pandey S and Jangir H (2018) Efficient Facial Expression Recognition System Based on Geometric Features Using Neural Network, In Information and Communication Technology for Sustainable Development, Springer, Singapore 181–190.

  37. Uddin MZ, Hassan MM, Almogren A, Alamri A, Alrubaian M, Fortino G (2017) Facial expression recognition utilizing local direction-based robust features and deep belief network. IEEE Access 5:4525–4536

    Article  Google Scholar 

  38. Uddin MZ, Hassan MM, Almogren A, Zuair M, Fortino G, Torresen J (2017) A facial expression recognition system using robust face features from depth videos and deep learning. Computers & Electrical Engineering 63:114–125

    Article  Google Scholar 

  39. Valstar MF, Sánchez-Lozano E, Cohn JF, Jeni LA, Girard JM, Zhang Z, Yin L and Fera PM (2017) 2017-addressing head pose in the third facial expression recognition and analysis challenge, in Automatic Face & Gesture Recognition (FG 2017), 2017 12th IEEE International Conference on IEEE, 839-847

  40. Verma M, Vipparthi SK, Singh G (2019) Hinet: hybrid inherited feature learning network for facial expression recognition. IEEE Letters of the Computer Society 2(4):36–39

    Article  Google Scholar 

  41. Videla SL, Rao MRN, Anand D, Vankayalapati HD and Razia S (2019) Deformable facial fitting using active appearance model for emotion recognition. In Smart Intelligent Computing and Applications, Springer, Singapore 135–144

  42. Xie W, Jia X, Shen L, Yang M (2019) Sparse deep feature learning for facial expression recognition. Pattern Recogn 96:106966

    Article  Google Scholar 

  43. Xu, Lei, Minrui Fei, Wenju Zhou, and Aolei Yang (2008) Face expression recognition based on convolutional neural network. In 2018 Australian & New Zealand Control Conference (ANZCC), pp. 115–118. IEEE

  44. Yang J and Wang S (2017) Capturing spatial and temporal patterns for distinguishing between posed and spontaneous expressions. In proceedings of the 2017 ACM on multimedia conference, 469-477

  45. Yang R-P, Liu Z-T, Zheng L-D, Wu J-P and C-C Hu (2018) Intelligent Mirror System Based on Facial Expression Recognition and Color Emotion Adaptation iMirror. In 2018 37th Chinese Control Conference (CCC), IEEE, 3227–3232.

  46. Yang Y, Fang D, Zhu D (2016) Facial expression recognition using deep belief network. Rev Tec Ing Univ Zulia 39(2):384–392

    Google Scholar 

  47. Yuan B, Cao H and Chu J (2012) Combining local binary pattern and local phase quantization for face recognition. In 2012 international symposium on biometrics and security technologies, IEEE 51-53

  48. Zeng N, Zhang H, Song B, Liu W, Li Y, Dobaie A (2018) Facial expression recognition via learning deep sparse autoencoders. Neurocomputing 273:643–649

    Article  Google Scholar 

  49. Zhang K, Huang Y, Du Y (2017) Facial expression recognition based on deep evolutional spatial-temporal networks. IEEE transactions on image processing, 4193-4203

  50. Zhang T, Zheng W, Cui Z, Zong Y, Yan J, Yan K (2016) A deep neural network-driven feature learning method for multi-view facial expression recognition. IEEE Transactions on Multimedia 18(12):2528–2536

    Article  Google Scholar 

  51. Zhang W, Zhao X, Morvan J and Chen L (2018) Improving shadow suppression for illumination robust face recognition. IEEE Trans Pattern Anal Mach Intell, 1–1

  52. Zhang Z, Luo P, Loy CC, Tang X (2018) From facial expression recognition to interpersonal relation prediction. Int J Comput Vis 126(5):550–569

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. IT Department, Vasireddy Venkatadri Institute of Technology, Nambur, Guntur, Dt. Andhra Pradesh, 522508, India

    Ramachandran Vedantham

  2. Principal, ANU College of Engineering & Technology, Acharya Nagarjuna University, Nambur, Guntur, Dt. Andhra Pradesh, 522510, India

    Edara Sreenivasa Reddy

Authors
  1. Ramachandran Vedantham
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Edara Sreenivasa Reddy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ramachandran Vedantham.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vedantham, R., Reddy, E.S. A robust feature extraction with optimized DBN-SMO for facial expression recognition. Multimed Tools Appl 79, 21487–21512 (2020). https://doi.org/10.1007/s11042-020-08901-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-08901-x

Keywords

Search

Navigation