Skip to main content
SpringerLink
Log in

Facial expression recognition based on local binary patterns

  • Applied Problems
  • Published:
Pattern Recognition and Image Analysis Aims and scope Submit manuscript

Abstract

In this paper, a novel approach to automatic facial expression recognition from static images is proposed. The face area is first divided automatically into small regions, from which the local binary pattern (LBP) histograms are extracted and concatenated into a single feature histogram, efficiently representing facial expressions—anger, disgust, fear, happiness, sadness, surprise, and neutral. Then, a linear programming (LP) technique is used to classify the seven facial expressions. Experimental results demonstrate an average expression recognition accuracy of 93.8% on the JAFFE database, which outperforms the rate of all other reported methods on the same database.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. M. Pantic and Leon J. M. Rothkrantz, “Automatic Analysis of Facial Expressions: The State and the Art,” IEEE Trans. on Pattern Analysis and Machine Intelligence 22(12), 1424–1445 (2000).

    Article  Google Scholar 

  2. B. Fasel and J. Luettin, “Automatic Facial Expression Analysis: A Survey,” Pattern Recognition 36(1), 259–275 (2003).

    Article  MATH  Google Scholar 

  3. W. Fellenz, J. Taylor, N. Tsapatsoulis, and S. Kollias, Comparing Template-Based, Feature-Based and Supervised Classification of Facial Expression from Static Images (Computational Intelligence and Applications, World Scientific and Engineering Society Press, 1999).

  4. I. Buciu, C. Kotropoulos, and I. Pitas, “ICA and Gabor Representation for Facial Expression Recognition,” in Proceedings Int. Conf. on Image Processing (2003), pp. 855–858.

  5. M. Gargesha and P. Kuchi, Facial Expression Recognition Using Artificial Neural Networks (EEE 511-Artificial Neural Computation Systems, Spring, 2002).

  6. Z. Zhang, “Feature-Based Facial Expression Recognition: Sensitivity Analysis and Experiment with a Multi-Layer Perceptron,” Int. J. on Pattern Recognition and Artificial Intelligence 13(6), 893–911 (1999).

    Article  Google Scholar 

  7. Z. Zhang, M. Lyons, M, Schuster, and S. Akamatsu, “Comparison between Geometry-Based and Garbor-Wavelet-Based Facial Expression Recognition Using Multi-Layer Perceptron,” in Proceedings 3rd Int. Conf. Automatic Face and Gesture Recognition (1998), pp. 454–459.

  8. M. Lyons, J, Budynek, and S. Akamastu, “Automatic Classification of Single Facial Images,” IEEE Trans. Pattern Analysis and Machine Intelligence 21(12), 1357–1362 (1999).

    Article  Google Scholar 

  9. M. Lyons, S. Akamastu, M. Kamachi, and J. Gyoba, “Coding Facial Expressions with Gabor Wavelets,” in Proceedings Third IEEE Conf, Face and Gesture Recognition (Nara, Japan. Apr. 1998), pp. 200–205.

  10. G. D. Guo and C. R. Dyer, “Simultaneous Feature Selection and Classifier Training via Linear Programming: A Case Study for Face Expression Recognition,” in Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (June, 2003), Vol. 1, pp. 346–352.

    Google Scholar 

  11. M. Pietikäinen, T. Nurmela, T. Mäenpää, and M. Turtinen, “View-Based Recognition of Real-World Textures,” Pattern Recognition 37(2), 313–323 (2004).

    Article  MATH  Google Scholar 

  12. T. Ojala. M. Pietikainen, and T. Mäenpää, “Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns,” IEEE Trans, on Pattern Analysis and Machine Intelligence 24(7), 971–987 (2002)

    Article  Google Scholar 

  13. T. Ahonen, A. Hadid, and M. Pietikäinen, Face Recognition with Local Binary Patterns (Lecture Notes in Computer Science 3021, Springer, 2004), pp. 469–481.

  14. D. Bolme, M. Teixeira, J. Beveridge, and B. Draper, “The CSU Face Identification Evaluation System User’s guide: Its Purpose, Feature and Structure,” in 3rd International Conf. on Computer Vision Systems (2003), pp. 304–313.

  15. K. P. Bennett and O. L, Mangasarian, “Robust Linear Programming Discrimination of Two Linearly Inseparable Sets,” Optimization Methods and Software 1, pp. 23–34 (1992).

    Article  Google Scholar 

  16. P. S. Bradley and O. L. Mangasarian, “Feature Selection via Concave Minimization and Support Vector Machines,” in Proceedings 5th Int. Conf. Machine Learning (1998), pp. 82–90.

  17. B. Fasel, “Head-Pose Invariant Facial Expression Recognition Using Convolutional Neural Networks,” in Proceedings of the Fourth Int. IEEE Conf. on Multimodal Interfaces (2002), pp. 529–534.

Download references

Author information

Authors and Affiliations

  1. Northwestern Polytechnic University, Xi’an, 710072, China

    X. Feng

  2. Machine Vision Group, Infotech Oulu and Department of Electrical and Information Engineering, University of Oulu, P.O. Box 4500, FIN-90014, Finland

    M. Pietikäinen & A. Hadid

Authors
  1. X. Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Pietikäinen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. A. Hadid
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to X. Feng.

Additional information

The text was submitted by the authors in English.

Xiaoyi Feng received her doctor of technology degree in electrical engineering from the Northwestern Polytechnic University, China, in 2001. She is currently an associate professor at the Northwestern Polytechnic University. Her research interests include computer vision, image processing, and pattern recognition. She has about 20 scientific publications.

Abdenour Hadid is finishing his doctoral research at the University of Oulu, Finland. He graduated and received his engineer diploma from the National Institute of Informatics (INI), Algeria, in 1997. His research interest includes computer vision and pattern recognition. He is currently focusing on face detection and recognition, color image analysis, and learning. He has authored about 20 papers in international conferences and one journal article. He has served as a reviewer to several international conferences and journals such as IEEE Transactions on Pattern Analysis and Machine Intelligence, International Journal of Information Visualization, and International Conference on Image Processing (ICIP). He is member of the Pattern Recognition Society of Finland.

Matti Pietikäinen received his doctor of technology degree in electrical engineering from the University of Oulu, Finland, in 1982. In 1981, he established the Machine Vision Group at the University of Oulu. The research results of his group have been widely exploited in industry. Currently, he is professor of information engineering, scientific director of Infotech Oulu research center, and leader of the Machine Vision Group at the University of Oulu. From 1980 to 1981 and from 1984 to 1985, he visited the Computer Vision Laboratory at the University of Maryland, USA. His research interests are in machine vision and image analysis. His current research focuses on texture analysis, face image analysis, learning in machine vision, and machine vision for sensing and understanding human actions. He has authored about 165 papers in international journals, books, and conference proceedings, and nearly 100 other publications or reports. He is associate editor of the journal Pattern Recognition and was associate editor of IEEE Transactions on Pattern Analysis and Machine Intelligence from 2000 to 2005, He was chairman of the Pattern Recognition Society of Finland from 1989 to 1992. Since 1989, he has served as a member of the governing board of the International Association for Pattern Recognition (IAPR) and became one of the founding fellows of the IAPR in 1994. He has also served on committees of several international conferences. He is a senior member of the IEEE and vice-chair of the IEEE Finland section.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Feng, X., Pietikäinen, M. & Hadid, A. Facial expression recognition based on local binary patterns. Pattern Recognit. Image Anal. 17, 592–598 (2007). https://doi.org/10.1134/S1054661807040190

Download citation

  • Received:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S1054661807040190

Keywords

Search

Navigation