Skip to main content
SpringerLink
Log in

Face detection and recognition of natural human emotion using Markov random fields

  • Original Article
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

This paper presents an integrated system for emotion detection. In this research effort, we have taken into account the fact that emotions are most widely represented with eye and mouth expressions. The proposed system uses color images and it is consisted of three modules. The first module implements skin detection, using Markov random fields models for image segmentation and skin detection. A set of several colored images with human faces have been considered as the training set. A second module is responsible for eye and mouth detection and extraction. The specific module uses the HLV color space of the specified eye and mouth region. The third module detects the emotions pictured in the eyes and mouth, using edge detection and measuring the gradient of eyes’ and mouth’s region figure. The paper provides results from the system application, along with proposals for further research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Besag JE (1974) Spatial interaction and statistical analysis of lattice systems. J R Stat Soc Ser B 36(2):192–236

    MATH  MathSciNet  Google Scholar 

  2. Celeux G, Forbes F, Peyrard N (2002) EM procedures using mean field-like approximations for Markov model-based image segmentation. Pattern Recognit 36(1):131–144

    Article  Google Scholar 

  3. Chai D, Ngan KN (1999) Face segmentation using skin-color map in videophone applications. IEEE Trans Circuits Syst Video Technol 9(4):551–564

    Article  Google Scholar 

  4. Chellapa P, Wilson C, Sirohey S (1995) Human and machine recognition of faces: a survey. In: Proceedings of IEEE, vol 83(5), pp 705–740

  5. Chellappa R, Jain A (eds) (1996) Markov random fields: theory and applications. Academic, New York

    Google Scholar 

  6. Chellappa R, Wilson CL, Sirohey S (1995) Human and machine recognition of faces: a survey. In: Proceeding of IEEE, vol 83(5), pp 705–741

  7. Cowie R, Douglas-Cowie E, Tsapatsoulis N, Votsis G, Kollias S, Fellenz W, Taylor J (2001) Emotion recognition in human-computer interaction. IEEE Signal Process Mag 18(1):32–80

    Article  Google Scholar 

  8. DeCarlo D, Metaxas D (2000) Optical flow constraints on deformable models with applications to face tracking. Int J Comput Visi 38(2):99–127

    Article  MATH  Google Scholar 

  9. Feraud R, Bernier OJ, Viallet JE, Collobert M (2001) A fast and accurate face detection based on neural network. IEEE Trans Pattern Anal Mach Intell 23(1):42–53

    Article  Google Scholar 

  10. Hjelm E, Low BK (2001) Face detection: a survey. Comput Vis Image Underst 83(3):236–274

    Article  Google Scholar 

  11. Ioannou S, Caridakis G, Karpouzis K, Kollias S (2006) Robust feature detection for facial expression recognition. EURASIP J Image Video Process. Special issue on Facial Image Processing

  12. Ioannou S, Raouzaiou A, Tzouvaras V, Mailis T, Karpouzis K, Kollias S (2006) Emotion recognition through facial expression analysis based on a neurofuzzy network. Special issue on emotion: understanding & recognition neural networks. Elsevier, Amsterdam, vol 18(4), pp 423–435

  13. Jones MJ, Rehg JM (2002) Statistical color models with application to skin detection. Comput Vis Pattern Recognit 46(1):274–280

    Google Scholar 

  14. Maio D, Maltoni D (2000) Real-time face location on gray-scale static images. Pattern Recognit 33(9):1525–1539

    Article  Google Scholar 

  15. Pantic M, Rothkrantz LJM (1996) Automatic analysis of facial expressions: the state of the art. IEEE Trans Pattern Anal Mach Intell 22(12):1424–1445

    Article  Google Scholar 

  16. Pantic M, Rothkrantz L (2004) Facial action recognition for facial expression analysis from static face images. IEEE Trans Syst Man Cybern Part B 34(3):1449–1461

    Article  Google Scholar 

  17. Smeulders AWM, Worring M, Santini S, Gupta A (2000) Jain R content-based image retrieval at the end of the early years. IEEE Trans Pattern Anal Mach Intell 22(12):1349–1380

    Article  Google Scholar 

  18. Sobottka K, Pitas I (1998) A novel method for automatic face segmentation, facial feature extraction and tracking. Signal Process Image Commun 12(3):263–281

    Article  Google Scholar 

  19. Wallace M, Ioannou S, Raouzaiou A, Karpouzis K, Kollias S (2006) Dealing with feature uncertainty in facial expression recognition using possibilistic fuzzy rule evaluation. Int J Intell Syst Technol Appl 1:3–4

    Google Scholar 

  20. Wang H, Chang SF (1997) A highly efficient system for automatic face region detection in MPEG video. IEEE Trans Circuits Syst Video Technol 7(4):615–628

    Article  MathSciNet  Google Scholar 

  21. Wang Y, Yuan B (2001) A novel approach for human face detection from color images under complex background. Pattern Recognit 34:1983–1992

    Article  MATH  MathSciNet  Google Scholar 

  22. Wechsler H, Phillips P, Bruce V, Soulie F, Huang T (eds) (1998) Face recognition: from theory to applications. Springer, Heidelberg

    MATH  Google Scholar 

  23. Winkler G (1995) Image analysis, random fields and dynamic Monte Carlo methods. Springer, Heidelberg

    MATH  Google Scholar 

  24. Wong KW, Lam KM, Siu WC (2003) A robust scheme for live detection of human faces in color images. Signal Process Image Commun 18(2):103–114

    Article  Google Scholar 

  25. Yang MH, Kriegman D, Ahuja N (2002) Detecting faces in images: a survey. IEEE Trans Pattern Anal Mach Intell 24(1):34–58

    Article  Google Scholar 

  26. Ying Li, Lai JH, Yuen PC (2006) Multi-template ASM Method for feature points detection of facial image with diverse expressions. Autom Face Gesture Recognit 435–440

  27. Zhao W, Chellappa R, Rosenfeld A, Phillips PJ (2003) Face recognition: a literature survey. CVL technical report, Center for Automation Research, University of Maryland at College Park. http://www.cfar.umd.edu/ftp/TRs/FaceSurvey.ps.gz

Download references

Author information

Authors and Affiliations

  1. Department of Information and Communication Systems Engineering, University of the Aegean, Karlovassi, 83200, Samos, Greece

    Ilias Maglogiannis, Demosthenes Vouyioukas & Chris Aggelopoulos

Authors
  1. Ilias Maglogiannis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Demosthenes Vouyioukas
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chris Aggelopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ilias Maglogiannis.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Maglogiannis, I., Vouyioukas, D. & Aggelopoulos, C. Face detection and recognition of natural human emotion using Markov random fields. Pers Ubiquit Comput 13, 95–101 (2009). https://doi.org/10.1007/s00779-007-0165-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-007-0165-0

Keywords

Search

Navigation