Abstract
The automatic recognition of human emotions is used to support several computing paradigms, like affective, positive and pervasive computing. Histograms of oriented gradients (HOG) have been successfully used with such a purpose, by processing facial images. However, the results of using HOG vary depending on the position of the facial components in the image used as input. This paper presents an extension to the HOG method, which was named Landmark-based Histograms of Oriented Gradients (LaHOG), that not only calculates HOG blocks in the whole face, but also in specific positions around selected facial landmarks. In this sense, the new method is more robust than its predecessor. In order to evaluate the capabilities and limitations of this proposal, we used it to recognize emotions in face images from the FACES database. In such a process we used two classification strategies: support vector machines and logistic regression. The results show that the extended method significantly surpasses the performance of HOG in the tested database.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Ahlberg, J.: Candide-3 - an updated parameterized face. Technical report, LiTH-ISY-R-2326, Department of Electrical Engineering, Linkping University, Sweden (2001)
Anderson, K., McOwan, P.W.: A real-time automated system for the recognition of human facial expressions. IEEE Trans. Syst. Man Cybern. B Cybern. 36(1), 96–105 (2006)
Bai, Y., Guo, L., Jin, L., Huang, Q.: A novel feature extraction method using pyramid histogram of orientation gradients for smile recognition. In: 2009 16th IEEE International Conference on Image Processing (ICIP), pp. 3305–3308, November 2009
Bartlett, M., Littlewort, G., Frank, M., Lainscsek, C., Fasel, I., Movellan, J.: Recognizing facial expression: machine learning and application to spontaneous behavior. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 2, pp. 568–573, June 2005
Bettadapura, V.: Face expression recognition and analysis: the state of the art. Technical report, College of Computing, Georgia Institute of Technology (2012)
Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 1–27 (2011). http://www.csie.ntu.edu.tw/~cjlin/libsvm
Chen, J., Chen, Z., Chi, Z., Fu, H.: Facial expression recognition based on facial components detection and hog features. In: Scientific Cooperations International Workshops on Electrical and Computer Engineering Subfields, Istanbul, Turkey, August 2014
Dahmane, M., Meunier, J.: Emotion recognition using dynamic grid-based hog features. In: FG, pp. 884–888. IEEE (2011)
Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: International Conference on Computer Vision & Pattern Recognition, vol. 2, pp. 886–893, June 2005
Dhall, A., Asthana, A., Goecke, R., Gedeon, T.: Emotion recognition using phog and lpq features. In: FG, pp. 878–883. IEEE (2011)
Ebner, N.C., Riediger, M., Lindenberger, U.: Faces. a database of facial expressions in young, middle-aged, and older women and men: development and validation. Behav. Res. Meth. 42(1), 351–362 (2010)
Ekman, P., Friesen, W. (eds.): The Facial Action Coding System. Consulting Psychologists Press, Palo Alto (1978)
Fasel, B., Luettin, J.: Automatic facial expression analysis: a survey. Pattern Recogn. 36(1), 259–275 (2003). http://www.sciencedirect.com/science/article/pii/S0031320302000523
Kołakowska, A., Landowska, A., Szwoch, M., Szwoch, W., Wróbel, M.R.: Emotion recognition and its applications. In: Hippe, Z.S., Kulikowski, J.L., Mroczek, T., Wtorek, J. (eds.) Human-Computer Systems Interaction: Backgrounds and Applications 3. AISC, vol. 300, pp. 51–62. Springer, Heidelberg (2014)
Kotsia, I., Pitas, I.: Facial expression recognition in image sequences using geometric deformation features and support vector machines. IEEE Trans. Image Process. 16(1), 172–187 (2007)
Li, Z., Ichi Imai, J., Kaneko, M.: Facial-component-based bag of words and phog descriptor for facial expression recognition. In: IEEE International Conference on Systems, Man and Cybernetics, SMC 2009, pp. 1353–1358, October 2009
Murphy, K.P.: Machine Learning: A Probabilistic Perspective. The MIT Press, Cambridge (2012)
Orrite, C., Gañán, A., Rogez, G.: HOG-based decision tree for facial expression classification. In: Araujo, H., Mendonça, A.M., Pinho, A.J., Torres, M.I. (eds.) IbPRIA 2009. LNCS, vol. 5524, pp. 176–183. Springer, Heidelberg (2009)
Pantic, M., Patras, I.: Detecting facial actions and their temporal segments in nearly frontal-view face image sequences. In: 2005 IEEE International Conference on Systems, Man and Cybernetics, vol. 4, pp. 3358–3363, October 2005
Pantic, M., Patras, I.: Dynamics of facial expression: recognition of facial actions and their temporal segments from face profile image sequences. IEEE Trans. Syst. Man Cybern. Part B Cybern. 36(2), 433–449 (2006)
Pantic, M., Rothkrantz, L.J.M.: Automatic analysis of facial expressions: the state of the art. IEEE Trans. Pattern Anal. Mach. Intell. 22(12), 1424–1445 (2000). http://dx.doi.org/10.1109/34.895976
Sinha, P.: Perceiving and recognizing three-dimensional forms. Ph.D. Thesis, Massachusetts Institute of Technology (1995)
Steffens, J., Elagin, E., Neven, H.: Personspotter-fast and robust system for human detection, tracking and recognition. In: Proceedings of the Third IEEE International Conference on Automatic Face and Gesture Recognition, pp. 516–521, April 1998
Tao, H., Huang, T.S.: A piecewise Bezier volume deformation model and its applications in facial motion capture. In: Advances in Image Processing and Understanding: A Festschrift for Thomas S. Huang (2002)
Tivatansakul, S., Ohkura, M., Puangpontip, S., Achalakul, T.: Emotional healthcare system: emotion detection by facial expressions using japanese database. In: 2014 6th Computer Science and Electronic Engineering Conference (CEEC), pp. 41–46, September 2014
Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)
Yu, X., Huang, J., Zhang, S., Yan, W., Metaxas, D.: Pose-free facial landmark fitting via optimized part mixtures and cascaded deformable shape model. In: 2013 IEEE International Conference on Computer Vision (ICCV), pp. 1944–1951, December 2013
Yun, W., Kim, D., Park, C., Kim, J.: Hybrid facial representations for emotion recognition. ETRI J. 35(6), 1021–1028 (2013)
Zheng, W., Zhou, X., Zou, C., Zhao, L.: Facial expression recognition using kernel canonical correlation analysis (KCCA). IEEE Trans. Neural Netw. 17(1), 233–238 (2006)
Aknowledgments
This work has been partially supported by Fondecyt (Chile), grant Nro. 1150252.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Guerrero, P., Pavez, M., Chávez, D., Ochoa, S.F. (2015). Landmark-Based Histograms of Oriented Gradients for Facial Emotion Recognition. In: Cleland, I., Guerrero, L., Bravo, J. (eds) Ambient Assisted Living. ICT-based Solutions in Real Life Situations. IWAAL 2015. Lecture Notes in Computer Science(), vol 9455. Springer, Cham. https://doi.org/10.1007/978-3-319-26410-3_27
Download citation
DOI: https://doi.org/10.1007/978-3-319-26410-3_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-26409-7
Online ISBN: 978-3-319-26410-3
eBook Packages: Computer ScienceComputer Science (R0)