Abstract
Many studies in automatic facial expression recognitions merely limit their focus on recognizing basic emotions, ignoring the fact that humans show various emotions in their daily life. Moreover, from psychological perspective humans express multiple emotions simultaneously. Up to now, researchers recognize two basic emotions at the same time, called mixed emotions. Nevertheless, the mixed emotion still does not reflect how humans express the emotion naturally. This paper advances the concept of mixed emotion into a generalized fuzzy emotion. Fuzzy emotion captures multiple emotions in a single image using fuzzy inference engine. We propose a fuzzy emotion framework which consists of processing system and knowledge system. The processing system extracts facial expression parameters, and the knowledge system employs a fuzzy knowledge-based engine, elicited from the psychologist knowledge to recognize facial expressions. Some advantages are offered: (1) no facial template comparison; (2) no training efforts needed; (3) moreover, fuzzy emotion can recognize ambiguous facial expressions adaptively. The experiment gives a recognition result with the highest accuracy rate of 0.90. A research agenda for future study of mixed emotion recognition is proposed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aly A, Tapus A (2015) An online fuzzy-based approach for human emotions detection: an overview on the human cognitive model of understanding and generating multimodal actions. Spring Tracts Adv Robot 106:185–212. https://doi.org/10.1007/978-3-319-12922-8_7
Chakrabarti D, Dutta D (2013) Facial expression recognition using eigenspaces. Proc Technol 10:755–761. https://doi.org/10.1016/j.protcy.2013.12.419
Chakraborty A, Konar A, Chakraborty UK, Chatterjee A (2009) Emotion recognition from facial expressions and its control using fuzzy logic. IEEE Trans Syst Man Cybern Part A Syst Humans 39(4):726–743. https://doi.org/10.1109/TSMCA.2009.2014645
Cootes TF, Edwards GJ, Taylor CJ (2001) Active appearance models. IEEE Trans Pattern Anal Mach Intell 23:681–685. https://doi.org/10.1109/34.927467
Du S, Tao Y, Martinez AM (2014) Compound facial expressions of emotion. Proc Natl Acad Sci 111:E1454–E1462. https://doi.org/10.1073/pnas.1322355111
Ekman P (1992) An argument for basic emotions. Cogn Emot 6:169–200. https://doi.org/10.1080/02699939208411068
Ekman P, Friesen WV (1975) Unmasking the face: a guide to recognizing emotions from facial clues. Prentice-Hall, Oxford
Ilbeygi M, Shah-Hosseini H (2012) A novel fuzzy facial expression recognition system based on facial feature extraction from color face images. Eng Appl Artif Intell 25:130–146. https://doi.org/10.1016/j.engappai.2011.07.004
Izard CE (1992) Basic emotions, relations among emotions, and emotion-cognition relations. Psychol Rev 99:561–565. https://doi.org/10.1037/0033-295X.99.3.561
Kamachi M, Lyons M, Gyoba J (1997) The Japanese female facial expression (jaffe) database. http://www.kasrl.org/jaffe.html. Accessed 3 Oct 2018
Larsen JT, McGraw A (2014) The case for mixed emotions. Soc Personal Psychol Compass 8:263–274. https://doi.org/10.1111/spc3.12108
Liliana DY, Widyanto MR, Basaruddin T (2016) Human emotion recognition based on active appearance model and semi-supervised fuzzy C-means. In: 2016 international conference on advanced computer science and information systems (ICACSIS), pp 439–445
Liliana DY, Basaruddin C, Widyanto MR (2017) Mix emotion recognition from facial expression using SVM-CRF sequence classifier. In: Proceedings of international conference on algorithms, computing and systems - ICACS’17, pp 27–31. https://doi.org/10.1145/3127942.3127958
Loconsole C, Miranda CR, Augusto G et al (2014) Real-time emotion recognition—novel method for geometrical facial features extraction. In: Proceedings of the 9th international conference on computer vision theory and applications, pp 378–385
Lucey P, Cohn JF, Kanade T et al (2010) The extended Cohn–Kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression. In: 2010 IEEE computer society conference on computer vision and pattern recognition-workshops, CVPRW, pp 94–101. https://doi.org/10.1109/cvprw.2010.5543262
Mamdani EH (1974) Application of fuzzy algorithms for control of simple dynamic plant. Proc Inst Electr Eng 121:1585–1588. https://doi.org/10.1049/piee.1974.0328
Mavadati S, Mahoor M, Bartlett K, Trinh P, Cohn J (2013) DISFA: A Spontaneous Facial Action Intensity Database. IEEE Trans Affect Comput 4(2):151–160. https://doi.org/10.1109/T-AFFC.2013.4
Nicolai A, Choi A (2015) Facial emotion recognition using fuzzy systems. In: 2015 IEEE international conference on systems, man, and cybernetics, pp 2216–2221. https://doi.org/10.1109/smc.2015.387
Sadeghi H, Raie A-A, Mohammadi M-R (2013) Facial expression recognition using geometric normalization and appearance representation. In: 2013 8th Iranian conference on machine vision and image processing (MVIP). IEEE, pp 159–163
Siddiqi MH, Ali R, Khan AM et al (2015) Human facial expression recognition using stepwise linear discriminant analysis and hidden conditional random fields. IEEE Trans Image Process 24:1386–1398. https://doi.org/10.1109/TIP.2015.2405346
Sujono, Gunawan AAS (2015) Face expression detection on Kinect using active appearance model and fuzzy logic. Proc Comput Sci 59:268–274. https://doi.org/10.1016/j.procs.2015.07.558
Tzimiropoulos G, Pantic M (2013) Optimization problems for fast AAM fitting in-the-wild. In: 2013 IEEE international conference on computer vision, pp 593–600
Vinciarelli A, Salamin H, Pantic M (2009) Social signal processing: understanding social actions through nonverbal behaviour analysis. In: 2009 IEEE computer society conference on computer vision and pattern recognition workshops, vol 3, pp 42–49. https://doi.org/10.1109/cvprw.2009.5204290
Zadeh LA (1965) Fuzzy sets. Inf Control 8:338–353. https://doi.org/10.1016/S0019-9958(65)90241-X
Acknowledgements
The authors would like to thank all the contributors as well as participants in this research. The first author would also like to thank the Indonesia Endowment Fund for Education (LPDP) for the doctoral study sponsorship.
Funding
This study was funded by the Indonesia Ministry of Research Technology and Higher Education through the Postgraduate Team Grant number 120/SP2H/PTNBH/DRPM/2018.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards.
Informed consent
Informed consent was obtained from all individual participants included in the study.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Handling editor: Tina Iachini (University of Campania Luigi Vanvitelli).
Reviewers: Igor Aleksander (Imperial College London), Antonio Jesus Bandera Rubio (University of Malaga).
Rights and permissions
About this article
Cite this article
Liliana, D.Y., Basaruddin, T., Widyanto, M.R. et al. Fuzzy emotion: a natural approach to automatic facial expression recognition from psychological perspective using fuzzy system. Cogn Process 20, 391–403 (2019). https://doi.org/10.1007/s10339-019-00923-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10339-019-00923-0