Skip to main content
SpringerLink
Log in

Eloquence of eyes and mouth of virtual agents: cultural study of facial expression perception

  • Original Article
  • Published:
AI & SOCIETY Aims and scope Submit manuscript

Abstract

This paper reports the results of a cross-cultural study on facial regions as cues to recognize facial expressions of virtual agents. The experiment was conducted between Japan and Hungary using 30 facial expressions of cartoonish faces designed by Hungarians. The results suggest the following: (1) cultural differences exist when using facial regions as cues to recognize cartoonish facial expressions between Hungary and Japan. Japanese weighed facial cues more heavily in the eye region than Hungarians, who weighed facial cues more heavily in the mouth region than Japanese. (2) The mouth region is more effective for conveying the emotions of facial expressions than the eye region, regardless of country. Our findings can be used not only to derive design guidelines for virtual agent facial expressions when aiming at users of a single culture, but as adaptation strategies in applications with users from various cultures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Breazeal C (2003) Emotion and sociable humanoid robots. Int J Hum Comput Stud 59(1–2):11–155

    Google Scholar 

  • Breazeal C (2007) Sociable robots. J Robotics Soc Jpn 24(5):591–593

    Article  MATH  Google Scholar 

  • Ekman P, Friesen WV (1971) Constants across cultures in the face and emotion. J Pers Soc Psychol 17:124–129

    Article  Google Scholar 

  • Ekman P, Friesen WV (1975) Unmasking the face: a guide to recognizing emotions from facial cues. Prentice Hall, Englewood Cliffs, NK

    Google Scholar 

  • Ekman P, Friesen WV (1978) Facial action coding system: a technique for the measurement of facial movement. Consulting Psychologists Press, Palo Alto

    Google Scholar 

  • Ekman P, Friesen WV, O’Sullivan M, Chan A, Diacoyanni-Tarlatzis I, Heider K, Krause R, LeCompte WA, Pitcairn T, Ricci-Bitti PE, Scherer KR, Tomita M, Tzavaras A (1987) Universals and cultural differences in the judgment of facial expressions of emotion. J Pers Soc Psychol 53:712–717

    Article  Google Scholar 

  • Elfenbein H, Beaupré M, Leveque M, Hess U (2007) Toward a dialect theory: cultural differences in expressing and recognizing facial expressions. Emotion 7:131–146

    Article  Google Scholar 

  • Fridlund AJ (1994) Human facial expression: an evolutionary review. Academic Press, San Diego, CA

    Google Scholar 

  • Ishiguro H (2005) Android science: toward a new cross-disciplinary framework. In: CogSci-2005 Workshop: Toward Social Mechanisms of Android Science, pp 1–6

  • Jack RE, Blais C, Scheepers C, Schyns PG, Caldara R (2009) Cultural confusions show facial expressions are not universal. Curr Biol 19(18):1543–1548

    Article  Google Scholar 

  • Koda T, Ishida T, Rehm M, Andre E (2009) Avatar culture: cross-cultural evaluations of avatar facial expressions. J AI Soc 24(3):237–250

    Article  Google Scholar 

  • Koda T, Ruttkay Z, Nakagawa Y, Tabuchi K (2010) Cross-cultural Study on facial regions as cues to recognize emotions of virtual agents. In: Ishida T (ed) Culture Comput., Lecture Notes in Computer Science. Vol. 6259Springer, Berlin, pp 16–27

    Chapter  Google Scholar 

  • Matsumoto D, Ekman P (1989) American-Japanese cultural differences in intensity ratings. Motivation Emotion 13:143–157

    Article  Google Scholar 

  • Mori M: Bukimi no Tani (1970) The Uncanny Valley (trans: MacDorman KF, Minato T). Energy, 7(4):33–35 (Originally in Japanese)

  • Rickel J, Marsella S, Gratch J, Hill R, Traum D, Swartout B (2002) Towards a new generation of virtual humans for interactive experiences. IEEE Intelligent Systems, pp 32–38

  • Ruttkay Z (2009) Cultural dialects of real and synthetic emotional facial expressions. J AI Soc 24(3):307–315

    Article  Google Scholar 

  • Ruttkay, Z., Noot, H. (2000) Animated chartoon faces. In: International Symposium on Non-photorealistic Animation and Rendering, 91–100

  • Van Breemen AJN (2005) iCat: Experimenting with Animabotics. In: AISB Creative Robotics Symposium

  • Yuki M, Maddux WW, Masuda T (2007) Are the windows to the soul the same in the East and West? J Exp Soc Psychol 43:30–311

    Article  Google Scholar 

Download references

Acknowledgments

This research is supported by a Grant-in-Aid for Scientific Research (C) 20500196 (2008–2010) from the Japan Society for the Promotion of Science. CharToon was developed by Zsofia Ruttkay and Han Noot.

Author information

Authors and Affiliations

  1. Department of Information Science and Technology, Osaka Institute of Technology, 1-79-1 Kitayama, Hirakata, Osaka, 573-0196, Japan

    Tomoko Koda

  2. Moholy-Nagy University of Art and Design, Zugligeti ut 9-25, Budapest, Hungary

    Zsofia Ruttkay

Authors
  1. Tomoko Koda
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zsofia Ruttkay
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tomoko Koda.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Koda, T., Ruttkay, Z. Eloquence of eyes and mouth of virtual agents: cultural study of facial expression perception. AI & Soc 32, 17–24 (2017). https://doi.org/10.1007/s00146-014-0571-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00146-014-0571-6

Keywords

Search

Navigation