Skip to main content
SpringerLink
Log in

Reconsidering the Duchenne Smile: Formalizing and Testing Hypotheses About Eye Constriction and Positive Emotion

  • RESEARCH ARTICLE
  • Published:
Affective Science Aims and scope Submit manuscript

Abstract

The common view of emotional expressions is that certain configurations of facial-muscle movements reliably reveal certain categories of emotion. The principal exemplar of this view is the Duchenne smile, a configuration of facial-muscle movements (i.e., smiling with eye constriction) that has been argued to reliably reveal genuine positive emotion. In this paper, we formalized a list of hypotheses that have been proposed regarding the Duchenne smile, briefly reviewed the literature weighing on these hypotheses, identified limitations and unanswered questions, and conducted two empirical studies to begin addressing these limitations and answering these questions. Both studies analyzed a database of 751 smiles observed while 136 participants completed experimental tasks designed to elicit amusement, embarrassment, fear, and physical pain. Study 1 focused on participants’ self-reported positive emotion and Study 2 focused on how third-party observers would perceive videos of these smiles. Most of the hypotheses that have been proposed about the Duchenne smile were either contradicted by or only weakly supported by our data. Eye constriction did provide some information about experienced positive emotion, but this information was lacking in specificity, already provided by other smile characteristics, and highly dependent on context. Eye constriction provided more information about perceived positive emotion, including some unique information over other smile characteristics, but context was also important here as well. Overall, our results suggest that accurately inferring positive emotion from a smile requires more sophisticated methods than simply looking for the presence/absence (or even the intensity) of eye constriction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. Note that an earlier version of Study 1 was previously published as a conference paper (Girard et al., 2019).

  2. FACS includes instructions for determining whether an image includes AU6, AU12, or both (Ekman et al., 2002, pp. 188–193).

  3. Varying effects are also called “random effects.”

  4. Population-level effects are also called “fixed effects.”

  5. In this context, a simplex is a vector where each element is a real number between 0 and 1 and all the elements add up to 1.

  6. Inter-rater reliability under ICC model 2A equals \( {\hat{\sigma}}_o^2/\left({\hat{\sigma}}_o^2+\left({\hat{\sigma}}_r^2+{\hat{\sigma}}_e^2\right)/k\right) \) where \( {\hat{\sigma}}_o^2 \) is the estimated object (i.e., video) variance, \( {\hat{\sigma}}_r^2 \) is the estimated rater variance, \( {\hat{\sigma}}_e^2 \) is the estimated residual variance, and k is the number of raters whose scores are being averaged per object.

  7. McNeish and Wolf (2020) provide compelling arguments for why this approach of estimating a CFA model with freely estimated factor loadings and residuals is preferable to using a simpler approach, such as sum or mean scores, even with highly inter-correlated indicators.

  8. Testing this theory is beyond the scope of this paper (and would require looking beyond just smiles). However, we note that the non-significant partial effects of eye constriction in the joke task are problematic for this theory.

References

  • Arslan, R. C., Walther, M. P., & Tata, C. S. (2020). formr: a study framework allowing for automated feedback generation and complex longitudinal experience-sampling studies using R. Behavior Research Methods, 52, 376–387.

    Article  PubMed  Google Scholar 

  • Barrett, L. F., Adolphs, R., Marsella, S., Martinez, A. M., & Pollak, S. D. (2019). Emotional expressions reconsidered: challenges to inferring emotion from human facial movements. Psychological Science in the Public Interest, 20(1), 1–68.

    Article  PubMed  PubMed Central  Google Scholar 

  • Bergtold, J. S., Spanos, A., & Onukwugha, E. (2010). Bernoulli regression models: Revisiting the specification of statistical models with binary dependent variables. Journal of Choice Modelling, 3(2), 1–28.

    Article  Google Scholar 

  • Bürkner, P.-C. (2017). brms: an R package for Bayesian multilevel models using Stan. Journal of Statistical Software, 80(1), 1–28.

    Article  Google Scholar 

  • Bürkner, P.-C. (2018). Advanced Bayesian multilevel modeling with the R package brms. The R Journal, 10(1), 395–411.

    Article  Google Scholar 

  • Bürkner, P.-C., & Charpentier, E. (2020). Modelling monotonic effects of ordinal predictors in Bayesian regression models. British Journal of Mathematical and Statistical Psychology.

  • Bürkner, P.-C., & Vuorre, M. (2019). Ordinal regression models in psychology: a tutorial. Advances in Methods and Practices in Psychological Science, 2(1), 77–101.

    Article  Google Scholar 

  • Darwin, C. (1872). The expression of emotions in man and animals (3rd ed.). Oxford University.

  • DeCoster, J., Iselin, A.-M. R., & Gallucci, M. (2009). A conceptual and empirical examination of justifications for dichotomization. Psychological Methods, 14(4), 349–366.

    Article  PubMed  Google Scholar 

  • Devlieger, I., & Rosseel, Y. (2017). Factor score path analysis: an alternative for SEM? Methodology, 13, 31–38.

    Article  Google Scholar 

  • Duchenne, G. B. (1862). The mechanism of human facial expression (R. A. Cuthbertson, Ed. & Trans.). Cambridge University Press.

  • Ekman, P. (1985). Telling lies: clues to deceit in the marketplace, politics, and marriage. Norton.

  • Ekman, P., Davidson, R. J., & Friesen, W. V. (1990). The Duchenne smile: emotional expression and brain physiology: II. Journal of Personality and Social Psychology, 58(2), 342–353.

    Article  PubMed  Google Scholar 

  • Ekman, P., & Friesen, W. V. (1982). Felt, false, and miserable smiles. Journal of Nonverbal Behavior, 6(4), 238–252.

    Article  Google Scholar 

  • Ekman, P., Friesen, W. V., & Hager, J. (2002). Facial action coding system: a technique for the measurement of facial movement. Research Nexus.

  • Ekman, P., Friesen, W. V., & O’Sullivan, M. (1988). Smiles when lying. Journal of Personality and Social Psychology, 54(3), 414–414.

    Article  PubMed  Google Scholar 

  • Fernández-Dols, J.-M., & Ruiz-Belda, M.-A. (1997). Spontaneous facial behavior during intense emotional episodes: artistic truth and optical truth. In J. A. Russell & J.-M. Fernández-Dols (Eds.), The psychology of facial expression (pp. 255–274). Cambridge University Press.

  • FFmpeg Development Team. (2019). FFmpeg (Version a7245ad) [Computer software]. https://ffmpeg.org/.

  • Fletcher, R. H., Fletcher, S. W., & Fletcher, G. S. (2014). Diagnosis. In Clinical epidemiology: The essentials (5th ed., pp. 108–131). Lippincott Williams & Wilkins.

  • Fox, J. (2019). polycor: Polychoric and polyserial correlations (R package version 0.7-10) [computer software]. https://CRAN.R-project.org/package=polycor.

  • Frank, M. G., & Ekman, P. (1993). Not all smiles are created equal: the differences between enjoyment and nonenjoyment smiles. Humor - International Journal of Humor Research, 6(1).

  • Frank, M. G., Ekman, P., & Friesen, W. V. (1993). Behavioral markers and recognizability of the smile of enjoyment. Journal of Personality and Social Psychology, 64(1), 83–93.

    Article  PubMed  Google Scholar 

  • Fridlund, A. J. (1994). Facial reflexes and the ontogeny of facial displays. In Human facial expression: An evolutionary view (pp. 99–122). Academic press.

  • Garnier-Villarreal, M., & Jorgensen, T. D. (2020). Adapting fit indices for Bayesian structural equation modeling: Comparison to maximum likelihood. Psychological Methods, 25(1), 46–70.

    Article  PubMed  Google Scholar 

  • Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2014). Bayesian data analysis (3rd ed.). CRC Press.

  • Gelman, A., Goodrich, B., Gabry, J., & Vehtari, A. (2019). R-squared for Bayesian regression models. The American Statistician, 73(3), 307–309.

    Article  Google Scholar 

  • Gelman, A., Lee, D., & Guo, J. (2015). Stan: A probabilistic programming language for Bayesian inference and optimization. Journal of Educational and Behavioral Statistics, 40(5), 530–543.

    Article  Google Scholar 

  • Girard, J. M. (2020). Agreement: an R package for the tidy analysis of agreement and reliability. https://github.com/jmgirard/agreement.

  • Girard, J. M., Shandar, G., Liu, Z., Cohn, J. F., Yin, L., & Morency, L.-P. (2019). Reconsidering the Duchenne smile: indicator of positive emotion or artifact of smile intensity? Proceedings of the 8th International Conference on Affective Computing and Intelligent Interaction (ACII), 594–599. https://doi.org/10.1109/ACII.2019.8925535.

  • Gosselin, P., Perron, M., & Beaupré, M. (2010). The voluntary control of facial action units in adults. Emotion, 10(2), 266–271.

    Article  PubMed  Google Scholar 

  • Graham, J. W., Taylor, B. J., Olchowski, A. E., & Cumsille, P. E. (2006). Planned missing data designs in psychological research. Psychological Methods, 11(4), 323–343.

    Article  PubMed  Google Scholar 

  • Gunnery, S. D., & Hall, J. A. (2015). The expression and perception of the Duchenne smile. In A. Kostic & D. Chadee (Eds.), The social psychology of nonverbal communication. Palgrave Macmillan.

  • Gunnery, S. D., Hall, J. A., & Ruben, M. A. (2013). The deliberate Duchenne smile: individual differences in expressive control. Journal of Nonverbal Behavior, 37(1), 29–41.

    Article  Google Scholar 

  • Gunnery, S. D., & Ruben, M. A. (2016). Perceptions of Duchenne and non-Duchenne smiles: a meta-analysis. Cognition and Emotion, 30(3), 501–515.

    Article  PubMed  Google Scholar 

  • Gwet, K. L. (2014). Handbook of inter-rater reliability: The definitive guide to measuring the extent of agreement among raters (4th ed.). Advanced Analytics.

  • Harris, C. R., & Alvarado, N. (2005). Facial expressions, smile types, and self-report during humour, tickle, and pain. Cognition and Emotion, 19(5), 655–669.

    Article  Google Scholar 

  • Hoffman, M. D., & Gelman, A. (2014). The No-U-Turn Sampler: adaptively setting path lengths in Hamiltonian Monte Carlo. Journal of Machine Learning Research, 15, 1593–1623.

    Google Scholar 

  • Hunt, W. A. (1941). Recent developments in the field of emotion. Psychological Bulletin, 38(5), 249–276.

    Article  Google Scholar 

  • Jakobs, E., Manstead, A. S. R., & Fischer, A. H. (1999). Social motives, emotional feelings, and smiling. Cognition & Emotion, 13(4), 321–345.

    Article  Google Scholar 

  • Keltner, D. (1995). Signs of appeasement: evidence for the distinct displays of embarrassment, amusement, and shame. Journal of Personality and Social Psychology, 68(3), 441–441.

    Article  Google Scholar 

  • Klineberg, O. (1940). Social psychology. Holt.

  • Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163.

    Article  PubMed  PubMed Central  Google Scholar 

  • Kraut, R. E., & Johnston, R. E. (1979). Social and emotional messages of smiling: an ethological approach. Journal of Personality and Social Psychology, 37(9), 1539–1539.

    Article  Google Scholar 

  • Krumhuber, E. G., & Manstead, A. S. R. (2009). Can Duchenne smiles be feigned? New evidence on felt and false smiles. Emotion, 9(6), 807–820.

    Article  PubMed  Google Scholar 

  • Kruschke, J. (2014). Doing Bayesian data analysis: a tutorial with R, JAGS, and Stan (2nd ed.). Academic Press.

  • Kunz, M., Meixner, D., & Lautenbacher, S. (2019). Facial muscle movements encoding pain—a systematic review. PAIN, 160(3), 535.

    Article  PubMed  Google Scholar 

  • Landis, C. (1924). Studies of emotional reactions: II. General behavior and facial expression. Journal of Comparative Psychology, 4(5), 447–510.

    Article  Google Scholar 

  • Lewandowski, D., Kurowicka, D., & Joe, H. (2009). Generating random correlation matrices based on vines and extended onion method. Journal of Multivariate Analysis, 100(9), 1989–2001.

    Article  Google Scholar 

  • Makowski, D., Ben-Shachar, M., & Lüdecke, D. (2019). bayestestR: Describing effects and their uncertainty, existence and significance within the Bayesian framework. Journal of Open Source Software, 4(40), 1541.

    Article  Google Scholar 

  • Malek, N., Messinger, D., Gao, A. Y. L., Krumhuber, E., Mattson, W., Joober, R., Tabbane, K., & Martinez-Trujillo, J. C. (2019). Generalizing Duchenne to sad expressions with binocular rivalry and perception ratings. Emotion, 19(2), 234–241.

    Article  PubMed  Google Scholar 

  • Martin, J., Rychlowska, M., Wood, A., & Niedenthal, P. (2017). Smiles as multipurpose social signals. Trends in Cognitive Sciences, 1–14.

  • Matsumoto, D., & Willingham, B. (2009). Spontaneous facial expressions of emotion of congenitally and noncongenitally blind individuals. Journal of Personality and Social Psychology, 96(1), 1–10.

    Article  PubMed  Google Scholar 

  • McElreath, R. (2016). Statistical rethinking: a Bayesian course with examples in R and Stan. CRC Press.

  • McKelvey, R. D., & Zavoina, W. (1975). A statistical model for the analysis of ordinal level dependent variables. The Journal of Mathematical Sociology, 4(1), 103–120.

    Article  Google Scholar 

  • McNeish, D., & Wolf, M. G. (2020). Thinking twice about sum scores. Behavior Research Methods.

  • Mehu, M., Grammer, K., & Dunbar, R. I. M. (2007). Smiles when sharing. Evolution and Human Behavior, 28(6), 415–422.

    Article  Google Scholar 

  • Merkle, E. C., & Rosseel, Y. (2018). blavaan: Bayesian structural equation models via parameter expansion. Journal of Statistical Software, 85(1), 1–30.

    Google Scholar 

  • Messinger, D. S., Mattson, W. I., Mahoor, M. H., & Cohn, J. F. (2012). The eyes have it: Making positive expressions more positive and negative expressions more negative. Emotion, 12(3), 430–436.

    Article  PubMed  Google Scholar 

  • Neal, R. M. (1993). Probabilistic inference using Markov Chain Monte Carlo methods (technical report CRG-TR-93-1). University of Toronto.

  • Prkachin, K. M., & Solomon, P. E. (2008). The structure, reliability and validity of pain expression: Evidence from patients with shoulder pain. Pain, 139(2), 267–274.

    Article  PubMed  Google Scholar 

  • Ruan, Q.-N., Liang, J., Hong, J.-Y., & Yan, W.-J. (2020). Focusing on mouth movement to improve genuine smile recognition. Frontiers in Psychology, 11, 1126.

    Article  PubMed  PubMed Central  Google Scholar 

  • Rychlowska, M., Jack, R. E., Garrod, O. G. B., Schyns, P. G., Martin, J. D., & Niedenthal, P. M. (2017). Functional smiles: Tools for love, sympathy, and war. Psychological Science, 28(9), 1259–1270.

    Article  PubMed  PubMed Central  Google Scholar 

  • Sheather, S. J. (2009). Diagnostics and transformations for multiple linear regression. In A modern approach to regression with R (pp. 151–225). Springer-Verlag.

  • Tagiuri, R. (1968). Person perception. In G. Lindzey & E. Aronson (Eds.), Handbook of social psychology (pp. 395–449). Addison-Wesley.

  • Weber, S. J., & Cook, T. D. (1972). Subject effects in laboratory research: an examination of subject roles, demand characteristics, and valid inference. Psychological Bulletin, 77(4), 273–295.

    Article  Google Scholar 

  • Zhang, Z., Girard, J. M., Wu, Y., Zhang, X., Liu, P., Ciftci, U., Canavan, S., Reale, M., Horowitz, A., Yang, H., Cohn, J. F., Ji, Q., & Yin, L. (2016). Multimodal spontaneous emotion corpus for human behavior analysis. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 3438–3446. https://doi.org/10.1109/cvpr.2016.374.

Download references

Author information

Authors and Affiliations

  1. University of Kansas, Lawrence, KS, USA

    Jeffrey M. Girard

  2. University of Pittsburgh, Pittsburgh, PA, USA

    Jeffrey F. Cohn

  3. Binghamton University, Binghamton, NY, USA

    Lijun Yin

  4. Carnegie Mellon University, Pittsburgh, PA, USA

    Louis-Philippe Morency

Authors
  1. Jeffrey M. Girard
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jeffrey F. Cohn
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lijun Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Louis-Philippe Morency
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

JFC and LY designed, collected, and provided consultation about the BP4D+ dataset. JFC and JMG managed the process of annotating the dataset for facial action units. JMG conceived the present studies, wrote the code, collected the perceptual ratings, ran the statistical analyses, and wrote the initial draft of the manuscript under the advisement of LPM. All authors contributed to editing and approved the submitted manuscript.

Corresponding author

Correspondence to Jeffrey M. Girard.

Ethics declarations

Funding

This material is based upon work partially supported by the National Science Foundation (1629716, 1629898, 1722822, 1734868) and National Institutes of Health (MH096951). Any opinions, findings, and conclusions expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation or National Institutes of Health, and no official endorsement should be inferred.

Data Availability

The BP4D+ dataset, including videos and FACS annotations, is available from Binghamton University. Because access to this data requires signing a license, instead of providing direct access to the data as part of this work, we are providing code to reproducibly derive our data from the distributed dataset. This code, as well as other materials not covered by the BP4D+ license, is available online at https://osf.io/k3g2e/ under an open source license.

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

The collection of the BP4D+ dataset was approved and overseen by the Institutional Review Board at Binghamton University. The collection of perceptual ratings of videos from the BP4D+ dataset was approved and overseen by the Institutional Review Board at Carnegie Mellon University.

Consent to Participate

All participants in the BP4D+ dataset provided informed consent to participate and to have their data shared with other researchers, shown to participants in further studies, and printed in scientific journals.

Code Availability

All code used to derive our measures from the BP4D+ dataset and analyze the data is available online at the aforementioned link.

Additional information

Handling Editor: Jonathan Gratch

Supplementary Information

ESM 1

(DOCX 13 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Girard, J.M., Cohn, J.F., Yin, L. et al. Reconsidering the Duchenne Smile: Formalizing and Testing Hypotheses About Eye Constriction and Positive Emotion. Affec Sci 2, 32–47 (2021). https://doi.org/10.1007/s42761-020-00030-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42761-020-00030-w

Keywords

Search

Navigation