Skip to main content
SpringerLink
Log in
SpringerLink
Log in

NovA: Automated Analysis of Nonverbal Signals in Social Interactions

  • Conference paper
Human Behavior Understanding (HBU 2013)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 8212))

Included in the following conference series:

Abstract

Previous studies have shown that the success of interpersonal interaction depends not only on the contents we communicate explicitly, but also on the social signals that are conveyed implicitly. In this paper, we present NovA (NOnVerbal behavior Analyzer), a system that analyzes and facilitates the interpretation of social signals conveyed by gestures, facial expressions and others automatically as a basis for computer-enhanced social coaching. NovA records data of human interactions, automatically detects relevant behavioral cues as a measurement for the quality of an interaction and creates descriptive statistics for the recorded data. This enables us to give a user online generated feedback on strengths and weaknesses concerning his social behavior, as well as elaborate tools for offline analysis and annotation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Mehrabian, A.: Silent messages: Implicit Communication of Emotions and Attitudes. Wadsworth Publishing Co Inc., Belmont (1981)

    Google Scholar 

  2. Eagle, N., Pentland, A.: Reality mining: Sensing complex social signales. J. of Personal and Ubiquitous Computing 10(4), 255–268 (2006)

    Article  Google Scholar 

  3. Wittenburg, P., Brugman, H., Russel, A., Klassmann, A., Sloetjes, H.: Elan: A professional framework for multimodality research. In: Proc. of the Fifth International Conference on Language Resources and Evaluation (LREC), pp. 879–896 (2006)

    Google Scholar 

  4. Kipp, M.: Anvil: The video annotation research tool. In: Handbook of Corpus Phonology. Oxford University Press, Oxford (2013)

    Google Scholar 

  5. Schmidt, T.: Transcribing and annotating spoken language with exmaralda. In: Proc. of the LREC-Workshop on XML Based Richly Annotated Corpora, Lisbon 2004, pp. 879–896. ELRA, Paris (2004)

    Google Scholar 

  6. Curhan, J., Pentland, A.: Thin slices of negotiation: predicting outcomes from conversational dynamics withing the first 5 minutes. J. of Applied Psychology 92(3), 802–811 (2007)

    Article  Google Scholar 

  7. Pentland, A.: Automatic mapping and modelling of human networks. Physica A(378), 59–67 (2007)

    Google Scholar 

  8. Schuller, B., Müller, R., Eyben, F., Gast, J., Hörnler, B., Wöllmer, M., Rigoll, G., Höthker, A., Konosu, H.: Being bored? recognising natural interest by extensive audiovisual integration for real-life application. Image Vision Comput. 27(12), 1760–1774 (2009)

    Article  Google Scholar 

  9. Rich, C., Ponsleur, B., Holroyd, A., Sidner, C.L.: Recognizing engagement in human-robot interaction. In: Proc. of the 5th ACM/IEEE Intl. Conf. on Human-Robot Interaction, HRI 2010, pp. 375–382. IEEE Press, Piscataway (2010)

    Google Scholar 

  10. Zeng, Z., Pantic, M., Roisman, G.I., Huang, T.S.: A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE Trans. Pattern Anal. Mach. Intell. 31(1), 39–58 (2009)

    Article  Google Scholar 

  11. Pianesi, F., Mana, N., Cappelletti, A., Lepri, B., Zancanaro, M.: Multimodal recognition of personality traits in social interactions. In: Proc. of the 10th International Conference on Multimodal Interfaces, ICMI 2008, pp. 53–60. ACM, NY (2008)

    Google Scholar 

  12. Dong, W., Lepri, B., Cappelletti, A., Pentland, A.S., Pianesi, F., Zancanaro, M.: Using the influence model to recognize functional roles in meetings. In: Proc. of the 9th International Conference on Multimodal Interfaces, ICMI 2007, pp. 271–278. ACM, New York (2007)

    Google Scholar 

  13. Hung, H., Gatica-Perez, D.: Estimating cohesion in small groups using audio-visual nonverbal behavior. Trans. Multi. 12(6), 563–575 (2010)

    Article  Google Scholar 

  14. Sandbach, G., Zafeiriou, S., Pantic, M., Yin, L.: Static and dynamic 3d facial expression recognition: A comprehensive survey. Image Vision Comput. 30(10), 683–697 (2012)

    Article  Google Scholar 

  15. Caridakis, G., Raouzaiou, A., Karpouzis, K., Kollias, S.: Synthesizing gesture expressivity based on real sequences. In: Workshop on Multimodal Corpora: from Multimodal Behaviour Theories to Usable Models. LREC, Genoa (2006)

    Google Scholar 

  16. Vogt, T., André, E., Bee, N.: Emovoice - a framework for online recognition of emotions from voice. In: André, E., Dybkjær, L., Minker, W., Neumann, H., Pieraccini, R., Weber, M. (eds.) PIT 2008. LNCS (LNAI), vol. 5078, pp. 188–199. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  17. Kleinsmith, A., Bianchi-Berthouze, N.: Form as a cue in the automatic recognition of non-acted affective body expressions. In: D’Mello, S., Graesser, A., Schuller, B., Martin, J.-C. (eds.) ACII 2011, Part I. LNCS, vol. 6974, pp. 155–164. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  18. Kim, J., André, E.: Emotion recognition based on physiological changes in music listening. IEEE Trans. Pattern Anal. Mach. Intell. 30(12), 2067–2083 (2008)

    Article  Google Scholar 

  19. Camurri, A., Volpe, G., De Poli, G., Leman, M.: Communicating expressiveness and affect in multimodal interactive systems. IEEE MultiMedia 12(1) (2005)

    Google Scholar 

  20. Scherer, S., Marsella, S., Stratou, G., Xu, Y., Morbini, F., Egan, A., Rizzo, A(S.), Morency, L.-P.: Perception markup language: Towards a standardized representation of perceived nonverbal behaviors. In: Nakano, Y., Neff, M., Paiva, A., Walker, M. (eds.) IVA 2012. LNCS, vol. 7502, pp. 455–463. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  21. Simpson, J.A., Harris, B.A.: Interpersonal attraction. In: Weber, A.L., Harvey, J.H. (eds.) Perspectives on Close Relationships, pp. 45–66. Prentice Hall (1994)

    Google Scholar 

  22. McGinley, H., LeFevre, R., McGinley, P.: The influence of a communicator’s body position on opinion. J. of Personality and Social Psychology 31(4), 686–690 (1975)

    Article  Google Scholar 

  23. Schouwstra, S., Hoogstraten, J.: Head position and spinal position as determinants of perceived emotional state. Perceptual and Motor Skills 81, 673–674 (1995)

    Article  Google Scholar 

  24. Sidner, C.L., Kidd, C.D., Lee, C., Lesh, N.: Where to look: a study of human-robot engagement. In: IUI 2004: Proc. of the 9th International Conference on Intelligent User Interfaces, pp. 78–84. ACM Press, New York (2004)

    Google Scholar 

  25. Pease, A.: Body Language. Sheldon Press, London (1988)

    Google Scholar 

  26. Bandura, A.: Self Efficacy: The Exercise of Control. Palgrave Macmillan, New York (1997)

    Google Scholar 

  27. Forgas, J.P., Cooper, J., Crano, W.D.: The Psychology of Attitudes and Attitude Change. Taylor & Francis Group, New York (2010)

    Google Scholar 

  28. Wagner, J., Lingenfelser, F., Baur, T., Damian, I., Kistler, F., André, E.: The social signal interpretation (ssi) framework - multimodal signal processing and recognition in real-time. In: Proceedings of the 21st ACM International Conference on Multimedia, Barcelona, Spain (2013)

    Google Scholar 

  29. Kistler, F., Endrass, B., Damian, I., Dang, C., André, E.: Natural interaction with culturally adaptive virtual characters. Germany Journal on Multimodal User Interfaces Heidelberg/Berlin (2012)

    Google Scholar 

  30. Wallbott, H.: Bodily expression of emotion. European Jrl. of Social Psychology (28), 879–896 (1998)

    Google Scholar 

  31. Ruf, T., Ernst, A., Küblbeck, C.: Face detection with the sophisticated high-speed object recognition engine (shore). In: Microelectronic Systems, pp. 243–252. Springer (2011)

    Google Scholar 

  32. Hammer, T.: Mental health and social exclusion among unemployed youth in scandinavia. a comparative study. Intl. Jrnl. of Social Welfare 9(1), 53–63 (2000)

    Article  Google Scholar 

  33. Pan, X., Gillies, M., Barker, C., Clark, D.M., Slater, M.: Socially anxious and confident men interact with a forward virtual woman: An experiment study. PLoS ONE 7(4) (2012) e32931

    Google Scholar 

  34. Damian, I., Baur, T., André, E.: Investigating social cue-based interaction in digital learning games. In: Proc. of the 8th International Conference on the Foundations of Digital Games, SASDG (2013)

    Google Scholar 

  35. Porayska-Pomsta, K., Anderson, K., Damian, I., Baur, T., André, E., Bernardini, S., Rizzo, P.: Modelling users’ affect in job interviews: Technological demo. In: Carberry, S., Weibelzahl, S., Micarelli, A., Semeraro, G. (eds.) UMAP 2013. LNCS, vol. 7899, pp. 353–355. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Human Centered Multimedia, Augsburg University, Universitätsstr. 6a, 86159, Augsburg, Germany

    Tobias Baur, Ionut Damian, Florian Lingenfelser, Johannes Wagner & Elisabeth André

Authors
  1. Tobias Baur
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ionut Damian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Florian Lingenfelser
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Johannes Wagner
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Elisabeth André
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Engineering, Bogaziçi University, 34342, Bebek, Istanbul, Turkey

    Albert Ali Salah

  2. Delft University of Technology, Delft, The Netherlands

    Hayley Hung

  3. Idiap Research Institute, Martigny, Switzerland

    Oya Aran

  4. Queen Mary University of London, UK

    Hatice Gunes

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer International Publishing Switzerland

About this paper

Cite this paper

Baur, T., Damian, I., Lingenfelser, F., Wagner, J., André, E. (2013). NovA: Automated Analysis of Nonverbal Signals in Social Interactions. In: Salah, A.A., Hung, H., Aran, O., Gunes, H. (eds) Human Behavior Understanding. HBU 2013. Lecture Notes in Computer Science, vol 8212. Springer, Cham. https://doi.org/10.1007/978-3-319-02714-2_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-02714-2_14

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-02713-5

  • Online ISBN: 978-3-319-02714-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation