Skip to main content
SpringerLink
Log in

Applying a Text-Based Affective Dialogue System in Psychological Research: Case Studies on the Effects of System Behaviour, Interaction Context and Social Exclusion

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

This article presents two studies conducted with an affective dialogue system in which text-based system–user communication was used to model, generate and present different affective and social interaction scenarios. We specifically investigated the influence of interaction context and roles assigned to the system and the participants, as well as the impact of pre-structured social interaction patterns that were modelled to mimic aspects of “social exclusion” scenarios. The results of the first study demonstrate that both the social context of the interaction and the roles assigned to the system influence the system evaluation, interaction patterns, textual expressions of affective states, as well as emotional self-reports. The results observed for the second study show the system’s ability to partially exclude a participant from a triadic conversation without triggering significantly different affective reactions or a more negative system evaluation. The experimental evidence provides insights on the perception, modelling and generation of affective and social cues in artificial systems that can be realized in different modalities, including the text modality, thus delivering valuable input for applying affective dialogue systems as tools for studying affect and social aspects in online communication.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. Participants believe that they communicate with a dialogue system, while responses are actually provided by a human operator. In the presented experiments, the operator was asked to conduct a realistic and coherent dialogue and provided free text input to user utterances.

  2. Studies 2 and 3 included three 7-min-long interactions. No simulation of the thinking and typing delays was used. In both experiments participants were aware that they interact with an artificial system.

  3. Artificial Intelligence Markup Language (AIML).

  4. Consistent affective characteristics are achieved by modifying most of the generated response candidates. Modifications include removing, adding or replacing discovered positive or negative expressions, words and/or emoticons. E.g. for the negative profile, the component removes phrases that contain words, classified as “positive” (e.g., glad, happy, welcome, great, sir, please).

  5. In the following, “negative system”, “neutral system” refer to the specific type of affective profile applied, i.e. negative and neutral, respectively (ref. 3.1, Control Layer).

  6. Applied Annotation Tools and Resources: the analysis of the presented data-set was conducted with a set of natural-language processing and affective processing tools and resources, including: Linguistic Inquiry and Word Count dictionary, ANEW dictionary based classifier, Lexicon-Based Sentiment Classifier, and Support Vector Machine Based Dialogue Act classifier. Further, we analysed timing information and surface features of communication style such as wordiness and usage of emoticons. While the application of such tools and resources cannot always guarantee that all the expressions of affect, linguistic and discourse related cues are correctly detected and classified, in the recent years this set of tools was successfully applied in numerous psychological experiments and extensively evaluated and validated [9, 12, 39, 41, 42, 51, 61, 62] supporting their application for the automatic analysis of text in different domains, such as online and offline texts.

  7. http://www.cyberemotions.eu/.

References

  1. Andre E, Dybkjaer L, Minker W, Heisterkamp PE. Affective dialogue systems, tutorial and research workshop (ads 2004). In: Lecture notes in computer science. Berlin: Springer; 2004.

  2. Andre E, Rehm M, Minker W, Bühler. Endowing spoken language dialogue systems with emotional intelligence. Affect Dialogue Syst. 2004;3068:178–87.

  3. Arnold J. Affect in language learning. Cambridge: Cambridge University Press; 1999.

    Google Scholar 

  4. Bevacqua E, De Sevin E, Hyniewska SJ, Pelachaud C. A listener model: introducing personality traits. J Multimodal User Interfaces. 2012;6:27–38.

    Article  Google Scholar 

  5. Bhatt K, Argamon S, Evens M. Hedged responses and expressions of affect in human/human and human/computer tutorial interactions. In: Proceedings of the 26th annual conference of the cognitive science society (CogSci04), 2004, pp. 114–119.

  6. Blankenship V, Hnat SM, Hess TG, Brown DR. Reciprocal interaction and similarity of personality attributes. J Soc Pers Relat. 1984;1:415–32.

  7. Blascovich J. A theoretical model of social influence for increasing the utility of collaborative virtual environments. In: Proceedings of the 4th International conference on collaborative virtual environments. 2002; doi:10.1145/571878.571883.

  8. Blascovich J, Loomis J, Beall AC, Swinth KR, Hoyt CL, Bailenson JN. Immersive virtual environment technology as a methodological tool for social psychology. Psychol Inq. 2002;13:103–24.

    Article  Google Scholar 

  9. Bradley M, Lang P. Affective norms for English words (anew): stimuli, instruction manual and affective ratings. University of Florida; 1999.

  10. Buschmeier H, Kopp S. Co-constructing grounded symbols—feedback and incremental adaptation in human–agent dialogue. Künstliche Intelligenz. 2013;27:137–43.

    Article  Google Scholar 

  11. Calvo R, D’Mello S. Affect detection: an interdisciplinary review of models, methods, and their applications. IEEE Trans Affect Comput. 2010;1:18–37.

    Article  Google Scholar 

  12. Calvo RA, Kim S. Emotions in text: dimensional and categorical models. Comput Intell. 2012;29(3)527–43.

  13. Chang C, Lin C. LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol. 2011; 2:27:1–27:27.

  14. Cohen J. A power primer. Quantitative methods in psychology. Psychol Bull. 1992;112:155–9.

  15. Davis MA. A multidimensional approach to individual differences in empathy. JSAS Catalog of Selected Documents in Psychology 1980;10:85.

  16. Epley N, Akalis S, Waytz A, Cacioppo JT. Creating social connection through inferential reproduction. Psychol Sci. 2008;19:114–20.

    Article  PubMed  Google Scholar 

  17. Fast L, Funder D. Personality as manifest in word use: correlations with self-report, acquaintance report, and behavior. J Pers Soc Psychol. 2008;94(2):334–46.

    Article  PubMed  Google Scholar 

  18. Forsyth E, Martell C. Lexical and discourse analysis of online chat dialogue. In: Proceedings of the first IEEE international conference on semantic computing, 2007, pp. 19–26.

  19. Garas A, Garcia D, Skowron M, Schweitzer F. Emotional persistence in online chatting communities. Sci Rep. 2012;2(402).

  20. Gligorijevic V, Skowron M, Tadic B. Structure and stability of online chat network built on emotion-carrying links. Phys A Stat Mech Appl. 2012; 392: 538–543.

  21. Gobron S, Ahn J, Quentin S, Thalmann D, Rank S, Skowron M, Paltoglou G, Thelwall M. An interdisciplinary VR-architecture for 3d chatting with non-verbal communication. In: Proceedings of joint virtual reality conference of EuroVR, 2011.

  22. Gros C. Cognition and emotion: perspectives of a closing gap. Cogn Comput. 2010;2:78–85.

    Article  Google Scholar 

  23. Isbister K, Nass C. Consistency of personality in interactive characters: verbal cues, non-verbal cues, and user characteristics. Int J Hum Comput Stud. 2000;53:251–67.

    Article  Google Scholar 

  24. Jin B. How lonely people use and perceive Facebook. Comput Human Behav. 2013;29:2463–70.

  25. Kappas A. Appraisals are direct, immediate, intuitive, and unwitting… and some are reflective. Cogn Emot. 2006;20:952–75.

    Article  Google Scholar 

  26. Kappas A, Kuester D, Theunis M, Tsankova E. Cyberemotions: subjective and physiological responses to reading online discussion forums. Psychophysiology 2010;47:S101.

  27. Kassner MP, Wesselmann ED, Law AT, Williams KD. Virtually ostracized: studying ostracism in immersive virtual environments. Cyberpsychol Behav Soc Netw. 2012;15(8):399–403.

    Article  PubMed Central  PubMed  Google Scholar 

  28. Minear M, Park DC. A lifespan database of adult facial stimuli. Behav Res Methods Instrum Comput. 2004;36:630–3.

    Article  PubMed  Google Scholar 

  29. Moon Y. Intimate exchanges: using computers to elicit self-disclosure from consumers. J Consum Res. 2000;26:324–40.

    Article  Google Scholar 

  30. Moon Y, Nass C. How real are computer personalities? Psychological responses to personality types in human-computer interaction. Comm. Res. 1996;23:351–74.

  31. Nass C, Isbister K, Lee EJ. Truth is beauty: researching embodied conversational agents. In: Embodied conversational agents, 2000, pp. 374–402.

  32. Nass C, Moon Y. Machines and mindlessness: social responses to computers. J Soc Issues. 2000;56(1):81–103.

    Article  Google Scholar 

  33. Nass C, Moon Y, Carney P. Are respondents polite to computers? Social desirability and direct responses to computers. J Appl Soci Psychol. 1999;29:1093–110.

    Article  Google Scholar 

  34. Nass C, Moon Y, Fogg BJ, Reeves B, Dryer DC. Can computer personalities be human personalities? Int J Hum Comput Stud. 1995;1995(43):223–39.

    Article  Google Scholar 

  35. Nass C, Moon Y, Green N. Are computers gender-neutral? Social desirability and direct responses to computers. J Appl Soci Psychol. 1997;27:864–76.

    Article  Google Scholar 

  36. Nijholt A. Conversational agents and the construction of humorous acts, conversational informatics: an engineering approach. 2007, pp. 21–47.

  37. Paiva A, Dias J, Sobral D, Aylett R, Woods S, Hall L, Zoll C. Learning by feeling: evoking empathy with synthetic characters. Appl Artif Intell. 2005;19:235–66.

    Article  Google Scholar 

  38. Paltoglou G, Gobron S, Skowron M, Thelwall M, Thalmann D. Sentiment analysis of informal textual communication in cyberspace. In: Proceedings of engage 2010, Springer LNCS State-of-the-Art Survey, 2010, pp. 13–25.

  39. Paltoglou G, Theunis M, Kappas A, Thelwall M. Predicting emotional responses to long informal text. IEEE Trans Affect Comput. 2013;4:106–22.

    Article  Google Scholar 

  40. Pennebaker J, King L. Linguistic styles: language use as an individual difference. J Pers Soc Psychol. 1999;77(6):1296–312.

    Article  CAS  PubMed  Google Scholar 

  41. Pennebaker J, Francis M, Booth R. Linguistic inquiry and word count: LIWC 2001. Lawrence: Erlbaum Publishers; 2001.

  42. Pennebaker J, Mehl M, Niederhoffer K. Psychological aspects of natural language use: our words, our self. Annu Rev Psychol. 2003;54:547–77.

    Article  PubMed  Google Scholar 

  43. Pillutla M, Murnighan J. Unfairness, anger, and spite: emotional rejections of ultimatum offers. Organ Behav Hum Decis Process. 1996;68(3):208–24.

    Article  Google Scholar 

  44. Pittermann J, Pittermann A, Minker W. Handling emotions in human-computer dialogues. Dordrecht: Springer; 2009.

    Google Scholar 

  45. Reeves B, Nass C. The media equation: how people treat computers, television, and new media like real people and places. New York: Cambridge University Press; 1996.

    Google Scholar 

  46. Sanfey AG, Rilling JK, Aronson JA, Nystrom LE, Cohen JD. The neural basis of economic decision-making in the Ultimatum Game. Science. 2003;300:1755–8.

    Article  CAS  PubMed  Google Scholar 

  47. Schröder M, Bevacqua E, Cowie R, Eyben F, Gunes H, Heylen D, Maat M, McKeown G, Pammi S, Pantic M, Pelachaud C, Schuller B, Sevin E, Valstar M, Wöllmer M. Building autonomous sensitive artificial listeners. IEEE Trans Affect Comput. 2012;3:165–83.

    Article  Google Scholar 

  48. Sienkiewicz J, Skowron M, Paltoglou G, Hołyst JA. Entropy growth in emotional online dialogues. J Phys: Conf Ser. 2013;410(1):012096.

    Google Scholar 

  49. Skowron M. Affect listeners. Acquisition of affective states by means of conversational systems. In: Proceedings of COST 2102 training school, LNCS; 2010, pp. 169–181.

  50. Skowron M. Affect listeners. Acquisition of affective states by means of conversational systems. In: Development of multimodal interfaces—active listening and synchrony, Springer, Lecture Notes in Computer Science, 2010; pp. 169–181.

  51. Skowron M, Paltoglou G. Affect bartender—affective cues and their application in a conversational agent. In: IEEE symposium series on computational intelligence 2011, IEEE workshop on affective computational intelligence, 2011.

  52. Skowron M, Irran J, Krenn B. Computational framework for and the realization of cognitive agents providing intelligent assistance capabilities. In: ECAI. The 6th international cognitive robotics workshop, 2008, pp. 88–96.

  53. Skowron M, Pirker H, Rank S, Paltoglou G, Ahn J, Gobron S. No peanuts! Affective cues for the virtual bartender. In: Proceedings of the Florida artificial intelligence research society conference. USA: AAAI Press; 2011.

  54. Skowron M, Rank S. Interacting with collective emotions in e-Communities. In: Von Scheve C, Salmela M editors Collective emotions, perspectives from psychology, philosophy, and sociology. Oxford: Oxford University Press; 2014.

  55. Skowron M, Rank S, Theunis M, Sienkiewicz J. The good, the bad and the neutral: affective profile in dialog system-user communication. In: Proceedings of fourth bi-annual affective computing and intelligent interaction conference—ACII 2011, LNCS, 2011, pp. 347–356.

  56. Skowron M, Theunis M, Rank S, Borowiec A. Effect of affective profile on communication patterns and affective expressions in interactions with a dialog system. In: Proceedings of fourth bi-annual affective computing and intelligent interaction conference—ACII 2011, LNCS, 2011, pp. 337–346.

  57. Skowron M, Theunis M, Rank S, Kappas A. Affect and social processes in online communication—experiments with affective dialogue system. IEEE Trans Affect Comput. 2013;4:267–79.

    Article  Google Scholar 

  58. Squartini S, Schuller B, Hussain A. Cognitive and emotional information processing for human-machine interaction. Cogn Comput. 2012;4(4):383-5

  59. Stone P, Dexter C, Marshall S, Ogilvie D. The general inquirer: a computer approach to content analysis. Cambridge: MIT Press; 1966.

    Google Scholar 

  60. Tadic B, Gligorijevic V, Skowron M, Suvakov M. The dynamics of emotional chats with bots: experiment and agent-based simulations. ScienceJet. 2014;3:50.

    Google Scholar 

  61. Thelwall M, Buckley K, Paltoglou G, Cai D, Kappas A. Sentiment strength detection in short informal text. J Am Soc Inf Sci Technol. 2010;61:2544–58.

    Article  Google Scholar 

  62. Thelwall M, Buckley K, Paltoglou G, Skowron M, Garcia D, Gobron S, Ahn J, Kappas A, Kuster D, Holyst JA. Damping sentiment analysis in online communication: discussions, monologs and dialogs. In: Computational linguistics and intelligent text processing, 2013, pp. 1–12..

  63. Tatai G, Laufer L. Extraction of affective components from texts and their use in natural language dialogue systems. Acta Cybernetica. 2004;16:625–42.

    Google Scholar 

  64. ter Maat M, Truong KP, Heylen D. How agents’ turn-taking strategies influence impressions and response behaviors. Presence. 2011;20(5):412–30.

    Article  Google Scholar 

  65. Traum D, DeVault D, Lee J, Wang Z, Marsella S. Incremental dialogue understanding and feedback for multiparty, multimodal conversation. In: Proceedings of the 12th international conference on intelligent virtual agents, 2012, pp. 275–288.

  66. Valdesolo P, DeSteno D. Moral hypocrisy: social groups and the flexibility of virtue. Psychol Sci. 2007;18:689–90.

    Article  PubMed  Google Scholar 

  67. van’t Wout M, Kahn RS, Sanfey AG, Aleman A. Affective state and decision making in the ultimatum game. Exp Brain Res. 2006;169:564–568.

  68. Vinciarelli A, Pantic M, Heylen D, Pelachaud C, Poggi I, D’Errico F, Schröder M. Bridging the gap between social animal and unsocial machine: a survey of social signal processing. IEEE Trans Affect Comput. 2012;3:69–87.

    Article  Google Scholar 

  69. von der Pütten AM, Krämer NC, Gratch J, Kang S. It doesn’t matter what you are! - Explaining social effects ofagents and avatars. Comput Human Behav. 2010;26:1641–50.

  70. Ward AF, Olsen AS, Wegner DM. The harm-made mind: observing victimization augments attribution of minds to vegetative patients, robots, and the dead. Psychol Sci. 2013;24:1437–45.

    Article  PubMed  Google Scholar 

  71. Wang Z, Lee J, Marsella S. Towards more comprehensive listening behavior: beyond the bobble head. In: IVA, 2006, pp. 216–227.

  72. Waytz A, Epley N, Cacioppo JT. Social cognition unbound: insights into anthropomorphism and dehumanization. Curr Dir Psychol Sci. 2010;19:58–62.

    Article  PubMed Central  PubMed  Google Scholar 

  73. Waytz A, Epley N. Social connection enables dehumanization. J Exp Soc Psychol. 2012;48:70–6.

    Article  Google Scholar 

  74. Wilks Y. Is there progress on talking sensibly to machines? Science. 2007;318(9):927–8.

    Article  CAS  PubMed  Google Scholar 

  75. Williams KD, Cheung CKZ, Choi. Cyberostracism: effects of being ignored over the Internet. J Pers Soc Psychol. 2000; 79/5: 748–762.

  76. Williams KD, Govan CL, Croker V, Tynan D, Cruickshank M, Lam A. Investigations into the differences between social- and cyberostracism. Group Dyn Theor Res Pract. 2002;6:65–77.

  77. Williams KD, Yaeger DS, Cheung CKT, Choi W. Cyberball (version 4.0) [Software], 2012. Available from https://cyberball.wikispaces.com.

  78. Yarkoni T. Personality in 100,000 words: a large-scale analysis of personality and word use among bloggers. J Res Pers. 2010;44(3):363–73.

    Article  PubMed Central  PubMed  Google Scholar 

  79. Ziemke T, Lowe R. On the role of emotion in embodied cognitive architectures: from organisms to robots. Cogn Comput. 2009;1:104–17.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Austrian Research Institute for Artificial Intelligence, Freyung 6/6/3a, Vienna, Austria

    Marcin Skowron

  2. Westphal College of Media Arts and Design, Drexel University, 3141 Chestnut Street, Philadelphia, PA, USA

    Stefan Rank

  3. Jacobs University Bremen, Campus Ring 1, 28759, Bremen, Germany

    Aleksandra Świderska, Dennis Küster & Arvid Kappas

Authors
  1. Marcin Skowron
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stefan Rank
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aleksandra Świderska
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dennis Küster
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Arvid Kappas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marcin Skowron.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Skowron, M., Rank, S., Świderska, A. et al. Applying a Text-Based Affective Dialogue System in Psychological Research: Case Studies on the Effects of System Behaviour, Interaction Context and Social Exclusion. Cogn Comput 6, 872–891 (2014). https://doi.org/10.1007/s12559-014-9271-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-014-9271-2

Keywords

Search

Navigation