Simo Järvelä
Giulio Jacucci
Skip Abstract Section
Abstract
In a novel experimental setting, we augmented a variation of traditional compassion meditation with our custom-built VR environment for multiple concurrent users. The presence of another user’s avatar in shared virtual space supports social interactions and provides an active target for evoked compassion. The system incorporates respiration and brainwave-based biofeedback to enable closed-loop interaction of users based on their shared physiological state. Specifically, we enhanced interoception and the deep empathetic processes involved in compassion meditation with real-time visualizations of: breathing rate, level of approach motivation assessed from EEG frontal asymmetry, and dyadic synchrony of those signals between two users. We manipulated these interventions across eight separate conditions (dyadic or solo meditation; brainwave, breathing, both or no biofeedback) in an experiment with 39 dyads (N=8), observing the effect of conditions on self-reported experience and physiological synchrony. We found that each different shared biofeedback type increased users’ self-reported empathy and social presence, compared to no-biofeedback or solo conditions. Our study illustrates how dyadic synchrony biofeedback can expand the possibilities of biofeedback in affective computing and VR solutions for health and wellness.
- Adobe Systems. 2014. Photoshop CC. Retrieved from https://adobe.com.Google Scholar
- L. I. Aftanas and S. A. Golocheikine. 2001. Human anterior and frontal midline theta and lower alpha reflect emotionally positive state and internalized attention: High-resolution EEG investigation of meditation. Neurosci. Lett. (2001). Retrieved from http://www.sciencedirect.com/science/article/pii/S0304394001020948.Google Scholar
- Lauri Ahonen, Benjamin Cowley, Jari Torniainen, Antti Ukkonen, Arto Vihavainen, and Kai Puolamäki. 2016. Cognitive collaboration found in cardiac physiology: Study in classroom environment. PLoS One (14 July 2016).Google Scholar
- Lauri Ahonen, Benjamin Ultan Cowley, Arto Hellas, and Kai Puolamäki. 2018. Biosignals reflect pair-dynamics in collaborative work: EDA and ECG study of pair-programming in a classroom environment. Nat. Sci. Rep. 8, 1 (2018). DOI:https://doi.org/10.1038/s41598-018-21518-3.Google Scholar
- M. Alger. 2015. Visual design methods for virtual reality. Retrieved from http://aperturesciencellc.com/vr/VisualDesignMethodsforVR_MikeAlger.pdf.Google Scholar
- J. J. B. Allen, J. A. Coan, and M. Nazarian. 2004. Issues and assumptions on the road from raw signals to metrics of frontal EEG asymmetry in emotion. Biol. Psychol. (2004). Retrieved from http://www.sciencedirect.com/science/article/pii/S0301051104000377.Google Scholar
- C. Daniel Batson, Jim Fultz, and Patricia A. Schoenrade. 1987. Distress and empathy: Two qualitatively distinct vicarious emotions with different motivational consequences. J. Personal. 55, 1 (Mar. 1987), 19–39. DOI:https://doi.org/10.1111/J.1467-6494.1987.TB00426.X.Google ScholarCross Ref
- Mathias Benedek and Christian Kaernbach. 2010. Decomposition of skin conductance data by means of nonnegative deconvolution. Psychophysiology 47, 4 (2010), 647–658. DOI:https://doi.org/10.1111/j.1469-8986.2009.00972.x.Google Scholar
- Herbert Benson, Helen P. Klemchuk, and John R. Graham. 1974. The usefulness of the relaxation response in the therapy of headache. Headache: J. Head Face Pain 14, 1 (Apr. 1974), 49–52. DOI:https://doi.org/10.1111/J.1526-4610.1974.HED1401049.X.Google ScholarCross Ref
- Frank Biocca and Chad Harms. 2002. Defining and measuring social presence: Contribution to the networked minds theory and measure. In Proceedings of PRESENCE 2002. 7–36.Google Scholar
- Frank Biocca and C. Harms. 2003. Guide to the Networked Minds Social Presence Inventory v. 1.2: Measures of co-presence, social presence, subjective symmetry, and intersubjective symmetry. Retrieved from http://cogprints.org/6743/1/2002_guide_netminds_measure.pdf.Google Scholar
- Frank Biocca, Chad Harms, and Judee K. Burgoon. 2003. Toward a more robust theory and measure of social presence: Review and suggested criteria. Presence: Teleoperators and Virtual Environments 12, 5 (Oct. 2003), 456–480. DOI:https://doi.org/10.1162/105474603322761270. Google ScholarDigital Library
- Blender Foundation. 2017. Blender. Retrieved from https://blender.org.Google Scholar
- M. M. Bradley and P. J. Lang. 1994. Measuring emotion: The self-assessment manikin and the semantic differential. J. Behav. Therap. Exper. Psychi. 25, 1 (1994), 49–59.Google ScholarCross Ref
- J. T. Cacioppo, L. G. Tassinary, and G. G. Berntson. 2007. Handbook of Psychophysiology. Cambridge University Press, New York, NY.Google Scholar
- Kristine S. Calderon and Winifred W. Thompson. 2004. Biofeedback relaxation training: A rediscovered mind-body tool in public health. Amer. J. Health Stud. 19, 4 (2004), 185.Google Scholar
- Claudia Carissoli, Daniela Villani, and Giuseppe Riva. 2015. Does a meditation protocol supported by a mobile application help people reduce stress? Suggestions from a controlled pragmatic trial. Retrieved from http://www.liebertpub.com/cyber (Jan. 2015).Google ScholarCross Ref
- A. Chiesa and A. Serretti. 2010. A systematic review of neurobiological and clinical features of mindfulness meditations. Psychol. Medic. 40, 8 (2010), 1239–1252. DOI:https://doi.org/10.1017/S0033291709991747.Google ScholarCross Ref
- L. Chittaro and A. Vianello. 2014. Computer-supported mindfulness: Evaluation of a mobile thought distancing application on naive meditators. Int. J. Hum.-comput. Stud. (2014). Retrieved from http://www.sciencedirect.com/science/article/pii/S107158191300195X. Google ScholarDigital Library
- C. Coelho, J. G. Tichon, T. J. Hine, and G. M. Wallis. 2006. Media presence and inner presence: The sense of presence in virtual reality technologies. In From Communication to Presence: Cognition, Emotions and CultureTtowards the Ultimate Communicative Experience. IOS Press, 25--45.Google Scholar
- J. S. Coke, C. D. Batson, and K. McDavis. 1978. Empathic mediation of helping: A two-stage model.J. Personal. Soc. (1978). Retrieved from http://psycnet.apa.org/journals/psp/36/7/752/.Google Scholar
- A. D. Craig. 2002. How do you feel? Interoception: The sense of the physiological condition of the body.Nat. Rev. Neurosci. 3, 8 (Aug. 2002), 655–66. DOI:https://doi.org/10.1038/nrn894.Google ScholarCross Ref
- Benjamin M. P. Cuff, Sarah J. Brown, Laura Taylor, and Douglas J. Howat. 2016. Empathy: A review of the concept. Emot. Rev. 8, 2 (Apr. 2016), 144–153. DOI:https://doi.org/10.1177/1754073914558466.Google ScholarCross Ref
- B. D. Dunn, H. C. Galton, R. Morgan, D. Evans, C. Oliver, M. Meyer, R. Cusack, A. D. Lawrence, and T. Dalgleish. 2010. Listening to your heart–How interoception shapes emotion experience and intuitive decision making. Psychol. Sci. 21, 12 (2010), 1835–1844. DOI:https://doi.org/10.1177/0956797610389191.Google ScholarCross Ref
- Inger Ekman, Guillaume Chanel, Simo Järvelä, J. Matias Kivikangas, Mikko Salminen, and Niklas Ravaja. 2012. Social interaction in games: Measuring physiological linkage and social presence. Simul. Gam. 43, 3 (Oct. 2012), 321–338. DOI:https://doi.org/10.1177/1046878111422121. Google ScholarDigital Library
- Jennifer Edson Escalas and Barbara B. Stern. 2003. Sympathy and empathy: Emotional responses to advertising dramas. J. Consum. Res. 29, 4 (Mar. 2003), 566–578. DOI:https://doi.org/10.1086/346251.Google ScholarCross Ref
- Tiffany Field and Miguel Diego. 2009. Maternal depression effects on infant frontal eeg asymmetry. Retrieved from http://dx.doi.org/10.1080/00207450701769067 (2009).Google Scholar
- Diane Gromala, Xin Tong, Amber Choo, Mehdi Karamnejad, and Chris D. Shaw. 2015. The virtual meditative walk: Virtual reality therapy for chronic pain management. In Proceedings of the 33rd ACM Conference on Human Factors in Computing Systems. 521–524. DOI:https://doi.org/10.1145/2702123.2702344. Google ScholarDigital Library
- Eddie Harmon-Jones. 2003. Clarifying the emotive functions of asymmetrical frontal cortical activity. Psychophysiology 40, 6 (Nov. 2003), 838–848. DOI:https://doi.org/10.1111/1469-8986.00121.Google ScholarCross Ref
- C. Harms and F. Biocca. 2004. Internal consistency and reliability of the networked minds measure of social presence measure. In Proceedings of the 7th International Workshop: Presence 2004, M. Alcaniz and B. Rey (Eds.). Retrieved from http://cogprints.org/7026/.Google Scholar
- Hinterberger and Thilo. 2011. The sensorium: A multimodal neurofeedback environment. Adv. Hum.-Comput. Interact. 3. DOI:https://doi.org/10.1155/2011/724204. Google ScholarDigital Library
- Alicia J. Hofelich and Stephanie D. Preston. 2011. The meaning in empathy: Distinguishing conceptual encoding from facial mimicry, trait empathy, and attention to emotion. DOI:https://doi.org/10.1080/02699931.2011.559192.Google Scholar
- Stefan G. Hofmann, Paul Grossman, and Devon E. Hinton. 2011. Loving-kindness and compassion meditation: Potential for psychological interventions.Clin. Psychol. Rev. 31, 7 (Nov. 2011), 1126–32. DOI:https://doi.org/10.1016/j.cpr.2011.07.003.Google ScholarCross Ref
- C. A. Hutcherson, E. M. Seppala, and J. J. Gross. 2008. Loving-kindness meditation increases social connectedness.Emotion (2008). Retrieved from http://psycnet.apa.org/journals/emo/8/5/720/.Google Scholar
- Joris H. Janssen. 2012. A three-component framework for empathic technologies to augment human interaction. J. Multimod. User Interf. 6, 3 (Nov. 2012), 143–161. DOI:https://doi.org/10.1007/s12193-012-0097-5.Google ScholarCross Ref
- N. A. Jones, T. Field, M. Davalos, and S. Hart. 2009. Greater right frontal EEG asymmetry and nonempathic behavior are observed in children prenatally exposed to cocaine. DOI:https://doi.org/10.1080/00207450490422786.Google Scholar
- Simo Järvelä, Jari Kätsyri, Niklas Ravaja, Guillaume Chanel, and Pentti Henttonen. 2016. Intragroup emotions: Physiological linkage and social presence. Front. Psychol. 7 (2016). DOI:https://doi.org/10.3389/fpsyg.2016.00105.Google Scholar
- W. Klimesch. 1999. EEG alpha and theta oscillations reflect cognitive and memory performance: A review and analysis. Brain Res. Rev. (1999). Retrieved from http://www.sciencedirect.com/science/article/pii/S0165017398000563.Google Scholar
- W. Klimesch, M. Doppelmayr, and H. Russegger. 1998. Induced alpha band power changes in the human EEG and attention. Neuroscience (1998). Retrieved from http://www.sciencedirect.com/science/article/pii/S0304394098001220.Google Scholar
- Ilkka Kosunen, Antti Ruonala, Mikko Salminen, Simo Järvelä, Niklas Ravaja, and Giulio Jacucci. 2017. Neuroadaptive meditation in the real world. In Proceedings of the ACM Workshop on An Application-oriented Approach to BCI out of the laboratory (BCIforReal’17). ACM Press, New York, New York, 29–33. DOI:https://doi.org/10.1145/3038439.3038443. Google ScholarDigital Library
- Ilkka Kosunen, Mikko Salminen, Simo Järvelä, Antti Ruonala, Niklas Ravaja, and Giulio Jacucci. 2016. RelaWorld: Neuroadaptive and immersive virtual reality meditation system. In Proceedings of the 21st International Conference on Intelligent User Interfaces (IUI’16). ACM Press, New York, New York, 208–217. DOI:https://doi.org/10.1145/2856767.2856796. Google ScholarDigital Library
- Joseph J. LaViola, Jr.2000. A discussion of cybersickness in virtual environments. SIGCHI Bull. 32, 1 (Jan. 2000), 47–56. DOI:https://doi.org/10.1145/333329.333344. Google ScholarDigital Library
- R. W. Levenson and A. M. Ruef. 1992. Empathy: A physiological substrate.J. Personal. Soc. Psychol. 63, 2 (1992), 234. DOI:https://doi.org/10.1037/0022-3514.63.2.234.Google ScholarCross Ref
- Antoine Lutz, Heleen A. Slagter, John D. Dunne, and Richard J. Davidson. 2008. Attention regulation and monitoring in meditation.Trends Cog. Sci. 12, 4 (Apr. 2008), 163–9. DOI:https://doi.org/10.1016/j.tics.2008.01.005.Google Scholar
- Marek Malik, J. Thomas Bigger, A. John Camm, Robert E. Kleiger, Alberto Malliani, Arthur J. Moss, and Peter J. Schwartz. 1996. Heart rate variability—Standards of measurement, physiological interpretation, and clinical use. Eur. Heart J. 17, 3 (Mar. 1996), 354–381. DOI:https://doi.org/10.1093/oxfordjournals.eurheartj.a014868.Google ScholarCross Ref
- C. D. Marci, J. Ham, E. Moran, and S. P. Orr. 2007. Physiologic correlates of perceived therapist empathy and social-emotional process during psychotherapy. J. Nerv. 195, 2 (2007), 103--111. Retrieved from http://journals.lww.com/jonmd/Abstract/2007/02000/Physiologic_Correlates_of_Perceived_Therapist.1.aspxGoogle ScholarCross Ref
- M. Mori, K. F. MacDorman, and N. Kageki. 2012. The uncanny valley [from the field]. IEEE Robot. Automat. Mag. 19, 2 (June 2012), 98–100. DOI:https://doi.org/10.1109/MRA.2012.2192811.Google ScholarCross Ref
- Lindsay M. Oberman, Piotr Winkielman, and Vilayanur S. Ramachandran. 2007. Face to face: Blocking facial mimicry can selectively impair recognition of emotional expressions. DOI:https://doi.org/10.1080/17470910701391943.Google Scholar
- OpenViBE Developers. 2017. OpenViBE. Retrieved from http://openvibe.inria.fr/.Google Scholar
- Richard V. Palumbo, Marisa E. Marraccini, Lisa L. Weyandt, Oliver Wilder-Smith, Heather A. McGee, Siwei Liu, and Matthew S. Goodwin. 2016. Interpersonal autonomic physiology: A systematic review of the literature. Personal. Soc. Psychol. Rev. (Feb. 2016). DOI:https://doi.org/10.1177/1088868316628405.Google Scholar
- Mikko Salminen, Simo Järvelä, Ville Harjunen, Antti Ruonala, Giulio Jacucci, Juho Hamari, and Niklas Ravaja. [n.d.]. Evoking physiological synchrony and empathy using social VR with biofeedback. IEEE Transactions on Affective Computing. DOI:10.1109/TAFFC.2019.2958657Google Scholar
- Corina Sas and Rohit Chopra. 2015. MeditAid: A wearable adaptive neurofeedback-based system for training mindfulness state. Person. Ubiq. Comput. 19, 7 (2015), 1169–1182. DOI:https://doi.org/10.1007/s00779-015-0870-z Google ScholarDigital Library
- Martijn J. Schuemie, Peter van der Straaten, Merel Krijn, and Charles A. P. G. van der Mast. 2004. Research on presence in virtual reality: A survey. DOI:https://doi.org/10.1089/109493101300117884.Google Scholar
- C. D. Shaw, D. Gromala, and A. F. Seay. 2007. The meditation chamber: Enacting autonomic senses. In Proceedings of ENACTIVE/07 4th International Conference on Enactive Interfaces Grenoble, France, November 19th-22nd, 2007.Google Scholar
- R. Sudsuang, V. Chentanez, and K. Veluvan. 1991. Effect of Buddhist meditation on serum cortisol and total protein levels, blood pressure, pulse rate, lung volume and reaction time. Physiol. Beh. (1991). Retrieved from http://www.sciencedirect.com/science/article/pii/003193849190543W.Google Scholar
- Unity Technologies. 2017. Unity3D. Retrieved from https://Unity3d.com.Google Scholar
- G. A. Van Kleef. 2010. The emerging view of emotion as social information. Soc. Personal. Psychol. Compass 4, 5 (2010), 331–343.Google ScholarCross Ref
- John Waterworth and Eva Lindh Waterworth. 2004. Relaxation island : A virtual tropical paradise. interactive experience. Retrieved from http://www.diva-portal.org/smash/record.jsf?pid=diva2%3A154544&dswid=4532.Google Scholar
- Robert B. Welch, Theodore T. Blackmon, Andrew Liu, Barbara A. Mellers, and Lawrence W. Stark. 1996. The effects of pictorial realism, delay of visual feedback, and observer interactivity on the subjective sense of presence. DOI:https://doi.org/10.1162/PRES.1996.5.3.263. Google ScholarDigital Library
- R. L. Woolfolk, L . Carr-Kaffashan, T. F. McNulty, and P. M. Lehrer. 1976. Meditation training as a treatment for insomnia. Behav. Therap. (1976). Retrieved from http://www.sciencedirect.com/science/article/pii/S0005789476800640.Google Scholar
- Xueyan Xu and Stephanie Schuckers. 2001. Automatic detection of artifacts in heart period data. J. Electrocard. 34, 4, Part B (2001), 205–210. DOI:https://doi.org/10.1054/jelc.2001.28876.Google ScholarCross Ref
Index Terms
- Augmented Virtual Reality Meditation: Shared Dyadic Biofeedback Increases Social Presence Via Respiratory Synchrony
Recommendations
Bio-adaptive Social VR to Evoke Affective Interdependence: DYNECOM
IUI '18: Proceedings of the 23rd International Conference on Intelligent User InterfacesThe effects of bio-adaptation-based visual cues in conveying affective information was studied while conducting social meditation in a virtual reality (VR) environment. In a laboratory study, 22 dyads practiced a short duration, modified version of ...
RelaWorld: Neuroadaptive and Immersive Virtual Reality Meditation System
IUI '16: Proceedings of the 21st International Conference on Intelligent User InterfacesMeditation in general and mindfulness in particular have been shown to be useful techniques in the treatment of a plethora of ailments, yet they can be challenging for novices. We present RelaWorld: a neuroadaptive virtual reality meditation system that ...
Emotional Experience in Real and Virtual Environments – Does Prior VR Experience Matter?
HCI International 2023 – Late Breaking PapersAbstractVirtual reality (VR) has a great potential to induce emotions effectively and naturally. There are already existing concepts to use this technology, e.g., for the therapy of anxiety disorders or to train emotional and social skills for different ...
Comments