Unestablished Boundaries: The Capabilities of Immersive Technologies to Induce Empathy, Tell Stories, and Immerse

Many


Introduction
Research on immersive technologies (such as virtual reality [VR]) is dominantly based on technologically deterministic assumptions.These assumptions arise from understanding immersion solely as a quality of the technology (Cummings & Bailenson, 2016).Examples include equating immersion with simply wearing the head-mounted display, or HMD (Shen et al., 2021).These deterministic assumptions may also reduce embodiment to the mere use of an avatar (Li & Kim, 2021), even though embodiment, as a sense of owning the virtual body (Forster et al., 2022), involves multiple elements beyond avatar attributes (Gonzalez-Franco & Peck, 2018).Another assumption is that using a first-person avatar makes users "someone else" by sharing identities and/or bodies (Li & Kim, 2021;Tan et al., 2022), e.g., of an ethnic minority (Chen et al., 2021).
However, there's no evidence to suggest users become someone else; individuals remain the same people with their own selves and biases (Yee & Bailenson, 2007).By doing this, many VR studies also overlook the ethical side of their research.In particular, by placing users into "bodies" of someone else, "immersive" technologies potentially promote identity tourism, as a superficial play with identities of others (Nakamura, 2013), instead of true perspective-taking.All these issues also stem from an overarching approach that doesn't differentiate between mental and sensory (immersive) experiences.
In the end, this is an outdated approach that the communication field overcame by rejecting the linear model of communication and technological determinism (Hall, 1973), and this is what immersion (technologies) scholarship should overcome because technology does not tell us how to feel; instead, we process and interpret the incoming stimuli (Shaw, 2017).This may also include the exclusion of the term "immersive technology" from everyday use and replace it with a term that describes what these technologies factually do, instead of what users are expected to feel.The following article also advocates for diversifying the concept of immersion based on the type of experiences, differentiating between sensory and mental immersions, which would help to encompass diverse media interactions.To further promote understanding of the differences between different immersive experiences, I suggest that sensory and mental immersions should be connected to presence and narrative engagement, respectively.Overall, this would mean excluding narratives as objectives of the presence research to facilitate a clearer distinction between different types of immersion and experiences, as well as their effects and determinants.

Immersion Is a Psychological State, not a Quality of Technology (Boundary 1)
Immersion (via technology) is a complex psychological state that depends on various factors.The idea that the quality of the technology by itself can induce a particular experience no matter what the content is, aside from being problematically deterministic, is challenging (Calleja, 2014).Technology alone doesn't create immersion for users.Therefore, it is also incorrect to suggest that some technologies are either low or non-immersive technologies compared to others (Bailey & Bailenson, 2017).Everything depends on the context in which the technology is being perceived.From this and other perspectives, immersion (via technology) is not different from the sense of presence.
Technology plays a role in immersion because (certain types of) immersion is a product of interaction with or perception of it.It does not necessarily matter how well the system is designed, or how advanced it is-it has to be noticed, paid attention to, and perceived.In other words: A thing must be not only perceptible, but perceived.And in order to be perceived, a thing must be subject to some minimal allocation of perceptual attention, even if fleeting and non-deliberate.Put plainly, one could not expect to feel present in a virtual environment if one's eyes are shut and ears are plugged.(Murphy & Skarbez, 2020, p. 182) There are always individual factors in play too, whether it is a mood or some personality characteristics, such as an immersive tendency (Witmer & Singer, 1998).Also, if a person is not open or willing to be immersed via certain technologies, it will not happen (Sas & O'Hare, 2003).
How do we then describe the quality of such technologies?Riecke and Schulte-Pelkum (2015) suggested describing it as "immersiveness" or "the medium's ability to afford the psychological process of immersion" (p.205).It still does not describe the quality of the technology per se.In the end, this quality arguably refers to fidelity.In most general terms, fidelity is "the extent to which the simulation replicates the actual environment" (Liu et al., 2008, p. 92).This quality differentiates "immersive" technologies, but it does not define what these technologies are and what they do.
While these technologies do not necessarily immerse users or make them someone else, they do objectively simulate experiences, environments, activities, entities, and so forth.These technologies produce simulations, or, more particularly, interactive simulations (Brey, 2008;Søraker, 2011).Simulation, in general, models systems (Brey, 2008), or the system of signs from the original behaviour system (Frasca, 2013).Not all interactive simulations are designed to reproduce reality and its cues-although the high-fidelity ones usually do.High-fidelity simulations are supposed to be more "immersive" (Calleja, 2014), which, however, only means that these simulations replicate more sensorial cues compared to low-fidelity simulations.
So, even though it is not as appealing to the industry as "immersive technology," it is more accurate to refer to these technologies as "simulation(al)" (Kukshinov, 2023).In that sense, as a technological domain, simulation is similar to mediation.All media or mediation technologies have the potential to induce a sense of social presence, whether it is an email or a complex teleconference system (Kojima et al., 2021); however, what these technologies de facto all do is they all mediate social communication.It is also necessary to remember that VR, and related technologies, are not always supposed to immerse users (McVeigh-Schultz & Isbister, 2022).
As a psychological state, immersion should not be considered a mere "engagement," or a technical occlusion from the real world (as in, e.g., Tran et al., 2019).It is a vague and non-operationalizable approach, which also devaluates the immersion's meaning and significance.The value of immersion, as a conception and a psychological state, lies in its capability to blur or merge experiences between media/technology and the real world (Martínez, 2014;Snodgrass et al., 2013).As a result, something that is not real, whether a location or a social situation, is experienced to some extent as real.However, there are at least two ways for our cognition to "misjudge" reality via immersion like that, which I describe in the next section.

Differences Between Sensory and Mental (Immersive) Experiences (Boundary 2)
The possibilistic model of consciousness suggests that cognition consists of perception and imagination that mutually define what is real and what is not (O'Connor & Aardema, 2005, 2012;O'Connor et al., 2005).In short, perception is processing what is "there" to be perceived, while imagination processes what is "not there," or possibilities (O'Connor et al., 2005).For example, when we see an object from one of its sides, we do not assume that nothing exists on the back of this object-we mentally simulate some possibilities of what it can be.So, we make sense of reality with both imagination and perception.Therefore, there are at least sensory and mental ways not only to "judge" but also to "misjudge" reality.
For example, we can misperceive reality via illusions (Ramakonar et al., 2011).Presence, as a perceptual illusion of non-mediation (Lombard & Ditton, 1997) and/or non-simulation (Kukshinov, 2024), is not just any immersive state (Behm-Morawitz et al., 2016).Presence is a form of sensory immersion because it relies on the perceptual process, on sensory cues, or fidelity, i.e., on how well and coherently the simulation replicates some aspect of reality (Murphy & Skarbez, 2020).Something has to be simulated to be perceived.
Otherwise, if it is not simulated, if it is not "there," it is left to the imagination as a possibility.Possibilities or uncertainty are beneficial for mental immersion as they stimulate the imagination, which is key to mental immersion.
Presence, as a sensory illusion, happens at the moment of perceiving the simulated, i.e., it is impossible to be immersed in VR by memory.For presence, this spatiotemporal sequence of sensory cues needs to be credible before it can be lived-in or experienced as real (O'Connor & Aardema, 2012).For mental immersion, it is the opposite.Mental immersion does not involve external simulation that is being perceived-it is based on the mental simulation, for instance, of narratives.Narrative, such as a book, a movie, or a song, is not a simulation that is perceived in which we may feel immersed-it is perceived to be mentally simulated.
Narrative engagement can be described as a mental reconstruction of the narratives (Busselle & Bilandzic, 2009).The narrative unveils its structure in the imagination as it is only through this process that it attains credibility (O'Connor & Aardema, 2012).So, individuals need to live through narratives and engage emotionally to feel them as something meaningful.Also, mental immersion does not have to happen in the actual process of consuming a narrative-it can persist after consuming a narrative (Martínez, 2014).Along with technology, narratives do not inevitably immerse their consumers (who are not "users").
As a result, there are at least two types of immersion, i.e., sensory and mental, which can be represented by the states of presence and narrative engagement, respectively.A dual approach to immersion is most common in video game research.For instance, some researchers distinguished sensory and imaginative immersions or diegetic and situated immersions (as mentioned in Veale, 2012) or psychological and sensory/perceptual immersions (Carr et al., 2006)."Narrative" and "technological" conceptualizations of immersion are also common, but they focus on the potential source of the immersion, not on its cognitive nature as with "sensory"/"mental" conceptualization.Nilsson et al. (2016) also distinguished challenge-based immersion, which may arise from challenges to the user's motor or mental skills.However, this potential form of immersion seems to be very similar to the state of flow, as the authors themselves stated (Nilsson et al., 2016).In any case, based on the dual cognitive approach, the mental and sensory immersions are fundamentally different and, sometimes, contradict each other as they are based on different types of cognitive processing and forms of attention.

Storytelling and Narrative Issues
Such contradictions appear, for instance, when immersion into narratives impacts action possibilities, vection (i.e., perceived self-motion), reality judgment, and, consequently, presence (Balakrishnan & Sundar, 2011).Research also indicates that external factors, such as control over media content (Oh et al., 2014), can disrupt immersion into narratives (Busselle & Bilandzic, 2009).This happens because sensory immersion, or presence, which relies on active involvement and perception of external stimuli, can disrupt mental immersion, which is based on internal focus and mental simulations.
As a result, without differentiation of immersive experiences, presence scholars assume it is possible to feel presence "in" narratives.This may sound more plausible when researchers refer to audio-visual narratives, such as movies, because these types of narratives involve more sensory output.However, they are still narratives that need to be mentally processed for immersion to appear.When it comes to textual narratives, presence scholarship faces the so-called "book problem" (Gysbers et al., 2004;Schubert & Crusius, 2002).
It is a problem because it becomes difficult for presence researchers to justify "presence in books" via sensorial terms, level of fidelity, and other associated factors.However, books and novels can be the most (mentally) immersive narratives as they engage imagination more than narratives represented via richer media modalities, such as movies (Green et al., 2008).Simultaneously, there is no research on "presence in cartoons" because presence researchers probably recognize this specific boundary.Overall, simulations can induce sensory immersive experiences, while narratives can induce mental immersive experiences.
It is not correct to combine them, as narratives and simulations represent different types of media (Calleja, 2014).
The lack of this differentiation creates another assumption based on blurring the difference between presence and narrative engagement.This assumption implies a natural capability of "immersive" technologies to convey stories.However, this approach does not consider that storytelling, as a reproduction of narratives, can be immersive by itself, and that simulation technology does not improve but impedes storytelling.In short, "immersive" technologies do not allow users to follow the storyline once users have a choice of what to perceive.As a result, the simulative nature of "immersive" technologies impedes storyline comprehension (Pressgrove & Bowman, 2021), hindering recall of the story (Szita et al., 2018).However, story understanding and retention are pivotal for narrative engagement (Busselle & Bilandzic, 2009).As a result, regular flat screens, as media technologies, are better suited to consume narratives uninterruptedly and emotionally engage with them (Baños et al., 2004).
Video games are one of the most coherent examples of "immersive" or simulated storytelling (Gröppel-Wegener & Kidd, 2019) because they are designed as both simulations and narratives (Frasca, 2013).The story in video games is coded to progress depending on the players.There might be other examples of media, such as art installations, that make it possible to experience both sensory immersion from simulation and mental immersion into narratives.

Empathy, Perspective-Taking, and Identification Issues
There are more issues in video game research that stem from a lack of immersion differentiation.For instance, characters, as driving story elements (Eder, 2010), are often conflated with avatars (as in Downs et al., 2019), which are virtual representations of players in simulated environments (Nowak & Fox, 2018).This leads to mistaking character identification and avatar embodiment as the same concepts and experiences, even though they are distinct psychological processes.This further exemplifies the confusion between sensory (via avatar control) and mental experiences (via character relationships).However, when we imagine characters and their situations (even when we imagine ourselves with them), we cannot alter the course of events, as we do not control characters or possess any agency when we engage with narratives; characters also do not react to us, no matter how strongly we identify with them (Calleja, 2014).Those relationships are parasocial, or one-sided and imagined.
Arguably, because of this confusion, sometimes researchers assume that players identify with characters due to the fact of playing as them (as in Lin, 2013).However, identification requires mental effort-it is about an emotional connection with the character and the character's behaviour, motives, and desires (Busselle & Bilandzic, 2009).Characters, whether in books or video games, should be relatable, so that players can identify with them; it is not enough to play as them.Alternatively, embodiment is sometimes equated with the fact of using an avatar because, as sometimes suggested, the "body transfer" happens (Herrera et al., 2018;van Loon et al., 2018).However, embodiment does not simply happen when we use an avatar; it depends on the combination of various senses that are not always related to an avatar, such as a sense of location (Gonzalez-Franco & Peck, 2018).
In VR studies, in particular, the blurring of mental and sensory processes can become problematic.This happens because many VR researchers confuse the conception of a point of view (a sensory viewpoint) with a perspective of another person (a mental attitude and understanding of them), which produces an idea that VR is naturally capable of evoking perspective-taking just because we "see through the eyes of the other." Perspective-taking is based on the mental effort to make inferences about and represent others' intentions, goals, and motives (Stietz et al., 2019), or feelings and thoughts of others in terms of affective and cognitive perspective-taking respectively (Healey & Grossman, 2018).Affective empathy is usually described as the ability to share the emotional experiences of others, while cognitive empathy is usually equated to affective perspective-taking (Cox et al., 2012;Healey & Grossman, 2018).These are mental processes of relating to and thinking about the other, which does not mean that the user of a technology becomes someone else.In any case, no matter how much time a person spends "in" a virtual body, it does not (necessarily) transform them; it is still the same person, who has certain attitudes and biases.
As a result, for instance, when a racist person is "placed" in the "body" of a person of colour, who is facing virtually simulated abuse, this (a) does not make them a different person with different experiences, and (b) does not automatically make them less racist due to this intervention.Instead, users of these technologies remain themselves, along with any stereotypes and biases these individuals may have.If there are any biases attributed to certain simulated identities, then they are reproduced in such situations.This effect is called the Proteus effect (Yee & Bailenson, 2007).So, the question is not whether these technologies help to reduce biases, but rather how much reproducing these expressions of biases and stereotypes reinforces the biases by reliving and practising them in the virtual settings.
In addition to that, the whole premise of this type of research is problematic as it promotes identity tourism (Nakamura, 2013), a form of "superficial, reversible, recreational play at otherness…[based on] an episodic experience" (p.55).What generally happens in such studies is a reduction of the human lives and tragedies into the virtually visualized pieces of information posed as "real experiences."As Nakamura (2013) discusses further, this glimpsing at other people's lives does not entail any real consequences or impact for users: In cyberspace, players do not ever need to look for jobs or housing, compete for classroom attention, or ask for raises.This ensures that identity tourists need never encounter situations in which exotic otherness could be a liability….Players who represent as members of a minority may get the impression that minorities "don't have it all that bad," since they are unlikely to find themselves discriminated against in concrete, material ways.This imperfect understanding of the specific "real life" social context of otherness can lead to a type of complacency backed up by the seemingly unassailable evidence of "personal experience."(pp. 56-57) As a result, such studies are not only incorrect in their underlying assumptions-they can also be dangerous in regard to the effects they have on the participants by deepening the ignorance towards traditionally marginalized groups.

Discussion and Conclusion
"Immersive technology" is neither a useful nor a precise term.It reflects a technologically deterministic (and marketing-driven) framework that is currently dominant in the scholarship.Technology use is not equal to any particular psychological state, including immersion or immersive experiences.At the same time, scholarship and forms of immersive experience are not bounded by certain technologies-immersion is a complex and diverse psychological state.Simulation technology can induce sensory immersion or presence by presenting a coherent set of sensory cues.Narratives can induce mental immersion by mentally simulating narrative possibilities.In other words, there are different forms of immersion, and they often contradict each other.The relationship between the different types of immersions can be tested using mixed methods, especially if questionnaires, such as presence questionnaires, are context-dependent (Kukshinov et al., 2024) and require clarifications from the questionnaire respondents.Phenomenological analysis may be the best way to examine different immersive experiences in detail.
Even though sensory and mental immersions may contradict each other, their combination through "immersive" or rather virtually simulated storytelling is possible.Video games are a great example of combining narratives and simulations.Still, there is a necessity to maintain a constant balance between the focus on the simulation and engagement with narratives, or players' agency and characters' relationships.
Alternatively, documentaries can be a better content to represent via simulation technologies.It is possible to minimize the impact of the simulation by "reducing" the narrative structure of the content.In other words, more informative and less fictional content can be better suited for virtual and simulation technologies, as it is less immersive, or, in other words, extractive (Kukshinov, 2023), which refers to information as media content (Humphreys et al., 2013).
In addition, it may be possible to assign different modalities for the simulation and narrative parts.
In particular, it can be useful to use the audio channel for the narrative and the video channel for the simulation (i.e., a 360-degree video).As a result, it would be possible to visually stimulate a sense of space through a simulation and provide a mentally engaging audio-recorded narrative.This way, the story is not intervened by the sensorial input-it is rather supplemented.
Even though simulation technologies are not naturally designed to tell stories, these technologies are very impactful and useful in certain areas that require stimulating senses/sensations or vividly informing users via sensory immersion or presence.These technologies are effective in simulating situations to train and stimulate learning (Grassini et al., 2020) or to prepare for new circumstances (Lanzieri et al., 2020).VR is extremely useful, for example, in terms of VR exposure therapy, when it is necessary to simulate virtual contexts that are perceived as less intimidating than real ones to treat phobias (Price & Anderson, 2007).In the end, simulation itself and control over simulation can be fun and entertaining (Grodal, 2000).Simulation technologies may not make stories more involving, or people less racist, but these technologies are still incredibly impactful in their own way.It is necessary to understand the boundaries and use the benefits of these technologies to the fullest extent.It is also crucial to remember that it is the processing of these technologies that can have an effect on their users, not the mere fact of using them.Moving forward, it is imperative to refine our understanding of immersion beyond technological determinism and a unified conceptual framework.By embracing this, we can harness the transformative potential of the simulation technologies while addressing critical questions about their societal, ethical, and psychological implications.