Presence, flow, and narrative absorption questionnaires: a scoping review

Background: This is a review and analysis of the questionnaires most used in empirical research on psychological phenomena labelled as “presence,” “flow,” and “narrative absorption,” mostly for experiences mediated by technology (printed books, screens for games and films, and virtual reality). Overlapping concepts have been formulated in different fields according to specific disciplinary interests and based on knowledge within each field. Objectives: This review focuses on how language is actually used in questionnaire items, rather than on how concepts are formulated top-down and associated with corresponding linguistic expressions that become items of a questionnaire. The goal is to highlight similarities and overlaps in order to show a possible interdisciplinary agreement about the core aspects of the psychological states elicited by mediated experiences. Eligibility criteria: Questionnaires developed or used for research about VR, video games, films, or books have been selected for analysis. They should be available in English and used in empirical research since the year 2000. Sources of evidence: A search has been performed through Google Scholar and two other disciplinary bibliographies edited by international learned societies. Charting methods: The items of each questionnaire are categorized based on their wordings, and thus independently from the conceptual models within which they have been developed. Based on this categorization, various domains to which the items can be ascribed are identified (e.g. space, realism, agency, etc.) and psychological phenomena are linked to them (e.g. presence, social presence, narrative absorption, etc.). Results: 308 items in 23 questionnaires have been found to have overlapping of wordings. Conclusions: A list of the core aspects of presence, social presence, flow, and narrative absorption is presented, together with a critical selection of items suitable to measure each construct.


Rationale
Experiences mediated by technology (e.g.printed books, screens, and virtual reality) are studied across a variety of disciplines, often with little cooperation.Different theorizations, models, and empirical tools have been developed, resulting in a fuzzy agglomerate of related and overlapping concepts, like presence (Lombard et al., 2015), flow (Csikszentmihalyi, 1990;Harmat et al., 2016), and narrative absorption (Hakemulder et al., 2017).A scoping review is a suitable method to identify and summarize the core aspects of these various concepts, since they are currently obscured by the heterogeneity of disciplines investigating them.I surveyed the questionnaires most used in empirical research regarding this kind of psychological phenomena and I categorized the items in each questionnaire based on their wordings, thus independently from the conceptual models within which they have been developed.Overlapping concepts have been formulated in different fields according to specific disciplinary interests and based on knowledge within each field, this review focuses on how language is actually used in questionnaire items, rather than on how concepts are formulated top-down and associated with corresponding linguistic expressions that become items of a questionnaire.

Objectives
The goal is to highlight similarities and overlaps between questionnaires' items in order to identify which are the most relevant aspects of the psychological phenomena labelled as "presence," "flow," and "narrative absorption."Based on this categorization, I suggested the domains to which each group of items can be ascribed (e.g.space, realism, agency, etc.) and I associated them to the respective psychological phenomena for which they are more frequently used (e.g.presence, social presence, narrative absorption, etc.).

Protocols and registration
I followed Arksey and O'Malley's framework for scoping reviews (Arksey & O'Malley, 2005), refined by Levac et al. (2010) and the Joanna Briggs Institute (Peters et al., 2015).I reported findings following the PRISMA-ScR (Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews) checklist (Tricco et al., 2018).

Eligibility criteria
The sources considered are questionnaires available in English, no year limit has been used.To be included in the review, questionnaires need to have been developed or used for research about one of the following media: VR, video game, film, book.I only included questionnaires measuring psychological states, not those measuring personality traits or broader psychological concepts (e.g.state empathy has been included, but not trait empathy).Validation and statistical reliability were not necessary criteria.
Information sources I performed the search in May 2020, using three sources: the aggregator Google Scholar, the bibliography of the International Society for the Empirical Study of Literature (IGEL), and the measurement guides provided by the International Society for Presence Research (ISPR).Additional useful comparisons of presence-related concepts can be found in Paiva de Oliveira et al., 2016, van Baren & IJsselsteijn (2004), and Skarbez et al. (2017); for narrative absorption and similar concepts, see Busselle & Bilandzic (2017); for games, see Reddy (2016).

Search
The queries used in Google Scholar are: "presence questionnaire," "immersion questionnaire," "flow questionnaire," "narrative questionnaire," "narrative engagement," "narrative absorption," "narrative transportation." Selection of sources of evidence I obtained information about questionnaires directly from published articles and also from reviews included in Master theses or PhD dissertations.The criterion used to consider a questionnaire eligible as a source of evidence is its application in recent years: once I identified a questionnaire, I checked its use in research starting from the year 2000.I made this selection also with the help of a review of the questionnaires most used in VR research in the years 2016-17 (Hein et al., 2018)).

Data charting process
When multiple versions of a questionnaire were available, I considered only the most recent or shortest version, since this is likely to be an improvement over previous or longer versions, with respect to the goal of this scoping review.I then recorded each item of the data in a spreadsheet and manually annotated them.

Data items
Being a data-driven bottom-up review, I did not define any specific variables a priori.Rather, I analyzed all questionnaires' items.Among the total items in all the questionnaires studied, I only grouped and categorized the items for which I found close similarities and overlap of wordings.

Amendments from Version 1
I have reworded some parts, following a reviewer's (Christoph Klimmt) suggestion, to better present the goal and utility of this scoping review.In particular, I have highlighted the need for further theoretical reflection and empirical validation of the proposed conceptual and methodological systematization.I would have liked to include all of Klimmt's commentary (available at the end of the article) in the main body of the article because it is an excellent theoretical integration to the more practical work that I have done with my scoping review.I invite readers to read it.

Any further responses from the reviewers can be found at the end of the article
Critical appraisal of individual sources of evidence From a preliminary screening, I found that some items inquire about more than one aspect of the target experience.During the analysis, I identified all such items and excluded them from the synthesis of results, in order to avoid confusion with respect to the aspect covered by each type of item.

Synthesis of results
I compared the items of the selected questionnaires and grouped them according to similarities in the wordings used.For instance, the narrative absorption item "When I was finished with reading the story it felt like I had taken a trip to the world of the story" (Kuijpers et al., 2014) strongly resembles the spatial presence item "After my experience of the displayed environment, I had a sense that I had returned from a journey" (Lessiter et al., 2001).Once I have identified various clusters of items, I labelled each group and linked it to the most relevant psychological phenomenon.When items were already originally grouped in subdimension of the broader psychological construct, I used the subdimensions as guidance for the classification.

Selection of sources of evidence
The process of selection is outlined in Figure 1.

Characteristics of sources of evidence
The questionnaires analyzed are listed in Table 1.Out of the 23 questionnaires included in the analysis, 8 have been developed to measure presence, 3 for flow, 6 for game immersion/engagement, and 6 for narrative phenomena (absorption, engagement, transportation, immersion, identification with characters, and empathy with characters).
Critical appraisal within sources of evidence Some items present a combination of more than one aspect, so I excluded them from the synthesis of the results in order to avoid confusion within each group of items.For instance, the item "I lose perceptions of time and the real world surrounding me, as if everything just stops" (Game Immersion Questionnaire, Cheng et al., 2015) asks about the perception of both time and space.I also excluded items inquiring about some of the aspects identified when they have

Results of individual sources of evidence
Table 1 reports the number of items selected in each questionnaire.

Synthesis of the results
The complete categorization of the questionnaire items can be found in the underlying data.A summary of the most frequent categories is reported in Table 2. Attention is undoubtedly the most relevant term for all the constructs considered, conceived as disregard for both thoughts and perceptions that are not part of the activity eliciting presence, flow, or absorption.Similarly, a distorted perception of time is in many cases considered to be a sign of the occurrence of all the considered phenomena.
With respect to categories specific to each concept, spatial presence is characterized by items related to space, agency, and a comparison with reality not mediated by technology.Social presence is characterized by the same categories that are relevant for spatial presence (space and agency) but in relation to the existence of other agents; additionally, some kind of cognitive attention to the other and emotional arousal elicited by them are also frequent.Flow is specifically characterized by the perception of a sense of challenge.Narrative absorption is characterized by a comparison with non-mediated reality (in terms of vividness of imagery), by an easy comprehension of content, and by emotions and thoughts anticipating possible outcomes (suspense).Lastly, there are two groups of items explicitly asking about the user's perception of involvement/ engagement or absorption/immersion.

Summary of evidence
In all questionnaires, the most frequently recurring items concern attention and the sense of time.The isolation from external thoughts and perceptions is the main characteristic of presence-related phenomena, and such disconnection from stimuli unrelated to the undergoing experience probably leads to an alteration of the sense of time.Despite the evolution towards broad psychological conceptions of presence (Baños et al., 2000;Lee, 2004;Riva et al., 2015), a review (Hein et al., 2018) of the psychometric questionnaires used in VR research in the years 2016-17 found that the most used one is the Presence Questionnaire (Witmer & Singer, 1998), which heavily focuses on visual realism and naturalness of interaction.However, the broadest and most protracted collective effort aimed at clarifying how to measure presence (Hartmann et al., 2016;Vorderer et al., 2004) has excluded realism from the subdimensions of presence, keeping only "self-location" and "possible action" as core dimensions.Indeed, these two categories seem to be the two really specific to presence, since a comparison with non-mediated reality is also relevant for the "imagery" category, which concerns items related to narrative absorption.
Inquiring about the vividness of imagery or about the realism of a VR scene is a way to check how similar the imagined/mediated experience is to a non-mediated one.Both realism and vivid imagery are outcomes that can be associated with presence, but they are not particularly helpful to explain the underlying psychological processes that bring to the emergence of a sense of presence.
Many questionnaires also take into account the possibility that perceiving the existence of other agents can affect our sense of presence or, more broadly, that we can have intense experiences when interacting with others or following their actions.With a growing degree of complexity, such perception goes from merely noticing the existence of others, to interacting with them, to emotional and cognitive ways of responding to and understanding others' mental states.These groups of items, which I have associated with the concept of social presence, occur often together with spatial presence items and seem to entail it as the basis on top of which they can emerge.Indeed, they are all different expressions of a self-other relationship and can be conceptualized as forms of presence in co-participation.Analogously, questionnaires about flow experiences include items that I have here associated with spatial presence -and in some cases also items related to social presence -plus a specific group of questions regarding the perception of an experience as challenging.Similar wordings can be also found in items of narrative and game questionnaires.
Items that I specifically associated with the concept of narrative absorption regard imagery, the feeling of suspense triggered by the narrated events, and the comprehension of the content of the story, an aspect which can be connected to the sense of challenge of flow experiences, since the right match between the complexity of a story and the cognitive skills of the audience is relevant for narrative absorption.It is worth noting that questionnaires investigating narrative absorption include these three groups of items but also items related to spatial presence and social presence (with characters of a story), which can be considered subdimensions of narrative absorption.Given their metaphorical nature, items explicitly asking whether an experience elicited involvement, engagement, immersion, or absorption are not particularly useful for describing the psychological processes activated during the experiences they aim at qualifying.Moreover, the adjective "immersive" is used in VR research as a technical attribute of the medium -consistently with Sheridan seminal definition (Sheridan, 1992) -whereas in game and narrative studies it is a quality of the player or reader's experience (Jennett et al., 2008;Ryan, 2015;Stockwell, 2019).
Another popular but quite heterogeneous group of questions concerns the emotional impact of mediated experiences.Ten questionnaires investigate this aspect in slightly different ways, so it is hard to say whether emotional impact is a component of any of the presence-related phenomena or a secondary effect elicited by them.
The recognition presented can be used to reflect on the extent to which wording similarities among items from different questionnaires actually result from similarities between the underlying conceptualizations.One possible outcome is a cross-disciplinary systematization of concepts, suggesting viable options for an interdisciplinary agreement about the core aspects of the psychological states elicited by mediated experiences.To sum up, attention and time distortion are common to all the considered phenomena, and spatial presence (space and agency) is the phenomenon with the narrowest scope, the core.Social presence and narrative absorption are phenomena of increasingly broader scope, each of them including the listed phenomena of narrower scope.Flow is a concept transversal to the other three, being more related to the balance between a person's skills and the complexity of the stimulus, rather than to a specific psychological dimension.
Following the above-mentioned strategy, in Table 3 I summarized the conceptual overlaps that can be inferred from the similarities between items, and I recommend the subdimension that best correspond to the various groups of items.Additionally, in Table 4, I present a selection of items that best correspond to the categories identified by my inductive process.
The use of such items to measure presence, social presence, and narrative absorption can help to achieve a more solid epistemic comparability among research on these phenomena.
In order to benefit from previous statistical validations, in case of similarities, I gave preference to items coming from the same questionnaire.Depending on the task/content with which the participants are engaging, only a part of these items may be relevant.

Limitations
Categorizing only 308 items, out of the total 484 found in the sampled questionnaires, this scoping review may have missed some aspects of presence and related concepts that are important to grasp the nuances of the phenomenal experience that may be specific to certain media.However, by focusing on items showing a recurring intersubjective agreement between researchers and disciplines, I think I have successfully identified and summarized the core aspects of the surveyed phenomena.However, it is worth remembering that the employment of measurement tools should always be justified by theoretical reflection and empirical validation.A scoping review is an aid for the systematization of knowledge, but it also produces new knowledge that requires further scrutiny and methodological testing before it can be deployed into experimental settings.

Conclusions
The categorization proposed here can be used to further refine existing questionnaires and possibly encourage a convergence of different disciplines towards a use of the same items, so that insight coming from different fields could be used for the advancement of knowledge in specific areas.For instance, empirical research on narrative could benefit from using existing items for presence and social presence, without "reinventing the wheel" and focusing rather on refining how to measure dimensions like suspense and imagery.Moreover, a shared agreement on basic items will enable better and more informative meta-analyses, as well as comparative media studies, a kind of research that is strongly relevant for all the disciplines that I mentioned here, since only a comparison between experiences with different media can help to account for the specificity of presence and related phenomena.This is such a fantastic article for its ability to synthesize the base information needed to determine the kind of questionnaire that may be required, or could be useful, for a lot of basic research in the articulated fields.I wish that this was available when I first started my research in narrative absorption!I hope I can add something meaningful beyond the other two excellent reviews.

Extended data
I just have a few notes for minor revisions or future considerations: There's a slight contradiction in the eligibility criteria section, "available in English, no year limit has been used."You do mention a year limit of 2000 in the abstract. 1.
I'm also interested in the data charting process.For the use of the shortest version of the questionnaire, is there any documentation for what may be lost?There is almost certainly something lost in many questionnaires for deciding to use the short version over the long version, is there anything of note here?2.
For "data items", was "close similarities" determined through inter-rater standards, or just what looked "close"?Perhaps this is just a nuance of the scoping method and can be ignored.

3.
For "critical appraisal of individual sources of evidence", is there any affordance for bivalent or complex experiences?Might this be included in a future study?

4.
Were all removed articles (or questionnaires -n=24) removed because of complexity?It would be useful to have a better sense of why these articles were removed.

5.
This is a fantastic first step, but I would love to see this article have at least one follow-up article -a tracing of theory in relation to linguistic use, as Christoph Klimmt points out, but it would also be useful to see how these concepts align with other pragmatic considerations (style of study, goal of study, etc.).This would not only make it easier for researchers to know which questionnaire might work best for their needs, but it would also allow for an easier way to branch out into related, but perhaps slightly different methodologies.The only reason I say, "partly" for "Are the conclusions drawn adequately supported by the results presented in the review?" is that I would like to see a little more discussion regarding future studies for how this work may be built upon and made into a study that makes that extra connection with practical use and theory.

6.
Are the rationale for, and objectives of, the Systematic Review clearly stated?Yes

Are sufficient details of the methods and analysis provided to allow replication by others? Yes
Is the statistical analysis and its interpretation appropriate?Yes

Are the conclusions drawn adequately supported by the results presented in the review? Partly
Competing Interests: No competing interests were disclosed.
I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Version 1
Reviewer Report 01 June 2021 https://doi.org/10.21956/openreseurope.14347.r27020 Department of Journalism and Communication Research, Hanover University of Music, Drama and Media, Hanover, Germany The motivation behind Federico Pianzola's analysis is the enormous number of questionnaires and scales that have been developed across various fields in which scholars investigate media users' state of being 'captured', 'absorbed', 'transported' or 'involved' by a message and / or its delivery technology.Because much media innovation aims at improving or intensifying such user experiences, thematic research is thriving and very productive.As a consequence, a hard-tooverview landscape of empirical measures has evolved, and oftentimes, these measures obviously utilize similar-sounding questions and items for assessing (presumably) different concepts.
Pianzola offers a scoping review of the items used in some 20+ such measures to investigate the extent to which such "overlaps" occur, to gain a better orientation of the measures available, and to derive recommendations about which measure to use for which concept.
The author's goal of serving an interdisciplinary research community that is confused by the many different published questionnaires with orientation knowledge is relevant and laudable, and the overview of measures, items and linguistic similarities among items is highly valuable.However, from a psychological-methodological point of view, the idea of scoping the items of different questionnaires *across* and *independent of their* theoretical concepts, is a kind of "reverse engineering" -because Pianzola tacitly tries to build an integrative *theory* of media absorption by synthesizing the *measures* used for different concepts.This does, in my humble view, not work at all.
First, the key reason why there are so many different questionnaires and measures out there is that there are so many concepts and different elaborations of the same concepts (e.g., there is a lot of competing understandings of "Presence"), and various author teams have tried to establish their conceptual understanding and an according measure.The large number of measures available on the market is thus a consequence of theoretical diversity (and also some chaos) -a normal situation in the social sciences in which there is no standard definition of entities, variables, or phenomena as it can be found in natural (hard) sciences.For instance, in social psychology, a huge number of "theories" is circulating, many of which differ from each other in similarly small nuances as conceptualizations of "Presence" differ in the current context.The multiplicity of similar concepts and of elaborations of the same concept is a notorious problem, and Pianzola is right that in the case of media absorption, in which many scholars from diverse fields are interested, the resulting complexity and chaos is particularly undesirable.-But this chaos cannot be resolved by distilling some 'optimal' choice of questionnaire items or scales from the abundance of available measures.This is because measures in psychology are developed *based on their theoretical foundation*.Using a specific conceptualization of, say, Spatial Presence, one author team has created their items to measure Presence and has done (more or less, high-or less-high-quality) research to validate their measure (against the background of their theory).It is certainly NOT the case that all research teams who have developed a questionnaire of Spatial Presence had the SAME basic conceptual understanding of Spatial Presence in mind when they crafted their items.It is therefore misleading that Pianzola writes "this review focuses on how language is actually used in questionnaire items, rather than on how concepts are formulated top-down and arbitrarily associated with corresponding linguistic expressions that become items of a questionnaire" -no author team would accept the assumption that they have "arbitrarily" linked item wordings to a concept; on the contrary, most authors have invested much thinking in finding item wordings that match their conceptual understanding.As a consequence, Pianzola's (implicit) idea that we can reshuffle items from different measures to come up with best-choice approaches to assess certain concepts or conceptual dimensions disconnects each questionnaire from its individual theoretical base.This is not how social-scientific measurement works -it must always be "theory firstmeasurement second".Pianzola, however, disregards theoretical differences and suggests a kind of meta-measures based on wording similarities.Such wording similarities among items from different questionaires, however, actually result from similarities between the underlying conceptualizations, and maybe in some cases, from poor operationalizations (i.e., items badly formulated so that they accidentally overlap with items validly designed to measure something profoundly different).
So I express a warning not to step over the necessity of theoretical integration and synthesis by simply searching for "good measures" based on linguistic similarity analysis.Measures only "work" within the context of their underyling theoretical substance, and this substance differs between all questionnaires, which cannot be ignored when comparing (or combining) measures.Hence, for epistemological reasons, the main objective that Pianzola is pursuing, cannot (and should not) be achieved.
But the author's review of the many similar-yet-different measures of the many similar-yet-(maybe?)-differentconcepts is nevertheless of great utility!First, the mere listing of available measures and the accompanying descriptions help to maintain an orientation of the existing diversity.And second, this scoping review must be understood by many scholars active in the field as a reminder that the notorious diversity and chaos that results from competing conceptualizations and individual desires to establish one's own theory and measure (in spite of the existence of many other similar theories and measures) is a huge problem.The precise analysis of which items and questionnaires display which kind of linguistic overlap in spite of (alleged) conceptual discrepancies is a great contribution by Pianzola, because it helps to detect those spots in media absorption research that seem to require particular effort of theoretical rethinking, integration, and synthesis.Practically speaking, scholars may refer to this scoping review when articulating (and justifying) their individual decision to use certain conceptualizations (and according measures) in their specific study, thus acknowledging the undesirable plurality of available concepts and measures and making an informed selection decision at the same time.So I encourage the author of this paper and all readers to reflect on the theory-measurementrelationship and to develop conclusions on how the highly diverse, cross-disciplinary research on media absorption phenomena can cope with the theoretical-methodological diversity and still maintain strong standards of excellence in social-scientific (self-report) measurement.
Clearly, this question is a key challenge that hopefully will inspire many young scholars to work towards greater theoretical integration, harmony, and parsimony.Based on my own experience with measuring user states that occur during media exposure but that are only assessed cumulatively after exposure, I end with the recommendation of not being too ambitious regarding the ability of assessing (theoretically) fine-grained differences in experience this way.Language that is comprehensible to laypersons is hardly capable to make experiential nuances (e.g., between "transportation" and "spatial presence") distinguishable.So one issue that Pianzola inspired me to reflect on is this specific aspect of the theory-measurement-relationship: Small theoretical discrepancies may simply not be possible to translate validly and precisely into item wording differences.So maybe media absorption research is in need of a new pragmatism in measuring concepts *in spite of* conceptual diversities.Authors of (slightly) differing concepts should thus feel invited to team up and debate whether they can agree on an integrated measure that would fit *both* their concepts.Now that would be a pathway of "bottom-up" integration of measures and hence a potentially viable way to move forward into the direction that Pianzola has shown to us.
Are the rationale for, and objectives of, the Systematic Review clearly stated?Yes Are sufficient details of the methods and analysis provided to allow replication by others?Yes Is the statistical analysis and its interpretation appropriate?Yes Are the conclusions drawn adequately supported by the results presented in the review?Partly Competing Interests: No competing interests were disclosed.
I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Some minor observations:
In the Rationale (if there is space): the text states: "Overlapping concepts have been formulated in different fields according to specific disciplinary interests and based on knowledge within each field".This is absolutely true, and it is the main problem tackled in this work.Could the author provide one or two examples of these different definitions? 1.
In the third paragraph of the discussion, change "and the comprehension of the content of the story" to "the understanding of the story's narrative and characters".

2.
The idea of "reality non mediated by technology", which appears in the synthesis of results, 3. refers to "actual, physical reality", as opposed to its virtual counterpart.Is this correct?Could the author include one or two sentences explaining the difference between mediated and non-mediated reality, and why is this different from differentiating "virtual" from "real"?Or, at least, provide a reference for further reading?None of these notes take away the relevance and quality of this work.I congratulate the author and have no further comments on this article.I agree that understanding characters is often considered an important aspect of subjective-phenomenal states related to narrative. 2.
I will elaborate on the difference between mediated and non-mediated experiences, a topic which I treated more at lengths in a complementary article based on this scoping review: Pianzola, F., Riva, G., Kukkonen, K., & Mantovani, F. (2021)

Figure 1 .
Figure 1.Flow diagram of the selection of sources process (adapted from Moher et al., 2009).
Are the rationale for, and objectives of, the Systematic Review clearly stated?YesAre sufficient details of the methods and analysis provided to allow replication by others?YesIs the statistical analysis and its interpretation appropriate?YesAre the conclusions drawn adequately supported by the results presented in the review?YesCompeting Interests: No competing interests were disclosed.Reviewer Expertise: I have worked with presence and flow, although I currently do not use these terms anymore (due to the exact reasons stated in this article).Currently, I work with technologymediated musical interactions and this article is highly interesting to my research.I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.Author Response 15 Jul 2021Federico PianzolaThank you very much for your positive feedback.Regarding your comments: I will add examples of how different definitions of the same concept are based on specific disciplinary knowledge.1.

Table 4 . Selection of questionnaire items (with minimal adaptation) recommended to achieve a more solid epistemic comparability among research on presence, social presence, and narrative absorption
. (R = reverse scored).