Head-mounted augmented reality to support reassurance and social interaction for autistic children with severe learning disabilities

1 Introduction

Autism is a neurodevelopmental condition characterized by social communication and interaction difficulties, repetitive patterns of behavior, sensory issues, and focused interests (American Psychiatric Association, 2013). The significance of these features varies along a spectrum. Indeed, some autistic people¹ can display Mild Learning disabilities and low support Needs (MLN), with difficulties initiating social interaction or performing academic tasks. Others can display Severe Learning disabilities and complex Needs (SLN), sometimes with associated Intellectual Disability (ID), involving limited communicational abilities (some being non-verbal) and significant difficulties participating in everyday activities (Bottema-Beutel et al., 2021).

While 50% of autistic people have associated ID, 94% of studies published in 2016 overlooked them (Russell et al., 2019). This gap stems from three main causes (Spiel et al., 2017; Tager-Flusberg et al., 2017; Russell et al., 2019): autistic people with SLN and/or ID are hard to reach, they can display challenging behaviors, and their limited verbal abilities generate methodological difficulties (e.g., in data collection and analysis). To analyze children’s experiences, studies advise collecting data from multiple sources (Dechsling et al., 2021) and considering the overall context (Spiel et al., 2017) (e.g., place and time). However, due to the under-representation of children with SLN and/or ID in studies, challenges remain in analyzing the quality of their experiences.

According to Happé and Frith (2020), technological advances can help to better include autistic individuals with SLN. However, most studies directly target social abilities with a rehabilitation focus, thus being inaccessible to children with SLN (Sandgreen et al., 2021). Many practitioners already use digital tools (e.g., tablets) (Sandbank et al., 2020) to complement their rehabilitation strategies (Sandgreen et al., 2021) or to support children’s daily needs (Mankoff et al., 2010; Ringland et al., 2019). Indeed, digital tools are promising because they are predictable, secure, individualizable, controllable, and often appealing (Sandgreen et al., 2021). However, they lack multisensory capabilities and, as such, are often ill-suited to extend the sensory approaches used for autistic children with SLN (Bauer et al., 2023).

Although recent reviews suggest that Virtual Reality (VR) and Augmented Reality (AR) could complement the existing approaches that are used for autistic individuals (Bozgeyikli et al., 2018; Berenguer et al., 2020; Karami et al., 2021), people with SLN remain under-studied (Karami et al., 2021; Bauer et al., 2023). In addition to the advantages of other digital tools, VR and AR display immersive multisensory interactive capabilities. While VR displays an entirely synthetic environment, AR supports adding virtual elements to the child’s surroundings while still seeing a human practitioner. In light of this capability, our previous analysis of stakeholders’ needs suggested using AR sensory-based mediation approaches for autistic children with SLN within clinical settings. Such approaches aim first to secure children through free-play activities and then reinforce the dyadic child-practitioner relationship (Bauer et al., 2023). Following these insights, the environment Magic Bubbles was designed in collaboration with eleven practitioners (Bauer et al., 2021). Pre-tests were conducted with ten children with neurodevelopmental conditions and ID to measure acceptability and usability (Bauer et al., 2022). To finally validate its potential to prompt reassurance² and social interaction, we conducted a long-term study with seven autistic children in a day hospital setting. Three research questions were addressed: (RQ1) To what extent can Magic Bubbles secure autistic children with SLN? (RQ2) To what extent can Magic Bubbles prompt the dyadic relationship between an autistic child with SLN and a practitioner? (RQ3) What is the overall quality of experience for autistic children with SLN when using Magic Bubbles?

This paper has three main contributions: empirical, methodological, and theoretical. First, empirical insights from a long-term AR field study with autistic children with SLN are reported. Then, it is one of the first AR studies focusing on autism to our knowledge that uses mixed methods (qualitative and quantitative) mainly relying on the grounded theory methodology (Glaser and Strauss, 1967; Charmaz, 2006). At last, a categorization of children’s experiences is presented that could inform future AR studies focusing on autism.

After presenting the related works and methodology, findings are detailed following the concepts and categories emerging from the grounded theory. At last, findings are discussed, and perspectives are drawn.

2 State of the art

2.1 Existing autism sensory approaches

Early, structured, and individualized approaches are recommended to address the heterogeneous needs of autistic children (Sandbank et al., 2020). In particular, clinic-based sensory strategies are often used, with two main objectives. First, practitioners can train the sensory integration of interactive processes to enhance developmental and cognitive abilities. To that end, Sensory Integration Therapy is often used for individuals across the entire spectrum with playful activities including various objects (e.g., sensory balls) (Ayres, 1972; Schoen et al., 2019). Second, sensory environments or tools are used to secure and engage children, as a prerequisite for accessing social interaction. To that respect, the Snoezelen approach is common, consisting of a multisensory space often including bubble columns, with promising outcomes for autistic individuals with SLN (Lancioni et al., 2002; Novakovic et al., 2019). Nevertheless, providing optimal care for autistic children remains challenging, particularly for those with SLN (Kouo and Kouo, 2021). Challenges can come from a lack of flexibility in the tools or environments, a lack of resources, time constraints, or cost, thus preventing practitioners from using some equipment or hindering children’s engagement over time (Kouo and Kouo, 2021). This study presents a technology-based approach via AR that intends to complement common sensory approaches used with autistic children with SLN.

2.2 Using AR headsets for autistic children with severe learning disabilities

Recent reviews suggest that AR is well accepted and used by autistic children, but children with SLN are under-represented (Berenguer et al., 2020; Khowaja et al., 2020; Karami et al., 2021). This could be due to a common focus on training socio-emotional abilities (Bauer et al., 2023), or to using too restrictive exclusion criteria (Karami et al., 2021). Indeed, most studies exclude participants with “critical contributing factors that could affect the outcome” (Karami et al., 2021, p. 21), such as restricted and repetitive behaviors, which are very common among individuals with SLN. It is worth noting that the under-representation goes beyond the autism field. Indeed, in Bailey et al. (2021)’s review about VR and AR applications for individuals with communication disabilities or neurodevelopmental conditions, only two out of 53 studies evaluated the feasibility and barriers of using VR/AR for individuals with ID. Thus, this paper complements previous findings by focusing on the AR experience of autistic children with SLN.

Most autism AR studies use phones or tablets without immersive multisensory capabilities (Berenguer et al., 2020; Khowaja et al., 2020; Karami et al., 2021). To our knowledge, only four studies used AR headsets for individuals with autism or a related neurodevelopmental condition, and associated ID (Washington et al., 2017; Aruanno et al., 2018; Daniels et al., 2018; Sahin et al., 2018). Aruanno et al. (2018) tested the Hololens with adults with neurodevelopmental conditions with positive results over acceptability and engagement. Sahin et al. (2018) showed the feasibility of using smartglasses for an autistic child with SLN. Daniels et al. (2018) and Washington et al. (2017) suggested that smartglasses that provide real-time social cues are promising for training socio-affective abilities for autistic children with SLN. Our paper complements these findings by using an AR video see-through display for autistic children with SLN.

2.3 Designing multisensory AR autism approaches

Designing meaningful AR environments for autistic individuals requires working with stakeholders to understand their specific needs (Parsons et al., 2020; Bauer et al., 2023). In particular, findings from 34 interviews with stakeholders, mainly including practitioners, suggest using AR sensory-based mediation approaches to complement existing approaches for children with SLN (Bauer et al., 2023). Such AR approaches must also support collaboration between a child and practitioner, as advised by Cavus et al. (2021). Furthermore, they must be conducted in familiar spaces, not in laboratories (Bauer et al., 2023). This focus is close to Happé and Frith (2020)’s statement: “Sensory issues would have been a major scientific focus much sooner if researchers had been working more closely with autistic people.” It also recalls the “positive technology” approach (Riva et al., 2012), which intends to enhance the quality of experience, adaptive behaviors, and positive functioning by focusing on affective quality, engagement, and connectedness. Our paper investigates the use of a sensory-based mediation approach within a clinical setting for autistic children with SLN.

While AR sensory-based mediation approaches are under-researched, some digitally-augmented multisensory spaces were designed in collaboration with clinical teams for autistic children, with successful outcomes over wellbeing, emotional regulation, engagement, and the dyadic relationship (Pares et al., 2005; Ringland et al., 2014; Mora-Guiard et al., 2017; Garzotto and Gelsomini, 2018; Basadonne et al., 2021). Various settings were used, i.e., a large space where physical items are augmented to trigger stimuli depending on users’ actions (Pares et al., 2005; Garzotto and Gelsomini, 2018; Basadonne et al., 2021), a projected floor (Mora-Guiard et al., 2017), or a projected screen and a Kinect camera (Ringland et al., 2014). Nevertheless, the equipment used is often bespoke, bulky, or too expensive for many clinical institutes for widespread adoption. Our approach that uses an AR headset intends to address these issues.

3 Materials and methods

To address the recommendations and findings mentioned above, we designed and developed an AR multisensory environment called Magic Bubbles, which seeks to secure autistic children with SLN and reinforce the dyadic relationship with their practitioner. Then, to answer our research questions about the feeling of being secure (RQ1), the dyadic relationship (RQ2), and the quality of experience (RQ3), we conducted a long-term study in a day hospital with seven autistic children. Section 3.1 introduces the design and development of this environment, and Section 3.2 presents the experimental design.

3.1 Presentation of the AR environment Magic Bubbles

3.1.1 User interactions

Magic Bubbles drew upon an analysis of 34 autism stakeholders’ interviews that suggested designing AR sensory-based mediation approaches to complement practitioners’ approaches with autistic individuals with SLN (Bauer et al., 2023). Then, it was adapted for a day hospital setting, where pre-tests with eleven practitioners validated its acceptability and usability for a clinical team (Bauer et al., 2021). After that, pre-tests with ten children with neurodevelopmental conditions and associated intellectual disabilities validated its acceptability and usability for autistic children with SLN (Bauer et al., 2022).

Children can interact with audio, visual, and tactile stimuli (e.g., vibrations) in an augmented 3×3 m space while seeing their practitioner (see Figure 1). Stimuli are displayed through appealing objects (e.g., bubble column), being either generic (e.g., soap bubble) or individualized (e.g., drawings). Practitioners can support exploration (e.g., verbal guidance) while perceiving what the child sees and hears through a screen monitor. Two conditions exist for the child, drawing upon clinical practices: spectator and actor. In the spectator condition, only practitioners can trigger the stimuli while the child can move in space. In the actor condition, the child can directly interact with everything and move in space. More information about the design is available in a previous paper (Bauer et al., 2021). Video presentations of the design (Bauer et al., 2021), pre-tests (Bauer et al., 2022), and some critical incidents that occurred during the long-term experiment presented in this paper are available at this link: https://valentinbauer.com/research/doctorat.

FIGURE 1. Design of Magic Bubbles Game—Content (A, B, C): 1 Bubble; 2 Bubble column; 3 Panel; 4 Water pond; 5 Music bubble; 6 Image panel; 7 Drawing panel; 8 Recording Bubble—Practitioner’s user interface (D): 1 Add/Remove objects; 2 Trigger stimuli; 3 Add/Remove simple feedback; 4 Display contextual information; 5 Show/Hide the practitioner’s user interface.

3.1.2 Apparatus

An AR video see-through system is used, with a Zed-Mini camera on a HTC Vive Pro headset (Figure 3 shows children using the system). The system enables adding virtual objects onto a video stream of the real environment. Two HTC Vive controllers and four Vive Lighthouse outside-in tracking systems are used. The size and weight of the HTC Vive headset (563 g) are not considered to be an issue for autistic children aged more than ten, as Newbutt et al. (2020)’s study with 31 autistic children with MLN aged from 6 to 16 presented positive outcomes in terms of acceptability, usability, and enjoyment. Though, to our knowledge, very few headset-based AR/VR studies have focused on individuals with SLN. In Garzotto et al. (2017)’ study, five children aged 6–10 years old with neurodevelopmental conditions tested two VR applications with a Google Cardboard headset. Findings suggested positive acceptability and usability, but two autistic children with SLN initially did not want to wear it. In Cai et al. (2013)’s study, fifteen autistic children interacted with virtual dolphins in a room-centric VR environment while wearing 3D shutter glasses. Despite encouraging outcomes, seven children with SLN and associated ID had difficulties wearing the glasses. Five children with SLN also required much parental guidance to interact, and four others were overwhelmed despite such guidance. Such challenges may be due to widespread tactile issues in autism (Robertson and Baron-Cohen, 2017). Therefore, when devising our protocol (see below), we considered any sign of physical discomfort that could arise with AR/VR headsets.

For our study, we decided to use this AR video see-through display in light of the requirements needed to work at the day hospital, as discussed with the clinical team and described in a previous paper (Bauer et al., 2021). In particular, this device was more robust and had a wider field of view than other AR headsets (e.g., Hololens). Moreover, Aruanno et al. (2018) created an AR application with the Hololens headset to teach domestic tasks. Fourteen adults with neurodevelopmental conditions and ID had positive acceptability and usability. However, six adults with the most severe disability “had enormous difficulties” regarding understanding and air-tap gestures. Therefore, to validate the suitability of our apparatus for such children, preliminary testing was conducted with eleven practitioners at the day hospital (Bauer et al., 2021). Preliminary testing with ten children with neurodevelopmental conditions also ensured that Magic Bubbles was positively accepted, easily used, and generated enjoyment (Bauer et al., 2022). Even if we did not find AR/VR studies focusing on cybersickness or sensory issues for autistic children with SLN at the time of our study, we adopted a protocol that intended to minimize such risks as described below (e.g., two psychologists who knew the children always being present to detect any sign of discomfort).

3.2 Experimental design

The experiment was conducted with autistic children with SLN at the day hospital André Boulloche since two researchers from our team also worked there as clinical psychologists and had known the children for a long time. This section describes the demographics, procedure, and data collection and analysis.

3.2.1 Demographics

Seven autistic children with associated intellectual disabilities (MA: 11 ± 1.15) were recruited among the day hospital patients, in agreement with the clinical team. They included six boys and one girl. This low number is due to the following inclusion criteria: the autism diagnosis and level of learning disabilities had to be confirmed according to the International Classification of Diseases version 10 (ICD-10) (World Health Organization, 1993) and the Childhood Autism Rating Scale (CARS-2) (Schopler et al., 2010), children had to be aged between 10 and 14, and not to display known symptoms of epilepsy. Regarding the age range, while some manufacturers (Oculus) recommend using AR/VR headsets from 13 years old, Newbutt et al. (2020)’s study suggested that headsets could be used from 6 years old for children with MLN if a suitable protocol ensures their safety. As children at the day hospital had SLN, it was decided with the clinical team to only recruit them from 10 years old and to devise such a protocol.

After children were recruited, investigators got access to their clinical evaluations. The psychologists conducted new evaluations if they were absent or too old. Cognitive profiles were measured using the Kaufman Assessment Battery for Children (KABC), with the Mental Processing index (MPI) (Luria model) or Non-Verbal Index (NVI), respectively, for verbal and non-verbal children (Rozenblatt, 2011). Moreover, semi-structured interviews were conducted with the psychologists to individualize the environment and understand children’s usual behaviors. They focused on: individualized appealing elements, expected children’s and caregivers’ behaviors, usual repetitive behaviors, attitude towards digital tools, ability to express anxiety, and anxiety markers. The two psychologists asked the educators for additional information when needed. For privacy issues, children’s names were anonymized by using identifiers such as IDX (where X is a number). The identifiers do not go from 1 to 7, as some were already assigned during the pre-tests (e.g., ID5 is a non-autistic child who participated in the pre-tests) (Bauer et al., 2022).

Table 1 summarises children’s profiles, and Table 2 their expected behaviors and the respective design choices. All children have SLN, as five have “severe” autism, and two have “moderate” autism, as stated in the CARS-2 evaluation. Six children are verbally limited, and ID4 is non-verbal. Based on KABC ratings, six children have significant or moderate (ID13) cognitive impairments. All children need structuration. They are expected to be able to express their anxiety verbally (ID1, ID3, ID10, ID13, ID3) or non-verbally (all). To do so, they could use autistic behaviors, e.g., echolalias (repetitions of words said by another person) or stereotypies (repetitive actions). Regarding sensory issues, only ID2 can be sensitive to sounds (e.g., covering his ears). All but ID4 are curious about digital tools, and they know how to use a tablet. ID1, ID2, and ID4 have done video game workshops with the psychologists for six years, one year, and once, respectively. ID4 and ID10 have also done robot workshops for one year. ID1 and ID2 have digital tools at home.

TABLE 1. Profiles of autistic children. Condition corresponds to the ICD-10 classification. The CARS scale assesses the significance of autism traits: 15–29,5 (non-autistic), 30–36,5 (“mild-moderate” autism), 37–60 (“severe” autism). KABC assesses the cognitive condition with the Mental Processing index (MPI) (Luria model) or the Non-Verbal Index (NVI): below 69 (very inferior to mean), 70–84 (inferior to mean), 85–115 (mean). “Ses” represents the number of sessions that children had. The differences in terms of the number of sessions that they performed are due to adaptations to the procedure (see Section 3.2.3). “OD” stands for the psychologist Olivier Duris.

TABLE 2. Design choices and expected children’s AR experiences.

3.2.2 Procedure

Before the experiment, a psychiatrist from our team explains the protocol, purposes, data collection, and analysis to the legal tutors of children and collects their informed consent. Indeed, children are not capable of doing it by themselves, according to the psychologists. Then, considering children’s usual behaviors and potential therapeutic benefits, psychologists assign half of them to the actor condition (ID1, ID3, ID10, and ID13) and the other half to the spectator condition (ID2, ID4, and ID12). Psychologists knew Magic Bubbles when the protocol begun, as they were involved in its design (Bauer et al., 2021). This involvement may have led to some bias during the analysis process. Setting up a control group with children not using Magic Bubbles was planned but abandoned due to the low number of children in this study. Despite these limits, we deemed this approach suitable due to the exploratory nature of this field study.

Each child participates in six weekly sessions from the time when they accept to wear the headset. If several weeks are needed, the duration is extended accordingly. Sessions take place in a large room and last between five (minimum) and twenty (maximum) minutes, depending on children’s acceptability, practitioners’ evaluation of their experience, and unexpected events (e.g., the child being late). Their duration is expected to increase over time, along with children’s acceptability. Several children’s sessions occur successively every week, spaced by a fifteen-minute break to clean the equipment and air the room according to Covid-19 security rules. Four investigators are present, i.e., two psychologists, one psychology intern, and the first author. Following usual clinical practices, educators can also come if interested or for reassurance. At the start, psychologists introduce the child to the setting and invite them to wear the headset. Children are not forced to wear it and can remove/wear it at will. Then, children experience free-play time. Practitioners monitor what they perceive through a screen and can interact with them (verbally or non-verbally). A session can end in two ways: the child expresses that they want to stop (e.g., removing the headset), or time runs out. In the latter case, the practitioners warn about the session’s ending (verbally and by triggering a gong sound) and suggest doing one last thing. Children then remove the headset, or practitioners help them to do it. At last, practitioners tell children to draw with pencils if they want to and that their drawing would be included in the AR scene the week after.

After the child leaves, psychologists complete a questionnaire about their experience, as children cannot do it by themselves, according to psychologists. A semi-structured focus group interview follows to ask the psychologists and the intern for further insights (see Section 3.2.4). Educators are then questioned about potential unusual events happening after the session compared to before it.

Poléthis Ethics Research Committee of Paris-Saclay University approved procedures with reference 226.

3.2.3 Adaptations to the procedure

Due to the clinical context of the study and the children’s specific needs, we had to adapt the protocol. ID2 needed four sessions to wear the headset, so we extended his protocol to ten sessions. ID10 also did a seventh session due to bugs occurring during session six (s6). Moreover, when the psychologists deemed it beneficial, a few sessions were added: ID4 and ID12 did a seventh session as actors, and ID2 did fourteen sessions (switching to actor condition at s11). In our data analysis, to compare children based on a similar number of sessions, the sessions added to be beneficial are only considered for complementary insights. At last, ID1 was excluded from the protocol after s3 since he had an epilepsy crisis one week after s3, although such risks were unknown by the clinical team.

3.2.4 Data collection

As most children could not directly answer self-report questionnaires, various data sources were collected to mitigate the bias associated with each source and get closer to the child’s view. They include practitioners’ observations (questionnaires and interviews), three video recordings (two cameras and a video recording of the child’s AR view), and children’s drawings.

As no validated questionnaire to be filled by practitioners existed, we used the semi-structured questionnaire we previously devised for our pre-tests (Bauer et al., 2022). First, six questions address the child’s state before and after the sessions. Nineteen questions then address the child’s experience. Practitioners can finally add comments. The questionnaire lasts between five and ten minutes. Each item consists of 1-5 or 1-7 Likert-type scales (depending on the references that it draws from) and optional comments. The questionnaire is detailed in Table 3.

TABLE 3. Semi-structured questionnaire answered by the psychologists about children’s states and experiences. In the column called “Answers”, “1–5” and “1–7” represent the types of Likert-type scales that psychologists had to answer. We used 1-5 or 1-7 scales depending on the literature references that the questions drew from. “cmt” means that practitioners could add optional comments.

Semi-structured focus-group interviews target engagement, reassurance, and readiness to start another activity when leaving. After six sessions, asking if the setting was holding was preferred over reassurance to match psychologists’ language. Indeed, holding defines a supportive and secure environment (e.g., physical and social space) for generating and supporting interactions (Winnicott, 1980). Two elements were questioned to elicit more insights: differences between the session’s beginning and ending and critical incidents (e.g., moving moments), following Flanagan (1954)’s critical incident technique.

3.2.5 Data analysis

Qualitative and quantitative analysis were used to focus on children’s actions and their underlying processes, e.g., the relationship with practitioners’ actions (Dechsling et al., 2021). In particular, the grounded theory inductive qualitative approach enabled us to consider psychologists’ observations (interviews and questionnaires’ comments), educators’ observations, and children’s drawings (Glaser and Strauss, 1967; Charmaz, 2006). This method consists of constructing meaningful phrasings and sorting them into concepts and categories gradually built through the analysis process³. During this coding process, to mitigate biases (e.g., preconceptions), constant comparison techniques are systematically used to compare the data with the emerging concepts and categories (Charmaz, 2006, p.54). The analysis stops when reaching data saturation (i.e., no new properties emerge from the categories). At the end of the process, answers to the questionnaire were analyzed quantitatively to confirm the emerging theory, except for items about presence (Q18-Q21). Indeed, once these questions were answered, we realized that externally assessing children’s feeling of presence through practitioners’ quantitative ratings was not reliable enough. Videos were only used to double-check other data in case of doubts during the analysis process (e.g., potential mistakes in the questionnaire answers). Grounded theory and quantitative analysis were performed with the software MaxQDA⁴ and Matplotlib⁵. Children’s views were considered, but practitioners’ interpretations prevailed as children were minimally verbal.

4 Results

This section first presents how we analyzed children’s AR experiences with the grounded theory approach. The findings are presented following the resulting categorization, with three subsections accounting for the three main categories and examples of critical incidents. Each subsection starts by summarizing the properties of a category and their relationship with environmental conditions. Before detailing them, the structure of each subsection is explained, as it differs for each category.

We differentiate between events happening during discovery (i.e., two first sessions when children wear the headset) and after discovery (i.e., from the third session with the headset on) due to autistic children’s difficulties with novelty. For clarity, the number of phrasings, percentages, children, the session numbers, and practitioners’ names are written in parenthesis as follows. Xp (X is a number) represents the number of phrasings. For percentages, X%/t and X%/c, respectively, indicate the percentage of phrasings compared to the total number of phrasings in the grounded theory and a category. For concepts, percentages are often preferred over the exact number of phrasings for clarity. Session numbers are written with sX. IDY-sX refers to a specific child and session (Y is the child identifier, and X is the session number). The number of children who fit in some concepts is indicated with X/7ch (as seven children participated in the study). Since psychologists’ names are essential for some findings, they are mentioned using OD for Olivier Duris and CL for Charlotte Labossière.

4.1 Conducting the analysis

The first author analyzed the data while often checking with OD and CL. He first gathered all qualitative data by transcribing observations made by the psychologists (interviews and comments to the questionnaire) and educators and scanning children’s drawings (see Figure 2). In doing so, the psychologists were asked to clarify some misunderstandings or to correct their quantitative ratings when not matching their observations by looking at the videos. Then, open coding was used to build initial concepts close to practitioners’ wordings while keeping track of first impressions in memos. Figure 3 displays four critical incidents that will be discussed hereafter. After that, focused coding was performed, i.e., ranking the concepts based on their relative significance while paying attention to negative experiences. Then, to build advanced concepts, axial coding enabled us to explore the conditions, actions, and consequences underlying each concept. We also compared a few concepts with some of Glaser and Strauss (1967)’s theoretical codes (e.g., identity-self, reciprocity) to refine their properties. Diagrams were finally drawn. As grounded theory is a bottom-up approach, the names of sub-concepts, concepts, and categories are close to practitioners’ wordings.

FIGURE 2. Evolution of children’s drawings over time. The abscissa axis refers to the session number, and the ordinate axis to children’s identifiers. Some boxes are left empty as children did not draw every week (only drawing if they wanted to). We can notice the following features: ID1 drew himself with the headset at s1 and on another boy’s shirt at s3; ID3 drew the setting multiple times and gradually added elements to it, reflecting his exploration; ID4 added a red dot at s5 when being unhappy with CL; ID10 represented his relatives when undergoing a challenging time at s2, s3, s4, and maybe the first author at s1 (after meeting him for the first time); ID13 may have represented himself at s2 and s6; ID2 just added himself a beard to himself to look like the practitioner on the drawing, reflecting a strong dyadic relationship; ID12 never drew.

FIGURE 3. Photographs of critical incidents that illustrate the concepts coming from the grounded theory analysis which are presented in the results: (A) ID2 managing to wear the headset alone at s5, (B) ID3 auto-stimulating by jumping at s5, (C) ID13 comparing his virtual drawing to the practitioner’s real drawing at s5, (D) ID4 trying to reach the practitioner’s hand which was hidden by a virtual water pond at s2.

In parallel to these last analytic stages, descriptive statistics were used to analyze the answers to the questionnaire. Quantitative results are presented per child following the emerging categories, as the diversity of children’s profiles prevented us from displaying them together. The analysis process considered children’s temporary difficulties, as reported by the clinical team. In particular, ID2, ID10, and ID4 faced personal challenging times, respectively between s4 and s7, s2 and s4, and s6 and s7. ID4 was also unhappy against CL at s4, and ID12 was tired when coming at s4 and s5 or scolded by his educator before s6.

4.2 The emerging categories

The overall grounded theory analysis comprises 2047 phrasings and four main categories. Three interconnected categories describe the child’s experience: Encountering the AR setting (927p, 45%/t), Sharing experience over time (407, 20%/t), and Ways of getting secure (230p, 11%/t). A fourth category, Conditions supporting the AR experience (483p, 24%), concerns environmental factors influencing the child’s experience. It has eight concepts: Therapeutic frame (22%/c), Caregivers’ strategies (19%/c), Self-Regulation (14%/c), Time (13.5%/c), AR setting (11%/c), Child’s state beforehand (11%/c), Contingencies (7%/c), and Actor vs Spectator (2.5%/c). For psychologists, encountering the setting refers to how children interacted with the AR setting (e.g., equipment, AR entities, space, people) during and after the discovery period. Although three children did the actor and three the spectator conditions, practitioners rarely used the actor or spectator terminologies. Instead, they stressed the influence of being able to interact over engagement, exploration, and agency. Figure 4 gives an overview of the grounded theory. The three main categories are detailed in the following three subsections and summarized in Figures 5, 7, and 9. Figures 6, 8A and B display the corresponding quantitative analyses. Below, we present the emerging categories, concepts, and sub-concepts, along with representative examples of children’s behaviors. The evolution per child over time is out of the scope of this paper and will be examined in a future paper.

FIGURE 4. Overview of the four emerging categories of the grounded theory: categories related to the child’s experience are displayed using rectangles filled with orange stripes and environmental conditions influencing the experience with a rectangle filled with green. Their sizes represent the percentage of phrasings they gather compared to the total number of phrasings in the grounded theory. Arrows indicate the main influences that categories have over others. The rectangle on the right connected with dash lines to the category about environmental conditions gives an inside view of its concepts. Concepts are represented with circles. Their sizes correspond to the percentage of phrasings they gather compared to the number of phrasings in the category.

FIGURE 5. Overview of the concepts (rectangles) and sub-concepts (circles) subsumed in the category called Encountering the AR setting that emerges from the grounded theory analysis of children’s experiences. The lines show which sub-concepts belong to which concepts. The arrows between concepts display their respective influences. The circles’ size represents the sub-concepts’ respective significance over this category’s total number of phrasings.

FIGURE 6. Quantitative ratings from the questionnaires corresponding to the emerging category called Encountering the AR setting, about (A) Acceptability, (B) Usability, (C) Agency, and (D) Engagement. Colors refer to the three encountering profiles: green for profile 1, orange for profile 2, and red for profile 3.

4.3 The category called Encountering the AR setting

Encountering the AR setting is the most reported category (927p, 45%/t). It has three main concepts, namely, getting engaged (30.7%/c), qualities of exploration (31.3%/c), and becoming agent (19%/c), and two minor concepts, namely, qualities of understanding (10.9%/c), and qualities of acceptability (8.1%/c). Becoming agent is larger than the actor and spectator conditions, as this concept entails all actions and factors related to the child’s ability to act intentionally. We define that children accepted the setting when they agreed to wear the headset. Moderate acceptability corresponds to briefly wearing it (between five and twenty seconds). We also define that children explored when they did not stay still and tried to interact. The terms “qualities of” are used to refer to exploration, understanding, and acceptability, as these concepts depend on many properties, as detailed through their sub-concepts.

Three encountering profiles were built, which correspond to quick (profile 1 - ID1, ID3, ID10, and ID13), gradual (profile 2 - ID2, ID4), or challenging encounters (profile 3 - ID12). Concepts’ properties vary according to these profiles, as shown in Figure 6. Regarding sub-concepts, enjoying (179p) is the major sub-concept (as enjoyment is a prerequisite to benefit from the experience), followed by exploring various interactive spaces (118p) and getting autonomous (102p). None of them belongs to qualities of acceptability, as practitioners stop mentioning acceptability as soon as the setting is accepted. Five main conditions influenced the encounter: caregivers’ strategies (69p), time (63p), therapeutic frame (63p), AR setting (34p), and self-regulation (33p). For the encountering profiles 2 and 3, caregivers’ strategies include physical and verbal guidance (e.g., wearing the headset before the child). The therapeutic frame means that the dyadic relationship supported exploration. Figure 5 presents an overview of the category. The five following subsections detail its five concepts.

4.3.1 Qualities of acceptability

Most children (6/7ch) accepted the setting, 5/7ch directly and 1/7ch gradually (ID2 wore the headset from s5), but ID12 accepted it moderately. Direct acceptability relates to feeling excited or secure during discovery. For instance, ID13 jumped towards the headset at s1, whereas for ID2 and ID12, the novelty was anxiety-provoking. Moreover, despite their limited verbal abilities, ID2 and ID12 could refuse to wear the headset (verbally or non-verbally). For instance, ID2 sang the lullaby Promenons-nous dans les bois (s5), which tells a story where children are afraid of a wolf. Six main conditions influenced acceptability: caregivers’ strategies (52p), the overall context (35p), the AR setting (28p), the child’s state beforehand (15p), contingencies (5p), and time (4p).

Caregivers’ strategies include wearing the headset while the child looks at the monitor (helpful for ID2) or using guidance (verbal or, physical). Segmenting tasks also supported ID2’s acceptability at s5: proposing to get closer to the monitor, introducing the music, and then proposing the headset. Moreover, after s4, ID2 agreed to stay and observe ID10’s session. According to OD, this was a critical acceptability step, as “ID10 enjoys” and there “is no expectations towards ID2”, contrary to the practitioners. Guidance was also used with ID12: switching off the monitor at s4 allowed him to wear the headset again.

The overall context supported ID2 and ID12’s acceptability, mainly due to the dyadic relationship (ID2, ID12) and the understanding that they were not forced to wear the headset (ID2). Indeed, ID2 accepted it after OD created a holding space at s5 (whispering and, being physically close). ID2 expressed it by adding a beard to OD’s drawing to look like him (s6) (see Figure 2). ID12 was first worried without his educator, e.g., only wearing the headset when he was nearby (s3). He then largely refused from s5 but came alone at s6, which still indicated some interest. Practitioners hypothesized that ID12 may have required a longer protocol and that it may have worked better if his educator had been here during all the sessions.

About the AR setting, the content had a positive influence, e.g., ID12 wore the headset after seeing the individualized image (s1). The headset was well-accepted, despite slight physical disturbances for ID3 and ID10 (scratching their nose), ID2 (s7-s8), and ID12 (s7), who sometimes struggled to wear it and hold the controllers.

The child’s state beforehand impacted acceptability. Indeed, ID4 refused the activity when being unhappy with CL at s4. ID12 refused it when arriving in a bad mood and showed repetitive behaviors at s5 and s6 (e.g., repetitively closing the windows).

Time played a significant role, although less reported, as obvious for practitioners. In particular, ID2 went through five stages: being in the room (s1), touching the headset and controllers (s2), controllers and cones⁶ (s3), controllers (s4), and wearing the headset (s5). Ritualization over time was particularly helpful. For instance, it allowed ID12 to wear the headset without his educator at s4.

4.3.2 Qualities of exploration

ID1, ID3, ID10, and ID13 directly explored (encountering profile 1), ID2 and ID4 relied on their caregivers to then explore (profile 2), and ID12 hardly explored due to acceptability difficulties (profile 3). Children explored four main aspects, with differences depending on their profiles: interactive spaces (42%/c), the shift in self-perception (23%/c), the real vs virtual (21%/c), and the limits (14%/c).

According to the psychologists who referred to Lheureux-Davidse (2014), three spaces were explored: the body (until 10 cm away), the grip (arm’s length), and the locomotion (arm’s length to the room’s limits). Children also explored the monitor as a window over AR. 5/7ch engaged their whole body, mainly after discovery. ID2 did it more than usual from s6: “It is amazing! When I said ‘we can move,’ I did not believe in it” (CL, s7). Moreover, ID3 and ID13 often performed self-stimulation (e.g., jumping). About the grip, 6/7ch tried to touch real and virtual objects, from (ID2, ID4, and ID13) or after (ID3, ID10, and ID12) discovery. About locomotion, ID2, ID4, ID12, and ID13 observed a lot. ID2 and ID12 used the monitor as a gradual way toward AR.

Four main behaviors accounted for an exploration of what is real and virtual. 6/7ch children tried to touch real and virtual objects (e.g., ID2 and ID4 held the practitioners tight). 5/7ch often wore/removed the headset during discovery. 3/7ch covered the camera with their face mask (ID4-s1) or hands (ID2-s6, ID13-s4). ID2 hyperventilated and scraped his throat to feel his body was real (s7, s9).

In AR, children may have perceived a shift in their self-perception, prompting them to explore their bodies in unusual ways. 3/7ch had a different gait (ID2, ID4, and ID12). For instance, ID4 (s1,s2) and ID2 (s5) mainly remained on the floor first and then gradually stood up and moved in space. Others expressed this shift. For instance, ID10 said, “I want my voice in my mouth” (s2). Some drew themselves with (ID1-s1, ID3-s2) or without (ID13) headset (see Figure 2). 6/7ch also wanted to see themselves. Indeed, they asked to be photographed (ID10-s1, ID12-s1) or to look in the mirror (ID4-s4, ID13-s6).

6/7ch explored four types of limits. First, 4/7ch checked the limits of the physical space (e.g., ID4 went behind the table), and 4/7ch the limits of the virtual space (e.g., ID3 went behind a virtual drawing). Then, 2/7ch explored advanced interactions, e.g., ID13 hit bubbles with his head (s2). 2/7ch also looked for artifacts, often for the mise en abyme phenomenon: ID1 drew himself on a child’s shirt (see Figure 2), and ID13 generated it by looking at the monitor with the headset on. At last, ID13 tried to trigger some bugs by looking at the mirror.

Four conditions supported exploration: time (40%/c), self-regulation (22%/c), caregivers’ strategies (20%/c), and the therapeutic frame (11%/c). ID3 showed this significance of time by drawing the setting from s2 to s6 and gradually adding what he explored (see Figure 2). Self-regulation helped him to remain secure while exploring. At last, the more exploration was observed, the more the acceptability and agency was gained.

4.3.3 Becoming agent

Becoming agent has two sub-concepts detailed below, i.e., getting autonomous (58%/c) and taking control over the setting (42%/c). Three agency profiles were built that are related to the three encountering profiles (see Figure 2C): becoming agent (4/7ch, profile 1), moderately agent (2/7ch, profile 2), and not becoming agent (1/7ch, profile 3). Children with profile 1 were agents from s3, profile 2 were always moderately agents (as in real life), and profile 3 were not agents due to acceptability issues.

Getting autonomous concerns initiative-taking about the setting (e.g., wearing the headset alone) or others (i.e., initiating shared play). Children with profile 1 first asked when needing something (e.g., cleaning the drawings) and took more initiative over time. Some contextual requests struck practitioners, e.g., ID3 asked to listen to a new music piece where the singer talks about being in a bubble (s6).

Taking control over the setting first concerns the content. To that end, seeing their new drawings every week gave children some control, thus supporting self-confidence. Indeed, ID13, ID10, and ID3 often looked at all of them and left the last one visible. Controlling the content increased over time along with usability (see Figure 6B). Moreover, 5/7ch children took control over the session’s unfolding. In particular, ID3 and ID13 performed sessions in two parts at the protocol’s beginning: being excited and calmer. ID2, ID4, and ID12 sometimes also left the room when needed.

Four factors influenced agency: time (46%/c), contingencies (23%/c), caregivers’ strategies (15.5%/c), and the therapeutic frame (15.5%/c). Agency increases over time, with 89% of phrasings being reported after discovery. About the therapeutic frame, ID2 and ID4 became agents after relying on their practitioners. At last, the more agency, the better the child’s experience in terms of acceptability, reassurance, exploration, engagement, and social interaction.

4.3.4 Getting engaged

Engagement is the most reported concept (287p), with three sub-concepts: enjoying (71%/c), being involved (16%/c), and being curious (12%/c). Its phrasings mainly appear after discovery (83%/c).

Enjoying largely emerged, concerning all children, with four properties: having fun (36%/c), not wanting to stop (33%/c), being happy to come (17%/c), and being excited (14%/c). Having fun relates to laughing and smiling (6/7ch, except ID12), dancing (ID3-s6, ID4-s6, ID13-s3), drawing the setting (ID3 from s2 to s6), or being happy to see their drawings in AR (especially ID3, ID10, and ID13). Moreover, 5/7ch children (except ID4 and ID12) did not want sessions to end after discovery. ID1 and ID3 also returned three times after the protocol ended to express that they missed the activity. Most children were happy to come after the discovery period: coming alone (6/7ch, except ID2), stopping their ongoing activity in order to come (ID2, ID3, ID13), or directly smiling or communicating (ID1, ID2, ID3, ID10). At last, being excited largely appears, mainly for ID3 and ID13, during discovery. For instance, ID13 said, “Oh, a headset !” and jumped toward it at s1.

All children but ID12 were involved in the sessions, five more than usual (ID1, ID2, ID4, ID10, and ID13). For instance, ID10 was involved at s4, despite personal difficulties preventing him from being engaged in other activities at the day hospital. ID13 was calmer than usual at s2 (e.g., not mumbling). ID2 and ID4’s involvement depended upon agency levels, i.e., it came up when “authorizing themselves” (after discovery) or first testing the actor condition (s11 and s7). Similarly, ID12 spent more time in AR when being an actor (s7). Moreover, while 5/7ch were directly interested (ID1, ID3, ID4 ID10, and ID13), 2/7ch were intrigued but somewhat worried (ID2 and, ID12). For instance, ID12 quickly wanted to stop at s2 but then came back and asked to try again. At last, we observed that children’s involvement in AR did not hinder social interaction.

Four main conditions influenced engagement: therapeutic frame (34%), being actor (32%), contingencies (26%), and being agent (6%). The therapeutic frame was mainly reported for children encountering profiles 2 and 3, e.g., relying on their practitioners to get engaged. Being actor was also positive: ID4 (spectator) may have been bored from s4, but ID2 and ID4 were excited when becoming actors (at s11 and s7). For instance, ID2 unusually vocalized his pleasure (s11). About contingencies, ID3 and ID4 wore the headset again after the activity ended, leading to entering a dark world with stars (i.e., the Steam welcome screen). However, they enjoyed it: ID4 unusually vocalized and smiled (s6), and ID3 asked to “look at the stars” at the end of every following session. At last, the more engagement, the more agency, since it leads to “being happy in their actions” (OD).

4.3.5 Qualities of understanding

Qualities of understanding has three sub-concepts: understanding the AR setting (71%-7/7ch), the context (17%-5/7ch), and how to act (12%-5/7ch).

Understanding the AR setting mainly consists of describing it (ID4, ID10, ID3). For instance, ID3 drew it from s2 to s6 and gradually added the new elements he understood (see Figure 2). Other children uttered context-related sounds, e.g., ID2 vocalizing the bubbles’ sounds (s6). 2/7ch named the activity when talking to their educator: ID3 called it the “Snow Queen” after his individualized music, and ID10 “video games.” 6/7ch understood what the monitor represents (for ID4, the practitioners were unsure). 3/7ch identified bugs (ID3, ID10, and ID13), sometimes trying to fix them by imitating what the investigator had previously done (ID13-s7).

Understanding the context concerns ID2, ID3, ID10, and ID13. Children connected sessions by looking at their drawings. Some understood when the protocol ended (e.g., ID3 hugged everyone). ID3 identified the adults’ roles, asking for help from the adult in charge of technical aspects when spotting a bug: “I find it incredible: it bugs, and he comes to you” (CL-ID3-s5). 2/7ch understood that other children used Magic Bubbles, e.g., ID13 imitated ID4 when seeing her drawings. ID1, ID4, and ID12 are not mentioned, maybe due to ID4’s limited verbal abilities, ID12’s acceptability issues, or since ID1 only had three sessions.

Understanding how to act mainly relates to understanding the interactions. Hence, it mainly refers to quickly learning the interactions in the actor condition (ID1, ID2, ID4, and ID10). In the spectator condition, children had less understanding of it, maybe because the setting did not afford these possibilities.

Understanding increases over time, being positively influenced by exploration, caregivers’ strategies, and contingencies. Caregivers’ strategies consist of explaining how things work and confirming interrogations (e.g., OD said, “Yes, the headset is real” when ID2 held the headset at s6). Contingencies are unexpected bugs that clarify that children interacted with a machine.

4.4 The category called Sharing experience over time

The category Sharing the AR experience emerged a lot (927p, 20%/t). Indeed, the “evolution of the dyadic relationship” (OD) is key for practitioners. They were particularly interested in the fact that it unfolds within a “shared world” (CL, OD) rather than between a child and a practitioner, as in other usual activities. Figure 7 gives an overview of this category.

FIGURE 7. Overview of the category called Sharing experience over time emerging from the grounded theory analysis of children’s experiences. The six concepts corresponding to stages of sharing over time are represented over the abscissa axis. Three sharing profiles are represented over the ordinate axis, corresponding to different ways of sharing over time. Arrows highlight the evolution from one stage to another, and the first and last stages children experienced according to their sharing profiles.

Three sharing profiles appeared: from an inner to a shared experience (profile 1, ID1, ID3, ID10, and ID13), from the dyadic relationship to a shared AR experience (profile 2, ID2 and, ID4), and not secure enough to share (profile 3, ID12). They correspond to the three encountering profiles. Moreover, they were confirmed by answers to the questionnaire about communication (see Figure 8A): profile 1 relates to communicating more over time, profile 2 to communicating at first (due to relying on practitioners) and, at the end, but less in-between, and profile 3 to hardly communicating. The three profiles follow different evolutions over time through six stages. For instance, stage 5 (“Towards a shared experience”) consists of requesting various aspects for profile 1 but mainly in wearing the headset again at the end for profile 2. Although with different profiles, ID4 and ID13 were the children who interacted the most. The following three subsections present the three sharing profiles.

FIGURE 8. Quantitative ratings from the questionnaire related to the categories: Sharing experience over time and Ways of getting secure. (A) Evolution of communication over time; (B) Pre versus post child’s state. Colors refer to the three sharing profiles: green for profile 1, orange for profile 2, and red for profile 3.

Sharing was mainly influenced by caregivers’ strategies (11p) and contingencies (11p). Strategies mainly consist of initiating social interaction for profile 1. For the other profiles, the trust in the practitioners is leveraged to spur shared exploration. Contingencies are unexpected bugs prompting interaction, e.g., ID10 gave the controller to practitioners at s2 because it was buggy, thus generating shared play.

4.4.1 From an inner to a shared experience

These four children (ID1, ID3, ID10, and ID13) were first calm, “in their bubble,” gradually shared more, and finally initiated interactions: “First, you focus on yourself. Then, you want to have fun with the adults” (OD). For instance, ID1 first “did his own thing” but was not bothered when practitioners tried to interact and even laughed about it (s2). ID3 expressed this “inner thing through his drawings” (OD) by representing himself alone (s2 to s6). All children gradually shared more by making requests. Two of them (ID1 and, ID10) started to communicate through the recording bubble by repeating what was said (ID10-s3) or directly talking (ID1-s1). For instance, ID1 teased OD at s1, saying, “Well done!”. Over time, most children socially interacted more than usual. For instance, ID10 played with all the adults at s7, saying: “You four must stop!”. ID13 initiated shared play by drawing in AR and regularly checking that OD, who was also drawing, imitated him well (s5, s6). ID10 also managed to express personal difficulties by drawing his relatives, even though he had never done it before with his educators (s3, s4) (see Figure 2).

4.4.2 From the dyadic relationship to a shared experience

The way ID2 and ID4 shared their experience varied over time. At first, ID2 was somewhat intimidated, and ID4 stayed still due to the setting’s novelty. ID2 made the first eye contact through the monitor (s5), allowing him to accept the headset then (stage 2). As ID4 was “more interested in the human than in technology” and less intimidated, she directly went to stage 3. At stage 3, they both relied on their practitioners, e.g., holding their hands to move (ID2-s6). For instance, OD said about ID4 at s3: “I felt her strength to bring it [my hand] back to her.” From that point, children had an inner experience: ID2 avoided touching others, and ID4 became stiller. Then, they started interacting more. For instance, ID2 asked to wear the headset again (s6, s7, s9, and s10). At last, the children shared a lot. For instance, ID4 said “hello” with eye contact (s5) or non-verbally asked practitioners to get closer (s6): “She stopped to chit chat and stared at me like … come back” (s6). Moreover, ID2 talked more than during his regular therapy sessions (s10).

4.4.3 Not sharing

After exploring minimally due to novelty (e.g., quickly wearing the headset at s1), ID12 accessed AR by looking at his educator through the monitor. He then relied a lot on him when present. Finally, he displayed an inner experience, e.g., accepting to wear the headset again when the monitor was off (s4). He then largely refused the activity from s5, thus not accessing the following stages.

4.5 The category called Ways of getting secure

The category called Ways of getting secure (927p, 11%/t) has two concepts: getting secure (80%/c) and getting insecure (20%/c) (see Figure 9). These concepts vary with the children’s encountering and sharing profiles. They can be analyzed through four time windows: during versus after sessions and during versus after discovery. For instance, the evolution of children’s states between before and after sessions (see Figure 8B) gives insights about their reassurance: ID2 and ID10 became more secure (even during challenging times); ID1, ID3, and ID13 remained secure; and ID12 became insecure (due to acceptability issues). Then, observing the evolution between during and after discovery suggests that ID2 gradually became more secure after overcoming the setting’s novelty. ID4 also improved despite undergoing a hard time at s6 (not visible in Figure 8B). The following two subsections present these two main concepts.

FIGURE 9. Overview of the category called Ways of getting secure emerging from the grounded theory analysis of children’s experiences. Its two concepts, getting secure and getting insecure, are represented in subfigures (A) and (B). Sub-concepts are presented with yellow circles. The two subfigures are organized following four time windows: during sessions (left part), after sessions (right part), during discovery (top part), or after discovery (low part). The circles’ size represents the sub-concepts respective significance over this category’s total number of phrasings.

Four main conditions influenced reassurance: therapeutic frame (41p), self-regulation (34p), child’s state beforehand (38p), and the AR setting (20p). The therapeutic frame corresponds to the holding environment induced by practitioners and the physical setting (e.g., carpets). Self-regulation consists of behaviors that children use to stay calm. Children’s states largely influenced how sessions unfolded, e.g., ID2 hyperventilated at s6 due to going through a hard time personally. Due to the equipment and content, the AR setting may also be holding in itself. For instance, the headset may have been holding for ID1, ID2, ID3, and ID10 as being heavy, holding the head, and inducing a visual shrinking. The practitioners also suggested that Magic Bubbles could complement Snoezelen by offering a holding space with free play rather than relaxing activities.

4.5.1 Getting secure

Being well (58%/c) is the most reported sub-concept, followed by feeling way better (26%/c) and being as usual when leaving (16%/c). Phrasings were classified as being well when differences with children’s usual behavior were observed and as feeling way better if children felt unwell when starting the session. These three sub-concepts have similar properties with various proportions: getting calm, handling contingencies well, and being ready for other tasks when leaving. The three sub-concepts are described below following these properties.

About getting calm, during and after discovery, 4/7ch had fewer stereotypies, with ID2 and ID10 sometimes feeling way better. In particular, ID2: hyperventilated less (s6), walked with flat feet (s7, s9, s10, s11), or used less hand flapping (s9). ID1 was less tactile than usual (s2) and strongly refused to look in the mirror at s3, although he usually does it very often. ID1 (all sessions), ID10 (s2), and ID13 (s2, s4) did fewer echolalias, and ID2 did not do any at s2. 4/7ch adopted new self-regulatory behaviors. Indeed, ID1 made repetitive gestures with the bubble column to get secure after socially interacting or exploring, as he confirmed at s3: “As soon as ID1 interacts with us, he then refocuses with the column, does not he?” (OD). “Yes, I do” (ID1). Similarly, ID3 and ID13 self-stimulated by moving a lot to generate vestibular stimulation and/or visual blur. Other children focused on virtual elements to shut out the surroundings, i.e., column (ID1 - all sessions), panel (ID2 - s7), various objects (ID4 - s1), or the stars at the end (ID3 - all sessions). Some children also became much calmer during (ID2 and, ID3) and after (ID2, ID3, and ID13) discovery. ID4 also often lay on the floor, indicating relaxation (s1,s2,s3, and s5).

Handling contingencies well mainly consists of waiting without frustration for the application to start or to be restarted (ID3, ID4, ID10, and ID13). Such behavior was unusual for ID13 at s3: “This had never happened (CL).” ID3 could also “handle well his frustration” (CL) at s3, although he was eager to do the activity. However, children only waited if meaningful. For instance, ID13 did not do it at s4 when being told to wait for OD, as OD entered the room simultaneously. Then, children could also tolerate unexpected bugs (ID10, ID3): “It is great that ID10 was not disturbed by the bugs” (CL - s6). Moreover, ID3 and ID13 more easily stopped the activity throughout the protocol.

Practitioners stressed that ID2 (s4, s5, s7, and s10) and ID10 (s2 and, s3) were sometimes calmer than usual and readier to start other tasks when the session ended. For instance, ID2 did fewer echolalias than usual after s3, and ID2, ID3, ID4, and ID13 often easily started activities after leaving. Moreover, the fact that ID3 could express that he wanted to go to the toilets right after s6 was striking for OD and CL.

4.5.2 Getting insecure

ID2 and ID12 were initially somewhat insecure: ID2 due to novelty and ID12 due to being overwhelmed by too much stimuli. Whereas ID2 gradually got secure (from s5), ID12 did not. Indeed, ID12 refused the headset during most sessions and showed anxiety, repeating, “I want to go to the canteen” (s5), sorting everything (all sessions), or wanting to leave (s5, s6). However, which element was anxiety-provoking was hard to identify (e.g., too many people around). According to the practitioners, ID12 still displayed a positive evolution, as he quickly wore the headset alone at s5 and s6 and came alone at s6.

5 Discussion

This paper had three main research questions related to the possibility for Magic Bubbles AR environment to secure autistic children with SLN (RQ1), to prompt the child-practitioner relationship (RQ2), and to evaluate the quality of children’s experiences (RQ3). To address them, we conducted a long-term field study with seven autistic children with SLN at a day hospital. Multiple data were collected: practitioners’ observations (questionnaires and interviews), children’s drawings, and educators’ observations. Using a grounded theory approach and descriptive statistics we built three interconnected categories that describe children’s experiences: Encountering the AR setting, Sharing experience over time, and Ways of getting secure. A fourth category, Conditions supporting the AR experience, represents environmental factors influencing this experience (see again Figure 4). The following four paragraphs summarize these categories.

As described in the category called Ways of getting secure, most children (6/7ch) became secure, either from the start (4/7ch) or after discovery with practitioners’ support (2/7ch) (see Figure 9). This finding matches practitioners’ initial expectations (see Table 2) and validates RQ1. ID2 and ID10 sometimes felt better when using Magic Bubbles, despite undergoing personal challenging times. Such reassurance enabled 6/7ch to gradually explore more (e.g., moving in space) and become more agent (e.g., authorizing themselves) and engaged (e.g., having fun).

Magic Bubbles also encouraged children to share their experiences, thus validating RQ2. Indeed, psychologists said that it “created some relationship.” Three sharing profiles emerged, with different time evolutions (see Figure 7). With profile 1 (4/7ch), children directly explored alone and gradually shared with others (e.g., shared play). Profile 2 (2/7ch) consisted of three steps: relying on practitioners, exploring alone, and sharing their experiences. With profile 3 (1/7ch), ID12 was often worried and refused to share. However, it may have been different if his educator had been around during all sessions. Magic Bubbles’s potential to bond with each other complements the findings from previous studies using digitally-augmented multisensory spaces (Pares et al., 2005; Ringland et al., 2014; Mora-Guiard et al., 2017; Garzotto and Gelsomini, 2018). Moreover, in light of these positive findings, practitioners expressed that it could complement the Snoezelen approach (Lancioni et al., 2002), by focusing more on free-play engaging aspects than relaxation.

To account for the quality of children’s experiences (Riva et al., 2012) and thus answer RQ3, the category called Encountering the AR setting was built⁷ (see Figure 5). It includes three encountering profiles that are related to the sharing profiles: profile 1 (4/7ch) consists of easily encountering the setting (e.g., exploring), profile 2 (2/7ch) consists of largely relying on practitioners at first and then encountering the setting, and profile 3 (1/7ch) of facing encountering issues (e.g., acceptability issues). Although with profile 3, ID12 still had a positive evolution: coming alone at s6 and wearing the headset alone at s5 and s6. The Encountering the AR setting category subsumes five concepts: getting engaged and qualities of exploration largely appear, followed by becoming agent, qualities of understanding, and qualities of acceptability⁸. The significance of engagement and agency echoes the insights coming from interviews with autism stakeholders (Parsons et al., 2020; Bauer et al., 2023), and autistic children’s experiences with technology (Spiel et al., 2017).

The overall setting largely influenced children’s experiences, mainly through the therapeutic frame and caregivers’ strategies. This observation confirms that analyzing autistic children’s AR experiences requires using methods that consider the overall context, as previously reported (Spiel et al., 2019; Parsons et al., 2020; Bauer et al., 2023).

5.1 Limits of the study

Some limits prevent us from generalizing our findings. First, the analysis mainly reflects practitioners’ interpretations, as children were minimally verbal. Despite efforts to mitigate this bias (e.g., considering children’s drawings), new methodologies should be devised to better account for their views (Spiel et al., 2017; Parsons et al., 2020). As no previous study to our knowledge focused on such AR sensory and mediation objectives, our research questions also remained quite broad. While enabling us to raise new insights, they prevented us from examining specific aspects. In addition, the low number of children limits the generalization of our findings. In the future, collaborating with several clinical institutes may help to include more children and thus to decrease the impact of their idiosyncrasies on the findings. Practitioners’ preference for the actor condition and their participation in the design process may also have influenced how they behaved and, thus, children’s behaviors. To further examine some of our emerging concepts (e.g., real-virtual understanding), future studies should thus include more diverse practitioners with no prior knowledge of the environment and more diverse children. The apparatus displays some drawbacks, as it was devised based on a trade-off between several needs related to the day hospital context (Bauer et al., 2021). Nevertheless, the visual latency (i.e., around 60 m) and resolution (i.e., 720p per camera) may have influenced children’s behaviors (e.g., how they moved). Therefore, future studies need to compare the influence of different AR headsets (e.g., HTC Vive Pro, Hololens) over children’s quality of experience with Magic Bubbles. Controllers may also be compared with free-hand interaction, as it is potentially more user-friendly for autistic users due to their sensory issues. At last, while discussions with educators gave insights into children’s state after AR sessions, our methodology did not enable us to understand how the observed behaviors could be generalized to real life. Future works should focus on keeping track of the evolution of specific behaviors between AR and real life (e.g., stereotyped behaviors).

As previously mentioned (see Section 3.2.2), the low number of children prevented us from using a control group, where children would do another sensory activity without (e.g., Snoezelen) or with technology (e.g., with the Kinect). Hence, we can only suggest insights worth exploring, but we cannot speak about their potential generalization to other children. For future testing with autistic children with SLN, three possibilities may help to include a baseline. First, a control group may be composed of children with related neurodevelopmental conditions. However, differences between conditions may lead to behavioral differences that may hinder the analysis process. Indeed, insights may be limited to behavioral differences between the two groups or acceptability and usability issues but would not include specific aspects of autistic children’s experiences (e.g., regarding exploration). Second, more autistic children could be recruited through multiple clinical institutes. Third, a single-subject withdrawal experimental design could be used, where the aspect under investigation is successively absent and present (Krasny-Pacini and Evans, 2018), as done in previous studies (Coyle and Cole, 2004; Neely et al., 2013). This approach would help to compare behaviors observed with Magic Bubbles to the baseline. For instance, all children could do an A-B-A-B experimental design (each step taking six weeks), where A is the baseline (e.g., Snoezelen) and B is Magic Bubbles. A second possibility is that half of the children do A-B-A and the other half B-A-B.

The qualitative analysis may contain inaccuracies due to misunderstandings or preconceptions, despite strategies used to mitigate them. While various aspects of children’s experiences and their relationships were uncovered, their causes often remain uncertain. For instance, the respective influence of ritualization, time, and Magic Bubbles on children’s reassurance have yet to be explored. Such a study may help better understand why and when children went from one sharing stage to another.

5.2 Perspectives

While the context, content, and equipment have led to securing children, two other aspects that may also have contributed would be worth exploring: the visual shrinking induced by the AR headset and its physical characteristics. Moreover, the visual shrinking may be related to another type of shrinking: using AR to remove real elements, as previously suggested (Bauer et al., 2023). Exploring if and which headsets can be reassuring also echoes Newbutt et al. (2020)’s study about autistic children’s acceptability, where some children expressed that they would like to use headsets to feel calm.

The positive quality of experience observed for most children calls for more exploration of the potential benefits of Magic Bubbles in autistic children’s daily life. To that end, participatory design sessions should be conducted, as previously suggested (Spiel et al., 2017; Parsons et al., 2020), with autistic individuals with MLN and SLN, families, and practitioners. The goal would be to understand better if AR holding spaces like Magic Bubbles could and should be included in children’s daily spaces and, if so, how. For instance, Magic Bubbles may serve as a virtual space that children can use to calm down when starting to feel over-aroused at school.

Then, future studies should pay more attention to the difference between repetitive behaviors accounting for anxiety or being used to calm down. For instance, in our study, one child performed repetitive movements with the bubble column after exploring or socially interacting. Doing so could help to better identify anxiety-provoking features. Moreover, future research should consider including self-regulatory areas within AR and VR spaces where children can actively go to calm down. Therefore, a research challenge involves designing such spaces while preventing them from inducing isolation, as previously suggested (Pares et al., 2005).

Children’s immersion and co-immersion call for more research, with three main avenues. First, devising methodologies to assess the perceived immersion of minimally verbal children could help to go beyond practitioners’ interpretations. Second, children’s understanding of real and virtual elements could be further examined. Parallels could be drawn with Winnicott (1980)’s concept of potential space, i.e., an intermediate free-play area between the objective and subjective experience, enabling access to reality and the other. Third, parallels could be drawn between immersion and the mise en abyme phenomena, which appeared in drawings or when children looked at the screen.

The exploration of the shift in self-perception, the real versus virtual, and the limits indicated a possible feeling of presence. While this analysis was not performed in detail, as our measures targeting presence were not adapted for autistic children with SLN, the feeling of presence should be further explored regarding its influence on children’s experiences. Indeed, to our knowledge, previous studies targeting presence focused on autistic individuals with MLN who could answer self-report questionnaires (Wallace et al., 2010), which was impossible in our case. To pursue such endeavors, other inquiry methods must be devised to collect and compare multiple data sources, for instance, observations, interviews, or physiological data.

Many findings relate to a perceived shift in self-perception. Many children were interested in their body image (e.g., asking to look at the mirror), one child refused to look at the mirror although usually doing it very often, and others had a different gait than usual (e.g., walking with flat feet). Examining what caused this shift (e.g., equipment, AR) is worth investigating, as it may open new research avenues in AR and psychology.

At last, the positive influence of unexpected features on children’s experiences could be more explored (e.g., bugs). This investigation may be related to how practitioners introduce unexpected elements in their structured sessions to prompt exploration (Bauer et al., 2023).

Data availability statement

The raw data supporting the conclusions of this article cannot be made available by the authors, due to patient confidentiality restrictions. Requests to access the datasets should be directed to valentin.bauer@lisn.fr

Ethics statement

The studies involving human participants were reviewed and approved by Poléthis Ethics Research Committee of Paris-Saclay University reference 226. Written informed consent to participate in this study was provided by the participants’ legal guardian/next of kin.

Author contributions

VB participated in all steps of the study, i.e., he designed and developed the AR setting, designed the experiment, ensured technical support during all tests, conducted data collection and analysis, and wrote the article. TF and PB supervised the design of the AR setting and the experiment, and participated in the writing. OD and CL participated in the design of the experiment, conducted all tests, and participated in the data analysis. M-NC participated in the design of the experiment. All authors contributed to the article and approved the submitted version.

Acknowledgments

We want to thank the children who participated in the protocol, and their parents who agreed with their participation. We also want to thank the entire clinical team of the day hospital André Boulloche who enabled us to conduct this experience. This work is part of the AudioXR4TSA project, funded by the DIM RFSI Ile de France. This work was also supported by French government funding managed by the National Research Agency under the Investments for the Future program (PIA) under grant ANR-21-ESRE-0030 / CONTINUUM.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

¹This paper adopts a terminology that is close to autism stakeholders’ preferences (Kenny et al., 2016; Bottema-Beutel et al., 2021), which includes identity-first language (e.g., autistic people), and excludes potentially offending medicalized terms (e.g., “deficit”, “disorder”, and “severe autism”). This choice aims to move away from understanding autism as something that could be “fixed”, coming from the “medical model” of disability (Kapp, 2019). It also underlies that autistic people’s difficulties stem both from inner and external challenges (e.g., social environment) and that they also display specific strengths (e.g., visual memory), as stated by the “social model” of disability (Oliver, 2013) and the “neurodiversity movement” (Kapp, 2019).

²The term “reassurance” used in this paper refers to the child’s feeling of “being secure”. Being related to both the social and physical spaces, it is thus close to the term holding, which is common in developmental psychology (Winnicott, 1980).

³The general term “code” refers to all the phrasings, sub-concepts, concepts, and categories created during the coding process.

⁴MaxQDA software: https://www.maxqda.com/.

⁵Matplotlib python library: https://matplotlib.org/.

⁶Activity cones are often used by psychomotor therapists. They are here used to highlight the limits of the virtual AR space.

⁷See the definition in Section 4.2.

⁸See the definition in Section 4.3.

References