Cognitive training with fully immersive virtual reality in patients with neurological and psychiatric disorders: A systematic review of randomized controlled trials

Cognitive impairment occurs across several neuropsychiatric diseases and impede everyday functioning and quality of life. Fully immersive Virtual Reality (VR) aid motivation and engagement and therefore has a potential to help overcome the obstacles in the field of cognitive rehabilitation. The aim of this systematic review is to investigate whether VR can be a useful intervention in cognitive rehabilitation transdiagnostically. We identified nine studies with randomized controlled trials following the PRISMA guidelines in databases Pubmed, Embase and PsychInfo. The trials were all evaluated through Cochrane Collaboration's Risk of Bias. The studies were conducted in patients with mild cognitive impairment (k=4), schizophrenia (k=3), ADHD (k=1), or stroke (k=1) and involved 6-12 weeks of training. Overall, results showed improvement in some domains of cognition, primarily executive function and attention. The studies were pilot studies with 6-34 participants per treatment group. Risk of bias was either high (k=3) or moderate (some concerns) (k=6). Key reasons were suboptimal statistical analyses and lack of clarification on randomization and blinding of participants and assessors. In conclusion, this review found promising evidence for VR cognitive rehabilitation for neuropsychiatric illnesses. However, larger and methodologically stronger studies are warranted to establish the full potential of VR.


Introduction
Cognitive impairment is a common feature across several neuropsychiatric disorders, including schizophrenia and mood disorders (Millan et al., 2012). Schizophrenia is often accompanied by marked impairments across several cognitive domains including attention, memory and executive functions (Fioravanti, 2012) and social cognition (Green et al., 2015). Likewise, broad cognitive impairments are common even during remitted phases of major depressive disorder (MDD) (Rock et al., 2014) and bipolar disorder (BD) (Bourne et al., 2013), although they are generally less pronounced than in schizophrenia (Reichenberg et al., 2009). Negative associations between cognitive functions and employment outcomes have been demonstrated across neuropsychiatric diseases (Tse et al., 2014;Bowie and Harvey, 2006;Wong et al., 2019;McIntyre et al., 2013). This indicates a clear clinical relevance of treating cognitive impairments to improve patients' functioning and quality of life. Cognitive impairments are also a core symptom in neurological diseases such as stroke (Cumming et al., 2013) and traumatic brain injuries (TBI) (Ponsford et al., 2014).
Cognitive rehabilitation interventions, including both compensatory strategy-based and more practice-based remediation interventions, have shown some efficacy on cognitive impairments in both psychiatric (Wykes et al., 2011;Strawbridge et al., 2020) and neurological disorders (Cumming et al., 2013;Cicerone et al., 2010). However, a key obstacle in the field of cognitive rehabilitation is the inconsistent and generally poor evidence for transfer effects of cognitive improvement to functional outcomes in patients' daily lives (Bowie and Harvey, 2006;Jensen et al., 2016;Hoffmann et al., 2010;Cicerone et al., 2010). Studies have shown that in order to improve functioning, interventions should include training of more specific daily-life skills and compensatory strategies Wykes et al., 2011;Lewandowski et al., 2017). Another critical barrier for effective treatment is a lack of motivation in the patients to participate the assigned training despite receiving encouragement and support (Glenthøj et al., 2020). Perhaps because of these problems with limited transfer and motivation, attrition rates are often high in cognitive rehabilitation programmes (Lewandowski et al., 2017).
Virtual Reality (VR) training could feasible to accommodate these problems because of its highly engaging and gamified format (Makransky et al., 2019;Makransky et al., 2020). Virtual reality can be defined as a naturalistic simulated environment with which the user can interact as if the user was present (Lee and Wong, 2014;Biocca, 1992). With the possibility for a fully controlled and safe environment the technology offers a more ecological valid environment for cognitive rehabilitation (Matijević et al., 2013, Tieri et al., 2018 as it enables a multimodal setting that is quite similar to situations that patients might encounter in their daily lives (Bohil, 2011). Thus, cognitive training can be integrated more easily with daily life functioning. It seems reasonable to assume that this could facilitate a larger transfer effect (Tieri et al., 2018). However, to create a real-life experience, it requires fully immersive VR such as Head Mounted Display (HMD) or Cave Automatic Virtual Reality (CAVE) rather than semi or non-immersive VR such as desktop-based VR (Makransky and Lilleholt, 2018). HMD and CAVE allow for the user to be fully surrounded by the virtual environment and effectively be shut out of the physical reality, thereby facilitating a greater feeling of presence (Cummings and Bailenson, 2016). Another major advantage of VR is the motivational factor. Immersive VR has proven to induce the feeling of presence and thereby increase engagement in users (Makransky and Lilleholt, 2018). In educational contexts, VR has also been demonstrated to induce motivation and the feeling of entertainment (Makransky et al., 2019;Makransky et al., 2020). Similar effects in VR-based cognitive rehabilitation programs could reduce attrition rates and instill greater motivation in patients.
There has been great recent interest in the use of VR for cognitive rehabilitation across several neurologic and psychiatric disorders however reviews investigating VR rehabilitation have focused on neurological diseases (Massetti et al., 2018;Tieri et al., 2018;Moreno et al., 2019). Most of the VR interventions included in these reviews did not have cognitive improvement as the main focus, but rather physical rehabilitation or symptom relief. Other reviews have covered specific disorders such as mild cognitive impairment (MCI; Wu et al., 2020;Kim et al., 2019), traumatic brain injury (TBI, Pietrzak et al., 2014), stroke, (Laver et al., 2017), age-related cognitive decline (Bauer and Andringa, 2020), pediatric neurorehabilitation (Wang and Reid, 2011) or attention deficit hyperactivity disorder (ADHD, Romero-Ayuso, 2021;Bashiri et al., 2017). One recent narrative review by Riva et al. (2020) focused on cognitive rehabilitation with VR across neurological diseases. The authors concluded that there is encouraging preliminary evidence for training-related improvement in executive functions and visuospatial abilities. However, the review included semi-immersive desktop-based VR training, which do not provide a realistic and life-like user experience (Makransky and Lilleholt, 2018). Further, the review was narrative and did not include quality assessments of the studies. Considering the significance of immersion in virtual learning, the present systematic review investigates both the efficacy and quality of randomized controlled trials (RCTs) using fully immersive VR interventions to improve cognition across neuropsychiatric and neurologic disorders. The aims of the review are two-fold: (I) to investigate the efficacy of fully immersive VR for cognitive rehabilitation across neuropsychiatric disorders, (II) to examine the particular interventions, including (a) specific components involved in efficacy, (b) feasibility of the interventions and (c) relationship between any cognitive improvements and improvements in daily-life functioning and quality of life.

Search strategy
The present systematic review followed the procedures of the Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) statement (Moher et al., 2009). A comprehensive systematic computerized search was performed on the databases PubMed/MED-LINE, EMBASE and PsychInfo from inception to 29th January 2021. The search profile included three elements; "Psychiatric disorders"/" Brain disorders", "Virtual Reality" and "intervention"/" rehabilitation" each specified through subcategories and alternative key words in the respective databases (see supplementary material 1 for details on the search profile). Both key word-searches and title/abstract-searches were conducted. We used endnote for data extraction and removal of duplicates.
The authors (FSJ and MS) independently conducted a primary title/ abstract screening for eligible articles and following this, a secondary full text screening was conducted. A hand-search was performed by screening reference lists and citations of the included articles for eligible undetected articles. The articles were through both screenings assigned a reason for exclusion. Agreement between the two authors was high (primary screening: 94%; secondary screening: 98%). Disagreements were solved through consensus in all cases after discussion with AEJ and KMW.

Selection criteria
The inclusion criteria were original peer-reviewed articles involving (a) individuals meeting diagnostic criteria for a psychiatric disorder or central nervous system disease or trauma; (b) fully immersive VR intervention with cognitive rehabilitation as the main outcome or key aim; (c) randomized controlled study design. All age groups were included. We excluded articles that were not in English or primarily focused on physical or motor training rather than cognitive rehabilitation. A protocol with these a priori in-and exclusion criteria was developed but not pre-registered online.

Risk of bias assessment
The included studies were subjected to risk of bias assessment following the Revised Cochrane risk-of-bias tool for randomized trials (RoB 2). After the primary title/abstract screening, 99 articles were evaluated in a secondary full-text screening. This resulted in inclusion of nine eligible articles. The most common reason (k=1208) for exclusion was that training primarily involved neurological rehabilitation of physical and motor-cognitive function (e.g. finger tapping, walking distance). Another common reason (k=524) for exclusion was uncontrolled study designs or absence of performance-based measures. A large number of studies (k=160) were also excluded due to use of non-immersive VR. The nine studies involved patients with schizophrenia (k=3), ADHD (k=1), MCI (k=4), or stroke (k=1). Tables 1-3 provides an overview of all studies including details on participants, cognitive domains and measures, types of training tasks and virtual environments. Table 1 covers the three studies in patients with schizophrenia with sample sizes ranging from 12-17 participants. One study with 17 patients (active VR: n=9, passive VR: n=8) (Vass et al., 2020) had its primary focus on theory of mind (ToM). Training sessions of 50 minutes each were conducted weekly over nine weeks and involved training on two tasks: (I) Conversation training in everyday situations and (II) emotion visualization, in which the participant could practice aligning verbalized and visualized facial expressions. The study found significant group by time interactions for eight of thirteen ToM outcomes and three of nine outcomes of other cognitive domains, such as visuospatial cognition and attention, in favour of the active VR group. All significant findings survived Bonferroni corrections for multiple comparisons. In addition, the authors reported no dropouts and high feasibility, but no improvement in quality of life-measures.

Patients with schizophrenia
Two other studies in 12 (VR: n=6, psychotherapy control: n=6) and 15 patients (VR: n=9, psychotherapy control: n=6) respectively (La Paglia et al., 2013;La Paglia et al., 2016) had primarily focus on attention and executive functions. Training sessions of 90 minutes each were conducted weekly over ten weeks and involved of three VR tasks; (I) a task in a park consisted of catching a ball on irregular time intervals; (II) task in a valley, in which patients had to identify and pick up a specific flower with gradual increase in difficulty of distraction and the complexity of the flower and; (III) a task on a beach, where patients had to pick up a specific type of bottle, while also having to be attentive to cues signalling new activities. La Paglia et al. (2016) introduced a fourth task at the supermarket, where patients had to shop for specific products while following rules. These two studies reported no group by time interaction analyses for VR versus control group. However, the studies revealed exploratory within group results suggesting improvements in three of eight cognitive outcomes (La Paglia et al., 2013) and four of seven cognitive outcomes (La Paglia et al., 2016), respectively. In contrast the control groups only improved in one outcome measure (TMT-B). In both studies, improvements were found in general cognitive function and planning, while La Paglia et al. (2013) also found improvement of sustained attention.

Patients with attention deficit hyperactivity disorder
One study was conducted in children diagnosed with ADHD (VR: n=16, pharmacological treatment: n=16, psychotherapy: n=19)      (Bioulac et al., 2018; Table 1). Thirty minutes VR training sessions were conducted twice weekly over six weeks. Training targeted attention and was conducted in a virtual classroom environment where the children performed a letter detection task. The virtual environment was filled with increasing numbers of distractors such as auditory distractors (pencils being dropped, footsteps), visual distractors (paper airplanes), and mixed distractors (a car driving by outside of the windows). The sessions gradually increased in difficulty by adding more and more distractors. The authors did not assess whether there were any group by time interactions, but merely compared the groups cross-sectionally pre-treatment and post-treatment. These exploratory results revealed significantly better sustained attention in the VR group compared to the psychotherapy group post-treatment and less commission errors on both tests post-treatment compared to the methylphenidate group. The post-hoc tests survived correction for multiple comparisons using Tukey's test. No simulation sickness was reported, and all VR-group participants completed the study. Table 2 shows the four identified studies of VR training in MCI patients with sample sizes ranging from 21-68 participants. The studies report only limited significant improvements on the wide range of cognitive domains included. The study conducted by Park et al. (2019) in 21 patients (VR training: n=10, computer training: n=11) involved 15 tasks with usual daily life activities performed in a virtual setting of four rooms: a living room, a children's room, a kitchen and bathroom. An adaptive learning style was implemented such that the difficulty of the task was adjusted according to the patient's former answers. Training sessions of 30 minutes were conducted three times per week for six weeks. A significant group by time interaction was observed in one of nine outcomes. This effect on the visuospatial memory outcome survived corrections for multiple comparisons using the Tukey test.

Patients with mild cognitive impairments
In another study of 68 patients (VR: n=34, educational program: n=34) the training consisted of four tasks: (I) juice making in kitchen (pick a recipe, memorize and recall fruits), (II) crow shooting at beachside (shooting flying crows), (III) a firework task (remembering and recalling the order in which the numbers burst into fireworks), and (IV) love house (remembering positions of objects that needed to be reorganized after they are misplaced in the house) (Thapa et al., 2020). Training sessions of 100 minutes were conducted three times weekly for eight weeks. A significant group by time interaction was found in two out of four cognitive outcomes (measuring cognitive flexibility) in favor of the VR group. The results were not reported corrected for multiple comparisons and none of the effects survived the Bonferroni corrections.
Training in two studies, both with 34 patients (VR: n=18, cognitive and physical training: n=16), consisted of four different virtual situations: a metro transit, finding a store marked on a map, preparing a meal and buying certain items in store (Liao et al., 2019;Liao et al., 2020). The complexity of the meals in the kitchen task were gradually increased. Training sessions of 20 minutes (supplemented with 40 minutes tai-chi) were conducted three times weekly for 12 weeks. Liao et al. (2019) found a significant group by time interaction in cognitive flexibility in one out of five cognitive outcomes. This effect survived corrections for multiple comparisons using the Tukey test. Liao et al. (2020) found no significant group by time interactions for any of the cognitive outcomes, however they found significant with-in group improvements in global cognition and memory in the VR group, which was not seen in the control group.

Patients with stroke
One study included 17 patients with stroke (VR: n=9, desk-top VR: n=8) (Gamito et al., 2014; Table 3). The virtual environments consisted of a small town, a two-room apartment, and a mini-market. The training consisted of daily-life activities such as morning hygiene, meal preparation and dressing (choosing the right clothes to wear) and other activities of daily living such as buying several items, finding alternative routes to the mini-market and detecting certain people with specific features in a crowd. Training sessions (unknown length) were conducted weekly for nine weeks. No significant group by time interaction was demonstrated. However, significant within-group improvements in all domains were observed, including working memory, selective attention and visuospatial memory, for both VR and the control group of desk-top VR. The results were not reported corrected for multiple comparisons but did survive Bonferroni corrections in two out of three domains (working memory and selective attention).

Risk of bias evaluation
In Table 4, the Cochrane risk of bias evaluations are presented for the nine trials included in the review. The final evaluations consist of a 'high-risk' assessment for three of the studies and 'some concerns' the remaining six studies. The key concerns of the methodology used in the studies were lack of information about assessor and participant blinding (k=3) or no blinding at all (k=3), inadequate information about randomization including concealment of allocation (k=9), and suboptimal statistical analyses (k=4). Other common concerns were lack of prespecified plans of testing and analyses of effect in the studies with dropouts. Overall, all studies suffered from several challenges according to the Cochrane guidelines and the risk of bias was therefore moderate to high.

Discussion
This is the first systematic review of randomized controlled intervention trials investigating pro-cognitive effects of cognitive training in fully immersive VR scenarios in patients with neurological or psychiatric disorders. Nine trials were identified, including one in ADHD, three in schizophrenia, four in MCI and one in stroke. Group sizes were mostly small, ranging from n=6-34 participants (89% of the studies had <20 per intervention arm). The length and intensity of the interventions varied greatly, with session durations from 30-100 minutes and number of sessions ranging from 1-3 per week over 6-12 weeks. Four studies reported significant group by time interaction effects for 2-8 of the assessed 4-13 cognition outcomes of which all except one (Thapa et al., 2020) survived correction for multiple comparisons. The cognitive domains which improved significantly after VR training were theory of mind (Vass et al., 2020), visual working memory (Vass et al., 2020;Park et al., 2019) and executive functions (Thapa et al., 2020;Liao et al., 2019;Vass et al., 2020). All studies had low attrition rates and feasibility was generally high. Cochrane risk of bias assessments indicated either 'some concerns' or 'high risk of bias' in all studies, which was mainly due to a lack of blinding of assessors, patients and/or trainers, inadequate statistical analyses and insufficient reporting of the methodology.
The nine identified studies had different primary cognitive outcome measures and targets for the interventions, which impedes inferences regarding which specific cognitive domains are the most responsive to VR training. However, executive functions were improved in three studies (Thapa et al., 2020;Liao et al., 2020;Vass et al. 2020). Attention (Vass et al., 2020;Bioulac et al., 2018) and visuospatial memory (Park et al., 2019;Vass et al., 2020) were also improved in several studies. The VR-related executive functioning improvements may have clinical relevance since executive functioning is important for optimal utilization of other cognitive skills (Blair, 2017). In particular better ability to plan, use appropriate problem-solving strategies and attention shifting may help patients succeed in activities of daily living (Bell-McGinty et al., 2002).
Importantly, the four studies that found significantly greater cognitive improvements in VR versus control groups all involved training of activities of daily living (ADL), like cooking a meal or shopping, which require use of executive functioning (Tables 1-3). Naturalistic settings Table 3 Studies included with stroke patients as population. in the VR intervention involving tasks that resemble daily life activities seemed to show most cognitive improvements and may also translate into better functioning due to their high validity. However, only two studies (Vass et al., 2020, Liao et al., 2020 that found significant cognitive improvement reported measures of a possible transfer effect to daily life functioning and quality of life. One study measuring daily life functioning (Liao et al., 2020) found correlations between improved cognitive flexibility and improved daily life functioning. In contrast, another study measuring quality of life (Vass et al., 2020) showed no transfer of emotional visualization and conversations with simulated people to patients' quality of life. Taken together, the scarcity of evidence regarding potential transfer effects of VR-based cognitive training hampers any firm conclusions. Future VR cognition trials should therefore include assessments of functional measures in parallel with cognitive outcomes. A notable limitation of some of the included studies (La Paglia et al., 2013;La Paglia et al., 2016) is the use of dementia screeners, such as the Mini Mental State Examination (MMSE), as an outcome of treatment effect, since several studies have found that the MMSE have suboptimal sensitivity in neuropsychiatric disorders as it underestimates cognitive impairments especially in younger, better functioning patients (Faustman et al., 1990;Fisekovic et al., 2012). Two of the five studies that did not show significant group by time interactions, used the mini-mental-state-examination (MMSE). This may have masked any improvement due to ceiling effects in psychiatric populations (Faustman et al., 1990;Fisekovic et al., 2012). This highlights a need for more careful selection of sensitive outcome measures in future studies. The findings of this review show that application of VR for cognitive training and rehabilitation is a small yet rapidly growing field. This may also explain the overall preliminary and hypothesis-generating nature of the findings in the field. The next necessary steps arebased on this promising preliminary evidenceto conduct larger studies, which adhere to the Cochrane recommendations, before firm conclusions can be drawn regarding the efficacy of immersive VR interventions. An interesting point, which can be drawn from the transdiagnostic approach in the present review is that VR seems to be useful for cognitive training across diagnoses. Indeed, the results suggest that future use of VR to improve cognitive abilities could target several cognitive domains independent of diagnostic categories. The potential implications are more personalized training, which is adjusted to the specific neuropsychological profile and cognitive difficulties as they present in everyday life of the individual patient. In addition, there is a potential to adjust the specific VR scenarios to fit specific cultural backgrounds and age groups. This would allow for a more authentic and relevant experience for the patient which could increase motivation and engagement during cognitive training (Tieri et al., 2018;Makransky and Lilleholt, 2018).
A particular promising aspect of VR which should be studied further is the efficacy of strategy-based learning as part of the VR training. Only one study in this review (Vass et al., 2020) describes a phase of learning metacognitive and cognitive techniques as an integrated part of the VR sessions. This study revealed cognitive improvements across several domains suggesting that strategy learning may improve many aspects of cognition and perhaps also aid transfer to cognitive challenges in daily life. Although the findings in this review must be considered preliminary, they can in this way be used to guide future research into immersive VR cognition treatments.
A limitation of this review is, that it did not involve a quantification of the effects of the interventions through meta-analysis. There would, however, be several problems with conducting a meta-analysis on the studies included in this review. First, the studies are scarce and still in pilot phase which is also reflected in the moderate to high risk of bias. Second, there was a high diversity between the VR training scenarios as well as the control groups in the studies, which would complicate the comparison of the effects across studies. Even so, the pursuit of this review was not to draw any solid conclusions on the magnitude of effects of VR-based training, but to provide an umbrella perspective on this emerging field which can guide the design of future VR-based cognitive interventions.
In conclusion, emerging evidence from a small number of randomized controlled pilot studies indicate that immersive VR-based cognitive training is highly feasible and may improve aspects of cognition including executive functioning across neuropsychiatric disorders. No firm conclusions regarding the efficacy can be drawn due to small sample sizes and moderate to high risk of bias in the identified studies, which reflects methodological challenges such as a lack of patient and assessor blinding, suboptimal statistical analyses, and lack of prespecifying one primary outcome. Based on the promising findings, larger studies that adhere to the Cochrane recommendations for RCT's are now warranted. If such studies document similar feasibility and treatment-related cognitive improvements following VR training, this would have potential to aid treatment for cognitive impairments across a range of neuropsychiatric disorders.

Declaration of Competing Interest
KWM reports having received consultancy fees from Lundbeck and Janssen-Cilag in the past three years. AEJ, MS, KO and FSJ report no conflicts of interest.

Supplementary materials
Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.psychres.2021.113928. Domain 1: Risk of bias arising from the randomization process; Domain 2: Risk of bias due to deviations from the intended interventions (effect of assignment to intervention); Domain 3: Risk of bias due to missing outcome data; Domain 4: Risk of bias in measurement of the outcome; Domain 5: Risk of bias in selection of the reported result.