Efficacy of Virtual Reality Rehabilitation after Spinal Cord Injury: A Systematic Review

Background Spinal cord injury (SCI) is often associated with long-term impairments related to functional limitations in the sensorimotor system. The use of virtual reality (VR) technology may lead to increased motivation and engagement, besides allowing a wide range of possible tasks/exercises to be implemented in rehabilitation programs. The present review aims to investigate the possible benefits and efficacy of VR-based rehabilitation in individuals with SCI. Methods An electronically systematic search was performed in multiple databases (PubMed, BVS, Web of Science, Cochrane Central, and Scielo) up to May 2019. MESH terms and keywords were combined in a search strategy. Two reviewers independently selected the studies in accordance with eligibility criteria. The PEDro scale was used to score the methodological quality and risk of bias of the selected studies. Results Twenty-five studies (including 482 participants, 47.6 ± 9.5 years, 73% male) were selected and discussed. Overall, the studies used VR devices in different rehabilitation protocols to improve motor function, driving skills, balance, aerobic function, and pain level, as well as psychological and motivational aspects. A large amount of heterogeneity was observed as to the study design, VR protocols, and outcome measures used. Only seven studies (28%) had an excellent/good quality of evidence. However, substantial evidence for significant positive effects associated with VR therapy was found in most of the studies (88%), with no adverse events (88%) being reported. Conclusion Although the current evidence is limited, the findings suggest that VR-based rehabilitation in subjects with SCI may lead to positive effects on aerobic function, balance, pain level, and motor function recovery besides improving psychological/motivational aspects. Further high-quality studies are needed to provide a guideline to clinical practice and to draw robust conclusions about the potential benefits of VR therapy for SCI patients. Protocol details are registered on PROSPERO (registration number: CRD42016052629).


Background
Spinal cord injury (SCI) is a common neurological condition that often results in long-term impairments in physical function and psychological and socioeconomic status [1]. Because of functional limitations in the sensory and motor systems [2], which may involve both lower and upper limb functions [3], SCI drastically affects independence and quality of life [4]. Different types of training and stimulation protocols are commonly used to induce or facilitate processes of neural regeneration and plasticity, which might lead to significant functional recovery after SCI [5]. erefore, appropriate rehabilitation strategies are highly needed to regain sensorimotor function and reduce symptoms such as spasticity, imbalance, and neuropathic pain.
Recently, studies have used virtual reality (VR) as a promising tool for clinical rehabilitation in a variety of neurological disorders. For instance, VR-based technologies have been demonstrated to improve cognitive function after traumatic brain injury [6] and to promote balance control and gait recovery after stroke [7,8], cerebral palsy [9], and SCI [10,11]. VR makes use of advanced technologies (such as computers and multimedia peripherals) to provide an interactive and multidimensional simulated environment that users perceive as comparable with real-life experiences [11,12]. e advantage of VR-based technologies over conventional rehabilitation therapies has been associated with increased motivation, engagement [13], and the wide range of possible tasks/exercises that might be implemented [10].
In fact, VR-based interventions in patients with SCI have been demonstrated to improve motor function [14][15][16][17], neuropathic pain [14,18], balance [14,19,20], and aerobic function [21,22]. erefore, the use of VR-based rehabilitation in SCI clinical practice shows great promise; however, the evidence has not yet been formally reviewed or synthesized. To our knowledge, to date, there are no systematic reviews that integrate different results of the putative efficacy of VR in promoting sensorimotor recovery, as well as in reducing impairments and symptoms in patients after SCI. In order to provide a detailed and critical description of the effects of immersive or nonimmersive VR-based rehabilitation after SCI, we conducted a systematic review of published and unpublished studies up to May 2019.

Methods
is systematic review was performed in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [23,24] and registered as a predefined review protocol in PROSPERO (CRD42016052629).

Search
Strategy. An initial search was performed electronically in PubMed, BVS, Web of Science, Cochrane Central, and Scielo databases in order to identify studies published between January 1, 1980, and May 1, 2019. e grey literature was also searched in ClinicalTrials.gov and Health Services Research projects, as well as in generic Internet research engines, to avoid missing relevant unpublished studies. e identified keywords and Medical Subject Headings (MeSH) were combined by boolean logic using the following terms: "virtual reality" OR "virtual reality immersion therapy" OR "virtual reality therapy" OR "reality therapy" OR "game(s)" AND "spinal cord injury(ies)" OR "spinal cord trauma" OR "paraplegia" OR "tetraplegia." Additionally, the reference lists of all relevant literature were hand-searched to identify any additional suitable studies. Two reviewers performed the search independently.

Eligibility Criteria.
Eligible studies included a sample with adults aged between 18 and 65 years (both genders) with traumatic or nontraumatic SCI who underwent immersive or nonimmersive VR-based rehabilitation. Only full scientific papers were included, regardless of the levels of lesion and the levels of disability (as assessed by American Spinal Injury Association (ASIA) Impairment Scale (AIS) classification) of the samples. Randomized controlled trials were included, along with nonrandomized controlled trials, quasiexperimental studies, and before and after studies. Studies reporting validity and/or development of VR games or devices as well as transversal comparisons designed to investigate physiological mechanisms rather than clinical efficacy assessments were excluded. Conference papers and abstracts, as well as papers written in languages other than English, Spanish, or Portuguese, were also excluded.
To increase confidence of the selection process, two reviewers (AA and JN) independently screened the title and abstract of each reference identified by the search strategy. e full-text article of all potentially relevant eligible references was subsequently retrieved and further examined independently. Discrepancies between reviewers were reconciled by discussion or a third independent reviewer (FM).
All identified studies were stored in Mendeley ® , and duplicates were removed.

Assessment of Risk of Bias.
Two authors independently assessed the risk of bias and the methodological quality of the studies based on the PEDro scale, the most acceptable scale for rehabilitation research [25]. e scale assesses the presence or absence of randomization, allocation concealment, blinding of participants and/or researchers, homogeneity of the groups, intention-to-treat analysis, and presentation of statistical analysis. We considered the level of evidence as "excellent," "good," "fair," or "poor" for PEDro scores in the ranges of 9-10, 6-8, 4-5, and <4, respectively [25].

Data Extraction.
Two independent reviewers (AA and JN) filled out a data collection form on a customized Excel ® spreadsheet. Data included information on participant characteristics (sample size, age, sex, cause of SCI, level of injury, type of SCI, ASIA impairment level, and time after injury), descriptive studies' characteristics (time of publication, continent/country, type of VR, and rehabilitation objectives), methodological details (study design, dropout rate, type of therapy, VR characteristics, number and time of sessions, frequency by week, follow-up time, and outcome measurements), VR effects (statistically significant or nonstatistically significant results), risk of bias, size effects, statistical power, and limitations. e results of extraction were compared, and divergences were resolved by consensus.
Study design was analyzed by considering the following aspects: randomization, blinding, presence of the control group, bias, internal and external validity, and statistical power. e statistical power was low when β − 1 < 80% (α � 0.05).
e participant characteristic data were grouped and expressed as mean or percentage for better visualization of the sample profile. To form the VR therapeutic guideline, we collected the following characteristics: type of VR (immersive or nonimmersive), type of the VR device (commercial or developed by authors), number and time of sessions, frequency of therapy by week, type of therapy (VR alone or combined with other interventions), motivational aspects, and adverse effects.
Finally, we observed the positive or negative effects of VR-based rehabilitation so as to perform considerations about the clinical practice based on statistical significance when p < 0.05.

Search and Selection of Studies.
We identified 721 titles from searches in all databases (i.e., published and unpublished) and coming from the screened list of references (n � 2) for VR-based rehabilitation studies in individuals with SCI. We also found 20 unpublished studies, but none of them met the eligibility criteria because full text was not available. One hundred ninety-three studies were excluded by duplicates, 447 by title, and 35 by abstract. After full-text screening, 21 studies were excluded. Figure 1 shows the selection process of studies identified and included, along with the reasons for exclusion. At the final stage, 25 studies were included in the qualitative analyses (Table 1).

Design of the Studies.
From the 25 studies analyzed, 24 were prospective. Twelve of them used a pre-post design without control group. irteen were controlled in a parallel or crossover design (see Table 2). Only eleven studies used randomization to equally distribute the participants between groups of intervention. As to the blinding aspect, only two studies were double-blinded [10,33], in which neither researchers nor participants knew how the sample was distributed within the groups. ree studies [18,35,39] were single-blinded, in which only researchers or participants knew group allocation (see Table 2). Another aspect that deserves attention is the sample size, which ranged from 6 to 54 participants. However, most of the studies had a small sample size and did not report statistical power associated with the observed effects. e only nonprospective study was from Pozeg et al. [34], who used a 2-level factorial, randomized, repeated-measures design to investigate changes in perception of body ownership and neuropathic pain before and after experimental paradigms that combined virtual visual and tactile input.

VR Characteristics.
e studies used either VR alone or VR paired with other therapy(ies). Fifteen applied treatment with only VR, while ten combined VR with occupational therapy and physiotherapy or conventional therapy. So, these studies show results about VR as an adjuvant treatment (see Table 2).
Despite different types of VR devices used in the studies (see Table 2), most of the protocols used the games to provide stimuli that encourage movements to improve motor function, balance, aerobic function, and pain. Some studies also used the walking control of an avatar in a virtual environment [17,18], development of daily activities [10,16], or training of driving skills [15,29]. Both commercial and noncommercial VR devices were used in the studies. e total number of VR rehabilitation sessions ranged from 1 to 36. e intervals between VR-based interventions also varied between studies, with a frequency ranging from 1 to 5 times a week (with sessions lasting from 1 to 90 minutes). However, some studies did not clearly report any of the following information: amount of sessions [15,18,21,22,26,29,39], duration [13,15,16,20,26,29,30], or frequency [15-18, 21, 26, 29] (see Table 2).
Finally, most of the studies reported results of the short-term effect of VR. Follow-up assessments to verify long-term effects were performed only in nine studies [13,14,16,27,31,[35][36][37]39]. e follow-up time ranged between 4 and 30 weeks, and the pooled studies had a mean of 13.7 weeks of follow-up time ( Table 2). Table 2 depicts all outcome measures used to assess the effects of the VR-based interventions. It is noteworthy that most of the studies used more than one instrument or scale. Moreover, many different scales were used to evaluate the motor function of lower or upper limbs, balance, and pain. Studies also quantified the independence and level of daily activities through the Spinal Cord Independence Measure Scale, Barthel Index, and Functional Independence Measure. On the other hand, all studies with quantification of the level of aerobic function in response to VR used similar measurements of the heart rate, oxygen consumption, and energy/ metabolic expenditure. Five studies also used the quantitative variables such as the score or kinematic aspects of games or VR devices to evaluate the performance of SCI subjects in response to VR rehabilitation [15,16,[28][29][30] (see Table 2).

Outcome Measurements.
Despite some studies have used the same outcome measurements, the variability in population characteristics, VR device aspects, and VR rehabilitation methods among the studies makes it very difficult and not relevant to perform a meta-analysis.

Participant Characteristics.
e pooled sample of all studies included a total of 482 individuals with SCI. e dropout rate of participants was low, ranging from one to eight dropouts in nine studies. e majority of participants were men (73%), and the mean age was 47.6 ± 9.5 years. e cervical level of injury was observed in 52% of the pooled sample, whereas 35% and 13% had thoracic and lumbar injuries, respectively. Incomplete lesions were more frequently observed (64.3% of the pooled sample). Regarding the ASIA impairment level, only 19 studies (n � 227 participants) presented complete data on this classification. In these studies, ASIA impairment level "A" was observed in 46% of the participants, whereas ASIA impairment levels "B," "C," and "D" were observed in 17%, 15%, and 22% of the participants, respectively. We highlight some studies did not present complete information about the cause of SCI (48%), level of injury (20%), type of SCI (16%), ASIA impairment level (24%), or time after injury (20%). In addition, two studies did not report most of the injury characteristics [20,22].     e mean time after injury was 5 years (pooled analysis), which corresponds to the chronic stage of SCI. Only three studies included participants in the subacute stage of SCI (from ∼2 months to 1 year after injury) [29,33,35]. e detailed information and absolute numbers of each participant characteristic are shown in Table 3.

VR Adverse Effects.
Most of the studies did not directly report any adverse effect after VR therapy (88%). Only a reduced number of participants had a transient musculoskeletal pain (n � 2) [14,17], physical fatigue (n � 4), and difficulties to maintain attention (n � 2) [17] because of the increased use of limbs during the sessions of therapy. Moreover, Carlozzi et al. [15] reported acute simulator sickness during the protocol in seven participants.

Risk of Bias.
Risk of bias assessment showed that most of the analyzed studies involving the investigation of VR effects after SCI presented some level of potential bias. Only seven studies presented a low risk of bias associated with an excellent or good level of evidence (see Table 5).

Limitations of the Studies.
All studies had design limitations (see Risk of Bias) and hence restriction on internal or external validity. Most of the studies did not perform randomization [13, 14, 16, 17, 19, 20, 22, 26, 29-32, 36, 37] or blinding [10, 13-17, 19, 22, 26, 28-32, 34, 36, 37]. e small and heterogeneous samples of SCI subjects with a wide range of injury levels, cause of injury, and time after injury were frequently observed in all studies. erefore, the results of the studies included in this review cannot be generalized for the whole population with SCI, which represents an external validity limitation [40].
When accessible, the power of the statistical tests was low because of the small sample size used in most of the studies [41,42]. In addition, only three studies included in this review reported effect sizes [33,34,39], an important estimator of clinical significance [43,44].
Another limitation was the incomplete description of VR protocol characteristics (number of sessions, treatment frequency, duration, and training activities). Furthermore, a large variety of outcome measurements were  reported among the studies, which preclude objective conclusions on specific aspects to be drawn.

Discussion
To the best of our knowledge, this systematic review is the first study aimed at investigating the effects of immersive or nonimmersive VR-based rehabilitation after SCI. erefore, the present review provides a systematic overview and important guidelines for future research on VR-based rehabilitation for SCI individuals.

Summary of Main Results, Overall Completeness, and
Applicability of Evidence. We included twenty-five studies involving a total sample of 482 subjects with SCI. e currently findings describe eighteen years of VR-based rehabilitation after SCI as an emerging research area. Based on the present results, reviewed studies applied VR therapy to (1) improve motor function or motor skills, (2) restore balance, (3) improve aerobic function, (4) reduce the pain level, or (5) provide better psychological/motivational aspects.

Kowalczewski et al. [39] ✓
Chen et al. [7] ✓ An and Park [32] ✓ * * ✓ ✓ Khurana et al. [33] ✓ * * * ✓ Pozeg et al. [34] ✓ # ✓ ✓ Prasad et al. [35] ✓ van Dijsseldonk et al. [36] ✓ ## ✓ ✓ Villiger et al. [37] ✓ ### ✓ ✓ ✓ Note. * e study had statistically significant results only in one functional aspect measured. * * Overall limits of stability significantly improved, but directional forward and backward limits of stability did not differ significantly after therapy. * * * Modified Functional Reach Test (mFRT) and self-care components of the Spinal Cord Independence Measure III (SCIM III) significantly improved, but t-shirt test did not differ significantly after therapy. # Significant pain reduction when the lower back was stimulated synchronously with the virtual legs but no significant reductions for other conditions. ## Significant effects on 4 out of 9 spatiotemporal and stability measures of gait. ### Significant improvements on LEMS, BBS, and TUG, but no significant changes on 6minWT, SCIM III, and WISC-III.
caution. Several studies also reported subjective positive results on motivational aspects of VR treatment [14,27,30,39]. Indeed, previous studies have been considering VR as an interactive tool that provides a motivational environment associated with high engagement, which favors adherence to treatment [13,45]. is is especially important when rehabilitation require repetitive movements or extensive protocols [46,47].
Improved aerobic function and physical activity have been reported as beneficial effects of VR [21,22,26] on SCI. However, as few studies were aimed at assessing these aspects, the body of literature would certainly benefit from further investigations about the effects of VR-based protocols on aerobic performance, as VR protocols can be performed in safe and comfortable environments [48], besides allowing the trainers/therapists to set up the level of physical activity according to the physical fitness or functional limitation of the patient.
Most of the studies with adequate experimental design and positive effects on motor function investigated VRbased protocols paired with conventional rehabilitation [27,28,39]. Indeed, some ethical issues may arise regarding clinical trials designed to investigate isolated effects of a single (and sometimes novel) therapy. Such an investigation would require control groups to receive no treatment at all (even for conditions in which the current evidence points to effective treatments options), which might be considered not ethically appropriate [49]. erefore, the present review suggests that VR might be an important adjunct tool for conventional therapy [50][51][52][53][54], considering benefits such as the wide variety of VR-based protocols that might be designed, the potential transferring of functional activities performed in the virtual environment to activities of daily living [55], the high adaptability of VR protocols according to the patient's limitations/preferences, and the positive effect of feedback during training on VR-based settings [10,55,56]. Additionally, some studies found positive effects of VR alone on motor function [13,14,17,29,31,32,36,37], balance [13,14,19,20,31,36,37], aerobic function [21,22,26], and pain level [14,34]. us, the present study cannot conclude whether VR-based rehabilitation is more effective than conventional therapy. Nevertheless, the positive effects reported provide the support to recommend the use of VR as an adjunct to conventional therapies in clinical practice.
Both commercial and noncommercial VR devices were used in the revised studies. e frequent use of noncommercial devices (i.e., customized and specifically built for the rehabilitation purposes at hand) is probably due to the requirement of contemplating the specific needs of the patients as a function of the level of their physical limitations after SCI [55]. Further studies are needed to determine whether there are different rehabilitation outcomes related to the use of specific types of VR devices.
Despite the large differences observed among the VR protocols used in the studies, both immersive and nonimmersive environments were able to induce the performance of a wide range of specific and global functional movements while promoting motivation to perform the activities [9,50,55,57] and a safe rehabilitation setting with no adverse effects [9,50,55]. Taken together with the beneficial effects of the VR commented above, these aspects increase the potential use of VR as a rehabilitation tool after SCI. Table 5: PEDro scale scores, assessment of the level of evidence, and risk of bias of individuals studies (n � 25).

Studies
PEDro scale score Level of evidence Risk of bias Villiger et al. [14] 5 Fair High Carlozzi et al. [15] 6 Good Low Dimbwadyo-Terrer et al. [10] 5 Fair High Dimbwadyo-Terrer et al. [27] 10 Excellent Low Fizzotti et al. [30] 4 Fair High Gaffurini et al. [21] 4.2. Heterogeneity. e present review found a great level of heterogeneity, also reported in other VR systematic reviews in neurological disorders [9,50,54,55]. Overall, the studies presented a wide range of VR characteristics and protocols. So, it remains unclear which device elements, VR type, number/frequency of sessions, and duration of VR-based rehabilitation are essential to induce optimal recovery after SCI. In addition, divergent outcome measurements were used in the studies reviewed. Similarly, heterogeneity in the injury level, lesion cause, injury type, and time after injury was commonly observed in the studies' samples. Indeed, SCI is a heterogeneous condition in nature, with nonlinear recovery [58,59], which makes it difficult to run studies with homogeneous samples so as to establish the specific characteristics associated with better clinical outcomes. However, we observed that all studies applied VR-based rehabilitation in subjects with chronic SCI. So, the conclusion drawn in the present study can only be applied to individuals with chronic SCI, as none of the included studies assessed acute and subacute stages of SCI.

Quality of the Evidence.
Despite the increased use of VR technology in neurorehabilitation study protocols [38,54], it is not possible to draw strong conclusions about the efficacy of VR-based rehabilitation for patients with SCI because of the overall lack of methodological quality and statistical power observed in the current body of literature. Unfortunately, only seven studies included in the present systematic review had an excellent or good level of evidence (low risk of bias) (see Table 5). e same issue has been observed in other systematic reviews involving VR [9,50,54,55]. e lack of adequate study design (randomized, controlled, and blinded studies), powered sample size, and absence of effect size report are the most important limitations of the studies reviewed here. Putative flaws in study design are associated with increased risk for selection, performance, or detection bias, thereby compromising internal validity [42,60,61]. Similarly, the lack of a control group in pre-post designs may have compromised the evidence of the treatment effect [62] and does not allow conclusions to be drawn about the nature of the observed effects [56]. In addition, studies with low statistical power and small samples might involve type II errors [41,63] and hence low certainty of the detection of treatment effects.
Furthermore, some of the revised studies did not include information about the characteristics of VR rehabilitation protocols, reducing the possibility of replication by future studies. Future studies shall include appropriate description of training duration, frequency by week, and duration of sessions and detailed information about the virtual activities so as to allow putative associations between the observed effects and the specific training characteristics. e limitations found in the revised studies preclude the detailed analysis of the effects of VR on SCI, as well as the grouping of results in a meta-analysis. Future studies should avoid methodological limitations and should use and report adequate statistic power so as to identify the effects of VR rehabilitation and ensure robustness for proper quantitative data analysis (meta-analysis).

Future Research.
e present systematic review has important implications for future research. First, studies with participants in the acute and subacute stages of SCI are warranted.
ese phases are associated with greater potential for recovery and plasticity as compared to the chronic stage [2,58] and hence might comprise an interesting scenario for VR-based rehabilitation. In this vein, future studies should be able to determine the VR applicability according to the injury level and comorbidities. Second, studies should explore the ideal VR characteristics related to success of rehabilitation and the effect duration (short-term and long-term effects). ird, studies should use the standardized outcome measurements. Fourth, statistical powered studies with adequate methods and design are warranted in order to reduce bias and provide reliable results. Finally, we highlighted the importance of the effect size report and detailed description of the VR protocol in future studies.

Potential Biases and Limitations in the Review Process.
ere are several limitations of this systematic review that must be pointed out: (1) although we conducted an extensive search in the published and unpublished literature, some relevant studies might not have been identified, (2) it is possible that publication biases exist in this field of research, (3) it was not possible to perform a meta-analysis because of heterogeneity of outcome measures and VR protocols, and (4) our findings are based on studies with a wide variety of methodological qualities and should therefore be interpreted with caution in terms of generalizability.

Agreements and Disagreements with Other Studies or
Reviews. To our knowledge, no systematic review addressing the effectiveness of VR-based rehabilitation in subjects with SCI has been performed. However, we identified some current reviews on other neurologic disorders, such as stroke [50,51,54,55], Parkinson's disease [55], and cerebral palsy [9].
Overall, these reviews reported positive effects of VR therapy on gait [50,51,54,55], balance [50,54], and motor function [9,47,52,54]. e systematic review by Malloy and Milling [64] suggests beneficial effects of VR to reduce the pain level in a variety of pathological conditions. erefore, VR emerges as a promising tool to improve the performance of daily activities and quality of life [55,65].
Some of the studies report positive results when VR is used as an adjuvant therapy (i.e., VR paired with conventional therapy) [47,[50][51][52]54], whereas other studies suggest that VR and conventional therapy may have similar effects [9,55]. With the present review of the literature, it is not possible to conclude whether VR-based therapy in association with conventional therapy might provide more significant benefits for patients with SCI.

Conclusion
Overall, the studies that were included in the present systematic review reported a beneficial effect of VR therapy alone or VR associated with conventional rehabilitation. Initial evidence of VR to improve motor function, motor skills, balance, and aerobic function and to reduce the pain level was observed. us, our findings provide important implications for the VR-based rehabilitation research field. However, further studies should explore VR effects on SCI subjects considering the injury stage, level of lesion, and comorbidities. In addition, the related effects in response to VR characteristics and their specific applications should be studied. Similarly, evidence is required to provide information about VR long-term effects. Finally, high-quality studies are needed to provide robust guidelines and to draw conclusions about the potential benefits of VR before its integration into rehabilitation protocols for subjects with SCI.

Conflicts of Interest
e authors declare no conflicts of interest.