Study design

This prospective, multicenter, single-blinded, parallel-group randomized trial was conducted in the outpatient departments of three rehabilitation hospitals in the German and French speaking parts of Switzerland: University hospital Inselspital Bern, Buergerspital Solothurn, and Reha Rheinfelden. In the study plan, each hospital was responsible for the recruitment, assessment, and therapy of 20 patients: 10 patients for the experimental group (EG) and 10 for the control group (CG), respectively.

More details regarding the study methodology can be found in the study flow chart in Fig 1 and the previously published study protocol strictly followed by each center (http://trialsjournal.biomedcentral.com/articles/10.1186/1745-6215-15-350) [11]. Ethics approval was warranted by the ethics committee of the Canton Aargau (2012/065) and the Canton Berne (220/12). The study was registered with ClinicalTrials.gov: NCT01774669 before the start of patient recruitment.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Patient flow chart.

BS = Buergerspital Solothurn, IS = Inselspital Bern, Reha Rheinfelden Measurement sessions: twice within one to two weeks before intervention start (BL, T0), once after eight (T1) and after 16 (T2) intervention sessions, and after a two months follow-up period (FU).

https://doi.org/10.1371/journal.pone.0204455.g001

Participants

In order to meet the inclusion criteria, each patient had to be at least six months after his or her first-ever stroke (ischemic or hemorrhagic) with a persistent motor deficit of the arm and hand, indicated by a Chedoke-McMaster Stroke Assessment (CMSA) score of three or greater on the arm subscale and two or greater on the hand subscale. If one of the CMSA subscales scored seven, the difference to the other subscale had to be at least two. Patients had to be able to sit in a normal chair without armrests or backrest support and to score at least one on the Box and Block Test, which was the primary outcome measure. Patients were excluded from the study if they had previous or current functional deficits of the arm and hand motor function not due to stroke, severe cognitive deficits indicated by a Mini-Mental State Examination score of 20 or lower, severe visual disorders, or a history of epileptic seizures triggered by visual stimuli within the past six months.

Patients were informed about the study in oral and written form and gave written informed consent before data collection began.

Furthermore, for descriptive purposes patients were evaluated with the Extended Barthel Index (EBI) [12], the Edinburgh Handedness Inventory (EHI) [13], the Mini Mental State Examination (MMSE) [14], and with the Line Bisection Test (LBS) [15]. Table 1 provides an overview of all outcome measures and measurement sessions.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Outcome measures and measurement sessions.

https://doi.org/10.1371/journal.pone.0204455.t001

Randomization and masking

If a patient met all eligibility criteria, he/she was randomly assigned to either the experimental group (EG) or the control group (CG) after the second baseline measurement session.

Group allocation (1:1 ratio) was based on one computer-generated randomization list for all centers. The randomization list was created on blocks of 10 and was generated by a researcher not involved in the study (MATLAB release 2007b; MathWorks, Natick, MA, USA).

The randomization list was stored at one clinic’s pharmacy only to avoid disclosing the group allocation. The treating therapist of the respective clinic called the pharmacy before the first therapy or training appointment of the respective patient to obtain the group allocation. Only study therapists, who were specifically trained in the study methodology, were allowed to call, which was verified by the pharmacist on duty. Group allocation was not recorded on any assessment document. Group allocation was only noticeable based on the selected training or therapy documentation sheets that were kept locked and separated from the assessment documents. A row of measures was implemented to keep group allocation concealed from the blinded assessors until the last follow-up measurement session of the last patient. Two patients disclosed their group allocation during the third out of five measurement sessions. In these cases, the measurement sessions three to five were video recorded to ensure objective and unbiased assessment scoring.

Procedures

Patients were assessed at five measurement sessions: twice within one to two weeks before intervention start (BL, T0), once after eight (T1) and after 16 (T2) intervention sessions, and after a two-month follow-up period (FU). For statistical analyses, average scores of BL and T0 were used as one pre-intervention score = (BL+T0)/2.

The intervention consisted of four 45-minute training sessions per week over a four-week period for patients in both groups. The EG underwent a virtual reality-based training for all 16 sessions and the CG underwent conventional physiotherapy or occupational therapy. The therapy and virtual reality-based training are described using the Template for Intervention Description and Replication (TIDieR) in Table 2 [16].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Study intervention description based on the TIDieR checklist [16].

https://doi.org/10.1371/journal.pone.0204455.t002

If patients received any kind of additional therapy before trial participation, it was reduced or suspended for the course of the study if the patient agreed. If the additional therapy had to be continued or could not be reduced, it was ensured that its focus was on lower extremity treatments.

Outcomes

Table 1 lists the primary outcome measure and all secondary outcome measures for each measurement session. Adverse events were registered and transferred to the responsible ethics committee if the applicable criteria for transfer were met. A detailed description of all outcome measures can be found in the published study protocol (https://trialsjournal.biomedcentral.com/articles/10.1186/1745-6215-15-350).

Changes in hand dexterity between T0 and T2 were measured with the BBT, which was described by Mathiowetz et al. in 1985 [20]. Patients were asked to grasp small wooden cubes and move them from one side of the box to the other as fast as possible within 60 seconds. The BBT provides normative data for healthy individuals in age groups ranging from 20 years to older than 75. A change of five or six cubes before and after an intervention seems to be the smallest real difference [21].

The CAHAI-13 was developed by Barreca et al. in 2004 [22–25]. It contains 13 bimanually performed real-life items. Scores represent the patient’s relative ability to independently perform stabilisation or manipulation in ADL with the impaired upper limb. A score of one represents total dependence on another person, and a score of seven indicates patient independence without time or safety concerns or necessary splints or devices.

The SIS is a questionnaire comprising questions regarding the impact of stroke on physical function, emotion, memory, communication and social participation. The SIS was developed by Duncan and colleagues and has been modified in recent years [26–28]. The current version, 3.0, consists of eight subscales (strength, hand function, mobility, ADL, emotion, memory, communication and participation) administered in a one-to-one interview. Patients can rate the level of their stroke’s impact on a 5-point Likert scale. The higher the score, the less affected the patient perceives his or her current status to be.

The Extended Barthel Index was used for patient evaluation of independence in ADL [12]. The EBI comprises 16 items on mobility, ADL and cognitive function. Scoring ranges from zero to four with four indicating the highest level of independence.

The CMSA was developed by Gowland et al. in 1995 for the evaluation of physical impairment and activity level of stroke patients [29]. We used the impairment subscales for hand and arm function that was scored on a seven-point scale (1 = hypoactive or absent muscle reflexes, 7 = no functional impairment detectable anymore, prestroke status) according to seven stages of motor recovery [30]. Additionally, the subscale of shoulder pain of the affected body side was administered on the same seven-point scale.

The Edinburgh Handedness Inventory was used to assess hand laterality [13]. The questionnaire included 12 daily activities were participants had to determine their preferred hand (right/left).

For cognitive screening the Mini-Mental State Examination (MMSE) was conducted. The MMSE comprised 30 items and patients could achieve zero to 30 points (indicating the highest scoring) [14].

The Line bisection test (LBT) is a paper-and-pencil test used to evaluate the presence of unilateral spatial neglect [15]. Patients were asked to mark the centre of 18 drawn lines on paper with a pencil.

Statistical analyses

The sample size was calculated based on the basis of an earlier efficacy study, in which the virtual reality-based system was tested in children with cerebral palsy [9]. A power analysis and a sample size calculation for the present study were performed using G*Power software version 3.1.5 [31]. In the cited study of children, the BBT (primary outcome measure) showed an effect size of Cohen’ s d = 0.98. Assuming a similar effect size for adult stroke patients, a total of 46 patients (23 per group) had to be included: two-tailed test, power = 0.9, significance level α = 0.05. Assuming a dropout rate of 20%, we thus planned to recruit a total of 60 patients across the participating centers.

All statistical analyses were conducted using the Statistical Package for Social Sciences (SPSS) version 23.0 (IBM, 290 Armonk, New York, USA) with a two-sided significance level of p≤0.05 as an intention-to-treat analysis.

Variables were checked for normal distribution with the Shapiro-Wilk test.

1. Due to observable mean differences at baseline we checked for baseline significant differences for age, time since stroke, BBT, CAHAI, and SIS subscale hand function (subscale 7). Due to non-normal distribution, we used the Mann-Whitney U-Test that did not reveal significant differences between EG and CG at BL.
2. Due to non-normal distribution of primary and secondary outcome measures we calculated the differences between each measurement session. Again, the differences were tested for normal distribution, which was not the case. Subsequently, the differences were tested using the Friedman test for repeated measures to determine changes over time for both groups. The Mann-Whitney U-Test was used to test for group differences of the changes for each measurement session.
3. All outcome analyses were intention-to-treat analyses with missing values replaced with two methods: carrying the last observable value forward or backward and by adding or subtracting the mean change of the group [32, 33]. This sensitivity analysis did not materially change the results. For the primary outcome analysis, the differences of the BBT scores of the paretic side for each measurement session were the dependent variables. For the secondary outcome analyses, dependent variables were the difference scores of the CAHAI, the SIS subscales 1, 5, 6, 7, 9, the SIS mobility index, and the BBT scores of the non-paretic side. To correct the p-value for multiple comparisons the Bonferroni adjustment (p = 0·05/k) was used, where k represents the number of tests for significance (k = 3) [34].
4. No post-hoc power analysis was conducted due to the lack of a group interaction effect. Nevertheless, we calculated the effect for the four-week training intervention for each group separately and the standardized mean difference between groups with the following formula: Kazis’ effect size = (pre-intervention score-post-intervention score)/standard deviation of pre-intervention score [35].

Comparison with other studies

Repetition of movement is one of the fundamental principles for motor re-learning and facilitating brain plasticity to improve motor function [37]. In our study, the differences in the subjectively improved strength measured with the SIS could be explained by the very high numbers of repetitions of arm and finger movements preformed during the YG games. The training in EG was very much focused on active finger, hand and arm movements that could lead up to 5000 grasp movements over 16 training sessions. However, only patients in the CG performed a resistance training with heavy objects, e.g. weights but with a lesser amount of repetitions.

The main scoring increase occurred within the first eight training sessions that were scheduled during the first two weeks of the intervention. During the second two intervention weeks, scores further increased albeit to a lesser degree or they remained on the same level. The Yerkes-Dodson Law, first explained by Yerkes and Dodson in 1908, is a dose-response framework that describes a relationship between arousal or motivation and performance and might help to explain that phenomenon [5]. It indicates that a low level of task difficulties elicits linear responses. Reaching a higher level of difficulty, the relationship becomes inverse and increases in arousal could cause decreases in performance. From our patients, we know that they enjoyed playing the virtual reality-based games and always reached for a higher score or a faster performance [38]. The intensive training could have reached a point when a higher intensity is necessary to push the functional improvements and patient motivation further, e.g. longer than 45 minutes per day or more than four times per week.

It is worth mentioning that patients in both groups scored approximately 20 points below the average score in the BBT with their non-paretic upper limb compared to healthy individuals of the same mean age. Mathiowetz et al. suggested a norm score above 70 for healthy women and men for the left and right side [20]. Patients in both groups improved their BBT scoring with 4.6 (EG) and 4.4 (CG) from baseline to follow-up. That distinct scoring change of the non-paretic upper limb over the four measurement sessions is somewhat surprising. One would assume that patients 2.5 to 3.5 years after stroke would use the non-paretic upper limb more throughout the day to compensate for the reduced motor function of the paretic side and would have developed even more hand and arm dexterity over time compared to healthy individuals. However, it could be hypothesized that the non-paretic hand/arm compensated for the non-use of the paretic hand/arm. Furthermore, the reduced overall activity and motor function after stroke and intensive carer assistance could have led to a more sedentary lifestyle and a learned non-use for the non-paretic side as well. A BBT performance difference of the paretic and non-paretic arm depending on side of brain lesion as suggested by Steward et al. could not be detected [39].

Overall, our results are in line with the previous publications comparing virtual reality-based training using a commercial consumer virtual reality training systems, e.g. Nintendo Wii, XBox, Playstation with conventional therapy including patients in a chronic stage after stroke [40–42]. One difference in our study was that we used a system that was specifically developed for patients with sensorimotor impairments. The system offers different display modes of the arms (a) real left/right hands control their virtual counterparts, (b) virtual mirror therapy, in which one real hand controls both virtual arms or the contralateral virtual hand in a mirrored fashion, or (c) virtual following that is the same as for virtual mirror therapy, but without mirroring [43].

Our results continue the work from Saposnik et al., who included 121 light to moderately impaired inpatients in a subacute stage after stroke [44]. They used a commercial virtual reality-based system for a two-week training program including ten 60-minute sessions as an adjunct to the multidisciplinary program and found significant improvements over time for both groups but no between-group differences.

Furthermore, our results are in line with findings from a recent systematic review on virtual reality-based training in stroke rehabilitation from Aminov et al. [45]. Researchers reported small to moderate effect sizes in favor of virtual reality-based training ideally performed with a purposed-designed virtual reality design system similar to the system used in our multi-center study. Supporting our study design the authors of the systematic review and meta-analysis reported higher effects in favor of virtual reality-based training for patients three or more months post stroke, with more than 15 training sessions in total including more than three training sessions per week adding up to more than 100 minutes per week. We also included outcome measures to evaluate all three main categories of the International Classification of Function: body structure and function, activity, and participation.

Future research directions should consider the potential of virtual reality-based training system to increase the efficiency of training in terms of human resources. Currently, the number of available therapists is not sufficient compared to the increasing number of patients. To supply the required dose of therapy to the number of patients after stroke experiencing motor deficits, it would be interesting if our results could be replicated with virtual reality-based group training sessions compared to individually supervised trainings. With virtual reality, most of the time of the therapy session could be automated and therefore could be completed without the constant supervision of a therapist. Furthermore, a system could even be installed in the patient’s home and the setup instructions given remotely, removing the burden of clinic visits.

Limitations and strengths

One limitation of our study is the number of patients per group. Despite the well-prepared randomization scheme the imbalances occurred by chance. Patients were randomized at study entry but held confidential until they passed both baseline assessments (BL, T0). However, it happened that patients did not pass the baseline assessments or had to be excluded for several reasons, which led to an uneven number of patients per group and could have therefore led to an under- or overestimation of the effect of the virtual reality-based training.

A common problem in RCTs investigating therapies or training procedures is in blinding the patients regarding their group allocation. In our study, only one patient withdrew study participation after being randomized into the control group. All other patients stated that they were happy with their group allocation and showed high compliance. The study had a very low dropout rate of 1.6%.

Our sample size calculation was based on a previous study that used the same VR-based training system and included children with cerebral palsy. Their effect size was higher compared to other VR-based training systems [8].

A remarkable strength of our study is the inclusion of moderate to severely impaired patients in both groups, who were able to move at least one wooden cube in the BBT. The adjustable virtual reality-based system was adaptable to amplify even very tiny movements and therefore enabled the patient to play or train with his/her severely impaired hand or arm. That could have led to an increased motivation and desire to move or use the paretic upper limb during the day more often.

From Kwakkel et al. and Verheyden et al. we know that recovery occurs mainly in the first six month [46, 47]. However, the majority of our patients started later than six months post stroke. Their improvement could reflect some training effect on a functional level or the use of behavioural adaptation strategies rather than by restitution of existing underlying impairments themselves.

In our study, it was an important aspect to open the black box of conventional therapy and provide the clinician with specific information on the therapy content of the CG. Therefore, we analyzed and summarized the content of the therapy down to a very detailed level based on the classification system laid out by Pollock and colleagues in their Cochrane review on interventions for improving upper limb function after stroke [48].