Decoding of motor intentions from epidural ECoG recordings in severely paralyzed chronic stroke patients

Objective. Recently, there have been several approaches to utilize a brain–computer interface (BCI) for rehabilitation with stroke patients or as an assistive device for the paralyzed. In this study we investigated whether up to seven different hand movement intentions can be decoded from epidural electrocorticography (ECoG) in chronic stroke patients. Approach. In a screening session we recorded epidural ECoG data over the ipsilesional motor cortex from four chronic stroke patients who had no residual hand movement. Data was analyzed offline using a support vector machine (SVM) to decode different movement intentions. Main results. We showed that up to seven hand movement intentions can be decoded with an average accuracy of 61% (chance level 15.6%). When reducing the number of classes, average accuracies up to 88% can be achieved for decoding three different movement intentions. Significance. The findings suggest that ipsilesional epidural ECoG can be used as a viable control signal for BCI-driven neuroprosthesis. Although patients showed no sign of residual hand movement, brain activity at the ipsilesional motor cortex still shows enough intention-related activity to decode different movement intentions with sufficient accuracy.

paralysis, a hemiparesis is the most common deficit after stroke and occurs in more than 80% of the patients [1]. In 30-66% of hemiparetic stroke patients, the paretic arm remains without function when measured six months after stroke [2]. Recent developments in BCI technology have introduced BCI as a new tool for rehabilitation in stroke patients [3][4][5][6][7][8]. By detecting the patients' intention to move then coupling this intention with feedback from an orthosis moving the hand, rehabilitation may be facilitated [3,4]. If stroke disrupts the connection between the sensorimotor cortex and the peripheral muscles, a coincident activation of the primary motor cortex and the sensory feedback loop may induce Hebbian plasticity [9] and thus support functional recovery [10].
Even though patients may not recover from the brain lesions caused by stroke, a BCI can serve as an assistive device to control artificial limbs and thereby compensate the loss of motor function in stroke patients. Furthermore, it has been shown recently in a double-blind study with severe chronic stroke patients that a well timed orthosis-BCI therapy coupled with physiotherapy can produce significant improvement in motor function [11].
While it is obvious that the patients profit from a high decoding accuracy of the movement intentions when using the BCI to control an artificial limb, a high accuracy and specific feedback may also be necessary when using BCI for rehabilitation. Since the main goal of BCI-guided stroke rehabilitation is to induce coincident activation of movement intention-related brain areas and the somato-sensory feedback corresponding to that same movement, it is important to detect intentional primary motor cortex activation as accurately and as specifically as possible.
A more specific (more degrees of freedom) coupling of haptic feedback with the patients' intention might further improve rehabilitation. Therefore, it could be advisable to differentiate between several movement intention states of the motor cortex and deliver appropriate sensory feedback according to its activation. Electrocorticography (ECoG) offers a higher spatial resolution and a better signal-to-noise ratio than electroencephalography (EEG) [12], thereby allowing for a more accurate and more specific BCI control than EEG [13,14]. That ECoG can be used for a more specific movement decoding was previously shown when movement of individual fingers [15], the arm movement direction [16,17], or the movement onset [18] was decoded. While these studies have been performed mainly in patients with an intact motor cortex, the literature is scarce with regard to ECoG for movement decoding in stroke patients.
However, it has been shown in one stroke patient that movement intention (movement vs. rest) can be decoded from ECoG with accuracies of about 90% [19]. Yanagisawa and colleagues [20] used ECoG from one stroke patient with residual hand movement to control a prosthetic hand, and three different types of movement could be discriminated with an accuracy of 68.3%.
Here we present decoding results from four severely affected chronic stroke patients with no residual hand movement during the execution of seven different hand movement intentions (six types of movement and rest) to introduce more accurate neuroprosthetic control for chronic stroke patients.

Patients
Four patients participated in this study. Patient P0 was a candidate for long-term cortical stimulation to reduce chronic pain. The patient underwent implantation of an electrode array for two weeks to determine treatment response and optimal stimulation sites for maximum pain reduction. Indications for implantation, array location, and duration of implantation were determined solely by clinical criteria. The patient also gave informed consent to a study exploring a BCI approach during this time, which was conducted in accordance with the Declaration of Helsinki and with the approval of the local ethics committee. Patients P1, P2, and P3 participated in a long-term investigational study for motor cortex stimulation with epidural implants concurrent with rehabilitation training to improve upper limb motor function after stroke. The study protocol, approved by the local ethics committee (Faculty of Medicine, University Hospital Tübingen), included an initial four week evaluation period immediately after implantation with externalized electrode leads to explore patients' individual cortical physiology for optimization of stimulation location and paradigms. The presented data was obtained during this evaluation period.
Patient P0 had a hemorrhagic stroke in the left thalamus. The ipsilesionally implanted electrode array consisted of 96 platinum electrodes (Ad-Tech Corp.) and was configured as an 8 × 12 electrode grid. The electrode pads had a diameter of 4 mm (2.3 mm exposed) with a center-to-center electrode distance of 5 mm. The grid was placed over the hand area of the ipsilesional primary motor cortex and also covered premotor and somatosensory areas, as can be seen in figure 1.
Patients P1, P2, and P3 presented lesions in cortical and subcortical areas on the right side and therefore suffered from left-sided chronic handparesis. Each of the three patients was implanted with 16 platinum disk electrodes (Medtronic Inc.) with a diameter of 4 mm, which were arranged in four strips with four electrodes each. The strips were placed in a grid-like fashion with a center-to-center distance of 1 cm. Although technically these are four strips, we will refer to them as one grid. These grids were placed above the hand area of the ipsilesional motor cortex and also covered premotor and sensory areas using an image-guidance approach [22], including intraoperative functional localizing of the cortical hand representation during surgery [23]. All strips were sutured separately to the dura to prevent accumulation of fluid underneath the electrodes and movement of the strips. The locations of the grids are shown in figure 1.
The upper limb motor function of all patients was scored using the Fugl-Meyer assessment [24]. A modified version of the assessment was used in which coordination and reflex activity was not tested. The results of the partial scores for upper extremity (FMA), wrist (FMB), and hand (FMC), as well as an overview of the demographic data, are shown in table 1.

Movement intention screening
Patients underwent one screening session to identify the most discriminating electrodes for the detection of movement intention. An outline of the paradigm is shown in figure 2.   Patients were asked to perform two different tasks with their paretic hand while the arm rests on their lap: a) the patient should either try to open their hand, hold the position, and go back to rest state (open-cycle), or b) try to close their hand, hold the closed position, and go back to rest state (closecycle). Each trial starts with a rest period of two seconds, during which the patient should not move and try to relax their hand. After this rest period, a three-second interval followed in which the patient was informed which of the two tasks would come next. During this phase the patient was instructed to stay relaxed. After the instruction phase (depending on the task), the patient had to try to open (or close) the hand for two seconds, hold this opened (or closed) hand position for another two seconds, then try to go back to the rest position by closing (or opening) the hand in the last two seconds. Both auditory and visual cues were given to help the patients with the timing. The recording took about 30 minutes and, on average, 93 ± 18 trials were recorded per patient. Patients were instructed to move despite their inability to move any part of the hand. During the screening, the ECoG signals were recorded with Brainamp DC (Brain Products GmbH, Munich, Germany) amplifiers at a sampling rate of 1000 Hz and a highpass filter at 0.16 Hz. To remove the 50 Hz noise caused by the power line, a notch filter was applied.

Offline analysis
Before decoding of movement intentions, the data obtained during the movement intention screening was analyzed offline to investigate which frequency ranges and electrodes carry more task-relevant information. Therefore, the signal was rereferenced to the common average by subtracting the mean over all channels at each time point. Subsequently, the power spectrum for each two second segment was estimated by an autoregressive model of order 50 with the Burg method [25]. Logarithmic power between 1-250 Hz in 1 Hz bins was used for the offline analysis. To determine which features allow a good discrimination between different movement intentions, we calculated the R 2 -values [26] between all relevant combinations of two movement intentions for each frequency bin at each electrode.

Offline decoding of movement intention
The data obtained during the movement intention screening was also used to estimate accuracies for the decoding of different movement intentions. The signal was rereferenced to the common average by subtracting the mean over all channels at each time point. As a next step, the power spectrum over each two-second segment was estimated by an autoregressive model of order 50 with the Burg method [25]. Logarithmic power of 5 Hz frequency bins were used from 5-150 Hz as features for classification. Before classification, R 2 -values were used to select the most important features [27]. The R 2 -values were calculated for each combination of two classes present in this classification step and averaged to obtain one R 2 -value for each feature. To reduce dimensionality of the input space, the mean R 2 -value averaged over all features was calculated, and only features with an R 2 -value larger than the mean were used for classification. For classification we used a Support Vector Machine (SVM) [28] implemented in the LibSVM Toolbox [29] with a linear kernel and the regularization parameter set to C = 1. To estimate accuracies we used a 10x10-fold cross-validation in which the data was permuted and partitioned into 10 blocks of equal size. In each of the 10 folds, nine blocks were used for training the classifier and tested on the one remaining block. Each block was used for testing once. This procedure was repeated 10 times, and the accuracy was averaged over all folds.

Spatial and spectral localization of features
To determine which features (electrodes and frequency range) are most important for the discrimination of different movement intentions, we performed an offline analysis of the data, and R 2 -values were calculated for all relevant class combinations and for each electrode and each frequency bin.
In general, a movement-related desynchronization was found in the frequencies below 40 Hz, and a movementrelated synchronization above 60 Hz. To document which features present the highest difference between classes and which frequency bands or electrodes hold the most information for discriminating between different movement types, exemplary R 2 -values are shown for patient P2 in figures 3 and 4. The figures for the remaining patients can be found in the supplementary materials.
In figure 3, the R 2 -values for different frequency bands are shown topographically, and the R 2 -values averaged over all electrodes and for the best electrode are also shown in more detail. The best electrode refers to the electrode with the highest R 2 -value averaged over all frequencies for a given patient. For patient P0, the best electrode is located over M1, while the best electrode for P1 and P2 is located over S1. For patient P3, the best electrode is located between S1 and M1.
For the topographic plots as well as the averaged plots, the R 2 -values have been calculated for combinations of two classes and were averaged over all relevant combinations. In general, the activity in the alpha band (8-12 Hz) and beta band (15-30 Hz) is widespread, and most electrodes over the grid contain movement-related information in those frequency bands. For the higher frequency ranges, especially in the range >80 Hz, the activity is localized, and only a few electrodes contain movement-related information in those higher frequency ranges, which is in line with previous ECoG results [13,16,30]. Figure 4 shows the R 2 -values for selected two-class combinations. When looking at two similar conditions like rest and hold, the R 2 -values are higher in the frequency range above 80 Hz and also more localized to a few electrodes. In addition, patterns are visible between similar movement

Decoding of movement intentions
To estimate how well different movement intentions can be decoded, different class combinations and parameters were tested by cross-validation.
Results from the cross-validation with combinations of three classes are presented in table 2. Only class combinations which could be used either in rehabilitation or for control of a prosthetic device are shown. The best accuracy was achieved for the combination rest, close hand, hold closed position, with an average of 88.3%.
The results from the cross-validation with combinations of four and five classes are shown in table 3, while  table 4 Figure 5 shows the confusion matrix for the cross-validation when decoding four classes. In general the rest and the open (close) condition could be classified best, while the hold open (hold close) and back to rest condition were misclassified more often. Confusion matrices for six and seven classes can be found in the supplementary materials.
An additional analysis was performed to investigate the influence of the different frequency ranges used for the decoding. Figure 6 shows classification results with seven classes, using different frequency ranges. Regarding the use of different frequency ranges for decoding, it can be seen that the range of 5-150 Hz gives the best accuracy, with an average of 61.33%. When looking at the different frequency ranges of 5-40 Hz, 40-80 Hz, and 80-150 Hz, the range of 5-40 Hz yields better results for subjects P0 and P3, while 40-80 Hz results in better decoding accuracies for subjects P1 and P2. On average, the frequency range of 40-80 Hz with 54.9% allows superior accuracy to the 5-40 Hz range with 51.3% or the 80-150 Hz with 49.1%. Since the accuracy for the frequency range 5-150 Hz is higher than for 5-80 Hz (with 59.5%), the frequency range 80-150 Hz still holds complementary information that improves the accuracy. The accuracy for 5-250 Hz is on average lower than 5-150 Hz, suggesting that the frequency range of 150-250 Hz holds no additional information. Classification using only the frequency range of 150-250 Hz yielded an average accuracy of 36.3%.

Discussion
In this paper we investigated if and how well seven different hand movement intentions can be decoded from epidural ECoG recordings in severely paralyzed chronic stroke patients. The fact that paralyzed patients can use ECoG as a control signal for a neuroprosthesis has been previously demonstrated [31,32]. While it has also been shown that two [19] or three different movement intentions [20] can be discriminated from ECoG in chronic stroke patients, we demonstrated that up to seven different hand movement intentions can be decoded with accuracies greater than 60%. Although the patients could not perform visible hand flexions and extensions, we still decoded seven different movement intentions of the paralyzed hand using epidural ECoG implanted over the ispsilesional hemisphere (contralateral to the paralyzed hand). It is also important to note that in our analysis we used very similar movement conditions like open hold and close hold that are extremely similar; perhaps fusing them into a one single movement condition could have improved our decoding results. However, we wanted to demonstrate that even subtle differences between motor conditions could be identified using ipsilesional epidural ECoG signals. This result is important for BCI-based stroke rehabilitation: 1) chronic patients with complete paralysis can still try to move the paralyzed limb, and this intentional neural activation can be decoded and used for prosthetic or orthetic control; and 2) the result indicates that less invasive and therefore less traumatic recordings (compared to subdural ones) could suffice for the decoding of functionally relevant movements that could be implemented and improve BCI-based stroke rehabilitation [11]. Furthermore, the results show that although motor-related brain areas are damaged by stroke, resulting in chronic severe motor impairment, in these patients there are enough intact ipsilesional brain structures available that produce electrical activity and allow decoding of different movement intentions of the same joint. Since electromyography (EMG) activity was not recorded, we can not completely rule out the possibility that despite hand paralysis, small muscle contractions produced somatosensory activation, which might have contributed to the movement intention decoding.  Although it would be interesting to see if there are any correlations between lesion size or location, functional scores and decoding performance, we refrained from doing such an analysis due to the insufficient number of patients and the heterogeneous nature of the patients' neurological condition.
Concerning the spatial and spectral distribution of the features that are most relevant for the differentiation of the different movement intentions, it is obvious that the activation in the alpha and beta band is rather widespread, while the activation in the higher frequency range (>40 Hz) is localized to only a few electrodes on the cortex. While frequencies >40 Hz play an important role in classification, even frequencies up to 150 Hz can yield important class-specific information, which is in line with reports from epilepsy patients with a healthy motor system [33,34] and was also shown for chronic stroke patients with moderate motor dysfunction [32].
In conclusion, we could show that up to seven different hand movement intentions can be decoded from ipsilesional epidural ECoG in chronic stroke patients, although the patients were not able to perform any voluntary movement with their paretic hand. These results show that ipsilesional epidural ECoG can be used as a viable control signal in chronic severely paralyzed stroke patients to drive a prosthetic or orthotic device. Whether implantation of epidural electrodes in severe chronic stroke allows a superior rehabilitation outcome to EEG-BCI-based rehabilitation programs-and outweighs the risks and costs of an invasive procedure-is an empirical question which still awaits a solution.