Decoding natural reach-and-grasp actions from human EEG

Objective. Despite the high number of degrees of freedom of the human hand, most actions of daily life can be executed incorporating only palmar, pincer and lateral grasp. In this study we attempt to discriminate these three different executed reach-and-grasp actions utilizing their EEG neural correlates. Approach. In a cue-guided experiment, 15 healthy individuals were asked to perform these actions using daily life objects. We recorded 72 trials for each reach-and-grasp condition and from a no-movement condition. Main results. Using low-frequency time domain features from 0.3 to 3 Hz, we achieved binary classification accuracies of 72.4%, STD  ±  5.8% between grasp types, for grasps versus no-movement condition peak performances of 93.5%, STD  ±  4.6% could be reached. In an offline multiclass classification scenario which incorporated not only all reach-and-grasp actions but also the no-movement condition, the highest performance could be reached using a window of 1000 ms for feature extraction. Classification performance peaked at 65.9%, STD  ±  8.1%. Underlying neural correlates of the reach-and-grasp actions, investigated over the primary motor cortex, showed significant differences starting from approximately 800 ms to 1200 ms after the movement onset which is also the same time frame where classification performance reached its maximum. Significance. We could show that it is possible to discriminate three executed reach-and-grasp actions prominent in people’s everyday use from non-invasive EEG. Underlying neural correlates showed significant differences between all tested conditions. These findings will eventually contribute to our attempt of controlling a neuroprosthesis in a natural and intuitive way, which could ultimately benefit motor impaired end users in their daily life actions.


Introduction
When asking a tetraplegic spinal cord injured (SCI) person, from which lost function they would benefit most, three quarters make regaining arm/hand function their priority choice [1,2] (2nd elimination of Dysreflexia, 3rd sexual function).
Yet up to this point this population is waiting for an intervention that will improve their functional ability [3].
Neuroprosthesis based on functional electrical stimulation (FES) may present a technical solution. Small electric pulses stimulate still innervated paralysed arm muscles. In this way, the neuroprosthesis is able to restore hand functions, especially different grasps [4,5] on demand. Studies from the early 2000s show that neuroprosthetic control can be achieved with non-invasive brain-computer interfaces (BCI) [6,7]. S Supplementary material for this article is available online (Some figures may appear in colour only in the online journal) Original content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.
These non-invasive BCIs enable its users to interact with their environment by means of changes in brain activity captured by the electroencephalographic signals (EEG). BCI control strategies usually rely on focused attention to an external stimuli [8][9][10][11] or on specific mental strategies [12][13][14].
Pfurtscheller et al [15] applied combined BCI-FES technology to restore left hand functions of a tetraplegic end user (complete SCI, lesion height C5, neither hand nor finger function, residual elbow function). After ten months of FES muscle training (five times a week, 45 min each session) and a period of four months of BCI training, the end user was able to perform a palmar grasp and use a glass using motor imagination (MI) of repeated foot movement as a control signal. In later studies [16,17], Rohm et al refined the approach by also including functional control of the elbow. They extended BCI control by introducing new mental tasks and temporal coded [18] BCI commands.
So far, BCI-based neuroprosthesis control has strongly relied on the (repeated) imagination of basic motor tasks e.g. repeated planar extension/flexion of both feet [15], or repeated MI of opening/closing left or right hand [16,17]. From a user's perspective it seems rather unnatural to perform specific foot MI for controlling one's hand functions. Even contralateral hand MI [6,16,17,19] feels unnatural and does not support a natural feeling of control.
We believe that for a more natural and intuitive control of an upper limb neuroprosthesis it is essential to focus on the successful decoding of more complex and natural hand/ arm movements, such as different grasp actions. In most daily life scenarios, a grasp is combined with a reaching movement towards an object. Studies investigating grasp kinematics imply that the hand preshapes already during the reaching phase, whereas maximum grip aperture occurs within 70% of movement completion [20,21]. Although the human hand incorporates a large amount of degrees of freedom, yet only three main grasps-palmar, pincer and lateral grasp-are necessary to perform most daily life actions [22].
Previously, it has been shown that it is possible to detect and discriminate different reach-and-grasp actions from human electrocorticogram (ECoG) [23,24]. Results indicated that discriminative information for movement detection and classification can be found in the amplitude modulations of frequencies below 6 Hz.
In the time domain, cortical activations in this frequency range are known as movement-related cortical potentials (MRCP). They are described by a negative shift in amplitude during movement preparation, reaching its maximum negativity imminently to the actual movement onset (Bereitschaftspotential). Thereafter a positive rebound occurs which ultimately returns to a baseline level [25]. The shape of these potentials may vary depending on various factors, such as the movement task [26][27][28], force [29,30] or movement speed [31].
EEG based studies investigating MRCPs [25] indicate that this information can also be exploited non-invasively, not only for directional information [26], but also for the analysis of hand shape during a grasping movement [32], to study the effects of grasp force [33] and ultimately to discriminate different grasps [31,[34][35][36]. The intention of movement has been investigated in palmar, pincer and lateral grasps [31].
Of particular interest are two studies from Agashe et al [32,37] who attempted low-frequency reach-to-grasp decoding and classification incorporating palmar and lateral-precision grasp on various objects. In a follow-up multi-session study incorporating two amputee end users controlling a robotic hand, they show the feasibility and success of their efforts.
In our current study we want to add up to their prior work and further investigate the whole reach-and-grasp process in a daily life setting using common objects of daily life.
In our experimental setup we chose three different reachand-grasp actions most commonly used in daily life: (i) palmar grasp, (ii) pincer grasp and (iii) lateral (key) grasp. We hypothesize that these executed reach-and-grasp actions can be discriminated significantly better than chance (I) against each other and (II) against the no-movement condition, based on low frequency EEG activity. We test our hypothesis in 15 healthy volunteers using binary and multiclass classification approaches and also show the difference between the low frequency neural correlates of the movements. Finally we discuss the potential of our results for online application to facilitate artificial control.

Subjects
The study was approved by the ethics committee of the Medical University of Graz (ek28-108 15/16). Fifteen righthanded subjects, seven male, eight female, aged between 23 and 37 years, participated in the experiment. Subjects were without any known medical conditions and had normal or corrected-to-normal vision. Each subject was explained the aim of the study, signed an informed consent and was paid for participating in the study.

Experimental task
Recordings took place in an electromagnetic and noise shielded room to facilitate consistent measurement conditions over all subjects. Subjects were seated in a comfortable chair. Right in front of them we placed a table with a built-in 22 inch screen. Subjects were asked to place their right hand on a pressure button located on the desk between them and the screen. We positioned four objects on pre-defined positions in a semicircle on the screen so that the distance to the right hand of the user was equidistant for all four objects (figure 1). The objects were (i) a glass (for palmar grasp), (ii) a needle (for pincer grasp), (iii) a key (for lateral grasp) and (iv) an empty plexiglass tile (for no-movement condition). Both the needle and the key were placed in plexiglass retainers not only to facilitate comfortable grasping conditions but to incorporate them in positions of everyday use: the key was placed in a keyhole ready to be turned, the needle was stuck at a 45 degree angle in the plexiglass retainer, ready to be picked up (figure 1).
In this setup we used a cue-based paradigm as shown in figure 2. At second 0, the subjects were presented with a fixation cross located in the center of the screen, together with an auditory beep. Subjects were instructed to look at the cross. After 2 s, one of the objects was randomly highlighted by illuminating the underneath tile in white. Subjects were instructed to focus their gaze on the highlighted object. After a varying time period of 1-1.75 s, the white illumination turned green for 3 s. Subjects were instructed to reach, grasp and hold the object for as long as the underneath tile was illuminated in green. Thereafter, subjects released the object and returned the hand to the starting position on the pressure button. In case the empty tile was illuminated, subjects were asked to focus on the tile and avoid any eye or body movement. After each trial we introduced a break for 2-3 s.
We recorded 72 trials for each condition over eight consecutive runs. After each run, objects were repositioned clockwise, so that every object was located on each position equally often. Furthermore we recorded a three times 1 min of rest as well as 1 min of eye-movements at the beginning, half-time and end of the session.

Data recording
For EEG recording we used four biosignal amplifiers with 61 active electrodes (g.tec medical engineering, Austria). Electrodes were positioned over frontal, parietal and temporal lobes (see figure 1, right). For reference we used the right earlobe and for ground the AFz channel. EEG was sampled with 512 Hz and pre-filtered between 0.01 and 200 Hz using an 8th order Chebyshev filter. To remove power line noise, we applied a notch filter at 50 Hz. Electrode positions were captured using a ELPOS system by Zebris (Zebris Medical GmbH, Germany).
In addition we used three active electrodes for recording electrooculographic signals (EOG). We positioned them above the nasion and below the outer canthi of the eyes to form a rectangular triangle [39], and used the same recording settings as for the EEG data. To record hand and finger movements during the experiments, we used a 5DT data glove (5DT, USA). For movement onset detection we used a pressure button. Data recording and synchronization was performed using MATLAB R2012b (Mathworks, Massachusetts, USA) and TOBI SignalServer [40].

Movement detection
For detecting the movement onset of each reach-and-grasp condition we used the rising flank (button release) of the pressure button. Time-locking the no-movement condition to the onset of the green cue would make the comparison with the reach-and-grasp conditions unfair. A cue-related visually  Experimental paradigm, sequential view. At trial start subjects focused on the cross shown on the screen (1), thereafter, the tile underneath one of the objects was highlighted in white and subjects switched their focus to the highlighted object (2). Once the highlighting turned green (3) subjects grasped the designated object and held it until the highlighting vanished. Thereafter they moved the hand back to the starting position (4). This figure was created and designed with friendly assistance of nu-art [38]. evoked potential could increase classification accuracies above chance-level not because of any motor potential involved, but because of the presence of the cue-related event.
Therefore, we calculated mean and standard deviation of the reaction time of each subject based on the period between the onset of the green cue and the movement onset indicated by the pressure button. Thereafter we added the mean plus a random percentage of the standard deviation to the onset of the green cue. In this way we allow fair comparison between movement and no-movement conditions.
In addition to the movement onset we were interested in the timing when subjects finished their grasps. For this matter, we investigated the data collected using the data glove. In this experiment we used 15 sensors of the data glove which were located at the joints of the finger phalanxes. To reduce the dimensionality of the movement data collected with the glove, we performed principle component analysis (PCA) for each reach-and-grasp condition. We used only the first PCA component for further analysis. We epoched the component according to movement onsets and calculated a subjectspecific mean for each reach-and-grasp condition. We determined the timepoint of the peak in the variance as the point where subjects finished their grasp. In a similar way, we determined the timepoint when subjects released their grasp, after holding the object.

Artefact avoidance and rejection strategies
EEG analysis, especially in the low frequency range is highly vulnerable to ocular based artefacts [41]. Our strategy in dealing with artefacts, especially eye movements, was based on artefact avoidance and trial rejection based on statistical parameters. To avoid unnecessary eye-movement during trial execution, we aligned the cue presentation and the target into the same field of view. In this matter we introduced the 'white phase' into our paradigm (see figure 2, step 2). Subjects were specifically asked to focus on the object above the tile highlighted in white. This procedure allowed the participants to maintain their focus on the designated object and to execute the designated task once the highlighting turned green (figure 2, step 3). We also repositioned each object clockwise after each run to minimize the impact of the position itself.
Preceding further analysis, we discarded all trials in which subjects did not lift their hand within 2 s after start of the green highlighting. We filtered EEG between 0.3 and 35 Hz using a zerophase 4th order Butterworth filter. Thereafter, we rejected artefact contaminated trials using statistical parameters: (1) amplitude threshold (amplitude exceeds ±125 µV), (2) abnormal joint probability (3) and abnormal kurtosis. As threshold for the last two we used four times the standard deviation (STD). Using similar statistics, we also performed channel based rejection. On average we rejected 12% of the the recorded trials and kept 59 EEG channels. Rejected channels were mainly located on the edges of the electrode grid on the right side. Our approach has no need for additional measurement channels and has already been used successfully in both offline [35] and online BCI scenarios [42][43][44].

Binary single trial classification
The aims of performing binary single trial classification were twofold. First, we were interested in the discriminability between reach-and-grasp actions. Second, we wanted to discriminate individual reach-and-grasp actions from the nomovement condition.
We common average referenced (CAR) the EEG to increase the signal to noise ratio and resampled the signal to 16 Hz to facilitate computational performance. Thereafter the signal was bandpass filtered from 0.3 to 3 Hz using a 4th order zerophase Butterworth filter. We epoched our trials based on the movement onset captured by the pressure button to define our time region of interest (tROI). The tROI started 2 s before and ended 3 s after movement onset. Using 10 times 5 fold cross-validation, we divided the recorded trials into test and training data. For further classification we used all available channels. For training the shrinkage based linear discriminant analysis classifier (sLDA) [45], we used a time window of 1 s taking amplitude values in 125 ms steps as features. In steps of 1/16 of a second we moved this window over the defined tROI of training and test trials. This means, that we trained and tested a classification model every 1/16 of a second in the tROI (in total 80 models over the whole tROI). In each crossvalidation fold, the classifier was trained based on training data and evaluated on the test data. We repeated the 5 fold cross validation 10 times and report the mean of the accuracies. Based on these accuracies, we also calculated the information transfer rate (ITR) according to Wolpaw's bit rate [46,47]. On average 6.7 trials were shown to each participant per minute.
For the binary classification strategy we performed this procedure for each possible class combination (6 in total).

Multiclass single trial classification
Our approach for multiclass classification was similar to the binary classification methodology, but we used a multiclass sLDA model instead [48]. Furthermore we investigated how the window size for feature extraction impacts on the overall performance. Here we analyzed four time-window sizes: one sample, 500 ms, 1000 ms and 1500 ms. Table 1 describes the windows and their features in detail.
Since our classification approach resulted in its own classification model every 1/16 of a second, we were also able to investigate time point specific confusion matrices. Therefore we calculated them for each subject for each time point and in grand average within the tROI. We also performed row-wise normalization so that for each row the sum of all predicted class rates adds up to 100%.
Additionally to the implications of the window size, we were interested in the classification performance when reducing electrodes. Therefore we performed multiclass classification not only with our full setup of 61 electrodes but also with three reduced setups (see also figure 9): For each setup we calculated the multiclass performance as already described using the 1000 ms feature window. Additionally, we statistically compared the layouts with respect to subjects peak performance using a one-way repeated measure ANOVA.

Movement-related cortical potentials (MRCPs)
Apart from classification we were also interested to analyze the underlying differences in the MRCP neural correlates of grasps.
To analyse MRCPs we used the CAR-filtered EEG data and resampled it to 16 Hz to ease computational effort. Thereafter we bandpassed the signal using a 4th order zerophase Butterworth filter between 0.3 to 3 Hz and epoched data from −2 s to 3 s with respect to the movement onset. For each condition we calculated the confidence interval (alpha = 0.05) across all trials of all subjects using nonparametric t-percentile bootstrap statistics. MRCP calculations were done for each channel separately, however we only show a selection of channels located primarily over the motor cortex.
In addition, we performed sample-wise statistical testing using the nonparametric Wilcoxon Rank Sum Test. We applied the false discovery rate (FDR) procedure to correct for multiple comparisons.

Behavioural analysis
In figure 3 we show a summary of the behavioural analysis of all subjects for each reach-and-grasp condition. All scaling is done relative to the movement onset of each subject, whereas the reach-and-grasp phase as well as the hold and release phase were calculated using data from the 5DT data glove. We observed similar timings for the reach-and-grasp-phases, as well as for the hold and release phases. We calculated a repeated-measures one-way ANOVA with the reach-andgrasp time as factor (three levels) for palmar, pincer and lateral grasp onset. Mauchly's test indicated that the assumption of sphericity was not violated. There was no significant effect for the grasp onsets F(2.28 = 1.473, p > 0.24). Figure 4 shows the confidence interval (alpha = 0.05) of the MRCPs for each condition for channels FCz, C1, Cz and C2 with respect to the movement onset (second 0). A broader selection of channels can be viewed in the supplementary material (stacks.iop.org/JNE/15/016005/mmedia).

Movement-related cortical potentials (MRCPs)
In the top left quadrant, all conditions are plotted together. For all grasping conditions a strong negative shift can be observed starting around 250 to 350 ms before the movement onset. Imminent to the movement onset the negative shift  In all comparisons between reach-and-grasp conditions, significant differences can be observed from −1 s seconds before until 1.5 s after the movement onset. Moreover, at the occurrence of the second positive rebound at around second 1, differences are most pronounced. These differences are the smallest in the palmar versus pincer comparison and only significant ipsilaterally at C2 (see top right quadrant). For the comparison of pincer and palmar grasp conditions versus the lateral grasp condition significant differences over C1, Cz and C2 can be observed. Moreover, a time difference between the positive peak of the second positive rebound can be observed among conditions, especially at central electrode Cz. The lateral grasp condition reaches peak rebound almost ~250 ms earlier than the pincer and the palmar grasp condition. In both cases this difference is significant in all central electrodes.

Binary classification approach
As a first step we performed one versus one classification of all task pairs. Figures 5 and 6 display classification results for all subjects and their grand average over all tasks pairs. We defined the time region of interest (tROI) from 2 s before to 3 s after the movement onset. The subject-specific chance level for binary classification is 63.3% (alpha = 0.05, adjusted Wald-interval) and is Bonferroni corrected for multiple comparisons of the sample-wise classification approach (see [49,50]). For the grand average over all subjects chance level is 53.5%. For this classification approach we used features from a time window of one second.
In figure 5 we show the subject-specific peak accuracies of the tROI. Grasps versus no-movement achieve average peak accuracies of 94.5%, while grasps versus grasp peak performance average at 75.8% (palmar versus pincer, blue), 75.9% (lateral versus palmar, red) and 72.3% (lateral versus pincer, green). Only the classification results of two subjects (S6: Lateral versus Palmar; S10: Palmar versus Pincer, Lateral versus Pincer) were below the subject-specific chance level in a grasp versus grasp condition.
In table 2 we show the peak performance of the grand average of each class combination percent and bits per minute as well as its time of occurrence relative to the movement onset.  Figure 6 shows the subject-specific classification performance and the grand average performance over the tROI. All grasps show similar classification behaviour against the no-movement condition (as shown in the top left plot). Better-than-chance classification performance could already be reached 1.4 s before the actual movement onset. Performance of at least 90% remained stable over the first second after the movement onset. Grand average peak performance was 93.5%. For grasp Figure 5. Binary classification results-peak accuracies over all trials. The first bar group displays the average peak accuracies of all subjects (grand average). Bar group S1 to S15 display subject-specific peak accuracies. Notice that only the accuracy of S6 and S10 are below the single subject significance threshold (dashed green line) of 63.2%. versus grasp conditions better-than-chance performance could be achieved on grand average up to 1 s before movement onset for palmar versus pincer, respective 400 ms and 300 ms for palmar versus lateral and lateral versus pincer. Grand average peak accuracies occurred after movement onset between 1.125 s and 1.375 s. Grand-average peak accuracy culminates at 73.7% (palmar versus pincer), 69.8% (lateral versus pincer) and 73.5% (lateral versus palmar).

Multiclass classification approach
Similar to the binary classification approach, we defined the time region of interest from 2 s before to 3 s after the movement onset. The subject-specific chance level for the multiclass approach lies at 33.5% (alpha = 0.05, adjusted Wald-interval, Bonferroni corrected for multiple comparisons). For the grand average, chance level lies at 27.1%. Figure 7 shows the grand average performance of the multiclass classification for the four different time windows tested. We were interested to evaluate the impact of window size on the overall performance. For all approaches better-than-chance classification was already possible in a time range between −0.5 s to 1 s relative to the movement onset (see figure 7, left plot). There is also almost no difference between the 1000 ms and the 1500 ms window in terms of performance. However, peak accuracy and its timing shifted with increasing window size as shown in table 3. We also investigated the subject-specific peak accuracies for each time window as shown in figure 7 (right plot). We conducted a one way repeated-measures ANOVA over the peak accuracies of all subjects per window size (4 levels). Mauchly's test for sphericity indicated correction of the p-values. Correction was done using the Greenhouse-Geisser criterion. Their differences between peak accuracies for each time window were significant (F(3, 42) = 84.6, p < 0.001).   Post-hoc test for multiple comparison using the Tukey-Kramer criterion revealed that window-based performance is significantly better than the one sample approach. We also found that the 1000 ms and the 1500 ms perform significantly better (p = 0.02) than the 500 ms window. For the rest of the analysis we chose the classification approach using the time window of 1000 ms. Figure 8 displays multiclass-classification results for all subjects using the 1000 ms time window. We attained the grand average peak accuracy of 65.9%. As shown in figure 7 better-than-chance classification was already possible more than 1 s before movement onset.
Apart from the classification results for participants S6 and S10, all peak accuracies exceeded 60%. In figure 8 top right, we show the subject-specific peak accuracies which result on average in 66.9%. This value differs from the grand average due to the variance in peak time per subject.
Since our method incorporates its own classification model every 1/16 of a second, we were able to show condition specific confusion matrices at several timepoints of interest ( figure 8, bottom). In this case we chose 1 s before movement onset (i, red), movement onset (ii, green) and 1 s after the cue (iii, yellow). While the true positive rate is similar for all contributing classes in (i), the true positive rate of the nomovement condition is almost twice as high in (ii) and almost a third higher in (iii). This resembles the results we presented previously in the binary classification approach (grasp versus no movement) and the underlying MRCPs, in which distinct differences between grasping movements and no movement could be seen on a broad time interval (−1 to 1 relative to the movement onset). However, in (iii) true positive rates for the grasp classes also reached values of almost 60%.
Incorporating the 1000 ms window for feature extraction, we also investigated three additional electrode setups with reduced number of electrodes, as can be seen in figure 9. For all tested layouts better than chance accuracies could be achieved. With increasing number of electrodes, performance increased for all subjects. We conducted a one-way repeated-measures ANOVA over the peak accuracies of all subjects per channel layout (four levels). Mauchly's test for sphericity indicated no need for correction. The differences between peak accuracies for each channel layout were significant (F(3, 42) = 78.505, p < 0.001). Post-hoc test for multiple comparison using the Tukey-Kramer criterion revealed that all channel-layouts are significantly different from each other.

Discussion
In this paper we show that it is possible to discriminate three executed reach-and-grasp actions prominent in everyday life using their EEG neural correlates. Furthermore, we show that these actions can be discriminated against no-movement with high accuracy. In the binary classification scenario, performance for grasp versus grasp conditions peaked on average at 72.4% (STD ± 5.8%), for grasps versus the no-movement peak performances of 93.5% (STD ± 4.6%) could be reached. For the multiclass classification scenario which incorporated all reach-and-grasp conditions and the no-movement condition, maximum performance (65.9%, STD ± 8.1%) could be reached using a feature window of 1000 ms. Underlying MRCPs of the reach-and-grasp actions investigated over the primary motor cortex showed significant differences starting from approximately 800 ms to 1200 ms after the movement onset which is the same time frame where classification performance reached its maximum.

Movement-related cortical potentials
Analysis of the MRCPs for each reach-and-grasp condition showed the maximum negative shift imminent to the movement onset, as previously described by Shibasaki et al in [25] and presented also in other studies that investigate the neural correlates of upper limb movements such as Gu et al [51,52], Jochumsen et al [30,53] or Oda et al [54]. Our analysis shows that the maximum negative shift occurs over the central motor cortex (Cz) imminent to the movement onset and is more pronounced on the contralateral side (C1) than on the ipsilateral side (C2). This effect can be seen further in the extended MRCPs analysis provided as supplementary material.
Comparisons between all conditions reveal that all reachand-grasp conditions show significant differences to the nomovement condition over the whole tROI.
In all grasp-versus-grasp conditions we found significant differences ( p < 0.05) emerging around one second before the movement onset, howevers most pronounced differences are found around 0.8 s ms to 1.2 s after the movement onset.
Interestingly we found a time shift in the peaks of this positive rebound potential for the reach-and-grasp conditions. This time shift is pronounced strongest in condition combinations involving the lateral grasp. This potential produced by the lateral grasp appears the earliest of all reach-and-grasp conditions. It can be seen not only at contralateral, but also at central and ipsilateral sides.
The behavioural analysis indicate that subjects finished grasping objects on average between 1.1 s and 1.25 s after the movement onset. This falls in the same time frame where the significant differences between the MRCPs can be found.

Single trial classification
Binary single trial classifications show high classification results for movement versus no-movement conditions with subject-specific accuracies over 90%. Even for subjects (S6, S10) with unfavourable grasp versus grasp classification performance detection rates reach performances of 85% and more. Peak accuracies were reached within the first second after the movement onset. The movement intention could be detected before the actual movement with performance rates exceeding 80% in grand average over all subjects. These performance results are similar to the findings reported by other studies regarding the detection of upper limb movement intention [31,34,55] and [56] (online).
Binary grasp versus grasp classification performance ranged from ~70% to 73% on grand average. Subjectspecific accuracies were usually higher (~+3%) due to variances in peak timing. Only for two subjects (S6, S10) any grasp versus grasp combination scored lower than chance (S6: Lateral versus Palmar; S10: Palmar versus Pincer, Lateral versus Pincer). We could also observe significant classification performance imminently before the movement onset ranging from 57%-59%. These results are in line with the findings of Jochumsen et al [31] who investigated strategies for discriminating grasp intentions. Using temporal features in the range of 0.01-5 Hz, they obtained similar performance results. However, the focus of their study was only on the movement intention and it does not reflect the whole movement process including reaching. We obtained grand average peak accuracies around 1-1.5 s after movement onset.
Peak accuracies correspond to changes in amplitude in the neural correlates in which we observed significant differences regarding the emerging of a positive rebound potential of the reach-and-grasp conditions as well as in their time shift.
Regarding the multi-class classification approach significant classification results could also be reached up to 1 s before the movement onset. In the multi-class classification approach confusion matrices reveal a disproportionate contrib ution to the true positive rate by the movements versus no-movement condition. Despite true-positive rates being significantly higher than chance level for all movement versus movement conditions, the movement versus no-movement condition contribute almost twice as high to the overall performance at movement onset. In the multiclass scenario, peak accuracy of 65.9% was reached one second after the movement onset. Again, confusion matrices show high contribution of movement versus no-movement conditions, however also grasp versus grasp conditions contributed equally to the overall performance (~ two third ratio).
With respect to the previously mentioned findings of Agashe et al [32], a direct comparison of performance results is difficult due to differences in the methodological approach and in the experimental setup. We also investigated the no-movement condition in the multiclass scenario, which was not incorporated in their study. However, in their study, participants reached peak accuracy for classification already 250 ms after the movement onset which is approximately 750 ms earlier than we report in our findings (1 s after movement onset). This suggests that they find their most discriminant information already within the early reaching phase to the object, while our results indicate peak accuracy during the grasping itself.

Implications of the window size and electrode setup
Regarding our investigations on different window sizes in the multiclass approach we show that all window-based approaches could outperform single sample based classification significantly (p < 0.001).
Interestingly, we also found a significant difference in performance (p = 0.02) between 500 ms and the 1000 ms window, which suggests that discriminative information for different reach-and-grasp actions is spread over a longer period of time than 500 ms. Our behavioural analysis also shows that all reach-and-grasp actions are on average slightly longer than 1000 ms (1100 ms to 1250 ms) which also implies that the 1000 ms window allows better coverage of the whole reach-and-grasp action than a shorter one. We also tested a 1500 ms window, though performance compared to the 1000 ms window was almost identical and no significant differences could be found (p > 0.86). However, we observed a delay in peak performance of around 400 ms.
Our investigation towards different electrode setups clearly showed that with increasing number of electrodes also peak performance increases. This effect is present for each subject and lead to the significant (p < 0.05) differences between all evaluated electrode setups. However, by reducing the number of electrodes (and therefore the feature space for the classifier) by almost two thirds to 25 electrodes, peak performance decreases by less than 3%, which suggests a possible trade-off between performance and usability in e.g. an out-of-the-lab scenario incorporating potential end users.

Limitations
In this study we used a cue-based protocol and conducted an offline analysis. In this regard we used zero-phase bandpass filtering EEG to compensate for the group delay. In an online scenario, non-causal filtering is not possible.
To allow a fair comparison between movement and nomovement conditions we time-locked the no-movement condition according to a virtual onset calculated from the subject's reaction time. In our opinion, these no-movement epochs are not comparable to real resting periods since these epochs are interspersed in a cue guided experiment setting which demands the subjects attention and action. In an online scenario, a real resting period would persist over a longer period or during a phase in which subjects intentionally do not attempt any form of control (e.g. while watching a movie).
With increasing window size the increasing number of features becomes an issue. The larger the number of features included the more unfavourable the trial to feature ratio, which ultimately results in an increased validation error. Empirical evaluation of this issue was already performed by Blankertz et al [45]. With a linear increase of features used, the dimensionality of the feature space grows exponentially. This results in a poor estimation of the covariance matrix for the classification model and has a high negative impact on classification performance ('curse of dimensionality').
One possibility to overcome this issue would be to use a feature reduction technique to keep only features containing high discriminative information, such as sequential forward selection (SFS, already applied in [31]) or the smooth and decimation approach used in [57]. In our experimental setup we used a causal sliding window for extracting features for classification. In an online scenario, any window based approach will introduce a delay with regard to the reach-and-grasp action of the subject. Although this is a static delay, increasing window size will also increase the delay and introduce an offset time in any BCI control scenario. However, this applies to any causal online scenario and is not solely a limitation of this specific experiment.

Transfer to online control and future work
So far we showed in offline analysis that three reach-andgrasp movements towards different objects can be discriminated from low-frequency EEG time domain features.
Our offline analysis showed better-than-chance performance in single trial classification, however the generation of robust motor commands needs to be investigated further in online setups. Our results show peak accuracies around 75% on grand average, which suggest that at best three out of four reach-and-grasp commands could be decoded correctly. In previous sensori-motor rhythm (SMR) based BCI studies [58][59][60] incorporating cerebral palsy end users, we investigated possibilities to accumulate multiple control commands for one single decision. This evidence accumulation strategy demanded three 'correct' commands out of e.g. five to finally trigger the correct action. Though the whole process of decision making is prolonged, erroneous commands are less probable.
Another idea for boosting BCI performance would be to incorporate error-related potentials (ErrP) into the decision process, as already briefly introduced by Kreilinger et al [61] using motor imagery tasks (MI). The idea here would be to use a hybrid combination of EEG based detectors for grasps and ErrP. Whenever a misclassification of the designated grasp happens, the triggered ErrP could be used for undo. We hypothesize that this combination could lead to increased overall performance, however data collection for calibration will require a more complex paradigm since not only grasps, but also ErrP data has to be collected.
In this study we rejected on average around 12% of trials due to artefact contamination which would affect at least every tenth grasp attempt in an online scenario. Though artefacts may not be avoided completely, we believe that with proper end user training this percentage can be decreased. Still, for robust grasp control these contaminated attempts need to be handled accordingly e.g. by signaling the end user to repeat the current action.
For this experiment we used a high density electrode grid of 61 electrodes placed on frontal, central and parietal areas over the scalp. Our investigations show that this grid can be reduced by almost two thirds to 25 electrodes, while still maintaining similar performance. This factor might become critical when attempting to leave a controlled laboratory environ ment and, for instance, when working together with end users in their own homes.
Successful online control requires reliable movement intention detection since the exploited MRCPs are time-locked to the movement onset. In this study, we showed high detection accuracies for movements versus no-movement conditions. In an online scenario, a hierarchical classification model could be used to detect the movement intention of the user and rely on this detection point as a timelock for grasp versus grasp discrimination. This approach has already been used offline in several studies incorporating complex hand movements [31,33,62].
Further studies incorporating high spinal cord injured end users will finally assess whether our current results can be translated to the targeted end user group. The command strategy in our study relied on executed movements and it is still unknown whether similar classification results could be achieved in end users. Studies from Blokland et al [63,64] and Verbaarschot et al [65] indicate that attempted movements may present a better command strategy than imagined movements. Also Lacourse et al [66] indicated higher correlations between attempted and imagined movements of tetraplegic end users than between executed and imagined movements of a healthy control group. Our first experiments incorporating high SCI end users performing attempted complex hand movements confirm [67] that attempted grasps of end users can be discriminated better than chance.

Conclusion
In this study we showed that it is possible to discriminate three executed reach-and-grasp actions prominent in people's everyday use from non-invasive EEG. Based on their neural correlates, we could show differentiation against each other and also against a no-movement condition. Furthermore we could identify significant differences in the underlying movement-related cortical potentials.
This findings will eventually contribute to our attempt of controlling a neuroprosthesis in a natural and intuitive way and a step closer to a successful and reliable intervention for end users with high spinal cord injury.