Stacked Sparse Autoencoders for EMG-Based Classification of Hand Motions: A Comparative Multi Day Analyses between Surface and Intramuscular EMG

Abstract

Advances in myoelectric interfaces have increased the use of wearable prosthetics including robotic arms. Although promising results have been achieved with pattern recognition-based control schemes, control robustness requires improvement to increase user acceptance of prosthetic hands. The aim of this study was to quantify the performance of stacked sparse autoencoders (SSAE), an emerging deep learning technique used to improve myoelectric control and to compare multiday surface electromyography (sEMG) and intramuscular (iEMG) recordings. Ten able-bodied and six amputee subjects with average ages of 24.5 and 34.5 years, respectively, were evaluated using offline classification error as the performance matric. Surface and intramuscular EMG were concurrently recorded while each subject performed 11 hand motions. Performance of SSAE was compared with that of linear discriminant analysis (LDA) classifier. Within-day analysis showed that SSAE (1.38 ± 1.38%) outperformed LDA (8.09 ± 4.53%) using both the sEMG and iEMG data from both able-bodied and amputee subjects (p < 0.001). In the between-day analysis, SSAE outperformed LDA (7.19 ± 9.55% vs. 22.25 ± 11.09%) using both sEMG and iEMG data from both able-bodied and amputee subjects. No significant difference in performance was observed for within-day and pairs of days with eight-fold validation when using iEMG and sEMG with SSAE, whereas sEMG outperformed iEMG (p < 0.001) in between-day analysis both with two-fold and seven-fold validation schemes. The results obtained in this study imply that SSAE can significantly improve the performance of pattern recognition-based myoelectric control scheme and has the strength to extract deep information hidden in the EMG data.

1. Introduction

Advances in myoelectric interfaces have the potential to revolutionise the use of wearable prosthetic devices as artificial substitutes for missing limbs. Active hand and arm prostheses are usually controlled by electromyography (EMG) signals. EMG signals can be recorded from remnant muscles using either invasive or non-invasive electrodes. In most commercially available upper limb prostheses, EMG is used for on-off control, which allows the prosthesis to move bi-directionally with constant velocity [1]. With the addition of proportional control, the velocity of a prosthetic function is proportional to the intensity of the EMG signal [2]. These clinical control schemes are based on the direct association (direct control) of EMG signals to the actuation of degrees of freedom (DoF) [1] and therefore require at least two independent EMG channels to drive each DoF. To overcome this limitation, control schemes based on pattern recognition (PR) [3,4,5,6] have been used to decode several prosthesis functions using supervised learning.

A number of algorithms have been tested for EMG classification, including artificial neural networks (ANN) [7,8,9,10,11], log linearized Gaussian mixture networks (LLGMN) [12,13,14,15], Fuzzy mean max NN [16], radial basis function [17], hidden Markov model [18], Bayesian network [16,19], random forest [20], k-nearest neighbors (KNN) [21,22], support vector machine (SVM) [23], and linear discriminant analysis (LDA) [24,25]. Some of these algorithms achieved classification accuracies above 95% for up to 10 classes when applied to temporal and frequency EMG features [26]. PR-based approaches are promising compared to conventional methods. However, their clinical usability is still in its infancy and thus the natural control of prostheses is still limited to a few basic movements [27,28]. This calls for better machine learning methods to improve usability of PR schemes.

The sophisticated deep learning algorithms impact several applied fields, such as computer vision [29] and speech recognition [30]. In addition to classical machine learning methods, deep learning algorithms, such as autoencoders (AE) and convolutional neural networks (CNN), have been used in biomedical signal applications [31], such as in electroencephalography (EEG) [32,33,34] and electrocardiography (ECG) [35,36,37,38]. Despite some applications of EMG processing [39,40,41,42], AE-based methods have not been extensively applied in myoelectric control. Several studies evaluated the CNN in myoelectric control; Park and Lee [43] decoded movement intention from EMG using a CNN that outperformed a SVM classifier. Atzori et al. [28] proposed a deep networks-based algorithm for classification of surface EMG (sEMG) associated to hand movements in the publicly available Ninapro database of intact-limb and amputee subjects [44]. This technique provided comparable performance with respect to KNN, SVM, random forest, and LDA. Other studies [27,45,46,47,48,49] evaluated deep learning methods and concluded that these methods either performed better or comparable to classical machine learning algorithms. Most of these studies were performed with datasets recorded in a single session, limiting the usefulness of deep networks. Hence, performance of deep networks over multiple sessions needs to be assessed with both sEMG and intramuscular (iEMG).

Whereas myoelectric control is most commonly applied with sEMG, iEMG has also been proposed as an approach to overcome some of the limitations of non-invasive systems [50]. For example, Kamavuako et al. [3] showed that the classification accuracy of a myoelectric control system with combined surface and intramuscular EMG was better than sEMG alone. Other studies [51,52,53,54,55] compared the individual performance of sEMG and iEMG for classification of different hand and wrist movements and generally found similar performance (no significant difference). However, all these previous studies reported results on able-bodied individuals in a single recording session. Hence nothing can be said about the performance with amputee subjects or over multiple days.

In this study, we apply stacked sparse autoencoders (SSAE) in a myoelectric control application and its performance is compared with the benchmark LDA that is widely used in myoelectric control research [24]. Moreover, we tested able-bodied as well as amputee subjects over multiple sessions on different days and we compared sEMG and iEMG classification.

2. Materials and Methods

2.1. Subjects

Ten able-bodied (male, mean age ± SD = 24.5 ± 22.02% years) and six transradial amputee subjects (male, three left and three right transradial amputation, mean age ± SD = 34.8 ± 32.7% years) participated in the experiments. One amputee regularly used a body-powered prosthesis, whereas the others did not use any prostheses. The procedures were performed in accordance with the Declaration of Helsinki and approved by the local ethical committee of Riphah International University (approval no.: ref# Riphah/RCRS/REC/000121/20012016). Subjects provided written informed consent prior to the experimental procedures.

2.2. Experimental Procedures

Surface and intramuscular EMG signals were collected concurrently. Six sEMG and six iEMG electrodes were used for able-bodied subjects. Three electrodes were placed on the flexor and three on the extensor muscles. The same number of electrodes were used for three of the amputees. In the other three amputees, it was only possible to use five surface electrodes and three to six intramuscular electrodes. Electrodes placement is shown in Figure 1. Intramuscular EMG signals were filtered with an analog bandpass filter of 100–900 Hz and sampled at 8 kHz. Surface EMG signals were filtered at 10–500 Hz and sampled at 8 kHz [25].

Each subject performed 11 hand motions in each experimental session: hand open, hand close, flex hand, extend hand, pronation, supination, side grip, fine grip, agree and pointer, and rest. Seven experimental sessions separated by 24 h were completed by each subject. For each session, each hand movement was repeated four times with a contraction and relaxation time of 5 s, as shown in Figure 2. Hence, a single session was 400 s long. The sequence of movements was randomized for each session.

After data collection and during offline data analysis, time drifting was found for the onset and offset duration of individual time periods. Therefore, individual time period labeling was performed with a semi-automatic technique using MATLAB 2016a. In this method, onset time was manually chosen using a cursor on the data and corresponding time periods were stored automatically. Each period was reduced to three seconds by removing the first and last second of data to remove transient parts.

2.3. Signal Processing

Surface and intramuscular EMG signals were digitally filtered with a 3rd order Butterworth bandpass filter with bandwidths of 20–500 Hz and 100–900 Hz, respectively, and a 3rd order Butterworth band-stop filter to suppress the 50 Hz powerline noise [41].

Four-time domain features, mean absolute value (MAV), waveform length (WL), zero crossing (ZC), and slope-sign change (SSC) [4], were computed from sEMG and iEMG signals in intervals of 200 ms at increments of 28.5 ms.

The signals were then classified with SSAE (as detailed below) [56,57] and a state-of-the-art LDA from a publicly available myoelectric control library (MECLAB) [24]. LDA was chosen for comparison because it is commonly used in the literature on myoelectric control, and even in online studies [58,59] and the commercial available prosthetic hand COAPT [60]. For the within-day analysis, a five-fold cross validation scheme was used for testing. For the between-days analysis, all pairs of days were compared with an eight-fold validation. Data were randomly divided into eight equal folds. Moreover, the classifiers were trained and tested on separate days with a two-fold cross validation where each day was used for training and testing separately. The data from the seven days were also tested with a seven-fold cross validation, in which six days were used for training and one day for testing with seven repetitions.

2.4. Stacked Sparse Autoencoders

AEs are deep networks trained in an unsupervised fashion to replicate the input at the output [61]. AEs consist of an encoder and a decoder. An encoder maps an input x to a new representation z, which is decoded back at the output to reconstruct the input x′:

Z = h (Wx + b)

(1)

X′ = g (W′z + b′)

(2)

where h and g are activation functions, W and W′ are weight matrices, and b, b′ are bias vectors for the encoder and decoder, respectively [62]. The error between the input k and the reconstructed input x′ is optimized as follows:

$${\min }_{\left(W,b,W^{\prime },b^{\prime }\right)}={{\displaystyle \sum }}_{i=1}^n||x_i-x_i^{\text{'}}||{}^2$$

(3)

In this work, stacked sparse autoencoders (SSAE) [57] were used with two hidden layers, consisting of 24 and 12 hidden units (Figure 3). For both layers, the logistic sigmoid and linear functions were used for the encoders and decoders, respectively. In SSAE, the output of one AE is fed to the input of another AE [39] and sparsity is encouraged by adding regularization to the cost function [63], which stands for the average output activation of a neuron. An average output activation for a neuron i can be formulated as:

$$\dot{\widehat{p_i}=\frac{1}{n}{{\displaystyle \sum }}_{j=1}^n{z}_i}\left(x_j\right)$$

(4)

where i is the ith neuron, n is the total number of training examples, and j is the jth training example. This regulariser is introduced to the cost function using the Kullback-Leibler divergence [64]:

$${\mathsf{\Omega }}_{sparsity}={{\displaystyle \sum }}_{i=1}^{d}p\log \left(\frac{p}{{\widehat{p}}_i}\right)+\left(1-p\right)\log \left(\frac{1-p}{1-{\widehat{p}}_i}\right)$$

(5)

where d is the total number of neurons in a layer [65] and p is the desired activation value, called sparsity proportion (SP). An L2 regularization term (L2R) is further added to the cost function to control the weights:

$${\mathsf{\Omega }}_{weights}=\frac{1}{2} {{\displaystyle \sum }}_l^L{{\displaystyle \sum }}_j^N{{\displaystyle \sum }}_i^K{\left(w_{ji}^{\left(l\right)}\right)}^2$$

(6)

where L represents the number of hidden layers, N is the total number of observations, and K is the number of features within an observation.

Therefore, by inserting the regularization terms from Equations (5) and (6) into the reconstruction error in Equation (3), the cost function can be formulated as follows:

$$E=\underset{mean square error}{\underbrace{\frac{1}{N}{{\displaystyle \sum }}_{n=1}^N{{\displaystyle \sum }}_{k=1}^K{\left(x_{kn}-{\widehat{x}}_{kn}\right)}^2}}+\lambda \ast \underset{\begin{array}{c}L2\\ Regularization\end{array}}{\underbrace{{\mathsf{\Omega }}_{weights}}}+\beta \ast \underset{\begin{array}{c}Sparsity\\ Regularization \left(SR\right)\end{array}}{\underbrace{{\mathsf{\Omega }}_{sparsity}}}$$

(7)

The three optimization parameters are λ (coefficient for L2R), which prevents overfitting; β (coefficient for sparsity regularization SR) that controls the sparsity penalty term; and p (SP), which sets the desired level of sparsity [33,66]. Parameter optimization for both layers is shown in Figure 4.

Both the AEs were trained with the scale conjugate gradient descent function [67] using greedy layer-wise training [68]. Finally, the softmax layer was trained in a supervised fashion, then stacked with the sparse AEs as shown in Figure 3 and the network was fine-tuned before final classification.

2.5. Statistical Tests

For performance comparison between SSAE and LDA classifiers, and surface vs. intramuscular EMG-based control schemes, Friedman’s tests with two-way layout were applied. Results of classification are presented as mean error with standard deviation. Statistical p-values less than 0.05 were considered significant.

3. Results

3.1. Parameter Optimization

Different combinations of optimization parameters (L2R, SR, and SP) were explored for both layers, whereas the best parameter values that minimized the mean squared error (MSE) were chosen for corresponding layers. Figure 4 shows the MSE for both layers (maximum epochs of 500) along with different combinations of the three parameters. The chosen parameter values were the same for the two layers (L2R = 0.0001, SR = 0.01, and SP = 0.5).

3.2. SSAE vs. LDA

Classification errors were computed for each day using five-fold cross validation, in which data of an individual day was divided randomly into five equal folds, ensuring that each fold had an equal number of all movements. Results were then averaged over seven days for each subject.

For this analysis, EMG data were arranged in four sets including sEMG and iEMG data of both healthy and amputee subjects. All four sets of data were classified with SSAE and LDA separately and the results are shown in Figure 5 as the average of 10 healthy and six amputee subjects.

SSAE achieved statistically less error rates than LDA for all cases (p < 0.001) (Figure 5).

3.3. sEMG vs. iEMG

The performance resulting from the use of sEMG and iEMG data was compared using four combinations of datasets, including healthy and amputee data classified with SSAE and LDA. The same cross-validation scheme was used as in Section 3.2. Results for this analysis are shown in Figure 6 as the average of the 10 healthy and six amputee subjects.

No significant difference was observed between iEMG and sEMG when using SSAE (p > 0.05), whereas sEMG outperformed iEMG with LDA (p < 0.05).

3.4. Analysis between Pairs of Days

Data from seven days were arranged into 21 unique pairs. Classification errors were calculated for each pair using eight-fold cross validation (eight repetitions for each movement per two days, so each repetition constitutes a separate fold), which were averaged over all pairs for each subject. Results for this analysis are shown in Figure 7 as the average of 10 healthy and six amputee subjects.

SSAE outperformed LDA by 11.93 and 21.59 percentage points for both healthy and transradial amputee subjects, respectively (Figure 6). Furthermore, SSAE achieved error rates similar to those of the corresponding data in the within-day analysis with percentage point differences of 3.55 and 11.26 for healthy and amputee subjects, respectively. Conversely, LDA performance between-days worsened significantly from the within-day analysis by 11.28 and 18.95 percentage points for healthy and amputee subjects, respectively. Mean classification errors for the 10 healthy and six amputee subjects for each pair of days are shown in

Table 1. Stacked sparse autoencoder (SSAE) vs. linear discriminant analysis (LDA) performance and surface electromyography (sEMG) vs. intramuscular (iEMG) data comparison for healthy subjects. In each matrix, the upper diagonal triangle represents the mean classification errors obtained with SSAE and the lower diagonal triangle represents errors obtained with LDA for corresponding pairs of days.

Healthy sEMG Data
Day	D1	D2	D3	D4	D5	D6	D7
D1	-	2.29	3.06	2.23	3.09	2.74	3.88
D2	10.28	-	3.8	3.15	3.23	2.95	3.65
D3	10.4	9.96	-	3.74	4.08	3.57	4.31
D4	11.22	11.37	11.02	-	3.02	2.61	3.45
D5	13.16	12.78	12.68	11.43	-	2.53	3.46
D6	12.74	12.94	12.66	12	11.11	-	2.69
D7	13.54	13.56	13	11.62	11.31	9.95	-
Healthy iEMG Data
Day	D1	D2	D3	D4	D5	D6	D7
D1	-	5.68	6.63	6.03	5.7	5.15	6.25
D2	21.69	-	5.36	4.75	4.79	3.93	4.54
D3	23.42	17.93	-	5.19	5.03	4.78	5.24
D4	21.96	19.5	18.16	-	5.16	5.2	5.33
D5	24.68	21.03	20.52	18.98	-	3.05	3.58
D6	23.59	20.19	20.96	18.73	17.66	-	3.12
D7	23.85	20.53	19.03	19.65	16.55	15.9	-

and

Table 2. SSAE vs. LDA performance and sEMG vs. iEMG data comparison for transradial amputee subjects. In each matrix, the upper diagonal triangle represents the mean classification errors obtained with SSAE and the lower diagonal triangle represents errors obtained with LDA for corresponding pair of days.

Amputee sEMG Data
Day	D1	D2	D3	D4	D5	D6	D7
D1	-	14.56	16.23	14.4	15.47	13.21	12.18
D2	34.75	-	14.32	13.51	12.96	11.54	11.48
D3	38.19	34.03	-	16.4	15.84	12.73	13.31
D4	32.27	31.77	34.36	-	13.49	11.83	12.11
D5	36.1	33.21	34.79	30.56	-	10.34	10.78
D6	31.45	30.42	30.76	27.42	24.18	-	8.57
D7	34.12	31.01	32.27	27.92	25.38	20.91	-
Amputee iEMG Data
Days	D1	D2	D3	D4	D5	D6	D7
D1	-	15.84	17.41	18.64	16	16.46	15.32
D2	36.15	-	14.96	16.99	13.39	13.76	13.53
D3	39.66	34.25	-	16.54	13.76	13.83	13.74
D4	38.16	33.58	33.91	-	15.11	15.2	14.43
D5	38	34.23	35.01	32.42	-	11.79	12.78
D6	36.07	33.56	34.9	32.55	29.47	-	13.16
D7	36.75	31.99	34.11	31.18	28.97	27.72	-

, respectively.

From Table 1, sEMG and iEMG classification were similar when using SSAE (percentage point difference of 1.76), whereas classification differed significantly when using LDA. sEMG was superior to iEMG with a percentage point difference 8.37. For transradial amputees, however, sEMG and iEMG were classified with similar accuracy with both SSAE and LDA (Table 2).

3.5. Between-Days Analysis

This analysis was performed with two-fold and seven-fold validation schemes. For two-fold validation, each pair of days were tested so that one day was used for training and the other for testing. The result was calculated as the average of all pairs. For seven-fold validation, six days were used for training and one day for testing with seven repetitions. Results for both schemes are shown in Figure 8 as the average of 10 healthy and six amputee subjects.

SSAE achieved lower error rates than LDA for both the two-fold and seven-fold validation. As the training data increased from one to six days, the error rate of SSAE decreased by 9.50 and 13.03 percentage points for sEMG and iEMG, respectively. The decrease in error rate with increasing training data was smaller for LDA (6.39 and 5.16). Moreover, in both the two-fold and seven-fold analyses, sEMG patterns proved to be more robust and outperformed the iEMG (p < 0.01).

4. Discussion

SSAE performed significantly better than LDA using both sEMG and iEMG data from able-bodied and amputee subjects (Figure 5 and Figure 7, respectively). Moreover, SSAE was more robust to between-day variability in signal features.

The number of hidden units in both layers was optimized and further increments did not significantly reduce the error. Performance at both layers was better when using non-linear and linear activation functions for encoders and decoders, respectively. Beside the architecture, during the training phase, we noted that some parameters of the individual layers played an important role in the error reduction for that layer. Layers were trained in a greedy layer-wise training fashion, in which each layer was trained independently of the others. At layer one, the MSE depended on SR, but was almost independent of L2R. Conversely, at layer two, MSE decreased with lower values of L2R.

The within-day analysis revealed that SSAE performed similarly when applied to iEMG and sEMG. Although both iEMG and sEMG achieved classification errors below 1% for able-bodied subjects, iEMG (σ² = 0.05) had less variance than sEMG (σ² = 0.26) for between-subject data. For LDA, the classification error was greater than with SSAE and differed between sEMG and iEMG, with lower error rates for sEMG. These results are consistent with previous studies that showed similar performance for classifying iEMG and sEMG or slight poorer performance for iEMG [52,53,54,55].

When analysing between pairs of days, the performance of SSAE was similar to that achieved for the within-day analysis. Conversely, LDA performance substantially worsened when analysing different days. Hence, unlike LDA, SSAEs generalized well when adding data from other days.

For between-days analysis with two-fold and seven-fold validation, SSAE outperformed (p < 0.001) the LDA. In this case, sEMG led to better performance than iEMG using both SSAE and LDA and revealed that sEMG patterns are comparatively more repetitive over days than iEMG patterns. This finding could be due to the fact that sEMG (non-invasive recording) represents global information from surrounding muscles, which increases its usability and robustness, unlike iEMG that includes local information from a specific muscle.

Moreover, when increasing the training set, the performance of SSAE relatively improved more than that of LDA (Figure 8).

Previous studies on myocontrol that used deep networks only focused on sEMG [28,43,49]. For sEMG, the results of this study are comparable with those from previous studies where either deep networks performed similar to or better than classical machine learning algorithms. The present study also included iEMG data and showed that deep networks outperform LDA classification when using both sEMG and iEMG with an increased robustness across days.

5. Conclusions

For both the within- and between-pairs-of-days analysis, SSAE significantly outperformed the state-of-the-art LDA classifier with both the sEMG and iEMG data. No difference was found between the sEMG and iEMG data of individual sessions. However, sEMG outperformed the iEMG in between-days analysis. The findings imply that deep networks are more robust across days and the performance improved with adding more sessions to the training data. Furthermore, sEMG patterns proved to be more consistent for long-term assessment compared with iEMG for both healthy and amputee subjects.