Edinburgh Research Explorer Combined influence of forearm orientation and muscular contraction on EMG pattern recognition

The performance of intelligent electromyogram (EMG)-driven prostheses, functioning as artiﬁcial alterna- tives to missing limbs, is inﬂuenced by several dynamic factors including: electrode position shift, varying muscle contraction level, forearm orientation, and limb position. The impact of these factors on EMG pat- tern recognition has been previously studied in isolation, with the combined effect of these factors being understudied. However, it is likely that a combination of these factors inﬂuences the accuracy. We inves- tigated the combined effect of two dynamic factors, namely, forearm orientation and muscle contraction levels, on the generalizability of the EMG pattern recognition. A number of recent time- and frequency- domain EMG features were utilized to study the EMG classiﬁcation accuracy. Twelve intact-limbed and one bilateral transradial (below-elbow) amputee subject were recruited. They performed six classes of wrist and hand movements at three muscular contraction levels with three forearm orientations (nine conditions). Results indicate that a classiﬁer trained by features that quantify the angle, rather than am- plitude, of the muscle activation patterns perform better than other feature sets across different contraction levels and forearm orientations. In addition, a classiﬁer trained with the EMG signals collected at multiple forearm orientations with medium muscular contractions can generalize well and achieve classiﬁcation accuracies of up to 91%. Furthermore, inclusion of an accelerometer to monitor wrist movement further improved the EMG classiﬁcation accuracy. The results indicate that the proposed methodology has the potential to improve robustness of myoelectric pattern recognition.


Introduction
An artificial hand is an example of a technology that can be used to help a person, with a congenital condition or after an injury that results in the loss of the limb, perform essential activities of daily living ( Nazarpour, Cipriani, Farina, & Kuiken, 2014 ). Nowadays, commercial prosthetic hands are highly sophisticated, offering individual finger movement. These prosthetic hands are controlled with an intelligent interface that infers movement intentions by deciphering the electrical activity of muscles, known as the surface electromyogram (EMG) signal. The EMG signals are recorded non-invasively from the skin surface of the remaining arm to which the prosthesis is attached. Myoelectric inter-faces based on EMG pattern recognition have gained considerable attention thanks to their naturalness enabling human intentions to be conveyed to control a machine.
EMG pattern recognition has been adopted in academic and commercial research, showing improvements in the control of hand prostheses ( Al-Timemy, Bugmann, Escudero, & Outram, 2013a; Asghari Oskoei & Hu, 2007;Atzori et al., 2014;Kuiken, Turner, Soltys, & Dumanian, 2014;Nazarpour, Sharafat, & Firoozabadi, 2007;Zardoshti-Kermani, Wheeler, Badie, & Hashemi, 1995 ). In this framework, a natural map from users' limb motion to analogous, but discrete, prostheses function is formed by extracting the movement intent from multi-channel EMG signals. A significant amount of research has been devoted to various aspects of EMG pattern classification systems such as preprocessing and filtering, feature extraction and reduction and classification ( Boostani & Moradi, 2003;Phinyomark et al., 2013 ). tion accuracy (generally > 90%) and real-time performance ( Rasool, Iqbal, Bouaynaya, & White, 2016 ). However, despite encouraging demonstrations, translation of this research into the clinic has been limited ( Jiang, Dosen, Müller, & Farina, 2012;Kuiken et al., 2014 ). This shortcoming may be because it is very difficult for the amputees to generate distinct activity patterns for different movement classes of hand or wrist movements. This will lead to overlap of the movement classes in the feature space which will in turn impair the classification. Advanced machine learning with induced artificial variability or, perhaps, with daily recalibration may improve the accuracy ( Liu, Sheng, Zhang, He, & Zhu, 2016 ). However, because it is not clinically viable to emulate all possible variations, during prosthesis use, incorrect classifications will likely take place in response to EMG patterns that were not observed during training or recalibration.
Moreover, EMG pattern recognition performance is influenced by many factors including, but not limited to, change in limb position (with respect to the body) ( Fougner, Scheme, Chan, Englehart, & Stavdahl, 2011;Park, Suk, & Lee, 2016 ), shift in EMG electrode location ( Hargrove, Englehart, & Hudgins, 2008;Spanias, Perreault, & Hargrove, 2016 ), change of forearm orientation ( Peng et al., 2013 ), varying the muscular contraction force level ( Al-Timemy, Bugmann, Escudero, & Outram, 2013b; Al-Timemy, Khushaba, Bugmann, & Escudero, 2015;He, Zhang, Sheng, Li, Zhu, 2015 ), and the changing characteristics of the myoelectric signals itself over a long time Liu, Zhang, Sheng, & Zhu, 2015 ), that in turn makes the long term myoelectric control an open challenge. MacIsaac et al. ( MacIsaac, Parker, Scott, Englehart, & Duffley, 2001 ), tested the effect of joint angle and muscle contraction levels individually by estimating the EMG mean frequency and conduction velocity. They showed that both factors contribute to the variability observed in EMG parameter estimation, with the effect of joint angle variation being more dominant than that of the force level. Similarly, Tkach et al. ( Tkach, Huang, & Kuiken, 2010 ) investigated the individual effects of electrode movement, variation in muscle contraction effort, and fatigue on the EMG pattern recognition accuracy. They showed that muscle fatigue has the smallest effect whereas electrode movement and force level variations can significantly affect the accuracy. In addition, several research groups have studied isolated effects of the forearm posture, position, and orientation ( Fougner et al., 2011;Peng et al., 2013 ). They showed that with employing accelerometers and using training data from all positions that the user expects to encounter during real-time usage ( Fougner et al., 2011;Peng et al., 2013 ) the isolated effect of forearm posture, position, and orientation can be reduced. Furthermore, Yang et al. ( Yang, Yang, Huang, & Liu, 2015 ) reported that paradigms in which the EMG signals are collected during dynamic arm postures with varying muscular contractions can largely reduce the misclassification rate. However they only used the mean absolute value of the EMG signals as feature and hence the effectiveness of other feature extraction methods was not quantified.
Previous research quantified the effect of dynamic factors on EMG pattern recognition performance in isolation. In this paper, we investigate the combined effect of multiple dynamic factors. We consider varying the muscle contraction effort s, the first dynamic factor, across three levels along each of three different forearm orientations, the second dynamic factor. We further study the classification performance at each contraction level and forearm orientation with a number of time-and frequency-based EMG features. This is a novel analysis step; not previously utilized in this direction with multiple factors. Recently, we introduced power spectral descriptor features of the EMG signals to mitigate the impact of muscular contraction levels on EMG power spectrum characteristics . Here, we compare the classification performance of this and five other EMG and ac-celerometry features. In addition, we test generalizability of the classification across different forearm orientations and muscle contraction effort s. A preliminary version of this work was reported in ( Khushaba, Al-Timemy, & Kodagoda, 2015 ).
In Section 2 , we present the methods of recording and analysis of the EMG signals and the details of our generalization protocols. Results are reported in Section 3 , before we discuss their significance and conclude in Section 4 .

Subjects
Twelve intact-limbed and one bilateral amputee subjects (age range: 20-33 years, 1 female) with average forearm circumference of 26.59 ± 2.41 cm, participated in this study. At the time of the experiment, the amputee subject (30 years old male) had lived for six years with his condition. Data for ten of the able-bodied subjects was collected at University of Technology, Sydney, Australia. Data from the three subjects (including one amputee subject) was collected in Babylon and Baghdad, Iraq. Participants had no previous experience with EMGcontrolled interfaces. Subjects were introduced to the system in a 20-min briefing session before the main experiment.
All participants observed the corresponding university's research ethics committee approvals and gave informed consent to participate in the study.

Data acquisition
Data recorded in Australia : Data from six EMG electrodes was recorded with a Bagnoli desktop EMG system (Delsys Inc., USA), with a gain of 10 0 0. Electrodes were equally spaced across the circumference of the forearm for the intact-limbed as shown in Fig. 1 a. A 2-slot adhesive skin interface was applied on each of the sensors to firmly attach them to the skin. A conductive adhesive reference electrode (dermatrode reference electrode) was placed near the subjects' wrist. An additional 3-D accelerometer (MPU-6050 from InvenSense) was attached to the participants' wrist as shown in Fig. 1 a to record wrist acceleration. A 12-bit analog-todigital converter (National Instruments, BNC-2090) was used to sample the signal at 40 0 0 Hz. The accelerometer data was resampled to match the sampling rate of the EMG signals, with an original sampling rate of 26.6 ± 0.3Hz.
Data recorded in Iraq : Data was recorded from three subjects in Iraq, one bilateral amputee and two intact-limbed subjects. Skin was first prepared with an abrasive gel (NuPrep, USA). For the amputee subject, eight pairs of Ag/AgCl electrodes (Tyco Healthcare GmbH, Germany) were placed around the amputee's right stump ( Fig. 1 b) and connected to an EMG amplifier (built in-house). As for the other two intact-limbed subjects, the EMG sensors were placed on the forearm. The EMG signals were then amplified with a custom built amplifier (gain: 10 0 0), bandpass filtered between 20-450 Hz with a 50Hz notch filter, and sampled at a rate of 2,0 0 0 Hz with a 16-bit ADC (National Instruments NI USB-6210). A Labview-based software was developed to acquire and display the EMG signals. All recorded signals were analyzed using MATLAB ® (Mathworks, USA).
We considered three forearm orientations: wrist fully supinated, at rest, and fully pronated, denoted by Orienta-  tions 1 to 3 in Fig. 2 . At each orientation, subjects repeated all six movements at three different muscular contraction levels: low, medium, and high. Three trials for each movement were recorded, with each beginning with the low contraction level and gradually increasing the intensity of contraction. Instructions appeared clearly on the computer screen using an image of the desired movement along with a text caption.
In total, 162 trials per subject were collected: 6 movement classes × 3 forearm orientations × 3 contraction levels × 3 trials per movement. The number of conditions was selected such that experiment remains short, and yet, it would be possible to conduct statistical analysis. To avoid fatigue, each trial lasted 5 s and subjects had up to 10 s of rest between trials.
The loss of proprioceptive feedback from the hand after amputation can make the task very challenging for the amputee subjects ( Pistohl, Joshi, Ganesh, Jackson, & Nazarpour, 2015 ). We therefore showed the raw EMG signals on the screen to help the amputee to generate the movement with the required muscular contraction level.

Data analysis
We investigated the combined effect of forearm orientation and muscle contraction level on the generalizability of the EMG pattern recognition. We therefore defined two schemes: 1. Within-orientation generalizability , in which we evaluated the classification performance when the training and testing data were selected from the same forearm orientation condition; and 2. Between-orientation generalizability , in which the classifier was trained and tested with data recorded in different orientation conditions.

Feature extraction
We investigated the effect of the extracted features on the generalizability of the EMG pattern recognition. Using an overlapping window size of 150 ms with 75 ms increments (50% overlap), we extracted the following features: Feature sets 1 to 4 have been used in myoelectric control widely ( Boostani & Moradi, 2003;Hakonena et al., 2015 ). The DFT (5) and TD-PSD (6) features were recently proposed in Khushaba et al., 2014 ). In the interest of completeness we briefly review the underlying concepts of the DFT and TD-PSD features. The interested reader is referred to Khushaba et al., 2014 ) for further information.
The DFT and TD-PSD features form a set of invariants to force level variations in two steps: (1) a set of features describing the EMG power spectrum are extracted either from the EMG power spectrum, for DFT features, or directly from the time-domain signal, for TD-PSD; (2) a cosine similarity function is employed to estimate the angle between the extracted power spectrum characteristics from the original EMG signals and their non-linear version in TD-PSD, while the DFT method considers the angle between force levels within each of predefined frequency bands. The resulting vector is then used as the final feature set. This mechanism has proven robust to the variability caused by force level variations on the intact-limbed  and amputees , with TD-PSD offering more powerful solutions for amputees, while performing equivalently to DFT on intact-limbed subjects. Furthermore, we investigated the potential enhancement of the classification accuracy when the RMS of the accelerometry sensor signal was included as an additional element in the above feature vectors.

Classification
We used a support vector machine (SVM) classifier with a radial basis kernel. Parameters were optimized based on training accuracy: C = 32 and γ = 0.0625. The utilized SVM classifier performed comparably to linear discriminant analysis (LDA) classifier.

Statistical analysis
To test the statistical significance of our results, we carried out repeated measures ANOVA, unless stated otherwise. Where required, post-hoc analysis with a Bonferroni correction was performed. Significance level was set to 0.05. All statistical analysis were performed in IBM ® SPSS Statistics (ver. 21).

Experimental results
In reporting the results, we first combine all results from all subjects ( n = 13 ). Following that we report the classification results for the amputee subject. Finally, we investigate the relative information in the EMG and accelerometer features in Section 3.5 .

Generation of graded muscular contraction levels
We first verified whether the subjects could produce EMG activity in all different classes of movement at three muscular contraction levels (low, medium, and high). This was carried out by analyzing the RMS of EMG activity across all channels, orientations and movements. Fig. 3 depicts a representative radar plot of the EMG signals RMS values collected from one subject performing wrist flexion (C4) when the wrist was fully pronated (Orientation 3). The RMS values from the different channels were then grouped in separate columns for each force level, thereby constructing an RMS matrix. A one-way ANOVA was utilized to validate statistical significance, treating each column of the RMS matrix as a separate group. We determined whether the population means of the columns are equal. A significant difference between the means of the different force levels was observed ( p = 0 . 003 ).

Within-orientation generalization
At each orientation condition, we compared the performance of different f eature sets when the classifier was trained with data recorded on a certain contraction level and was tested with data from all possible force levels. We selected the first two trials for training and the third was used for testing. The results of this experiment are shown in Fig. 4 . An 3-by-6 (orientations-by-features) ANOVA with repeated measures showed that there is no significant difference between the classification scores achieved in different forearm orientations ( n = 13 , F 2 , 24 = 3 . 08 , p = 0 . 06 ).
We however observed a significant effect of the extracted features ( n = 13 , F 1 . 78 , 21 . 39 = 10 . 03 , p < 10 −3 , Greenhouse-Geissercorrected). Bonferroni corrected post-hoc pairwise comparisons showed that the TD-PSD features result in significantly higher classification scores when compared to the TD (difference: 10 . 82% , p = 0 . 01 ), TDAR (difference: 5 . 9% , p = 0 . 01 ) and TDAR2 (difference: Fig. 5. Class-wise classification accuracy results, averaged across all subjects, using the TD-PSD features when training and testing data was acquired from the same forearm orientation. Error bars represent the standard errors. 5 . 7% , p = 0 . 02 ) features. However, the difference between the results achieved by the TD-PSD, the DFT and the wavelet transform were statistically insignificant ( p > 0.05). Interestingly, DFT features outperformed the classic TD features by 10.46% ( p = 0 . 04 ). As the TD-PSD feature set has previously shown to outperform DFT and other feature sets when tested on a group of amputees, we applied only TD-PSD for the remaining of the within-orientation analysis. Classification resulted in higher accuracies when the training data was collected at medium contraction effort s ( Fig. 4 ).
We investigated the classification accuracy of individual movement classes with the TD-PSD features ( Fig. 5 ). A 3-by-6 (orientations-by-class), ANOVA with repeated measures showed no differences across the three forearm orientations ( n = 13 , F 2 , 17 = 0 . 47 , p = 0 . 63 ), despite the observed differences in the class-wise recognition results.

Between-orientation generalization
We considered training the classifier with the TD-PSD features extracted from data collected in all contraction levels within a specific forearm orientation. We then tested the classifier with data from all contraction levels within the remaining orientations, that is, the training and testing samples were from different orientations.
The performance of the classifier in this case dropped significantly when compared to the case when the training and testing data was collected at the same forearm orientation ( n = 13 , F 1 , 35 = 96 . 37 , p < 10 −3 ). Results are shown in Fig. 6 . This finding is in line with the earlier work of Fougner et al. (2011) .
We further investigated the between-orientation generalization by repeating the classification for all feature sets according to the muscular contraction levels. To that end, the classifier was trained with data recorded at a certain contraction level from all orientations. Then it was tested with data from all muscular contraction levels from all orientations. In other words, both the training and testing data were from all orientations.
Classification scored are shown in Fig. 7 . Statistical tests revealed the main effects of training data (ANOVA, repeated measures, n = 13 , F 2 , 24 = 20 . 3 , p < 10 −3 ) and the extracted features (ANOVA, repeated measures, corrected with Greenhouse-Geisser, n = 13 , F 2 . 12 , 25 . 48 = 10 . 41 , p < 10 −3 ). A pair-wise post-hoc analysis of the main effect of the training data showed that the classifier achieved the highest classification results when trained with data Fig. 6. Class-wise classification accuracy, averaged across all subjects, using the TD-PSD features when the training and testing data was acquired from different forearm orientations. Error bars represent the standard errors. recorded at medium contraction level (medium-low difference: 7 . 6% , p < 10 −3 ) and (medium-high difference: 5 . 6% , p = 0 . 002 ).
A pair-wise post-hoc analysis of the main effect of the extracted features revealed that the scores achieved with the TD-PSD feature were significantly higher than that of all other features p < 10 −3 , but the DFT feature (difference: 2 . 21% , p = 0 . 34 ). In addition, the performance achieved with the DFT feature was significantly better than that achieved with the TD features only (difference: 8 . 79% , p = 0 . 03 ). All other differences were statistically insignificant ( p > 0.05). Fig. 8 illustrates the average confusion matrix of the classification results across all subjects with the TD-PSD features achieved at medium contraction levels. Between all classes, C2 (Hand Open) showed the largest error. The misclassification of C2 in this case might be because of the variability of opening with different contraction level. It may be that with further training, and with visual feedback, these results can be further enhanced across all forearm orientations and muscular contraction levels.

Table 1
Classification accuracy of the amputee subject's data only with TD-PSD and DFT features, with training and testing data from all contraction levels within each forearm orientation.

Classification results for the amputee subject
Finally, we report the classification scores for the amputee subject in the following two scenarios: 1. With the classifiers trained based on data of all contraction levels within each orientation and tested based on data of all muscular contraction levels from other orientations. The overall accuracy results for this scenario are shown in Table 1 ; 2. With training and testing data that belong to all orientations for each contraction level. The overall accuracy results for this scenario are shown in Table 2 .
The results in both tables are given for the TD-PSD and the DFT features only, as these were the most promising features in terms of generalization capability as demonstrated in Fig. 7 .   Fig. 9. Relative information in the EMG and accelerometer features. Inclusion of the accelerometer data improves the classification accuracy.

Relative information in the EMG and accelerometer data
We quantified the benefit of including the accelerometry information in the classification. We used data from subjects for whom the accelerometry data was recorded. The mean classification performance averaged across ten subjects and orientations for the three contraction level train-testing conditions are shown in Fig. 9 . The classification accuracy when using the combined TD-PSD and accelerometer features was significantly higher than that with the TD-PSD features only condition (ANOVA, repeated measures, n = 10 , F 1 , 9 = 31 . 37 , p < 10 −4 ).

Concluding remarks
With using a subset of well-known EMG feature extraction, we studeid the effects of combined effect of forearm orientation and muscular contraction level on the performance of EMG pattern recognition. Both of the TD-PSD and DFT features were previously used in  to reduce the effect of varying the contraction level on the classification accuracy of the EMG signals. However, neither of these feature sets were tested previously in experiments with more than one dynamic factor affecting the EMG signals. Our results show that the recently proposed TD-PSD and DFT features are superior, in terms of classification accuracy, to other features; with a tendency for the TD-PSD feature to produce higher results. Training the classifier with data from different forearm orientations and using the TD-PSD and DFT features offered promising solutions to the problem of identifying the hand and wrist movements when performed at different orientations and muscular contraction levels.
Analysis of the amputee data suggested, predictably, that training and testing a classifier with the data collected at the same forearm orientation provides the highest scores. In addition, our results showed that there is a considerable advantage in including data from all orientations in the training set to support the classifier with generalizability to novel data.
These results indicate that when training a classifier with data with low or high contraction levels, the classifier can generalize reasonably to other conditions. In addition, we showed that training the classifier with data recorded at medium contraction levels can significantly enhance generalization on novel data with medium and low contraction conditions, but not at the high contraction level. For amputees, generating large-amplitude EMGs for a long time may become very tiring, as they may have not used their muscles for a long time. Therefore, it is more practical to train the amputees on gestures at low or medium contraction levels to avoid fatigue and indeed misclassification. Moreover, our results suggest that changing the forearm orientation has a more profound impact on the classification results than changing the muscular contraction levels ( Figs. 5 and 6 ). Specifically, Fig. 5 showed no differences between the classification accuracy results when the training and testing data were acquired from the same orientation. However, training and testing on data from different orientations affected the classification results significantly ( Fig. 6 ). Thus, the collection of training data from multiple orientations was necessary to enhance generalization ( Fig. 7 ).
The presented results suggest several future research directions to further improve the performance of myoelectric-driven intelligent systems: • More efforts should be directed towards advancing the methods of feature extraction to overcome the influence of dynamic factors that limit the performance. The use of advanced machine learning methods such as deep neural networks and muscles synergies extraction should also be investigated on problems under the influence of multiple dynamic factors as such methods may provide substantial improvements upon the utilized time-and-frequency EMG feature extraction methods ( Diener, Janke, & Schultz, 2015;Ison, Vujaklija, Whitsell, Farina, & Artemiadis, 2016;. Meanwhile, we showed that the performance of the learning algorithms can be improved by using feature extraction methods that rely on the angular information of muscle activation patterns. Features such as the TD-PSD and the DFT proved more successful than others in reducing the impact of the two dynamic factors that we considered in this paper. Such features can be readily implemented into a prosthesis controller for real-time control, especially that the EMG pattern recognition systems are nowadays becoming available for clinical testing, e.g. the COAPT complete control system  1 . • More research efforts should be directed to investigate whether accelerometry could provide a reliable alternative or complementary medium for motor intention detection when the quality of the EMG signal deteriorates under many circumstances such as altered skin conditions, temperature change and sweat ( Asghari Oskoei & Hu, 2007;Farina et al., 2014;Jiang et al., 2012 ). We provided evidence that the inclusion of accelerometry information can significantly improve the classification scores in our experiments. The use of accelerometry in EMG decoding was first proposed by ( Fougner et al., 2011;Scheme, Fougner, Stavdahl, Chan, & Englehart, 2010 ). They demonstrated that when data was collected under multiple limb positions, the use of accelerometry information yielded a significant improvement in motion classification accuracy. Our results are in line with the work of Atzori et al. (2014) ; Krasoulis, Vijayakumar, and Nazarpour (2015) ; Kyranou, Krasoulis, Erden, Nazarpour, and Vijayakumar (2016) who showed that inertial measurements, e.g. accelerometry, information can yield higher classification and finger movement reconstruction performance when compared to decoding with EMG-features only. This finding could have a remarkable impact in clinical applications, where the use of multi-modal information improves robustness by inducing redundancy. • Moreover, we previously showed that it may be possible to control a prosthesis hand with arbitrary and abstract maps between upper-limb muscles and active joints of prosthetic hands, for example, grasping an object by contacting the index finger muscle only or by contracting a small group of muscles that do not control the grasp naturally ( Pistohl, Cipraini, Jackson, & Nazarpour, 2013 ). Future research may include fusion of the EMG and accelerometry in abstract control interfaces to test whether subjects can learn to control the prosthesis with accelerometry data as well as they can with EMG.
Finally, the main strength of this paper was that it provided a pioneering evaluation into the combined effect of multiple dynamic factors on the EMG classification accuracy using different features. To the authors' knowledge this has not been previously investigated. On the other hand, it is also important to discuss a few points that may be considered as limitations of the proposed research method, including: Firstly, a small number of hand movements were considered in this work. It should be noted here that three trials for each of these movements were performed at each of the muscular contraction levels within each specific forearm orientation. Following that, the whole set was repeated for the remaining two forearm orientations. Despite the resting periods included between the trials, some of the recruited 13 subjects reported that the inclusion of any further hand movements could be fatiguing. Hence, we limited the experiment to six classes of movements only. Secondly, we recruited only one amputee subject in this study because the length of residual forearm of our other amputee volunteers was not adequate to study the effect of forearm orientation. We continue to recruit more amputees to study the influence of forearm orientation and muscular contraction levels on EMG pattern recognition.