Pattern recognition of number gestures based on a wireless surface EMG system
Highlights
► We investigated 3 most popular feature sets and 4 classifiers for sEMG analysis. ► We proposed employing the MKL-SVM method and obtained super results. ► We proposed a novel application, number gesture recognition. ► We implemented a real-time system for the proposed application.
Introduction
Hand gesture recognition technology based on forearm surface electromyography (sEMG) has been an active research direction due to its broad applications in myoelectric control. With wired or wireless sEMG sensors, human motions can be captured non-invasively by sEMG signals and such sEMG signals can be intelligently recognized as control commands in many myoelectric systems such as multifunction prosthesis, wheelchairs, virtual keyboards, gesture-based interfaces for virtual reality games, etc. [1], [2]. Depending on the involved movements, current sEMG-based hand gesture recognition and classification researches can be divided into three main categories: gross hand, wrist and arm movement recognition; individual finger activation and movement detection; and multiple finger gesture classification.
The majority of previous research has been focused on gross hand, wrist and arm movement recognition and quite high recognition accuracy can be achieved. For instance, with two, four, five or eight sEMG channels, a high classification accuracy which is above 95% was reported for classifying four or six movements [3], [4], [5]. However, it is worth noting that these movements, such as palm extension and closure, wrist flexion and extension, wrist pronation and supination, represent relatively easy classification problems since they generate distinguishing sEMG activation patterns.
Individual finger activation and movement, such as finger typing, generally involves flexion and extension of the individual thumb, index, middle, ring and little fingers. Several recent works have been reported along with this research direction [6], [7], [8]. Researchers in [7] used five channels to collect forearm sEMG signals for a piano-tapping task, in which the subjects tapped a keyboard with each of the five fingers, and a 85% recognition accuracy was achieved by using artificial neural network classifiers. In [8], though a 98% high accuracy could be achieved for classifying 12 flexion and extension movements of individual fingers, 32 sEMG channels were used which is highly impractical for real-life online applications.
Regarding the research direction of multiple finger gesture classification, to our knowledge, relatively few research papers were published and there was no multiple finger gesture benchmark proposed yet. In [9], authors used blind source separation (BSS) and artificial neural network (ANN) techniques to off-line classify four subtle hand gestures by four wired sEMG channels and achieved 97% average accuracy for seven subjects, however, only three of the four gestures can be categorized as multiple finger gesture tasks. In this paper, we plan to define a set of multi-finger movement tasks which could be commonly used for benchmark testing. Based on our extensive preliminary investigation, we choose number gestures as an appropriate set of multi-finger movements because of the following reasons: they satisfy the basic definition of multi-finger movements; and people use number gestures frequently in real-life especially when there are language communication difficulties, which is indeed a major reason why originally number gestures were invented. In this work, we plan to study Chinese number gestures representing the numbers zero to nine, as illustrated in Fig. 1. We propose to recognize these 10 classes of subtle hand gestures based on a 4-channel wireless sEMG system. The recognition procedure is implemented in two phases. In Phase-1, we develop off-line recognition algorithms to study their recognition performances and check the feasibility of implementing a real-time recognition system. We first investigate the most popular feature extraction and classification algorithms; then based on observed performances we further propose to combine all three features together and employ multiple kernels to adapt the combined feature set structure by multiple kernel learning (MKL). In SVM, a kernel function implicitly maps samples to a feature space given by a feature map. It is often unclear what the most suitable kernel for the task at hand is, and hence the user may wish to combine several possible kernels. One problem with simply adding kernels is that using uniform weights is possibly not optimal. For instance, if one kernel is not correlated with the labels at all, then giving it a positive weight just adds noise [10]. MKL is an efficient way of optimizing kernel weights. In Phase-2, we implement a real-time sEMG recognition system for Chinese number gestures and demonstrate its online performance as a preliminary study for possible practical applications. For the two phases, we achieved 97.93% and 95% high average recognition accuracies respectively.
The main contributions of this work are summarized as below: (1) based on a 4-channel wireless system, the most popular feature sets and classifiers were investigated and a new classification method based on MKL was proposed for the defined gesture recognition problem; and (2) a real time recognition system was implemented and a preliminary study for possible practical applications in a controlled laboratory setting was demonstrated.
Section snippets
sEMG system
The architecture of the mentioned sEMG recording system is shown in Fig. 2. INA333 (Texas Instruments, Inc., Dallas, TX) is adopted for the data acquisition unit. TelosB mote (Crossbow, Inc., Milpitas, CA) based on IEEE 802.15.4/ZigBee is used in the analog-to-digital converter (ADC) and the wireless transmission unit. A software platform based on Java is installed on the computer. The maximum transmission distance indoors can achieve 20 m.
Electrode placement
Based on the movement tasks and the muscle anatomy of
Off-line recognition results and discussion
Three general feature types described in Section 2.3.2 and four popular classification algorithms described in Section 2.3.3, representing a total of 12 combinations, are first applied to our recognition problem of Chinese number gestures. Then based on their performances, we suggest to combine the three feature (3F) together and apply MKL-SVM method. To make a fair comparison between single kernel and multiple kernels, we also employ the single kernel SVM combined with 3F for classification.
Conclusion
In this paper, we first investigated the effects of three feature sets and four popular classification algorithms on the off-line recognition accuracy. Then we further proposed combining the three features together and employing the MKL-based method, which yields the highest accuracy 97.93%. Finally, a real-time recognition system for the number gestures was implemented by employing the TD features and the QDA classifier. Our real sEMG results show that all six subjects can achieve a real-time
References (36)
- et al.
Comparison of k-nearest neighbor, quadratic discriminant and linear discriminant analysis in classification of electromyogram signals based on the wrist-motion directions
Current Applied Physics
(2011) - et al.
Classification of the myoelectric signal using time-frequency based representations physics
Medical Engineering and Physics
(1999) - et al.
Classification of the myoelectric signal using time-frequency based representations
Medical Engineering and Physics
(1999) - et al.
Multi-channel surface EMG classification using support vector machines and signal-based wavelet optimization
Biomedical Signal Processing and Control
(2008) - et al.
Gestures as input: neuroelectric joysticks and keyboards
Pervasive Computing
(2003) - et al.
Myoelectric control systems—a survey
Biomedical Signal Processing and Control
(2007) - et al.
A fuzzy clustering neural network architecture for multifunction upper-limb prosthesis
IEEE Transactions on Biomedical Engineering
(2003) - et al.
A real-time EMG pattern recognition system based on linear-nonlinear feature projection for a multifunction myoelectric hand
IEEE Transactions on Biomedical Engineering
(2006) - et al.
Neural networks for online classification of hand and finger movements using surface EMG signals
- et al.
Classification of individual finger motions hybridizing electromyogram in transient and converged states
Decoding of individuated finger movements using surface electromyography
IEEE Transactions on Biomedical Engineering
Subtle hand gesture identification for HCI using temporal decorrelation source separation BSS of surface EMG
A statistical framework for genomic data fusion
Bioinfomatics
Pictorial Guide to Muscles and Surface Anatomy
Selection of sampling rate for EMG pattern recognition based prosthesis control
Research on gesture definition and electrode placement in pattern recognition of hand gesture action SEMG
A new strategy for multifunction myoelectric control
IEEE Transactions on Biomedical Engineering
A multi function myoelectric control strategy using an array of electrodes
Cited by (130)
Pattern recognition of hand movements based on multi-channel mechanomyography in the condition of one-time collection and sensor doffing and donning
2024, Biomedical Signal Processing and ControlOvercoming the effect of muscle fatigue on gesture recognition based on sEMG via generative adversarial networks
2024, Expert Systems with ApplicationsPrediction and classification of sEMG-based pinch force between different fingers
2024, Expert Systems with ApplicationsReconstruction of incomplete surface electromyography based on an adversarial autoencoder network
2023, Biomedical Signal Processing and ControlAdaptive detection of Ahead-sEMG based on short-time energy of local-detail difference and recognition in advance of upper-limb movements
2023, Biomedical Signal Processing and ControlSingle-channel surface electromyography signal classification with variational mode decomposition and entropy feature for lower limb movements recognition
2022, Biomedical Signal Processing and ControlCitation Excerpt :In this research, a quadratic polynomial kernel function was used to transform samples into high-dimensional space to improve the accuracy. The k-nearest neighbor (KNN) classifier aims to predict the classification result of the current sample by using the closest k training samples based on a certain distance metric [13,54]. Compared with other machine learning, the KNN classifier exhibits the advantages of high accuracy and insensitivity to abnormal points.