Pattern recognition of number gestures based on a wireless surface EMG system

https://doi.org/10.1016/j.bspc.2012.08.005Get rights and content

Abstract

Using surface electromyography (sEMG) signal for efficient recognition of hand gestures has attracted increasing attention during the last decade, with most previous work being focused on recognition of upper arm and gross hand movements and some work on the classification of individual finger movements such as finger typing tasks. However, relatively few investigations can be found in the literature for automatic classification of multiple finger movements such as finger number gestures. This paper focuses on the recognition of number gestures based on a 4-channel wireless sEMG system. We investigate the effects of three popular feature types (i.e. Hudgins’ time–domain features (TD), autocorrelation and cross-correlation coefficients (ACCC) and spectral power magnitudes (SPM)) and four popular classification algorithms (i.e. k-nearest neighbor (k-NN), linear discriminant analysis (LDA), quadratic discriminant analysis (QDA) and support vector machine (SVM)) in offline recognition. Motivated by the good performance of SVM, we further propose combining the three features and employing a new classification method, multiple kernel learning SVM (MKL-SVM). Real sEMG results from six subjects show that all combinations, except k-NN or LDA using ACCC features, can achieve above 91% average recognition accuracy, and the highest accuracy is 97.93% achieved by the proposed MKL-SVM method using the three feature combination (3F). Referring to the offline recognition results, we also implement a real-time recognition system. Our results show that all six subjects can achieve a real-time recognition accuracy higher than 90%. The number gestures are therefore promising for practical applications such as human–computer interaction (HCI).

Highlights

► We investigated 3 most popular feature sets and 4 classifiers for sEMG analysis. ► We proposed employing the MKL-SVM method and obtained super results. ► We proposed a novel application, number gesture recognition. ► We implemented a real-time system for the proposed application.

Introduction

Hand gesture recognition technology based on forearm surface electromyography (sEMG) has been an active research direction due to its broad applications in myoelectric control. With wired or wireless sEMG sensors, human motions can be captured non-invasively by sEMG signals and such sEMG signals can be intelligently recognized as control commands in many myoelectric systems such as multifunction prosthesis, wheelchairs, virtual keyboards, gesture-based interfaces for virtual reality games, etc. [1], [2]. Depending on the involved movements, current sEMG-based hand gesture recognition and classification researches can be divided into three main categories: gross hand, wrist and arm movement recognition; individual finger activation and movement detection; and multiple finger gesture classification.

The majority of previous research has been focused on gross hand, wrist and arm movement recognition and quite high recognition accuracy can be achieved. For instance, with two, four, five or eight sEMG channels, a high classification accuracy which is above 95% was reported for classifying four or six movements [3], [4], [5]. However, it is worth noting that these movements, such as palm extension and closure, wrist flexion and extension, wrist pronation and supination, represent relatively easy classification problems since they generate distinguishing sEMG activation patterns.

Individual finger activation and movement, such as finger typing, generally involves flexion and extension of the individual thumb, index, middle, ring and little fingers. Several recent works have been reported along with this research direction [6], [7], [8]. Researchers in [7] used five channels to collect forearm sEMG signals for a piano-tapping task, in which the subjects tapped a keyboard with each of the five fingers, and a 85% recognition accuracy was achieved by using artificial neural network classifiers. In [8], though a 98% high accuracy could be achieved for classifying 12 flexion and extension movements of individual fingers, 32 sEMG channels were used which is highly impractical for real-life online applications.

Regarding the research direction of multiple finger gesture classification, to our knowledge, relatively few research papers were published and there was no multiple finger gesture benchmark proposed yet. In [9], authors used blind source separation (BSS) and artificial neural network (ANN) techniques to off-line classify four subtle hand gestures by four wired sEMG channels and achieved 97% average accuracy for seven subjects, however, only three of the four gestures can be categorized as multiple finger gesture tasks. In this paper, we plan to define a set of multi-finger movement tasks which could be commonly used for benchmark testing. Based on our extensive preliminary investigation, we choose number gestures as an appropriate set of multi-finger movements because of the following reasons: they satisfy the basic definition of multi-finger movements; and people use number gestures frequently in real-life especially when there are language communication difficulties, which is indeed a major reason why originally number gestures were invented. In this work, we plan to study Chinese number gestures representing the numbers zero to nine, as illustrated in Fig. 1. We propose to recognize these 10 classes of subtle hand gestures based on a 4-channel wireless sEMG system. The recognition procedure is implemented in two phases. In Phase-1, we develop off-line recognition algorithms to study their recognition performances and check the feasibility of implementing a real-time recognition system. We first investigate the most popular feature extraction and classification algorithms; then based on observed performances we further propose to combine all three features together and employ multiple kernels to adapt the combined feature set structure by multiple kernel learning (MKL). In SVM, a kernel function implicitly maps samples to a feature space given by a feature map. It is often unclear what the most suitable kernel for the task at hand is, and hence the user may wish to combine several possible kernels. One problem with simply adding kernels is that using uniform weights is possibly not optimal. For instance, if one kernel is not correlated with the labels at all, then giving it a positive weight just adds noise [10]. MKL is an efficient way of optimizing kernel weights. In Phase-2, we implement a real-time sEMG recognition system for Chinese number gestures and demonstrate its online performance as a preliminary study for possible practical applications. For the two phases, we achieved 97.93% and 95% high average recognition accuracies respectively.

The main contributions of this work are summarized as below: (1) based on a 4-channel wireless system, the most popular feature sets and classifiers were investigated and a new classification method based on MKL was proposed for the defined gesture recognition problem; and (2) a real time recognition system was implemented and a preliminary study for possible practical applications in a controlled laboratory setting was demonstrated.

Section snippets

sEMG system

The architecture of the mentioned sEMG recording system is shown in Fig. 2. INA333 (Texas Instruments, Inc., Dallas, TX) is adopted for the data acquisition unit. TelosB mote (Crossbow, Inc., Milpitas, CA) based on IEEE 802.15.4/ZigBee is used in the analog-to-digital converter (ADC) and the wireless transmission unit. A software platform based on Java is installed on the computer. The maximum transmission distance indoors can achieve 20 m.

Electrode placement

Based on the movement tasks and the muscle anatomy of

Off-line recognition results and discussion

Three general feature types described in Section 2.3.2 and four popular classification algorithms described in Section 2.3.3, representing a total of 12 combinations, are first applied to our recognition problem of Chinese number gestures. Then based on their performances, we suggest to combine the three feature (3F) together and apply MKL-SVM method. To make a fair comparison between single kernel and multiple kernels, we also employ the single kernel SVM combined with 3F for classification.

Conclusion

In this paper, we first investigated the effects of three feature sets and four popular classification algorithms on the off-line recognition accuracy. Then we further proposed combining the three features together and employing the MKL-based method, which yields the highest accuracy 97.93%. Finally, a real-time recognition system for the number gestures was implemented by employing the TD features and the QDA classifier. Our real sEMG results show that all six subjects can achieve a real-time

References (36)

  • F. Tenore et al.

    Decoding of individuated finger movements using surface electromyography

    IEEE Transactions on Biomedical Engineering

    (2009)
  • G.R. Naik et al.

    Subtle hand gesture identification for HCI using temporal decorrelation source separation BSS of surface EMG

  • G. Lanckriet et al.

    A statistical framework for genomic data fusion

    Bioinfomatics

    (2004)
  • H.L. Lew et al.

    Pictorial Guide to Muscles and Surface Anatomy

    (2007)
  • Guanglin Li et al.

    Selection of sampling rate for EMG pattern recognition based prosthesis control

  • X. Zhang et al.

    Research on gesture definition and electrode placement in pattern recognition of hand gesture action SEMG

  • B. Hudgins et al.

    A new strategy for multifunction myoelectric control

    IEEE Transactions on Biomedical Engineering

    (1993)
  • S. Leowinata et al.

    A multi function myoelectric control strategy using an array of electrodes

  • Cited by (130)

    • Single-channel surface electromyography signal classification with variational mode decomposition and entropy feature for lower limb movements recognition

      2022, Biomedical Signal Processing and Control
      Citation Excerpt :

      In this research, a quadratic polynomial kernel function was used to transform samples into high-dimensional space to improve the accuracy. The k-nearest neighbor (KNN) classifier aims to predict the classification result of the current sample by using the closest k training samples based on a certain distance metric [13,54]. Compared with other machine learning, the KNN classifier exhibits the advantages of high accuracy and insensitivity to abnormal points.

    View all citing articles on Scopus
    View full text