Next Article in Journal
Dynamic Acetabular Cup Orientation during Gait: A Study of Fast- and Slow-Walking Total Hip Replacement Patients
Previous Article in Journal
Estimation of Validity of A-Mode Ultrasound for Measurements of Muscle Thickness and Muscle Quality
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Continuous Locomotion Mode and Task Identification for an Assistive Exoskeleton Based on Neuromuscular–Mechanical Fusion

1
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
2
Guangdong Provincial Key Lab of Robotics and Intelligent System, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
3
Guangdong-Hong Kong-Macao Joint Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
*
Author to whom correspondence should be addressed.
Bioengineering 2024, 11(2), 150; https://doi.org/10.3390/bioengineering11020150
Submission received: 17 November 2023 / Revised: 15 January 2024 / Accepted: 18 January 2024 / Published: 2 February 2024
(This article belongs to the Special Issue AI-Enhanced Wearable Technologies in Bioengineering)

Abstract

:
Human walking parameters exhibit significant variability depending on the terrain, speed, and load. Assistive exoskeletons currently focus on the recognition of locomotion terrain, ignoring the identification of locomotion tasks, which are also essential for control strategies. The aim of this study was to develop an interface for locomotion mode and task identification based on a neuromuscular–mechanical fusion algorithm. The modes of level and incline and tasks of speed and load were explored, and seven able-bodied participants were recruited. A continuous stream of assistive decisions supporting timely exoskeleton control was achieved according to the classification of locomotion. We investigated the optimal algorithm, feature set, window increment, window length, and robustness for precise identification and synchronization between exoskeleton assistive force and human limb movements (human–machine collaboration). The best recognition results were obtained when using a support vector machine, a root mean square/waveform length/acceleration feature set, a window length of 170, and a window increment of 20. The average identification accuracy reached 98.7% ± 1.3%. These results suggest that the surface electromyography–acceleration can be effectively used for locomotion mode and task identification. This study contributes to the development of locomotion mode and task recognition as well as exoskeleton control for seamless transitions.

1. Introduction

Wearable robotic devices—both exosuits and exoskeletons—show potential benefits in assisting humans in walking, reducing metabolism, and enhancing endurance. The design concept of wearable robotic devices is to provide appropriate joint torques and timing based on biomechanical gait analysis [1,2]. Both passive and active devices can contribute to a reduction in energy consumption.
Along with the advantages of exoskeletons comes the weight of systems and constraints on the human body, which results in uncomfortable sensations. Therefore, researchers are interested in exploring additional methods to significantly reduce energy expenditure to compensate for negative influences. In addition to structural improvement, control strategies have been a hot topic of research, essentially focusing on two areas: (1) personalized optimization using human-in-the-loop optimization algorithms [3,4,5] and (2) muscle dynamics [6], which are employed to adjust control parameters. Since the dynamics of human joints vary with the mode of locomotion, e.g., walking on level ground, a slope, or stair ascent/descent [7,8,9], accurate locomotion mode identification and real-time control strategies pose challenges with respect to the use of wearable devices.
Multiple sensors are applied to locomotion mode recognition. Sensors currently used for exoskeleton pattern detection systems can be categorized into three major groups: mechanical, biosignal, and vision sensors.
Many different strategies have been employed using mechanical sensors, such as the use of load cells [10] and inertial measurement units (IMUs) [11,12]. These methods predict locomotion modes by threshold or machine learning according to a human state, such as limb motions or heel contact forces [13]. Mechanical sensors are characterized by convenient, low cost, and relatively good recognition performance. However, mechanical sensors have a step delay, leading to uncomfortable sensations and increased risks of falls. Surface electromyography (sEMG) represents muscle activation and is widely applied in wearable devices for intention recognition [14]. The features of sEMG in the time or frequency domain are often extracted and input to several pretrained classifiers for mode identification. Compared to the responses of mechanical sensors to movements, sEMG signals precede limb motion by 20–150 ms [7,15], which can compensate for the delay of the system. sEMG is also associated with uncertainties, such as those relating to the electrode position, crosstalk, and noise, so measurements must be taken from multiple muscles to ensure a reasonable recognition rate. At present, the combination of mechanical and biosignal sensors is the most widely used method to improve the accuracy of recognition. Studies have shown that the fusion of mechanical and biosignal sensors can significantly improve the classification of gait modes [7,16].
Recently, the use of vision sensors has been proposed for the identification of terrain. Depth images are employed to detect user movement intent or to recognize real-life environmental context through edge detection or 3D point cloud classification [17,18]. The authors of a previous study achieved reliable confidence estimation, reporting a decision result 0.6 to 4 s ahead of the motion intention [19]. Vision sensors are generally large, and this method requires a great deal of computational power. In addition, visual sensors can only be used to recognize terrain (such as level ground, slopes, and stairs) and cannot consider the impact of tasks (such as speed and load) on assistive strategies.
Many experimental studies have pointed out that both kinematics and kinetics change considerably as speed and load change [9,20,21]. Therefore, it is critical to adopt different control strategies for corresponding tasks on specific terrain types. Spatiotemporal gait parameters during walking have been extensively studied in both healthy and pathological populations [22]. Walking speed is one of the most important parameters characterizing people’s daily mobility [23] and has been recognized as a proxy measure of ambulation quality. Consequently, speed recognition is a crucial parameter with respect to control strategies for assistive exoskeletons. Gait speed estimation is widely used in clinical applications to analyze the characteristics of walking-related diseases. Another central application area for speed estimation is in sports, where devices like smart watches are employed to predict the speeds for professional athletes or runners. The use of global navigation satellite systems is a highly accurate method for speed measurement [24]. These systems need to be used outside and away from tall buildings. Furthermore, communicating with satellites consumes a considerable amount of power, limiting operating time. Due to the low power consumption and portable characteristics of electrical sensors, inertial sensor-based methods have been developed for speed estimation [22,25]. Algorithms using inertial sensors to estimate walking speed can be grouped into two categories: black-box models based on machine learning [26] and human gait models [23].
Similar to walking speed, it has been demonstrated that carrying loads significantly affect human gait biomechanics [27,28]. Much of the current load carriage research has concentrated on the fundamental tasks of military personnel. Previous research has shown that carrying loads can impact the spatiotemporal parameters of human gait and influence the efficiency and safety of movement, increasing the risk of musculoskeletal injury [29,30]. Although the physiology and biomechanics of load-bearing carriage properties have been characterized [31], few teams have studied the identification of carriage weights for assistive exoskeletons. Weight can be measured initially, but such measurements are impossible during a mission, such as a march. Therefore, it is necessary to recognize load carriage in real time and provide an appropriate control strategy.
In order to assist human walking more efficiently and naturally, the exoskeleton must consider the modes and tasks of gait in the control strategies. An intuitive method is to utilize different control strategies for different gait movements. However, human walking parameters vary significantly based on terrains, speeds, and loads. Up to this point, the primary focus of most exoskeletons has been on terrain recognition [32]. This is mainly because the open-source dataset of human lower limb biomechanics emphasizes gait analysis [8,9] rather than the control strategies of exoskeletons. To our best knowledge, only one study has addressed the effect of walking tasks [6], but it only focused on speeds.
Accordingly, the objective of this study was to achieve the continuous recognition of locomotion modes (level/incline) and tasks (speed/load) through the fusion of neuromuscular–mechanical data for assistive exoskeletons. We formulated two hypotheses: (1) that combining accelerations (ACCs) and sEMG signals would prove effective for recognizing both mode and task, and (2) that incorporating a greater number of features would lead to higher estimation accuracy. To achieve this goal, we gathered a comprehensive dataset of human gait data from injury-free adult participants across various locomotion modes and tasks. This dataset served as the foundation for investigating the optimal algorithm, feature combinations, window length, and window increment. Furthermore, we conducted an assessment of the algorithm’s robustness, considering practical factors.

2. Materials and Methods

2.1. Participants and Protocol

Seven healthy volunteers (age: 24 ± 2, weight: 67 ± 6 kg, height: 175 ± 6 cm) without any physical disease were recruited for this experiment. All participants were healthy and injury free to both lower and upper limbs in the last six months.
Before the start of experiments, all participants were asked to sign an informed consent form. The experimental procedures, specifications, and risks were comprehensively explained to each participant. The experiments were conducted under the protocol of the ethics committee of Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences (SIAT-IRB-221215-H0632). Anthropometric and demographic information of the participants is shown in Table 1.
This study was conducted in low-power medical VLSI and Body Sensor Networks Laboratory Key Lab for Biomedical Informatics and Health Engineering, Chinese Academy Sciences. Accelerations in three axes, sEMGs signals from eight muscles, and ground reaction forces were recorded synchronously in real time. The acceleration and sEMG signals were captured using a 16-channel wireless EMG system sampled at 2000 Hz (Delsys Trigno® 1.2, Natick, MA, USA). Ground reaction forces were recorded using an instrumental treadmill sampled at 1000 Hz (AMTI, MA, USA). Eight muscles of the right leg were selected: gluteus maximus (GM), rectus femoris (RF), vastus lateralis (VL), vastus medialis (VM), biceps femoris (BF), tibialis anterior (TA), gastrocnemius medialis (GAS), and Soleus (SOL). The selection of muscles referred to results of Ding [33].
Six types of patterns were investigated: level ground walking at different speeds (0.75 m/s, 1.25 m/s, 2 m/s, and 2.75 m/s), 15% incline walking at 1.25 m/s, and 10 kg load walking at 1.25 m/s. An AMTI treadmill was used to achieve changes in speed and incline, and the load vest was used to achieve load changes. First, an experienced rehabilitation therapist performed the measurement and sensor-affixing work. Participants were asked to wear swimming trunks and sneakers. A tape measure was utilized to measure the leg length (from the anterior superior iliac spine to the medial malleolus) and body height. The weight was measured by a mass scale. The areas where electrodes were attached were wiped with alcohol-soaked cotton swabs, and any long body hair was trimmed. The EMG electrodes were pasted by double-side adhesive and secured by self-adhesive banding, as shown in Figure 1. The sEMG acquisition followed the SENIAM guidelines [34]. Then, the adjustment process was conducted. Patients adapted to different speeds and weight loads. When the patient felt ready for the experiment, the above-mentioned patterns were collected sequentially. The locomotion modes and tasks listed above were collected in sequence. Each pattern recorded at least 5 trials and each trial included at least 7 gait cycles. A gait cycle was defined from one heel strike of a foot to the next heel strike of the same foot. The first point through which the ground reaction force was greater than 15 N was regarded as the beginning of a gait cycle. There was a 2-min break between each pattern to prevent muscle fatigue.

2.2. Data Processing

This study aimed to design a method for continuously estimating locomotion modes and tasks, generating a decision stream for exoskeleton to apply control strategy. The system structure is depicted in a block diagram shown in Figure 1. The neuromuscular–mechanical fusion signals acquired simultaneously were streamed into a preprocessing session to remove the noise. Feature extraction was applied to the preprocessed signals to obtain a multimodal feature matrix. The matrices of sEMG and ACCs were then downsampled to match the length of GRF. Finally, the post-processing stage was conducted to obtain continuous control decisions and combine the classification results with the gait phase detector.
It is inevitable that unwanted noises would be produced during locomotion, which must be removed. Raw sEMG signals were filtered by a sixth-order bandpass Butterworth filter with cutoff frequencies of 15–500 Hz to remove electrical noise and motion artifacts. The mean value of the corresponding axis was subtracted from all ACC signals to reduce the direct current offset. Raw GRF signals were filtered by a fourth-order low-pass Butterworth filter with a cutoff frequency of 12 Hz. The zero-lag algorithm was applied to all filters to avoid phase shifts. All signals were then segmented by sliding analysis windows for continuous classification decision making. The length of the analysis window was initially set to 140 (equivalent to 70 ms) with a window increment of 10 (equivalent to 5 ms).
The preprocessed signals were then subjected to the feature extraction process. The features of sEMG and the mean value of ACCs were computed within an analysis window mentioned above. The preprocessed GRF signals were passed into the gait phase detector to segment gait cycles and define phases. A feature matrix of EMG and ACCs signals was finally formulated. The feature matrix was downsampled by a factor of two to match the length of GRF signals. The downsampled feature matrix was normalized from 0 to 1 to improve the performance of classifiers. Four gait cycles were used for the training set, and the final gait cycle was used for the test set. Support Vector Machine (SVM) [35], K-Nearest Neighbor (KNN) [36], Quotient-Difference Algorithm (QDA) [37], and Decision Tree (DT) [38] were verified successively. Cross-validation was conducted for each classifier ahead of classification.

2.3. Parameters on Performance

Classifier. The performance of locomotion mode estimation has been reported with different classifiers primarily including SVM, LDA, and ANN [7,14]. This study explored more classifiers for higher performance in locomotion mode and task estimation. We aimed to implemented functionality to reduce the latency of the recognition system. Furthermore, a total of four classifiers were explored, as mentioned earlier. Support Vector Machine (SVM), K-Nearest Neighbor (KNN), Quadratic Discriminant Analysis (QDA), and Decision Tree (DT) are all popular machine learning algorithms used for classification tasks. These algorithms work by analyzing the patterns in the training data and then using that information to make predictions for new, unseen data. In comparison to complex algorithms, these classifiers are known for their fast operation speed and high precision. The classifier of SVM was implemented using libsvm 3.3. The statistics and machine learning toolbox 12.0 [39] was used for the remaining classifiers. Since using more features results in longer running times, only RW, WL, and ACCs were selected for this process. A detailed introduction to the algorithms will not be provided in this paper.
EMG features. The selected features are closely related to the accuracy of estimation. Furthermore, the success of the recognition system depends on whether the selected features effectively reflect the problem under study. Many researchers have successfully achieved terrain estimation using root mean square (RMS), mean absolute value (MAV), number of zero crossings (ZC), waveform length (WL), and number of slope sign changes (SSC). Few researchers explored the individual contribution of each feature and the optimal combinations. In this study, we investigated a total of 15 individual features or combinations thereof. The confusion matrices of a signal feature and combinations were calculated. Table 2 displays the feature sets used in the estimation and abbreviations for feature combinations. The formulas for feature extraction are provided in Appendix A. Additionally, the Kruskal–Wallis one-way analysis of variance by ranks was employed to assess the improvement of ACCs. The reason for choosing Kruskal–Wallis was that we found that RMS, MAV, WL, and other features did not follow a normal distribution.
Length of the analysis window and increment size. The values of sEMG features would vary with the length of the analysis window. Furthermore, since sEMG is characterized as a broad–sense Gaussian random process [40], the window length should not exceed 200 ms to minimize variation [7]. To ensure synchronization between exoskeleton assistive force and human limb movement, we investigated window lengths ranging from 30 (15 ms) to 200 (100 ms) data points. The selected maximum window length was due to the fact that one gait cycle takes less than 160 ms when running at 2.75 m/s, and there were inevitable system delays. Additionally, sEMG signals typically precede limb movements by 30–150 ms; therefore, the window length can impact human–machine collaboration and should be adjusted considering system delay. We employed an overlapping windowing scheme to facilitate a quick decision making, improve signal utilization, and reduce abrupt signal jumps. Smaller window increments result in more decision points within a gait cycle, aiming to achieve smoother control. We compared classification errors and decision points across window increments ranging from 10 to 60 data points. The classification errors were determined by comparing the predicted class label of the final gait to its actual class label. In addition, the Kruskal–Wallis one-way analysis of variance by ranks was used to test window length and increment, respectively.
The robustness. The number of muscles greatly affects the practicability of the method used in real life. To better utilize exoskeleton scenes, the robustness of the SVM model was investigated. Muscles were separated into six groups according to function: gluteus, thigh, calf, anterior thigh (AT), posterior calf (PC), and all muscles. The muscles contained in each group are shown in Table 3.

3. Results

Both the classifier and the design parameters had a substantial impact on recognition accuracy. Figure 2 displays the classification performance of classifiers using an RMS, WL, and ACCs feature set with an analysis window of 140 and an increment of 10. SVM exhibited the lowest classification error, followed by KNN, QDA, and DT, with average error rates of 1.68%, 2.19%, 3.09%, and 6.96%, respectively. However, classifier performance was inconsistent and varied among participants. For instance, in the case of P2, P4, P6, and P7, KNN demonstrated better estimation accuracy than SVM. Among the results for P5, SVM was the sole classifier to achieve a superior result of 96.94%. Given that SVM maintained consistent accuracy across all participants’ tests, ranging form 95.46% to 99.75%, the SVM classifier was selected for the remainder of this study.
Figure 3 shows the influence of feature and feature combinations on classification accuracy when the SVM algorithm was employed to classify eight channels of data with 140 length of analysis window and 10 window increments. The definitions of each of these features are shown in Table 2. Regarding single signal features, WL achieved the highest accuracy in mode and task recognition (92.51%), while SSC had the lowest accuracy (67.63%). It is worth noting that RMS, MAV, and WL all had outliers at 25.73%, 25.90%, and 80.13%, respectively, all from P7. The ACCs feature also yielded relatively good results with an accuracy of 93.09%. The combination of features improved the classification accuracy. Interestingly, the accuracy of all feature combinations (five features of EMG and ACCs) was not the best (98.57%). The best results were obtained with the feature set of RMS, WL, and ACCs (98.32%). In other words, adding features may potentially decrease estimation accuracy. The accuracies of R/W, R/M/W, R/M/W/S feature set were 93.20%, 92.81%, and 91.63%, respectively. The impact of ACCs on the classification accuracy is detailed in Table 4. No significant differences were found between non-ACCs and ACCs feature sets. Overall, ACCs features improved accuracy except for R/W/A of P3, R/M/W/A of P1, and P3. Considering that the feature matrix of E/A had 64 dimensional while R/W/A had 40, and the difference in accuracy between the two sets was small (0.25%), R/W/A was selected for the algorithm, balancing response time and accuracy.
The effects of window increment are displayed in Figure 4 when the SVM algorithm was utilized to classify eight channels of data using a 200 ms window length and R/W/A feature set. Window increments had a minimal effect on the classification accuracy. The highest accuracy achieved was 99.25% when the increment was 60, while the lowest was 98.03% when the increment was 40. The most significant impact of window increments was on the number of decisions made, which directly affected the system’s decision delay. Decision points were counted at the highest speed for each participant within a gait cycle. Furthermore, the system selected the minimum decision count among all participants, which corresponded to the maximum decision delay, as displayed in Table 5. This process ensured that the system worked for everyone. Since there were also hardware delays, such as data transfer delays, the system needed to ensure that the decision delay did not exceed 15 ms. Considering the system delay, a window increment with higher accuracy (increment = 20) was chosen for the algorithm.
Figure 4 shows the effects of window length on classification accuracy. As the length decreased, the estimation accuracy generally showed a downward trend. The classifier produced favorable results within the window length range of 90 to 200 with the highest performance observed at a length of 170, reaching 98.79%. A linear correlation between accuracy and window length was observed in the range of 30 to 90. Table 6 contains the classification accuracy for each participant. The reduction in features and the enhancement of computer computing power minimized the impact of changes in window length on decision delay. Furthermore, there were no significant differences observed between window lengths.
The exploration of SVM algorithm robustness was performed as a separate step with a window length of 140, an increment of 10, and the R/W/A feature set. For this process, participants P1, P2, P4, and P5 were selected, and the statistical results are depicted in Figure 5. The anterior thigh muscles, namely RF, VL, and VM, achieved the highest accuracy of 94.95%, while Glu obtained the lowest accuracy at 70.53%. When considering only the posterior calf muscles, GAS and SOL, the accuracy was 87.08%. Upon adding TA, the accuracy improved to 91.61%. However, this improvement was not consistent for everyone, as P2’s accuracy increased from 94.21% to 97.71%.
According to the results, the SVM classification algorithm was selected for locomotion mode and speed estimation with the R/W/A feature set, a window length of 170, and an increment of 20. The confusion matrices for R/W/A, ACCs, and R/W are displayed in Figure 6. Analysis of the confusion matrix reveals that acceleration signals were advantageous in recognizing speeds, but they struggled to identify the load task accurately. Recognition accuracy for a 10 kg load and a speed of 1.25 m/s reached 87.4% and 83.4%, respectively. Within the R/W feature set, the lowest recognition accuracy was recorded at 66.0% for a speed of 0.75 m/s. In the R/W/A feature set, all tasks achieved commendable recognition rates ranging from 97.4% to 99.9%.
Figure 7 shows an instance of the identification and decision-making results for P1. The lines represent the desired assistive torque that the ankle exoskeleton should supply, while the dots represent the decision streams made by the system. The assistive torque functions were acquired by fitting 0.2 times the ankle biological torque curves in Gaussian form. The results revealed that the minimum number of decisions reached 32 at a speed of 2.75 m/s. This indicates that the system could provide adequate decisions for continuous control.

4. Discussion

The flexibility of exoskeletons in real-life scenarios is limited, which is primarily due to various environmental contexts and the complexity of human locomotion tasks. The most decisive factor in determining the operation mode of gait assistance is generally believed to be the terrain, including level walking, stair climbing, and ramp ascent [41]. However, human walking parameters also exhibit significant variability in different motor tasks, which many studies tend to overlook. In this study, an interface based on sEMG-ACCs signal fusion was developed, which could accurately classify a variety of locomotion modes (level/incline) and tasks (speed/load). Furthermore, the model exhibited high robustness and could still yield favorable results even with only three thigh muscles. Generally, joint sensors and IMUs data are processed by a machine learning or threshold-based method to recognize the situation [42]. This study adopted a fusion method of sEMG and acceleration signals, achieving accurate results in locomotion modes and tasks. By incorporating the biosensor, we eliminated the one-step delay [32]. Compared to the fusion of sEMG and ground reaction forces used by Huang [7], Kyeong [15], and others, the utilization of sEMG and acceleration signals improved the range and accuracy of recognition. The phase-dependent classifiers, as adopted by Kyeong in their study, were not employed in this research for two primary reasons. Firstly, the sEMG signals in a gait cycle exhibited considerable variation, which may lead to feature overlap and, consequently, reduced recognition accuracy. The phase-dependent method was originally designed to address this issue. Secondly, this method reduced multiple decision points and led to decision delays. The SVM classifier produced better classification accuracy, which aligns with the findings of Huang [7] and Kyeong [15]. The reason for this difference may be the varying sensitivity of algorithms to signal quality.
The study has demonstrated that recognition accuracy was not directly proportional to the quantity of features. Regarding signal features, WL achieved the best results, which was possibly because this feature provided information on waveform complexity, aligning with Kyeong’s findings [14]. However, the addition of RMS, MAV, and SCC reduced recognition accuracy. Such examples also existed in R/W/A and E/A, WL and EMG, etc. This may be because the increase in features resulted in overlaps between classes. Therefore, it is necessary to select the combination of features based on the specific problem, and in this study, the RMS, WL, and ACC were extracted. ACCs, as shown in Table 4, have generally improved recognition accuracy, although there was no significant difference. The confusion matrix revealed that ACCs had an advantage in recognizing speed but struggled with load recognition. Figure 6 illustrates that sEMG signals exhibited low recognition accuracy at low speeds, reaching only 66% at 0.75 m/s. This phenomenon arises from the reduced muscle activity at lower speeds, highlighting the significance of incorporating acceleration.
Discrepancies between individual and overall results were present in all investigations. There are two main reasons for this phenomenon: one is individual differences such as walking habits, muscle strength, etc., and the other is electrode position. The first reason is inevitable, so although the overall results improved, there was no significant difference, as indicated in Table 4, Table 5 and Table 6. The window increment mainly impacts the number of decision points (corresponding to the time between two decision points) within a gait cycle, and its determination should be based on specific requirements. A decision delay of 12 ms was chosen due to hardware latency. The window length corresponds to the time difference between the decision number point and muscle activation. Regarding model robustness, the thigh muscles yielded superior results compared to the calf muscles. Task recognition could be accomplished using three muscles on the anterior thigh.
While the method designed based on sEMG–ACCs signal fusion showed significant potential for continuous locomotion mode and task identification, this study also had several limitations. Firstly, this work represents an initial study, which investigated six types of locomotion modes and tasks, including level walking, slope, speed, and load. Secondly, we have only explored recognition and have not yet integrated it with exoskeletons. Thirdly, parameters were obtained based on offline data, and, in particular, the use, window increment, and length need to be re-evaluated for system performance. The aim of this study was to propose a suitable method that can help address task recognition, which is a commonly overlooked aspect in assistive exoskeletons. In future work, it will be essential to quantify varying speeds and loads as participants walk in realistic environments. Additionally, the recognition system would be integrated with the exoskeletons to develop appropriate control strategies. In the next phase of the study, it will be essential to comprehend the assist moment for different types of exoskeletons based on gait phases.

5. Conclusions

This study demonstrated an initial attempt to develop and evaluate a neuromuscular–mechanical fusion-based algorithm for identifying locomotion mode and task. Compared to traditional signal fusion, this study introduced acceleration to the locomotion task recognition. The fusion of sEMG and ACCs achieved the identification of speed and load, which are often overlooked in assistive exoskeletons but are very important. The optimal algorithm, feature combination, window length, window increment, and robustness were explored. The recognition system based on the SVM classifier, R/W/A feature set, 170 window length, and 20 increment produced over 98.7% accuracy for recognizing locomotion modes and tasks. These promising results may assist in applying assistive exoskeletons in real-world environments.

Author Contributions

Conceptualization, Y.L., C.C. and X.W.; methodology, Y.L. and C.C.; software, Y.L., Z.W. and Y.T.; validation, Y.L., Y.X. and S.W.; formal analysis, Y.L. and F.Y.; investigation, Y.L.; resources, Y.L.; data curation, Y.L. and Y.X.; writing—original draft preparation, Y.X.; writing—review and editing, C.C. and X.W.; visualization, Y.X.; supervision, C.C.; project administration, C.C.; funding acquisition, C.C. and X.W. All authors have read and agreed to the published version of the manuscript.

Funding

This work was partially supported by the National Key R&D Program of China (2022YFC3601704), the NSFC-Shenzhen Robotics Research Center Project (U2013207), the Natural Science Foundation of China (62273325, U191320, 62003327), the Natural Science Foundation of Guangdong Province (2019A1515010782), and the SIAT-CUHK Joint Laboratory of Robotics and Intelligent Systems.

Institutional Review Board Statement

The experiment was approved by the Medical Ethics Committee of Shenzhen Institute of Advanced Technology (SIAT-IRB-221215-H0632 (valid time from December 2022 to December 2025)), and the content of the experiment and its possible impact were explained to all the participants before conducting the experiment.

Informed Consent Statement

Informed consent was obtained from all participants involved in the study.

Data Availability Statement

The data that support the findings of this study are available on request from the corresponding author.

Acknowledgments

The authors would like to thank all participants who participated in experiments and the members of the SIAT exoskeleton team.

Conflicts of Interest

The authors declare no conflicts of interest.

Abbreviations

The following abbreviations are used in this manuscript:
sEMGsurface electromyogram
ACCsaccelerations in x, y, and z directions
GMgluteus maximus
RFrectus femoris
VLvastus lateralis
VMvastus medialis
BFbiceps femoris
TAtibialis anterior
GASgastrocnemius medialis
SOLsoleus
SVMSupport Vector Machine
KNNK-Nearest Neighbor
QDAQuotient-Difference Algorithm
DTDecision Tree
RMSroot mean square
MAVmean absolute value
ZCnumber of zero crossings
WLwaveform length
SSCnumber of slope sign changes
GRFground reaction force
ATanterior thigh
PCposterior calf

Appendix A

Appendix A.1

Time domain features: where N is the length of the analysis window and x n is the nth sample in the window.
R M S = i = 1 N x n 2 N
M A V = 1 N n = 1 N x n
Z C = N 1 n = 2 f ( n )
where
f ( n ) = 1 , if x n × x n + 1 < 0 and x n x n + 1 threshold 0 , otherwise
S S C = n = 2 N 1 f x n x n 1 x n x n + 1
where
f ( x ) = 1 , if x threshold 0 , otherwise
W L = N 1 n = 1 x n x n 1
The maximum time delay between two decision points. MG represents the maximum sampling point in the maximum speed in a gait cycle, and MG is the minimum decision point calculated by the recognition system.
T i m e D e l a y = M G F s × M N × 1000

References

  1. Quinlivan, B.T.; Lee, S.; Malcolm, P.; Rossi, D.M.; Grimmer, M.; Siviy, C.; Karavas, N.; Wagner, D.; Asbeck, A.; Galiana, I.; et al. Assistance magnitude versus metabolic cost reductions for a tethered multiarticular soft exosuit. Sci. Robot. 2017, 2, eaah4416. [Google Scholar] [CrossRef]
  2. Asbeck, A.T.; De Rossi, S.M.; Holt, K.G.; Walsh, C.J. A biologically inspired soft exosuit for walking assistance. Int. J. Robot. Res. 2015, 34, 744–762. [Google Scholar] [CrossRef]
  3. Ding, Y.; Kim, M.; Kuindersma, S.; Walsh, C.J. Human-in-the-loop optimization of hip assistance with a soft exosuit during walking. Sci. Robot. 2018, 3, eaar5438. [Google Scholar] [CrossRef] [PubMed]
  4. Zhang, J.; Fiers, P.; Witte, K.A.; Jackson, R.W.; Poggensee, K.L.; Atkeson, C.G.; Collins, S.H. Human-in-the-loop optimization of exoskeleton assistance during walking. Science 2017, 356, 1280–1284. [Google Scholar] [CrossRef] [PubMed]
  5. Gordon, D.F.; McGreavy, C.; Christou, A.; Vijayakumar, S. Human-in-the-loop optimization of exoskeleton assistance via online simulation of metabolic cost. IEEE Trans. Robot. 2022, 38, 1410–1429. [Google Scholar] [CrossRef]
  6. Nuckols, R.W.; Lee, S.; Swaminathan, K.; Orzel, D.; Howe, R.D.; Walsh, C.J. Individualization of exosuit assistance based on measured muscle dynamics during versatile walking. Sci. Robot. 2021, 6, eabj1362. [Google Scholar] [CrossRef] [PubMed]
  7. Huang, H.; Zhang, F.; Hargrove, L.J.; Dou, Z.; Rogers, D.R.; Englehart, K.B. Continuous locomotion-mode identification for prosthetic legs based on neuromuscular–mechanical fusion. IEEE Trans. Biomed. Eng. 2011, 58, 2867–2875. [Google Scholar] [CrossRef] [PubMed]
  8. Lencioni, T.; Carpinella, I.; Rabuffetti, M.; Marzegan, A.; Ferrarin, M. Human kinematic, kinetic and EMG data during different walking and stair ascending and descending tasks. Sci. Data 2019, 6, 309. [Google Scholar] [CrossRef]
  9. Bovi, G.; Rabuffetti, M.; Mazzoleni, P.; Ferrarin, M. A multiple-task gait analysis approach: Kinematic, kinetic and EMG reference data for healthy young and adult subjects. Gait Posture 2011, 33, 6–13. [Google Scholar] [CrossRef]
  10. Long, Y.; Du, Z.J.; Wang, W.D.; Zhao, G.Y.; Xu, G.Q.; He, L.; Mao, X.W.; Dong, W. PSO-SVM-based online locomotion mode identification for rehabilitation robotic exoskeletons. Sensors 2016, 16, 1408. [Google Scholar] [CrossRef]
  11. Kang, I.; Molinaro, D.D.; Duggal, S.; Chen, Y.; Kunapuli, P.; Young, A.J. Real-time gait phase estimation for robotic hip exoskeleton control during multimodal locomotion. IEEE Robot. Autom. Lett. 2021, 6, 3491–3497. [Google Scholar] [CrossRef]
  12. Jang, J.; Kim, K.; Lee, J.; Lim, B.; Shim, Y. Online gait task recognition algorithm for hip exoskeleton. In Proceedings of the 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany, 28 September–2 October 2015; pp. 5327–5332. [Google Scholar]
  13. Parri, A.; Yuan, K.; Marconi, D.; Yan, T.; Crea, S.; Munih, M.; Lova, R.M.; Vitiello, N.; Wang, Q. Real-time hybrid locomotion mode recognition for lower limb wearable robots. IEEE/ASME Trans. Mechatronics 2017, 22, 2480–2491. [Google Scholar] [CrossRef]
  14. Kyeong, S.; Shin, W.; Yang, M.; Heo, U.; Feng, J.r.; Kim, J. Recognition of walking environments and gait period by surface electromyography. Front. Inf. Technol. Electron. Eng. 2019, 20, 342–352. [Google Scholar] [CrossRef]
  15. Kyeong, S.; Feng, J.; Ryu, J.K.; Park, J.J.; Lee, K.H.; Kim, J. Surface electromyography characteristics for motion intention recognition and implementation issues in lower-limb exoskeletons. Int. J. Control Autom. Syst. 2022, 20, 1018–1028. [Google Scholar] [CrossRef]
  16. Young, A.; Kuiken, T.; Hargrove, L. Analysis of using EMG and mechanical sensors to enhance intent recognition in powered lower limb prostheses. J. Neural Eng. 2014, 11, 056021. [Google Scholar] [CrossRef]
  17. Zhang, K.; Zhang, W.; Xiao, W.; Liu, H.; De Silva, C.W.; Fu, C. Sequential decision fusion for environmental classification in assistive walking. IEEE Trans. Neural Syst. Rehabil. Eng. 2019, 27, 1780–1790. [Google Scholar] [CrossRef]
  18. Zhong, B.; Da Silva, R.L.; Li, M.; Huang, H.; Lobaton, E. Environmental context prediction for lower limb prostheses with uncertainty quantification. IEEE Trans. Autom. Sci. Eng. 2020, 18, 458–470. [Google Scholar] [CrossRef]
  19. Li, M.; Zhong, B.; Lobaton, E.; Huang, H. Fusion of human gaze and machine vision for predicting intended locomotion mode. IEEE Trans. Neural Syst. Rehabil. Eng. 2022, 30, 1103–1112. [Google Scholar] [CrossRef]
  20. Farris, D.J.; Sawicki, G.S. Human medial gastrocnemius force–velocity behavior shifts with locomotion speed and gait. Proc. Natl. Acad. Sci. USA 2012, 109, 977–982. [Google Scholar] [CrossRef]
  21. Fukuchi, R.K.; Fukuchi, C.A.; Duarte, M. A public dataset of running biomechanics and the effects of running speed on lower extremity kinematics and kinetics. PeerJ 2017, 5, e3298. [Google Scholar] [CrossRef]
  22. Yang, S.; Li, Q. Inertial sensor-based methods in walking speed estimation: A systematic review. Sensors 2012, 12, 6102–6116. [Google Scholar] [CrossRef]
  23. Soltani, A.; Dejnabadi, H.; Savary, M.; Aminian, K. Real-world gait speed estimation using wrist sensor: A personalized approach. IEEE J. Biomed. Health Inform. 2019, 24, 658–668. [Google Scholar] [CrossRef]
  24. Witte, T.; Wilson, A. Accuracy of non-differential GPS for the determination of speed over ground. J. Biomech. 2004, 37, 1891–1898. [Google Scholar] [CrossRef]
  25. Mannini, A.; Sabatini, A.M. Walking speed estimation using foot-mounted inertial sensors: Comparing machine learning and strap-down integration methods. Med. Eng. Phys. 2014, 36, 1312–1321. [Google Scholar] [CrossRef]
  26. Hannink, J.; Kautz, T.; Pasluosta, C.F.; Barth, J.; Schülein, S.; Gaßmann, K.G.; Klucken, J.; Eskofier, B.M. Mobile stride length estimation with deep convolutional neural networks. IEEE J. Biomed. Health Inform. 2017, 22, 354–362. [Google Scholar] [CrossRef]
  27. Demur, T.; Demura, S.i. Relationship among gait parameters while walking with varying loads. J. Physiol. Anthropol. 2010, 29, 29–34. [Google Scholar] [CrossRef]
  28. Attwells, R.L.; Birrell, S.A.; Hooper, R.H.; Mansfield, N.J. Influence of carrying heavy loads on soldiers’ posture, movements and gait. Ergonomics 2006, 49, 1527–1537. [Google Scholar] [CrossRef]
  29. Walsh, G.S.; Low, D.C. Military load carriage effects on the gait of military personnel: A systematic review. Appl. Ergon. 2021, 93, 103376. [Google Scholar] [CrossRef]
  30. Boffey, D.; Harat, I.; Gepner, Y.; Frosti, C.L.; Funk, S.; Hoffman, J.R. The physiology and biomechanics of load carriage performance. Mil. Med. 2019, 184, e83–e90. [Google Scholar] [CrossRef]
  31. Mexi, A.; Kafetzakis, I.; Korontzi, M.; Karagiannakis, D.; Kalatzis, P.; Mandalidis, D. Effects of Load Carriage on Postural Control and Spatiotemporal Gait Parameters during Level and Uphill Walking. Sensors 2023, 23, 609. [Google Scholar] [CrossRef]
  32. Qian, Y.; Wang, Y.; Chen, C.; Xiong, J.; Leng, Y.; Yu, H.; Fu, C. Predictive locomotion mode recognition and accurate gait phase estimation for hip exoskeleton on various terrains. IEEE Robot. Autom. Lett. 2022, 7, 6439–6446. [Google Scholar] [CrossRef]
  33. Ding, Y.; Galiana, I.; Asbeck, A.T.; De Rossi, S.M.M.; Bae, J.; Santos, T.R.T.; De Araujo, V.L.; Lee, S.; Holt, K.G.; Walsh, C. Biomechanical and physiological evaluation of multi-joint assistance with soft exosuits. IEEE Trans. Neural Syst. Rehabil. Eng. 2016, 25, 119–130. [Google Scholar] [CrossRef]
  34. Hermens, H.J.; Freriks, B.; Disselhorst-Klug, C.; Rau, G. Development of recommendations for SEMG sensors and sensor placement procedures. J. Electromyogr. Kinesiol. 2000, 10, 361–374. [Google Scholar] [CrossRef]
  35. Vapnik, V. The Nature of Statistical Learning Theory; Springer Science & Business Media: Berlin/Heidelberg, Germany, 1999. [Google Scholar]
  36. Cover, T.; Hart, P. Nearest neighbor pattern classification. IEEE Trans. Inf. Theory 1967, 13, 21–27. [Google Scholar] [CrossRef]
  37. Hastie, T.; Tibshirani, R.; Friedman, J.H.; Friedman, J.H. The Elements of Statistical Learning: Data Mining, Inference, and Prediction; Springer: Berlin/Heidelberg, Germany, 2009; Volume 2. [Google Scholar]
  38. Breiman, L. Classification and Regression Trees; Routledge: London, UK, 2017. [Google Scholar]
  39. MathWorks Inc. Statistics and Machine Learning Toolbox, Version R2020b; MathWorks Inc.: Natick, MA, USA, 2020. [Google Scholar]
  40. Hudgins, B.; Parker, P.; Scott, R.N. A new strategy for multifunction myoelectric control. IEEE Trans. Biomed. Eng. 1993, 40, 82–94. [Google Scholar] [CrossRef]
  41. Huo, W.; Mohammed, S.; Amirat, Y.; Kong, K. Fast gait mode detection and assistive torque control of an exoskeletal robotic orthosis for walking assistance. IEEE Trans. Robot. 2018, 34, 1035–1052. [Google Scholar] [CrossRef]
  42. Baud, R.; Manzoori, A.R.; Ijspeert, A.; Bouri, M. Review of control strategies for lower-limb exoskeletons to assist gait. J. NeuroEng. Rehabil. 2021, 18, 119. [Google Scholar] [CrossRef]
Figure 1. Experiment protocols. (A) Experiment setting (B) Data windowing scheme (C) Architecture of the neuromuscular–mechanical fusion system for locomotion mode and task recognition.
Figure 1. Experiment protocols. (A) Experiment setting (B) Data windowing scheme (C) Architecture of the neuromuscular–mechanical fusion system for locomotion mode and task recognition.
Bioengineering 11 00150 g001
Figure 2. Classification performance of classifiers and participants. (A) The classification performance of SVM, KNN, QDA, and DT. Error bars in the graph represent the 95% confidence interval. (B) The recognition accuracy of four classifiers for participant 1–7. The accuracy rate is equal to 100% minus the classification error rate.
Figure 2. Classification performance of classifiers and participants. (A) The classification performance of SVM, KNN, QDA, and DT. Error bars in the graph represent the 95% confidence interval. (B) The recognition accuracy of four classifiers for participant 1–7. The accuracy rate is equal to 100% minus the classification error rate.
Bioengineering 11 00150 g002
Figure 3. Influence of sEMG features and feature combinations on classification accuracy of SVM with 140 length of analysis window and 10 window increment.
Figure 3. Influence of sEMG features and feature combinations on classification accuracy of SVM with 140 length of analysis window and 10 window increment.
Bioengineering 11 00150 g003
Figure 4. The influence of increment and length of analysis window of SVM with R/W/A feature set. (A) The effect of window increment on classification accuracy. Error bars in the graph represent the 95% confidence interval. (B) The black line represents the effect of window increment on decision number in one gait cycle. The red line represents the time delay between two decision numbers. (C) The effect of window length on classification accuracy. Error bars in the graph represent the 95% confidence interval.
Figure 4. The influence of increment and length of analysis window of SVM with R/W/A feature set. (A) The effect of window increment on classification accuracy. Error bars in the graph represent the 95% confidence interval. (B) The black line represents the effect of window increment on decision number in one gait cycle. The red line represents the time delay between two decision numbers. (C) The effect of window length on classification accuracy. Error bars in the graph represent the 95% confidence interval.
Bioengineering 11 00150 g004
Figure 5. The robustness of the system. Error bars in the graph represent the 95% confidence interval. Glu = gluteus; Thi = thigh; AT = anterior thigh; PC = posterior calf. The components of each muscle group are shown in Table 3.
Figure 5. The robustness of the system. Error bars in the graph represent the 95% confidence interval. Glu = gluteus; Thi = thigh; AT = anterior thigh; PC = posterior calf. The components of each muscle group are shown in Table 3.
Bioengineering 11 00150 g005
Figure 6. Confusion matrix for SVM classifier with 170 window length and 20 increment. The confusion matrix from (AC) is R/W/A, R/W, and ACCs.
Figure 6. Confusion matrix for SVM classifier with 170 window length and 20 increment. The confusion matrix from (AC) is R/W/A, R/W, and ACCs.
Bioengineering 11 00150 g006
Figure 7. Continuous locomotion mode and task identifications with assistive torque decision results for P1.
Figure 7. Continuous locomotion mode and task identifications with assistive torque decision results for P1.
Bioengineering 11 00150 g007
Table 1. Anthropometric and demographic information of participants.
Table 1. Anthropometric and demographic information of participants.
AgeWeight (kg)Height (cm)Leg Length (cm)
P12367.3017380
P22461.7017488
P32364.5017586
P42372.6517483
P52367.6018095
P62666.7017384
P72671.0018089
Table 2. Abbreviation and explanation of feature sets used in locomotion mode and task recognition.
Table 2. Abbreviation and explanation of feature sets used in locomotion mode and task recognition.
Feature SetComponentsDim of 1 Channel
ACCACCs3
RMSRMS1
MAVMAV1
WLWL1
SSCSSC1
ZCZC1
R/MRMS, MAV2
R/WRMS, WL2
M/WMAV, WL2
R/M/WRMS, MAV, WL3
R/W/ARMS, WL, ACC5
R/M/W/SRMS, MAV, WL, SSC4
R/M/W/ARMS, MAV, WL, ACC6
EMGRMS, MAV, WL, SSC, ZC5
ALLRMS, MAV, WL, SSC, ZC, ACC8
ACC = accelerations in x, y, and z directions; RMS = root mean square; MAV = mean absolute value; WL = waveform length; SSC = number of slope sign changes; ZC = number of zero crossings.
Table 3. Muscle groups and components used in the investigation of system robustness.
Table 3. Muscle groups and components used in the investigation of system robustness.
Muscle GroupComponentsNumber in Total
GluteusGM1
ThighRF, VL, VM, BF4
CalfTA, GAS, SOL3
Anterior ThighRF, VL, VM3
Posterior CalfGAS, SOL2
GM = gluteus maximus; RF = rectus femoris; VL = vastus lateralis; VM = vastus medialis; BF = biceps femoris; TA = tibialis anterior; GAS = gastrocnemius medialis; SOL = soleus.
Table 4. The influence of ACCs on classification accuracy based on SVM with 140 length of analysis window and 10 window increment.
Table 4. The influence of ACCs on classification accuracy based on SVM with 140 length of analysis window and 10 window increment.
ParticipantR/WR/W/AR/M/WR/M/W/AEMGE/A
P199.2899.7399.0097.9296.3799.27
P293.3699.7593.8799.5095.8899.24
P398.7795.4696.6896.2597.9099.30
P498.8098.7198.6199.2698.8999.91
P591.7796.9489.9295.9790.0095.97
P689.8098.4490.5895.8794.4297.37
P780.6299.1981.0399.1093.0098.94
Mean93.2098.3292.8197.7095.2198.57
Table 5. The window increment, minimum decision number, and maximum time delay in participants.
Table 5. The window increment, minimum decision number, and maximum time delay in participants.
102030405060
P199.73 (119)99.64 (60)99.73 (40)100 (30)100 (24)100 (20)
P299.75 (139)99.50 (70)99.25 (47)99.00 (35)99.58 (28)100 (24)
P395.46 (109)99.13 (55)98.96 (37)95.50 (28)99.14 (22)100 (19)
P498.71 (125)99.45 (63)99.72 (42)99.26 (32)98.89 (25)99.91(21)
P596.94 (147)96.63 (74)98.07 (49)95.97 (37)90.00 (30)95.97 (25)
P696.82 (143)97.62 (72)97.62 (48)95.87 (36)94.42 (29)97.37 (24)
P799.19 (133)99.51 (67)99.27 (45)99.10 (34)93.00 (27)98.94 (23)
Mean (Min)98.08 (109)98.78 (55)98.95 (37)97.70 (28)95.21 (22)98.57 (19)
Delay (ms)5.8811.6517.3222.8829.1233.71
Table 6. The influence of window length on classification accuracy.
Table 6. The influence of window length on classification accuracy.
20017015013011090705030
S199.6410010010010010010099.3399.00
S299.599.5099.1898.5598.5599.5299.2198.2898.14
S399.1310010010010099.5198.8698.8797.60
S499.4599.6499.4698.2498.2498.9599.3198.9898.99
S596.6396.9894.3393.9793.9792.9690.4489.4786.89
S697.6296.5597.9897.7197.7197.8897.0096.1495.73
S799.5198.8898.8998.4398.2898.4597.8697.8896.69
Mean98.7898.7998.5498.0898.1198.1897.5296.9996.15
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Liu, Y.; Chen, C.; Wang, Z.; Tian, Y.; Wang, S.; Xiao, Y.; Yang, F.; Wu, X. Continuous Locomotion Mode and Task Identification for an Assistive Exoskeleton Based on Neuromuscular–Mechanical Fusion. Bioengineering 2024, 11, 150. https://doi.org/10.3390/bioengineering11020150

AMA Style

Liu Y, Chen C, Wang Z, Tian Y, Wang S, Xiao Y, Yang F, Wu X. Continuous Locomotion Mode and Task Identification for an Assistive Exoskeleton Based on Neuromuscular–Mechanical Fusion. Bioengineering. 2024; 11(2):150. https://doi.org/10.3390/bioengineering11020150

Chicago/Turabian Style

Liu, Yao, Chunjie Chen, Zhuo Wang, Yongtang Tian, Sheng Wang, Yang Xiao, Fangliang Yang, and Xinyu Wu. 2024. "Continuous Locomotion Mode and Task Identification for an Assistive Exoskeleton Based on Neuromuscular–Mechanical Fusion" Bioengineering 11, no. 2: 150. https://doi.org/10.3390/bioengineering11020150

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop