Skip to main content

ORIGINAL RESEARCH article

Front. Neurorobot., 29 November 2019
Volume 13 - 2019 | https://doi.org/10.3389/fnbot.2019.00099

Kinematic Synergy of Multi-DoF Movement in Upper Limb and Its Application for Rehabilitation Exoskeleton Motion Planning

Shangjie Tang1 Lin Chen1,2 Michele Barsotti3 Lintao Hu1,4 Yongqiang Li1,4 Xiaoying Wu1,2 Long Bai2,5 Antonio Frisoli3* Wensheng Hou1,2,4*
  • 1Key Laboratory of Biorheological Science and Technology, Ministry of Education, Bioengineering College, Chongqing University, Chongqing, China
  • 2Chongqing Key Laboratory of Artificial Intelligence and Service Robot Control Technology, Chongqing, China
  • 3PERCRO Laboratory, TeCIP Institute, Scuola Superiore Sant'Anna, Pisa, Italy
  • 4Chongqing Engineering Research Center of Medical Electronics Technology, Chongqing, China
  • 5College of Mechanical Engineering, Chongqing University, Chongqing, China

It is important for rehabilitation exoskeletons to move with a spatiotemporal motion patterns that well match the upper-limb joint kinematic characteristics. However, few efforts have been made to manipulate the motion control based on human kinematic synergies. This work analyzed the spatiotemporal kinematic synergies of right arm reaching movement and investigated their potential usage in upper limb assistive exoskeleton motion planning. Ten right-handed subjects were asked to reach 10 target button locations placed on a cardboard in front. The kinematic data of right arm were tracked by a motion capture system. Angular velocities over time for shoulder flexion/extension, shoulder abduction/adduction, shoulder internal/external rotation, and elbow flexion/extension were computed. Principal component analysis (PCA) was used to derive kinematic synergies from the reaching task for each subject. We found that the first four synergies can explain more than 94% of the variance. Moreover, the joint coordination patterns were dynamically regulated over time as the number of kinematic synergy (PC) increased. The synergies with different order played different roles in reaching movement. Our results indicated that the low-order synergies represented the overall trend of motion patterns, while the high-order synergies described the fine motions at specific moving phases. A 4-DoF upper limb assistive exoskeleton was modeled in SolidWorks to simulate assistive exoskeleton movement pattern based on kinematic synergy. An exoskeleton Denavit-Hartenberg (D-H) model was established to estimate the exoskeleton moving pattern in reaching tasks. The results further confirmed that kinematic synergies could be used for exoskeleton motion planning, and different principal components contributed to the motion trajectory and end-point accuracy to some extent. The findings of this study may provide novel but simplified strategies for the development of rehabilitation and assistive robotic systems approximating the motion pattern of natural upper-limb motor function.

Introduction

Hemiplegia, or unilateral paresis, is currently reported to be one major cause of physical disability in the middle-aged and elderly (Paul et al., 2002; Gert et al., 2003). Patients often experience autonomic difficulties in daily life due to the unilateral motor dysfunction. Recent works on upper-limb robotic training experiments exhibited substantial improvements on joint motions in either the horizontal plane or three-dimensional space for the patients of motor impairment with wearable exoskeletons (Kwakkel et al., 2008; Jarrassé et al., 2014). Therefore, exoskeleton technology has attracted extensive attentions for its noteworthy value of assisting multi-joint rehabilitation movement and daily life performance (such as reaching and grasping) (Frisoli et al., 2012). Pilot studies suggested that spatiotemporal motion patterns that well match the nature of upper-limb joint movement plays a key role in the long lasting effects in rehabilitation (Liu et al., 2018). A typical upper limb movement is fulfilled with spatiotemporal motor coordination of multiple joints, and kinematic synergy among limb joints have been widely explored as a control principle for motor function (Tomita et al., 2017). However, little is known how to effectively use kinematic synergies of upper limb movement in exoskeleton motion planning for motor assistive purpose.

Numerous robotic devices for upper limb rehabilitation have been developed with kinematic patterns mimicking actual human arm movements with multiple degree of freedoms (DoF). The kinematics of multi-joint coordination were extracted and implemented in exoskeleton to conduct activities of daily life (ADLs). Johnson et al. (2001) designed a 5-DoFs motorized assistive device with a 3-DoFs cable driven shoulder structure to enable patients with arm disorders to perform controlled movements, strengthening exercises, and continuous passive motion according to selective modes. Xu and Qiu (2013) proposed an alternative design of a flexible continuum exoskeleton apparatus to collaborate with human movements for the shoulder joint only. Most of these devices were programmed to certain trajectories for rehabilitation training without full considerations of actual continuous multi-joint movement modes, and may obstruct smooth performance of consecutive upper limb movements (Ding et al., 2014). To overcome the kinematic redundancy in reaching movements, Li et al. introduced an exponential method of human motor control strategy, which demonstrated accuracy improvements for real-time motion control of designed upper limb exoskeleton (Li et al., 2015). However, the regulation strategies in central nervous system (CNS) for multi-joint coordination in upper limb tasks were correlated to complex movement patterns (Scano et al., 2017). A simulated framework based on muscular and postural synergy indicated that a robotic arm with multi-DoF could generate arbitrary trajectories similar to human natural movements for the end-effector (Liu and Xiong, 2013), but also increased redundancy that deteriorated the motion control. Several studies attempted to simplify the motor control of assistive device for upper limb actuating planar or three-dimensional motion performance, such as the impedance control (Hogan et al., 1992) and mirror-image movement enabler technique (Lum et al., 2004).

The temporal and spatial inter-limb coordination plays a key role in upper limb rehabilitation. The surface electromyography (sEMG) signal was frequently employed to control multi-DoF mechanical arm (Hang et al., 2014), however, the sEMG signal did not directly reflect those complex temporal multi-joint coordination across limbs (Merad et al., 2018). Alternative motion kinematics model has also derived via fitting a high-order polynomials based on sEMG analysis to estimate the multi-joint angles of upper limb movements (Ding et al., 2014). However, sEMG-based muscle activities were generally represented in high dimensional feature space. It was difficult to meet the robustness requirement of complicated multi-DOF arm motions (Ding et al., 2016). Other strategies have also been developed to optimize the near-natural trajectories of assistive equipment. For example, a mathematical model based on a criterion function to characterize planar two-joint arm movements was formulated by Flash and Hogan, and the predicted trajectories matched experimental observations of real human performance well in a horizontal plane (Flash and Hogan, 1987). Nevertheless, how to temporally represent motion patterns of the multi-joint coordination similar to actual trajectory of human arm movements still left an open question.

Kinematic synergy based on classical neuromechanic theories (Bernstein, 1967) is another important concept in motor coordination. Existing evidence suggests that the CNS generates motor command to co-activate multiple muscles working with specific extents, which was generally referred to as “synergy” (Turvey, 2007; Mukta et al., 2015). During motion behaviors, the neural control strategies were modulated to regulate synergistic patterns dynamically to meet the task requirements (Burns et al., 2017). Several pilot studies have employed kinematic synergies to evaluate the inter-joint movement patterns of different human motor approaches (Tresch et al., 2006; Chen and Xiong, 2013). Artificial neural network (Devi et al., 2013), linear discriminant analysis (LDA) (Ramana Vinjamuri et al., 2015), and several other algorithms have been introduced to derive synergies; however, principal component analysis (PCA) is most frequently employed for kinematic synergies analysis (Ramana et al., 2010; Chen and Xiong, 2013; Patel et al., 2016; Burns et al., 2017). Burns et al. (2017) successfully used PCA method to derive time-varying kinematic synergies of bilateral upper arm reaching movement. Bockemühl et al. (2010) analyzed the kinematics complexity of human catching movement, and found that the first three principal components captured more than 97% of variance. Further study confirmed that human upper limb trajectories can be reconstructed by a linear combination of few principal time-dependent functions (Averta et al., 2017). Also, researchers employed synergy analysis as inputs to control the rehabilitation mechanical system, such as upper limb exoskeleton rehabilitation robot (Liu and Xiong, 2013) or dexterous hand for grasping movements (Catalano et al., 2014). However, few efforts have been made to manipulate the motion control based on human kinematic synergies. This work analyzed the spatiotemporal kinematic synergies of right arm reaching movement and investigated their potential usage in upper limb assistive exoskeleton motion planning (see Figure 1). Therefore, we employed PCA algorithm to analyze the spatiotemporal kinematic synergistic pattern of the shoulder and elbow joint in reaching activities and simulated the motion planning for upper limb assistive exoskeleton with kinematic synergies.

FIGURE 1
www.frontiersin.org

Figure 1. Block diagram of this work.

Subjects and Methods

This study was conducted at the Perceptual Robotics Laboratory (Pisa, Italy). Ten healthy subjects (eight males and two females, with the age of 26 ± 2) participated in the study after informed consent was obtained. All subjects were right handed and confirmed free of any arm neuromuscular disorder or previous joint injury. The experimental procedures were conducted in accordance with the Declaration of Helsinki and approved by the Ethical Review Board of Scuola Superiore Sant'Anna.

Data Acquisition

An IMU-based motion capture system (Perception Neuron, Noitom Technology Ltd., Beijing, China) was used to record motion data of the subjects' upper limb during tasks. Four IMU sensors were placed on the subjects, as shown in Figure 2C (P1–P4): left shoulder, right shoulder, right elbow, right wrist. Then, the spatial position of shoulder, elbow, wrist, and their trajectory of upper limb motion could be recorded during test. Referred as an experimental report (Burns et al., 2017), frame calibrations were performed before the data acquisition as: (1) positive x extended backward away from the subject; (2) positive y extended to the right side of subject, and (3) positive z extended upwards toward the ceiling. The axis x, y, and z were illustrated in Figure 2A. Therefore, the data collected by this motion capture system can scale the right arm motion pattern with respect to subject's torso. The sampling frequency is 120 samples per second.

FIGURE 2
www.frontiersin.org

Figure 2. (A) Subject interacting with the Perception Neuron device (incorporated with IMU), sitting at the desk while wearing the wearable system and performing tasks. (B) The cardboard used in the experiment: the red dotted circles represent the 10 positions where the targets can be placed (named from “a” to “j”). (C) Kinematic model with specific locations of the Perception Neuron sensors. The green circle indicates the sensor (IMU) in the corresponding position has been activated.

Experimental Procedure

The participants were asked to reach the target button placed on a cardboard in front of them. Three buttons (yellow, blue, and green) were designed as targets and a white button was assigned as the start point. A LED (light-emitting diode) was embedded in white button to cue a reaching task beginning, whereas a LED was embedded in yellow, blue or green button to show which target should be reached in a trial. Five linear tracks in different directions were defined by the angles from the central track (Figures 2B,C) as: −60, −45, 0, 45, and 60°. The central track (i.e., 0°) was directly in front of the subject. The target buttons were placed on the selected 3 tracks. We assigned the yellow button to one of the contralateral two tracks (−60 or −45°), blue button to the central track (0°), and green button to one of the ipsilateral two tracks (45 or 60°). On each track, 6 different locations with 5 cm interval were designed for the placement of target buttons, however, only 2 locations were used (the second and the fifth scales) in the experiments. Therefore, a total of 10 possible target locations, named in alphabetical order as “a” to “j,” were involved (Figure 2B). The distance between the start and the target position was designed as about 80% of the maximum distance that the participant can reach.

The reaching task trial was conducted as: (1) pressing the start button (white); (2) pressing a button located at ipsilateral (green), central (blue) or contralateral (yellow) as the reaching task required; (3) returning to the start position (white). An acoustic trigger was used as the start cue for participants to reach and press the target button. Each target was pressed six times, but the target order was randomly assigned. The instructions for reaching movements were given with a randomized interval between 1 and 3 s. Each of these ten Participants were required to avoid any redundant movement during the whole procedure of task performance. A more detailed experimental procedure was described in a previous report (Tang et al., 2018).

Kinematics Measurement of Right Upper Limb

During our reaching task, the upper limb and joints performed spatial motions, which could be projected to the calibrated frame of axes. Then, the motion pattern of limb and joint can be evaluated through calculating the angular course at each time point. As illustrated in Figure 2C, the kinematics chain of arm was modeled with four segments passing through the five aforementioned joints to evaluate the following angles: shoulder flexion/extension (SFE), shoulder abduction/adduction (SAA), shoulder internal/external rotation (SIR), and elbow flexion/extension (EFE). The kinematics is defined by the following vectors:

pi=[xi,yi,zi]    (1)
s¯12=p1-p2    (2)
s¯32=p3-p2    (3)
s¯43=p4-p3    (4)

where s¯n  is a normal vector to the sagittal plane, and pi(xi, yi, zi) is the spatial coordinates of each joint. The angle of SFE, SAA, SIR, and EFE are then calculated as θsf, θsa, θsi, and θef, respectively. s¯ijx=0 denotes the x coordinate in vector s¯ij set to 0.

θsf=cos-1s¯32·s¯ns¯32s¯n, s¯32z=0    (5)
θsa=cos-1s¯32·s¯ns¯32s¯n, s¯32x=0    (6)
θsi=cos-1s¯12·s¯32s¯12s¯32, s¯12y=0, s¯32y=0    (7)
θef=cos-1s¯42·s¯23s¯43s¯23    (8)

Synergy Derivation

The time-markers of action performance (button pressed/released and LED on/off states) were recorded through a Simulink model in MATLAB (the Mathworks, Natich, MA, USA). A complete recording set was segmented between the release and re-press of the white start button (Figure 3). The PT.F (forward-performance time) was used to segment the angular data of forward movement of reaching task trial by trial.

FIGURE 3
www.frontiersin.org

Figure 3. Data segmentation process. “PT” for performance time, “F” and “B” for forward and backward, respectively. When the start button or target button was pressed, the LED was turned off.

After data segmentation, the angular velocities were calculated by the derivative of the temporal joint angle profiles. For further processing, the angular velocity data of a reaching trial was resampled to 150 samples for each DOF. Then, the ith sample of a trial was used to construct a 6 × 4 sub-matrix as following:

W(i)=[ω11(i)ω21(i)ω31(i)ω41(i)ω12(i)ω22(i)ω32(i)ω42(i)ω16(i)ω26(i)ω36(i)ω46(i)]    (i=1, 2, 3, , 150)    (9)

Where the subscript (row variable) and superscript (column variable) represent the joint DOFs and task repetitions, respectively. Then, a 6 × 600 matrix W composed of all 150 sub-matrixes was constructed to represent the six repetitions of a target reaching task to each subject.

W=[W(1)W(2)W(150)]    (10)

Thus, W was 6-by-600 matrix, which was then given as input to the PCA algorithm to extract spatiotemporal synergies. Then, singular value decomposition (SVD) algorithm was performed on angular velocity matrix W to derive kinematic synergies. Three-component matrices U, Σ, and S were computed from W, as shown in:

W=UΣS    (11)

U and S are orthogonal matrices, and Σ is a diagonal matrix. In this case, U is a 6-by-6 matrix, which has orthonormal columns and S is a 600-by-600 matrix with orthogonal rows. The diagonal matrix Σ is of 6-by-600 and the diagonal elements of Σ correspond to the singular values (λi) of W.

Σ=[λ100000000λ200000000λ300000000λ400000000λ500000000λ600]    (12)

S matrix is defined as in equation below:

S=[s11(1)s41(1)s11(tmax)s41(tmax)   s1m(1)s4m(1)s1m(tmax)s4m(tmax)   s1M(1)s4M(1)s1M(tmax)s4M(tmax)]    (13)

The first m rows of S are called the first m principal components, or “synergies” and M was the maximal number of synergies. The approximation matrix W~ can be composed by the first m columns of U, m-by-m of Σ (the other values are replaced by zeros), and m rows of matrix S. The product Umdiag1, λ2, ⋯ , λm} is called the weight matrix.

W~=Umdiag{λ1,λ2,,λm}Sm    (14)

The fraction of sum-squared variance can be calculated from the diagonal elements of Σ and this index was denoted as K.

K=λ12+λ22++λk2λ12+λ22++λM2(k=1, 2, 3, , M)    (15)

K was used to determine how many principal components were sufficient to represent the whole data. The index threshold of 94% variance was used to determine the best number of synergies.

The angular velocity signal was reconstructed by selecting different numbers of synergies. The reconstruction error e between measured ωj(t) and reconstructed ωj~(t) across joints (J = 4) was computed for each reaching movement. One-way analysis of variance (ANOVA) was performed based on normalized reconstruction error according to different number of recruited synergies.

e=j=1J0t(ωj(t)ω˜j(t))2j=1J0tωj(t)2    (16)

Motion Planning for Upper Limb Assistive Exoskeleton

To evaluate the impact of kinematic synergy on arm motion control, an upper limb rehabilitation exoskeleton model was built in SolidWorks (Dassault Systemes, Massachusetts, USA). The simulated exoskeleton was used to guide the joints of right arm moving in 3-DoFs rotation for shoulder joint and 1-DoF flexion/extension for elbow joint. As illustrated in Figure 4, the exoskeleton was driven by four stepping motors within the controlled range of motion (ROM). The ROMs of EFE, SFE, SAA, and SIR joint for reaching movements were set as 0–135, 0–100, 0–90, and 0–105°, respectively. Motor I was assigned to drive the device rotating horizontally to achieve internal/external rotations of the shoulder joint. Motor II could enable the rotation and slide of bracket 2 on the chute 3 to perform the shoulder abduction/adduction. Motor III was used to control arm bracket 4 rotating around the bracket 2 to execute the flexion/extension. Motor IV rotated the forearm bracket 5 around the upper-arm bracket 4 to perform flexion/extension of the elbow joint. The description of motors in this model were detailed in Table 1.

FIGURE 4
www.frontiersin.org

Figure 4. The assistive exoskeleton model and its coordinate system. (A) Assistive exoskeleton model for right upper limb assistive exoskeleton model; (B) the coordinate system of assistive exoskeleton for Denavit-Hartenberg (D-H) model. The red circle dot in left panel was the endpoint position in D-H model during reaching movement (the end point of bracket 5).

TABLE 1
www.frontiersin.org

Table 1. Description of motors used in model.

In order to evaluate the effect of kinematic synergy on arm movement control, an exoskeleton Denavit-Hartenberg (D-H) model was established. This model consisted of five coordinate systems (see Figure 4B). The base coordinate system {0} was set at the shoulder joint center, whereas the coordinate system {1}, {2}, {3}, and {4} were specified with SIR, SAA, SFE, and EFE, respectively. In the Denavit-Hartenberg (D-H) model, the origins of coordinate system {0}, {1}, and {2} were located at the center of the shoulder joint; the origin of coordinate system {3} was located at the position of motor for flexion and extension, and the origin of coordinate system {4} was located at the motor position of elbow joint. The origin of coordinate system {5} was located at the position of the end of the exoskeleton (see the red circle dot in Figure 4A).

To quantify the effect of kinematic synergy on reaching movement control, we calculated the spatial position of the end-point of the exoskeleton model when angular variables was applied to D-H matrix. The Denavit-Hartenberg (D-H) matrix for our exoskeleton model was specified as

[cosθi-sinθi0a(i-1)sinθicosα(i-1)cosθicosα(i-1)-sinα(i-1)-sinα(i-1)disinθisinα(i-1)cosθisinα(i-1)cosα(i-1)cosα(i-1)di0001]    (17)
T05=T0112T23T34T45T    (18)

Where ai−1 was the distance between Zi−1 to Zi along Xi−1, αi−1 was the angle at which Zi−1 to Zi rotated around Xi−1, di was the distance measured from Xi−1 to Xi along Zi, θi was the angle at which Xi−1 to Xi rotated around Zi. The Denavit-Hartenberg (D-H) parameters were listed in Table 2.

TABLE 2
www.frontiersin.org

Table 2. Denavit-Hartenberg (D-H) parameter.

Results

Synergy Extraction Using PCA

All designed experiments have been successfully repeated 6 times at each target location in randomized order, and the kinematic data of right upper limb are collected during reaching task being conducted. Spatiotemporal kinematic synergies are derived using PCA subject by subject. Figure 5 presents an example of angular velocity profile based on the first four synergies of subject 9 performing the task for reaching target “e.” Here, the first synergy (column 1) involving shoulder flexion (SFE), abduction (SAA) and external rotation (SIR) clearly demonstrate a forward reaching movement. In the same time, an elbow extension (EFE) was conducted and guide his/her right arm moving to press a target. Synergy 2 (column 2) shows a similar shoulder kinematic trend, with higher amplitudes and longer duration time for the shoulder abduction (SAA) and flexion (SFE). Synergy 3 (column 3) consists of a shoulder extension (SFE), a combination of a shoulder adduction followed by an abduction (SAA), and a combination of internal rotation followed by a slight external rotation (SIR). The corresponding elbow angular velocity profile (EFE) illustrates the procedure of “flexion-extension-flexion.” Synergy 4 (column 4) tends to behave similarly as synergy 2 (column 2), except that the elbow movement exhibits repeated flexion/extension with a shorter execution time (EFE).

FIGURE 5
www.frontiersin.org

Figure 5. Angular velocity profile based on the first four synergies of subject 9 performing the task of reaching target “e” (target location was illustrated in Figure 2B). Each column corresponds to a specific synergy which have a duration time of 1 second (x-axis), and each row corresponds to a DoF (EFE, SFE, SAA, and SIR, respectively). The Positive/Negative angular velocity amplitudes (y-axis) indicates the movements of elbow flexion/extension (EFE), shoulder flexion/extension (SFE), shoulder abduction/adduction (SAA), and shoulder internal/external rotation (SIR), respectively.

Figure 6A presents the explained variance of PCs (or “synergies”). As shown in Figure 6B, four PCs are sufficient to capture more than 94% of the data's variance for all target positions (a-j). Moreover, the first three PCs account for at least 87% of the variance, and the first two PCs are also able to explained at least 64% of the variance. Specifically, the first two PCs at position “f” accounting for variance even achieve 78%.

FIGURE 6
www.frontiersin.org

Figure 6. (A) Explained Variance for all target positions (a–j) of subject 10. (B) Explained Variance of first four PCs at all reaching position across all subjects. (C) Correlation analysis of the first PC (synergy 1) across subjects according to different DoFs.

Correlation analysis of the first PC (synergy 1) has been carried out for each DoF of upper limb across subjects (Figure 6C). Ipsilateral reaching tasks (target position at g–j) shows higher correlations than the central and contralateral tasks (a–f), which suggests that personalized kinematic parameters vary less in tasks of reaching the closer or more skilled positions. The high level of coefficient value for SFE (red dots) also reveals that the shoulder flexion/extension exhibits better stability across subjects than the other three DoFs (EFE, SAA, and SIR).

Figure 7A give an example of the angular velocity reconstruction based on different numbers of PCs (synergies) for shoulder abduction reaching target “a” (Figure 2B). As expected, the reconstructed kinematic profiles for upper limb joints get closer to the actual measurement as the number of synergies augmented. The reconstructed angular velocities with the first 2 synergies (PC1 and PC2 only) can represent similar kinematic trends, such as acceleration/deceleration phases of the real movements, while higher orders of synergies (PC3 and PC4) can supplement and refine the reconstruction to match the true measurements with higher accuracy.

FIGURE 7
www.frontiersin.org

Figure 7. (A) Comparison between the measured (solid) and reconstructed (dashed) angular velocity for shoulder abduction of task “a” (Figure 2B) (upper), and the corresponding acceleration of movement (lower). The reconstructed angular velocities presented on the upper plots are calculated based on different numbers of PCs. (B) Reconstruction errors of angular velocities within stable and changing periods, respectively. (C) Relationships of normalized reconstruction errors against number of PCs (synergies) for angular velocity reconstruction. (D,E) Grouped comparison of the impact of synergy number and target position on reconstruction errors in “changing” or “stable” periods. (F,G) Grouped comparison of the impact of synergy number and shoulder DOFs on reconstruction errors in “changing” or “stable” periods (*indicates p < 0.05).

We segment the movements into two specific periods (changing and stable) depending on the value of acceleration: the “changing period” refers to periods when the absolute acceleration value is higher than 20% of the maximum, while the rest period is considered as “stable period” (Figure 7A). We compare the difference of normalized reconstruction errors between the stable and changing period for reconstructed angular velocities (Figure 7B), and significant differences are observed (p < 0.05). The normalized errors tends to decrease as the number of PCs (synergies) augmented. In detail, the first 2 synergies yield a normalized reconstruction error of 0.38 in changing period and 0.28 in stable period, while the errors reduce to 0.18 in changing period and 0.05 in stable period when the first 4 PCs (synergies) were involved. Moreover, the normalized errors also vary with movement phases, as the error level for the first 3 PCs in changing period (0.29 ± 0.06) is similar to that for the first 2 PCs in the stable period (0.28 ± 0.05) (see the black dashed line in Figure 7C). In addition, Figure 7C also indicates that the impact of recruited synergy numbers on reconstruction error is greater in the changing period than that in the stable period. When comparing the impact of recruited synergies on reconstruction error among different reaching target positions, it showed similar trend; in other words, either for “changing” or “stable” period, the reconstruction error of upper limb moving decreased with the recruited synergy numbers (see Figures 7D,E). However, as illustrated in Figure 7F, the kinematic synergies showed different roles to reconstruct the moving pattern, and more synergies significantly reduced the reconstruction error in “changing” periods for DOF of shoulder abduction/adduction (SAA). In “stable” period, DOF of shoulder internal/external rotation (SIR) exhibited significant reconstruction error when only the first synergy or the first two synergies were recruited (Figure 7G).

The reaching task conducted by right arm are normalized as an angular trajectory. To simplify the simulation, the reaching movement is divided into 3 equal periods (0–33.3, 33.3–66.6, 66.7–100%T), the angular value for shoulder joints (EFE, SFE, and SAA) and elbow joint (SIR) are obtained through integrating angular velocity with time. Then, the angular values of the DOFs of EFE, SFE, SAA, and SIR are used to drive motor I, motor II, motor III, and motor IV, respectively. An example visualization of simulated postures at four specific time-points for subject 9 are presented in Figure 8. The postures are in accordance with the angular velocity profiles with different synergies as shown in Figure 5.

FIGURE 8
www.frontiersin.org

Figure 8. Simulated posture visualization for first four PCs (synergies) at four specific time points of subject 9.

Motion Simulation Based on Kinematic Synergies

As shown in Figure 9A, the movement profile of exoskeleton are simulated when applied to assist subject 5 reaching target “a” in SolidWorks. The first four columns demonstrate the postures of assistive exoskeleton when different order of kinematic synergies are employed to control exoskeleton movement at different time point of reaching task. The last column is the movement pattern of assistive exoskeleton when the original kinematic is used to drive the exoskeleton. It can be observed that, with the angular input of shoulder internal/external rotations, shoulder abduction/adduction, shoulder variable for the flexion/extension, and elbow flexion/extension, the assistive exoskeleton moves to target “a” step by step. Figure 9B visually illustrates the endpoint positions of assistive exoskeleton when different number of PCs are involved in D-H model for the subject 5. The distance deviations between simulated reaching points and actual target position are calculated, and the results also show that distance errors of exoskeleton endpoint decrease as the kinematic synergies augmented (Figure 9C).

FIGURE 9
www.frontiersin.org

Figure 9. (A) The upper limb assistive exoskeleton model driving by the first four PCs (synergies) at three specific time points of subject 5. (B) The calculated endpoint positions for subject 5 corresponds to reconstructions based on 1–4 PCs recruitment, with the true position of target “a” (location illustrated by Figure 2B) shown as ⋆. (C) Distance errors between calculated and actual target positions for all participants, and its changing trends as the recruited PCs (synergies) augmented.

Discussion

Motor control optimization is one of the most important issues in assistive arm device design. In our study, the time-varying kinematics synergies of shoulder and elbow joints during upper limb reaching tasks were analyzed using PCA algorithm. Our results showed that the first four principal components can sufficiently represent the dynamical profile of upper limb joint angles, and principal components with different scales exhibited different contributions to the multi-joint motion behaviors. Specifically, the first two synergies (PC1 and PC2) could reflect the direction and motion range of the movement, while the higher order synergies (PC3 and PC4) could smooth the motion trajectory during acceleration period. The motion planning simulation of right arm assistive exoskeleton further confirmed that principal components with different scales played different roles in the motion trajectory and end-point accuracy.

Synergy is an assemble of individual functional units performing motor behaviors in relatively independent DoFs (Turvey, 2007), and can be represented as multi-joint coordination in angular velocity space (Burns et al., 2017). Our results indicated that the first four PCs can be used as surrogates to describe synergistic characteristics since the four PCs were able to explain major proportion of variance. Moreover, we found that the level of variance explained by PCs was significantly reliant on the target position, and the joint coordination patterns were dynamically regulated over time as the number of kinematic synergy (PC) increased (Figures 6A,B). Bockemühl et al. (2010) suggested that the proportion of each principal component was modulated to compensate for the disadvantage of different catching positions. Verrel et al. (2013) proved that the proportion of variance explained by the first three PCs increased when more skilled upper limb movements were performed. In addition, Côté et al. (2002) reported that additional synergy modes were involved when adapting to motion task complexity. Therefore, our results were in accordance with these previous studies, and the correlation coefficient (Figure 6C) further revealed that the kinematic synergy of upper limb was associated with positions of reaching target.

Upper limb movement is conducted through coordination of multiple joints both in time and space (Tomita et al., 2017). As illustrated in Figure 7B, the reconstruction error varied in different motion periods (i.e., changing and stable) defined by angular velocity profiles. This result agreed to a previous finding, in which Ahmad et al. observed that hand motion stability depends on the motion period (Nadzri et al., 2014). Moreover, Mukta et al. reported temporal compensation during the motion starting period under different reach-to-grasp conditions (Mukta et al., 2015). In our study, the error performance in Figure 7C indicated that the reconstruction results could significantly be improved as the number of recruited PCs (synergies) increased. Furthermore, as observed in Figure 7A, the low-order synergies appeared to show the overall change trend of motion, while the high-order synergies reflected the details at the special movement phase, which suggested potential implications for the level assessments of motor function and rehabilitation.

The multi-DoF synergistic pattern of upper limb movements can help to simplify the motion planning of assistive device for human rehabilitation. In fact, synergy has been considered as an effective method to improve motion smoothness of the rehabilitation device as human behavior (Burns et al., 2017). Therefore, implementing the time-varying principal component analysis into the upper limb prosthesis and rehabilitation device was potential to fulfill the requirements. Moreover, recent study (Tsai et al., 2018) investigated joint kinematics regulation of postural system, and confirmed the value of PCs in evaluating the contributions of individual joint. Our results also demonstrated that the low-order synergies or PCs, generally the first two PCs, represented major variance of original kinematics, and could fulfill common movements for rehabilitation devices. On the other hand, to improve the endpoint accuracy of reaching movement, additional high order components (such as the third and fourth PCs) should be involved in. Therefore, specific to daily life activities which generally require precise control, the high-order PCs (synergies) would have advantages on accurate control strategies for assistive device development. Altogether, the simulation results in our study implied the high potential of synergy being implemented for motion planning, and the corresponding precision of endpoints can be improved by synergy augments.

The present work has confirmed that the motion coordination of shoulder and elbow joint is manifested in the coordination of kinematic synergies. With the data from right arm reaching tasks, it can be concluded that different synergies have specific contributions to the upper limb movement, the low-order synergies represented the overall trend of motion patterns, while the high-order synergies described the fine motions at some moving phases. The results of exoskeleton movement simulation further confirmed that kinematic synergies could be used for exoskeleton motion planning, and different principal components contributed to the motion trajectory and end-point accuracy with some specific extent. The findings of this study may provide novel but simplified strategies for the development of rehabilitation and assistive robotic systems approximating the motion pattern of natural upper-limb motor function.

Data Availability Statement

The raw data supporting the conclusions of this manuscript will be made available by the authors, without undue reservation, to any qualified researcher.

Ethics Statement

The experimental procedures were conducted in accordance with the Declaration of Helsinki and approved by the Ethical Review Board of Scuola Superiore Sant'Anna.

Author Contributions

ST collected the data and analyzed the data. AF and WH designed the work. ST, LC, MB, and YL drafted the work. WH, XW, and LB interpreted the data. LH worked on the D-H model.

Funding

This work was supported by the National Natural Science Foundation of China (31771069, 31800824, 31971287), Graduate Research and Innovation Foundation of Chongqing of China (Grant No. CYB17038), and the Chongqing Science and Technology Program (cstc2018jcyjAX0390).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors would like to thank the staff of the Perceptual Robotics Laboratory, Sant'Anna School of Advanced Studies, for their continued support of ongoing research and assistance in performing the experiment. The authors also would like to thank all the subjects who participated in this study.

References

Averta, G., Santina, C. D., Battaglia, E., Felici, F., and Bicchi, A. (2017). Unvealing the principal modes of human upper limb movements through functional analysis. Front. Robot. AI 4:37. doi: 10.3389/frobt.2017.00037

CrossRef Full Text | Google Scholar

Bernstein, N. (1967). On coordination and regulation of movements. J. Neuropathol. Exp. Neurol. 27:348. doi: 10.1097/00005072-196804000-00011

CrossRef Full Text | Google Scholar

Bockemühl, T., Troje, N. F., and Dürr, V. (2010). Inter-joint coupling and joint angle synergies of human catching movements. Hum. Mov. Sci. 29, 73–93. doi: 10.1016/j.humov.2009.03.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Burns, M. K., Patel, V., Florescu, I., Pochiraju, K. V., and Vinjamuri, R. (2017). Low-dimensional synergistic representation of bilateral reaching movements. Front. Bioeng. Biotechnol. 5:2. doi: 10.3389/fbioe.2017.00002

PubMed Abstract | CrossRef Full Text | Google Scholar

Côté, J. N., Mathieu, P. A., Levin, M. F., and Feldman, A. G. (2002). Movement reorganization to compensate for fatigue during sawing. Exp. Brain Res. 146, 394–398. doi: 10.1007/s00221-002-1186-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Catalano, M. G., Grioli, G., Farnioli, E., Serio, A., Piazza, C., and Bicchi, A. (2014). Adaptive synergies for the design and control of the Pisa/IIT SoftHand. Int. J. Robot. Res. 33, 768–782. doi: 10.1177/0278364913518998

CrossRef Full Text | Google Scholar

Chen, W., and Xiong, C. (2013). A Principle of Mechanical Implementing the Kinematic Synergy for Designing Anthropomorphic Hand. Busan: Springer.

Google Scholar

Devi, B. N., Krishnan, M., Venugopalan, R., and Mahapatra, B. K. (2013). Artificial neural network model for synergy analysis of input markets in ornamental fish trade in Mumbai. Agric. Econ. Res. Rev. 26, 83–90. doi: 10.22004/ag.econ.152076

CrossRef Full Text | Google Scholar

Ding, Q., Zhao, X., and Han, J. (2014). EMG-based estimation for multi-joint continuous movement of human upper limb. Robot. 36, 469–476. doi: 10.13973/j.cnki.robot.2014.0469

CrossRef Full Text | Google Scholar

Ding, Q. C., Xiong, A. B., Zhao, X. G., and Han, J. D. (2016). A review on researches and applications of sEMG-based motion intent recognition methods. Acta Autom. Sinica. 42, 13–25. doi: 10.16383/j.aas.2016.c140563

CrossRef Full Text | Google Scholar

Flash, T., and Hogan, N. (1987). The coordination of arm movements: an experimentally confirmed mathematical model. J. Neurosci. 5:1688. doi: 10.1523/JNEUROSCI.05-07-01688.1985

PubMed Abstract | CrossRef Full Text | Google Scholar

Frisoli, A., Procopio, C., Chisari, C., Creatini, I., Bonfiglio, L., Bergamasco, M., et al. (2012). Positive effects of robotic exoskeleton training of upper limb reaching movements after stroke. J. Neuroeng. Rehabil. 9:36. doi: 10.1186/1743-0003-9-36

PubMed Abstract | CrossRef Full Text | Google Scholar

Gert, K., Kollen, B. J., Jeroen, V. D. G., and Prevo, A. J. H. (2003). Probability of regaining dexterity in the flaccid upper limb: impact of severity of paresis and time since onset in acute stroke. Stroke 34, 2181–2186. doi: 10.1161/01.STR.0000087172.16305.CD

CrossRef Full Text | Google Scholar

Hang, P., Ariga, Y., Tominaga, K., Oku, T., Nakayama, K., Uemura, M., et al. (2014). Extraction and implementation of muscle synergies in neuro-mechanical control of upper limb movement. Adv. Robot. 28, 745–757. doi: 10.1080/01691864.2013.876940

CrossRef Full Text | Google Scholar

Hogan, N., Krebs, H. I., Charnnarong, J., Srikrishna, P., and Sharon, A. (1992). “MIT-MANUS: a workstation for manual therapy and training. I,” in Proceedings of SPIE (Roman), 161–165.

Google Scholar

Jarrassé, N., Proietti, T., Crocher, V., Robertson, J., Sahbani, A., Morel, G., et al. (2014). Robotic exoskeletons: a perspective for the rehabilitation of arm coordination in stroke patients. Front. Hum. Neurosci. 8, 1845–1846. doi: 10.3389/fnhum.2014.00947

PubMed Abstract | CrossRef Full Text | Google Scholar

Johnson, G. R., Carus, D. A., Parrini, G., Scattareggia, M. S., and Valeggi, R. (2001). The design of a five-degree-of-freedom powered orthosis for the upper limb. Proc. Inst. Mech. Eng. H 215, 275–284. doi: 10.1243/0954411011535867

PubMed Abstract | CrossRef Full Text | Google Scholar

Kwakkel, G., Kollen, B. J., and Krebs, H. I. (2008). Effects of robot-assisted therapy on upper limb recovery after stroke: a systematic review. Neurorehabil. Neural Repair 22:111. doi: 10.1177/1545968307305457

PubMed Abstract | CrossRef Full Text | Google Scholar

Li, Z., Milutinovic, D., and Rosen, J. (2015). Spatial map of synthesized criteria for the redundancy resolution of human arm movements. IEEE Trans. Neural Syst. Rehabil. Eng. 23, 1020–1030. doi: 10.1109/TNSRE.2014.2382105

PubMed Abstract | CrossRef Full Text | Google Scholar

Liu, K., and Xiong, C. (2013). A novel 10-DoF exoskeleton rehabilitation robot based on the postural synergies of upper extremity movements. Adv. Robot. 8102, 363–372. doi: 10.1007/978-3-642-40852-6_37

CrossRef Full Text | Google Scholar

Liu, K., Xiong, C.-H., He, L., Chen, W.-B., and Huang, X.-L. (2018). Postural synergy based design of exoskeleton robot replicating human arm reaching movements. Robot. Auton. Syst. 99, 84–96. doi: 10.1016/j.robot.2017.10.003

CrossRef Full Text | Google Scholar

Lum, P. S., Burgar, C. G., and Shor, P. C. (2004). Evidence for improved muscle activation patterns after retraining of reaching movements with the MIME robotic system in subjects with post-stroke hemiparesis. IEEE Trans. Neural Syst. Rehabil. Eng. 12:186. doi: 10.1109/TNSRE.2004.827225

PubMed Abstract | CrossRef Full Text | Google Scholar

Merad, M., De, M. É., Touillet, A., Martinet, N., Roby-Brami, A., and Jarrassé, N. (2018). Can we achieve intuitive prosthetic elbow control based on healthy upper limb motor strategies? Front. Neurorobot. 12:1. doi: 10.3389/fnbot.2018.00001

PubMed Abstract | CrossRef Full Text | Google Scholar

Mukta, V., Konrad, K., Maryam, S., Kazutaka, T., and Hatsopoulos, N. G. (2015). Neural coordination during reach-to-grasp. J. Neurophysiol. 114, 1827–1836. doi: 10.1152/jn.00349.2015

CrossRef Full Text | Google Scholar

Nadzri, A. A. B. A., Ahmad, S. A., Marhaban, M. H., and Jaafar, H. (2014). Characterization of surface electromyography using time domain features for determining hand motion and stages of contraction. Australas. Phys. Eng. Sci. Med. 37, 133–137. doi: 10.1007/s13246-014-0243-3

CrossRef Full Text | Google Scholar

Patel, V., Burns, M., and Vinjamuri, R. (2016). Effect of visual and tactile feedback on kinematic synergies in the grasping hand. Med. Biol. Eng. Comput. 54, 1217–1227. doi: 10.1007/s11517-015-1424-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Paul, M., Elizabeth, G., Klag, M. J., and Josef, C. (2002). Trends in stroke prevalence between 1973 and 1991 in the US population 25 to 74 years of age. Stroke 33:1209. doi: 10.1161/01.STR.0000015031.57955.D1

CrossRef Full Text | Google Scholar

Ramana Vinjamuri, V. P., Powell, M., Mao, Z.-H., and Crone, N. (2015). Candidates for synergies: linear discriminants versus principal components. Comput. Intell. Neurosci. 2014:9. doi: 10.1155/2014/373957

PubMed Abstract | CrossRef Full Text | Google Scholar

Ramana, V., Mingui, S., Cheng-Chun, C., Heung-No, L., Sclabassi, R. J., and Zhi-Hong, M. (2010). Temporal postural synergies of the hand in rapid grasping tasks. IEEE Trans. Inform. Technol. Biomed. 14, 986–994. doi: 10.1109/TITB.2009.2038907

CrossRef Full Text | Google Scholar

Scano, A., Chiavenna, A., Malosio, M., Tosatti, L. M., and Molteni, F. (2017). Muscle synergies-based characterization and clustering of poststroke patients in reaching movements. Front. Bioeng. Biotechnol. 5:62. doi: 10.3389/fbioe.2017.00062

PubMed Abstract | CrossRef Full Text | Google Scholar

Tang, S., Barsotti, M., Stroppa, F., Frisoli, A., Wu, X., and Hou, W. (2018). “Upper limb joint angular velocity synergies of human reaching movements,” in IEEE International Conference on Cyborg & Bionic Systems. (Shenzhen: IEEE).

Google Scholar

Tomita, Y., Mrm, R., and Levin, M. F. (2017). Upper limb coordination in individuals with stroke: poorly defined and poorly quantified. Neurorehabil. Neural Repair 31:885. doi: 10.1177/1545968317739998

PubMed Abstract | CrossRef Full Text | Google Scholar

Tresch, M. C., Cheung, V. C. K., and Andrea, D. A. (2006). Matrix factorization algorithms for the identification of muscle synergies: evaluation on simulated and experimental data sets. J. Neurophysiol. 95:2199. doi: 10.1152/jn.00222.2005

PubMed Abstract | CrossRef Full Text | Google Scholar

Tsai, Y. Y., Chang, G. C., and Hwang, I. S. (2018). Adaptation of kinematic synergy and postural control to mechanical ankle constraint on an unsteady stance surface. Hum. Mov. Sci. 60, 10–17. doi: 10.1016/j.humov.2018.04.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Turvey, M. T. (2007). Action and perception at the level of synergies. Hum. Mov. Sci. 26, 657–697. doi: 10.1016/j.humov.2007.04.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Verrel, J., Lindenberger, U., Pologe, S., et al. (2013). Coordination of degrees of freedom and stabilization of task variables; in a complex motor skill: expertise-related differences in cello bowing. Exp. Brain Res. 224, 323–334. doi: 10.1007/s00221-012-3314-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Xu, K., and Qiu, D. (2013). “Experimental design verification of a compliant shoulder exoskeleton,” in IEEE International Conference on Robotics and Automation (Karlsruhe: IEEE), 3894–3901.

Google Scholar

Keywords: kinematic synergies, upper limb movements, principal component analysis, motion planning, inter-joint coordination

Citation: Tang S, Chen L, Barsotti M, Hu L, Li Y, Wu X, Bai L, Frisoli A and Hou W (2019) Kinematic Synergy of Multi-DoF Movement in Upper Limb and Its Application for Rehabilitation Exoskeleton Motion Planning. Front. Neurorobot. 13:99. doi: 10.3389/fnbot.2019.00099

Received: 08 March 2019; Accepted: 14 November 2019;
Published: 29 November 2019.

Edited by:

Naser Mehrabi, University of Washington, United States

Reviewed by:

Jing Jin, East China University of Science and Technology, China
Mahdokht Ezati, University of Waterloo, Canada
Borna Ghannadi, University of Waterloo, Canada

Copyright © 2019 Tang, Chen, Barsotti, Hu, Li, Wu, Bai, Frisoli and Hou. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Antonio Frisoli, a.frisoli@sssup.it; Wensheng Hou, w.s.hou@cqu.edu.cn

These authors have contributed equally to this work

Download