Towards Wearable Comprehensive Capture and Analysis of Skeletal Muscle Activity during Human Locomotion

Background: Motion capture and analyzing systems are essential for understanding locomotion. However, the existing devices are too cumbersome and can be used indoors only. A newly-developed wearable motion capture and measurement system with multiple sensors and ultrasound imaging was introduced in this study. Methods: In ten healthy participants, the changes in muscle area and activity of gastrocnemius, plantarflexion and dorsiflexion of right leg during walking were evaluated by the developed system and the Vicon system. The existence of significant changes in a gait cycle, comparison of the ankle kinetic data captured by the developed system and the Vicon system, and test-retest reliability (evaluated by the intraclass correlation coefficient, ICC) in each channel’s data captured by the developed system were examined. Results: Moderate to good test-retest reliability of various channels of the developed system (0.512 ≤ ICC ≤ 0.988, p < 0.05), significantly high correlation between the developed system and Vicon system in ankle joint angles (0.638R ≤ 0.707, p < 0.05), and significant changes in muscle activity of gastrocnemius during a gait cycle (p < 0.05) were found. Conclusion: A newly developed wearable motion capture and measurement system with ultrasound imaging that can accurately capture the motion of one leg was evaluated in this study, which paves the way towards real-time comprehensive evaluation of muscles and joint motions during different activities in both indoor and outdoor environments.


Introduction
Human posture control and motion require sufficient coordination of central nervous and musculoskeletal systems. In the musculoskeletal system, the ankle plantar-flexors generate necessary force for foot propulsion during walking [1] and running [2], as well as maintaining the static and dynamic balance control [3] in human. The function of plantar-flexor has been suggested as one strong indicator of gait performance [4]. Weakness [1] and contracture [5] of plantar-flexor tend to adversely affected balance and gait, leading to higher risk of falls [6].
Various measurement modalities have been used to evaluate the functions of plantar-flexors, including surface electromyography (EMG) measuring muscle electrical activity [7], goniometer measuring joint ankles [7], surface mechanomyography (MMG) measuring muscle vibration [8], infrared motion capture and analyzing system with cameras and reflective markers measuring Gwent, UK), a wireless Wi-Fi transmitter module (802.11.n), and two rechargeable lithium batteries (SNP-4200, D.seven Co., Ltd., Shenzhen, China). It was capable of providing the real-time B-mode ultrasound image of muscles, and measuring the muscle activation, muscle vibration, foot plantar force, and joint angle simultaneously. The ultrasound probe and sensors were wirelessly connected to a laptop, which collected and analyzed data. The frame rate of ultrasound images was 10 Hz, and the sampling frequency of the other sensors was 5 kHz. The multiple sensors were synchronized by a system clock from the field-programmable gate array (FPGA). The highperformance FPGA was used to control the data acquisition, conversion, and transfer of all the channels and ultrasound image.
As shown in Figure 1a, an ultrasound probe, a MMG sensor, and one set of EMG electrodes were put along the direction of muscle fiber of lateral head of gastrocnemius [7]. They were used to capture the real-time B-mode ultrasound image, the vibration motion on the skin surface generated by the muscle activity and the electrical activities of the muscle of lateral head of gastrocnemius, respectively. The other set of EMG electrodes were put along the direction of the muscle fiber of the tibialis anterior. The reference electrodes of both EMG channels were put on the bony prominence of patella. One end of goniometer was put at the medial foot along the direction of the first array, and the other end of goniometer was put at the medial shank along the direction of shank, to capture the ankle joint angle. The three force sensors were put at the plantar surface of the first metatarsal head, the second metatarsal head, and the heel, by a certified orthotist, to capture the force at plantar surface of the foot. The force sensors were attached to the right-side insole of a pair of flat insoles (2 mm thickness) manufactured using medium firm (30-35 Shore A hardness) ethylene-vinyl acetate (EVA, Foot Specialist Footcare & Products Co. Ltd., Hong Kong, China) by a CERTIFIED ORTHOTIST ( Figure 1a). Figure 1b demonstrates how the sensors and the probe were attached to the subject. Note that during the experiments, the sensors were attached to the right leg instead of the left leg, and the goniometer was attached to the medial shank instead of the lateral shank.

Experimental Procedure and Data Acquisition.
This study was conducted in a university locomotion laboratory. After explaining to the subjects about the experimental design, all the sensors were attached to each subject's right foot and shank ( Figure 1). Then each subject was asked to walk along a 7-m long level walkway with a comfortable speed for three walking trials consecutively.
The plantar-flexor muscle activity during walking was evaluated by the ultrasound image, MMG, and EMG signals of the mobile SMG system. The plantarflexion and dorsiflexion motion during walking was assessed by the joint angle measured by the goniometer, and the plantar force measured by the force sensors. The muscle area of plantar-flexor was measured from the ultrasound image, and was used to calculate the changes during walking.
An eight-camera three-dimensional (3D) motion capture system with good accuracy and reliability [33,34] (Vicon Nexus 2.5.1, Vicon NexusTM, Vicon Motion Systems Ltd., Yarnton, UK) sampling at 250 Hz and two floor-mounted force plates (OR6, Advanced Mechanical Technology, Inc., Watertown, MA, USA) sampling at 1000 Hz, was used to measure the 3D kinematic and kinetic data, specifically the plantar force and ankle joint angle, of the right side in subjects during walking. A built-in lower limb marker set (plug-in gait model) was adopted, in which 16 infrared reflective markers were affixed to both sides at the heels, foot dorsum, lateral malleolus, lateral femoral condyles, middle of thighs/shanks, anterior superior iliac spines, and posterior superior iliac spines. The kinematic and kinetic data were measured and analyzed using the plug-in gait model in the Vicon system.

Signal Processing
The data were analyzed using the Matlab (version 2016b, The MathWorks Inc., Natick, MA, USA). A flow chart for the process is included in Figure 2. An example showing the user-interface (UI) of the mobiles SMG system is illustrated in Figure 3.
MMG and EMG data was filtered using the 4th-order Butterworth band-pass filter of 5-50 Hz and 30-500 Hz, respectively. The signals were then rectified and filtered with a moving-average filter of temporal window of 0.101 s [35]. Baseline offsets of the recorded angles and forces were removed with subtractions of the steady-state baseline values. In the ultrasound images, muscle boundaries of the gastrocnemius muscle were indicated manually by drawing two lines in each frame by a trained practitioner ( Figure 3). The muscle area was then calculated from the area between the two lines in each frame. The measured muscle area at heel strike was set as the baseline to calculate the changes in muscle area in a gait cycle. The step of interest was extracted manually from the time-series data with reference to the plantar forces. In calculation of stance and swing time, the end of stance phase is regarded as the time when the force detected fell below 1% of their peak values. The data was then resampled to 0-100% of gait cycle using piecewise cubic interpolation. EMG and MMG signals were normalized to their peak values during walking [36]. Percentage change of muscle area with respect to the beginning of a gait cycle was calculated. The forces measured at the first metatarsal head, the second metatarsal head, and the heel were summed to calculate the plantar force of full foot, and normalized by dividing by each subject's body mass.
Vicon data was extracted to the step of interest with reference to the foot progress angle in the sagittal plane. In calculation of stance and swing time, the end of stance phase is regarded as the time when the force detected by the force plate returned to zero. The step was then resampled to 0-100% of gait cycle using piecewise cubic interpolation. Therefore, the step data from two systems were synchronized by the defined gait cycle.
The plantar force data captured by the force sensors in the wearable mobile sensing system were compared to the plantar force captured by the floor-mounted force plate in the Vicon system. The ankle joint angles (plantarflexion/dorsiflexion and inversion/eversion) captured by the goniometers in the wearable mobile sensing system were compared to the corresponding joint angle captured by the Vicon system.

Statistical Analysis
Statistical analysis was performed using Statistical Package for Social Sciences (SPSS, version 25.0, IBM Corporation, Armonk, NY, USA). Each gait cycle was divided into 21 time points from 0% to 100% (5% interval) for statistical analysis. Paired t-test was performed to examine the existence of significant change between two consecutive time points in each channel of the data captured by the mobile SMG system, to identify the statistically significant features of muscle activity and joint motion in a gait cycle. Pearson correlation test was performed to compare the ankle kinetic data captured by the mobile SMG system and the Vicon system, to examine the validity of the mobile SMG system. Intraclass correlation coefficient (ICC) test was performed to evaluate the test-retest

Statistical Analysis
Statistical analysis was performed using Statistical Package for Social Sciences (SPSS, version 25.0, IBM Corporation, Armonk, NY, USA). Each gait cycle was divided into 21 time points from 0% to 100% (5% interval) for statistical analysis. Paired t-test was performed to examine the existence of significant change between two consecutive time points in each channel of the data captured by the mobile SMG system, to identify the statistically significant features of muscle activity and joint motion in a gait cycle. Pearson correlation test was performed to compare the ankle kinetic data captured by the mobile SMG system and the Vicon system, to examine the validity of the mobile SMG system. Intraclass correlation coefficient (ICC) test was performed to evaluate the test-retest

Statistical Analysis
Statistical analysis was performed using Statistical Package for Social Sciences (SPSS, version 25.0, IBM Corporation, Armonk, NY, USA). Each gait cycle was divided into 21 time points from 0% to 100% (5% interval) for statistical analysis. Paired t-test was performed to examine the existence of significant change between two consecutive time points in each channel of the data captured by the mobile SMG system, to identify the statistically significant features of muscle activity and joint motion in a gait cycle. Pearson correlation test was performed to compare the ankle kinetic data captured Sensors 2019, 19,195 6 of 13 by the mobile SMG system and the Vicon system, to examine the validity of the mobile SMG system. Intraclass correlation coefficient (ICC) test was performed to evaluate the test-retest reliability in each channel of the data captured by the mobile SMG system. The level of significance was set at 0.05.

Results
A total of 10 healthy subjects participated in this study (seven males and three females, aged 26.4 ± 2.4 years, height 169.8 ± 12.2 cm, and weight 65.1 ± 14.9 kg).
The measured ankle activities in a gait cycle measured by the newly developed mobile SMG system and the Vicon system of 10 healthy subjects were illustrated in Figure 4. The trend of ankle plantar-/dorsi-flexion and in-/e-version measured by the mobile SMG device was similar to that measured by the Vicon system. Significantly high correlation between the goniometer of the introduced mobile SMG system and the Vicon system was found in peak plantarflexion (R = 0.703, p = 0.023) and peak dorsiflexion (R = 0.707, p = 0.022) during swing phase, as well as the peak eversion (R = 0.638, p = 0.047) during initial stance phase.  . Ankle activities in a gait cycle measured by Vicon and the newly developed mobile SMG system of ten healthy subjects. P: P value; R: Pearson's correlation coefficient; : significantly high correlation between the Vicon and mobile SMG systems; : significant changes in trend in consecutive 5% intervals; : significantly high correlation in intraclass correlation coefficient (ICC) among three trials; The bolded orange line illustrates the averaged data measured by the mobile SMG system; The bolded blue line illustrates the averaged data measured by the Vicon system; The thin dashed line illustrates the standard deviation (SD) of each corresponding bold line.  . Ankle activities in a gait cycle measured by Vicon and the newly developed mobile SMG system of ten healthy subjects. P: P value; R: Pearson's correlation coefficient; : significantly high correlation between the Vicon and mobile SMG systems; : significant changes in trend in consecutive 5% intervals; : significantly high correlation in intraclass correlation coefficient (ICC) among three trials; The bolded orange line illustrates the averaged data measured by the mobile SMG system; The bolded blue line illustrates the averaged data measured by the Vicon system; The thin dashed line illustrates the standard deviation (SD) of each corresponding bold line.  . Ankle activities in a gait cycle measured by Vicon and the newly developed mobile SMG system of ten healthy subjects. P: P value; R: Pearson's correlation coefficient; : significantly high correlation between the Vicon and mobile SMG systems; : significant changes in trend in consecutive 5% intervals; : significantly high correlation in intraclass correlation coefficient (ICC) among three trials; The bolded orange line illustrates the averaged data measured by the mobile SMG system; The bolded blue line illustrates the averaged data measured by the Vicon system; The thin dashed line illustrates the standard deviation (SD) of each corresponding bold line.
: significantly high correlation in intraclass correlation coefficient (ICC) among three trials; The bolded orange line illustrates the averaged data measured by the mobile SMG system; The bolded blue line illustrates the averaged data measured by the Vicon system; The thin dashed line illustrates the standard deviation (SD) of each corresponding bold line. Figure 4, the intensity of MMG signal increased since heel-strike and reached a peak during loading response (1.90 ± 1.19, p < 0.05), followed by a decrease until reaching a plateau during terminal stance phase. The intensity of MMG signal increased again and reached a second peak during pre-swing phase (1.56 ± 0.80, p < 0.05), followed by another decrease until the end of a gait cycle.

As shown in
The intensity of EMG signal at gastrocnemius increased and reached a peak during loading response phase (0.41 ± 0.16, p < 0.05). It then decreased and increased again, reaching a second peak during terminal stance phase (1.13 ± 0.46, p < 0.05). The intensity of EMG signal at tibialis anterior increased first and reached a peak during loading response significantly (0.70 ± 0.30, p < 0.05), followed by a decrease during mid-stance phase. Such EMG signal then increased in the following gait cycle, and decreased again since initial swing phase, reaching a minimal EMG signal intensity during mid-swing phase ( Figure 4).
As shown in Figure 4, the normalized plantar force of full foot increased significantly since the initial contact and reached a peak during loading response (1.64 ± 1.27 N/kg, p < 0.05). It then decreased significantly and reached a trough during mid-stance phase, followed by a significant increase that reached a second peak during terminal stance phase (1.15 ± 0.38 N/kg, p < 0.05). Such plantar force then decreased to 0 during pre-swing phase of a gait cycle. The first peak normalized plantar force of full foot was contributed by the peak plantar force of heel, and the second one was contributed by the peak plantar force of forefoot in a gait cycle ( Figure 4).
As shown in Figure 5, the gastrocnemius muscle area kept being below the baseline muscle area during the whole gait cycle. It firstly reduced since heel-strike, and reached a trough (6.28% ± 3.85%, p = 0.001) in mid-stance, followed by an increase until the terminal-stance phase. This muscle area reduced again from the pre-swing phase and reached a minimal muscle area (13.6% ± 3.8%, p = 0.002) during initial swing phase, followed by an increase until the terminal-swing phase in subjects. The minimal muscle area of a gait cycle was found in initial swing phase, which was 142.4% less than the trough in mid-stance phase (p < 0.001). Table 1 and Figure 4 summarize the ICC test results examining the test-retest reliability of the integrated sensors of the mobile SMG device during a gait cycle, including the ultrasound probe, MMG sensor, goniometer, EMG electrodes, and force sensors. Generally, the test-retest reliability of the various sensors of the mobile SMG device ranged from moderate to good. Specifically, the averaged significant ICC values (<0.05) suggested that there was moderate test-retest reliability in the measured changes in muscle area (by ultrasound probe, ICC = 0.552), muscle vibration (by MMG sensor, ICC = 0.653), ankle in-/e-version (by goniometer, ICC = 0.746), electrical activity of gastrocnemius (by EMG electrode, ICC = 0.745); and good test-retest reliability in electrical activity of tibialis anterior (by EMG electrode, ICC = 0.773), ankle plantar-/dorsi-flexion (by goniometer, ICC = 0.782), and plantar force of full foot (by force sensors, ICC = 0.781) during a gait cycle.

Discussion
A novel wearable mobile sensing system with real-time ultrasound imaging for human motion capture and analysis was developed and applied for gait analysis in this study. In addition to supporting the moderate to good test-retest reliability of this system, the findings of this study also unveiled some features of muscle activity, especially the gastrocnemius, during walking upon a comprehensive data capture and analysis.
Moderate to good test-retest reliability of various measurement channels of the mobile SMG system and strong correlation between this system with the Vicon system regarding the ankle plantar-/dorsi-flexion angle measurement was found. The averaged ICC values of EMG, goniometer, and force sensors were found to be 0.75 or above, suggesting good test-retest reliability in these measurement channels. Meanwhile, moderate test-retest reliability was found in MMG and ultrasound imaging (0.5 < ICC < 0.75, p < 0.05). The ankle plantar-/dorsi-flexion angle measured by the mobile SMG device was comparable to that of the Vicon system, and [35,37] strong correlation in peak dorsiflexion and peak plantarflexion angle during swing phase was found between the two systems (R > 0.5, p < 0.05). The ankle inversion/eversion angle measured by the mobile SMG device was not quite comparable to that of the Vicon system. One possible reason could be due to the different axis of rotation in angle measurement, i.e., Vicon system estimated the angle from the built-in model, while the goniometer measured the angle from the medial side of the ankle joints.
The trends of the EMG [35,37], MMG [37], muscle area [12], plantar force [38,39], and ankle angle [38,39] were comparable to previous studies. A large peak during loading response and a second peak during initial swing in MMG signal were found in this study. This is in accordance with a previous study identifying a similar trend in MMG signal during walking at slow walking speed [37]. Similar to previous studies, a large peak during terminal stance in EMG signal of the gastrocnemius was also found in this study [35,37].
While previous studies reported that the time lag between the maximal EMG and MMG signal was larger than the electromechanical delay in gastrocnemius [35], this study further identified that the time lag between the EMG signal, joint angle, and reduction of muscle area were also larger than the electromechanical delay in gastrocnemius during walking. A time lag was found between the peak of EMG signal (approximately 45% of gait cycle) and other signals of ankle plantarflexion (approximately 65% of gait cycle), MMG signal (approximately 70% of gait cycle), and maximal reduction of muscle area (approximately 70% of gait cycle). However, due to the relatively low frame rate of the ultrasound probe in this study, future studies adopting faster imaging modalities of muscles are needed to verify this.
Co-activation of the gastrocnemius and tibialis anterior was observed during most time of a gait cycle by the newly developed mobile SMG system used in this study. This was supported by the EMG signals of the two muscles, and was also similar to the findings of previous studies [35,40].
The muscle area of lateral head of gastrocnemius muscle was found to decrease during the swing phase and reach a trough during mid-swing phase in a gait cycle. This finding is in accordance previous studies investigating the changes of muscle fascicle length of the gastrocnemius muscle, which also identified that the muscle fascicle length decreased and reached a trough during swing phase [11,12].
A mobile motion analysis system with real-time ultrasound imaging would enable the comprehensive investigation of real-time muscle's behavior and adaption during different activities in vivo in humans. The success of this project will pave the way to real-time in-viva analysis of muscle activity in both indoor and outdoor environments, which offers tremendous potential health benefits. This device can be widely applied to evaluate the human posture and motion, as well as the treatment outcomes of patients in the future. It is a systematic extension of our previous research, which utilizes the team's unique expertise in ultrasound imaging technology, as well as posture and gait analysis in various populations.
This study has several limitations. The sample size of this study is relatively small. The frame rate of the current prototype was only 10 Hz. Future efforts could be put on optimizing this protype by increasing the frequency and capacity. The ultrasound image quality also can be further improved so as to produce image with better quality and facilitate more complexed image processing. The current prototype still needed wires to connect multiple sensors. This could be improved with flexible, conformable devices that are tailored to make such measurements in a way that is mechanically invisible in the future. This study has used the peak value during a walking trial as the reference value for normalization of the EMG and MMG signal [36]. Future studies may consider using other reference values, such as the peak value during maximum voluntary contraction (MVC) through the manual muscle testing (MMT), and compare if any differences in results existed between these methods.

Conclusions
This study introduced a newly developed wearable motion capture and measurement system with ultrasound imaging (mobile SMG system), evaluated its reliability and validity, and applied it to monitor muscle activities and joint motion in a gait cycle. Moderate to good test-retest reliability of the various channels of this system was found. This study also identified some interesting changes in lateral head of gastrocnemius during a gait cycle by comprehensively evaluating the muscle activity with the multiple measurement modalities integrated in the mobile SMG system. This paves the way towards real-time comprehensive evaluation of muscles during different activities in both indoor and outdoor environments. It is also expected that such kind of wearable motion capture and measurement system can be used in clinical field. Some potential applications could be diagnosis of skeletal muscle problems, objective evaluation of musculoskeletal disorders, and assessment of balance and gait control before and after rehabilitation trainings or orthotic management.
Author Contributions: C.Z.-H.M. contributed to the conception and design of this study, the system calibration, the acquisition, analysis and interpretation of data, and drafted the manuscript; Y.T.L. contributed to the system calibration, creation of the new graphical user-interface for data acquisition and analysis, the acquisition, analysis and interpretation of data, and preparation and revision of the manuscript; Q.T.K.S. contributed to the acquisition and interpretation of data, and revision of the manuscript; X.-Y.W. contributed to the concept of wearable motion analysis system, participation in the system development, and revision of the manuscript; L.-K.W. contributed to the hardware development of the system and the data acquisition and analysis software; Y.-P.Z. contributed to the conception and design of this study, revision of the manuscript, and supervision of the whole project.