Assisting walking balance using a bio-inspired exoskeleton controller

Background Balance control is important for mobility, yet exoskeleton research has mainly focused on improving metabolic energy efficiency. Here we present a biomimetic exoskeleton controller that supports walking balance and reduces muscle activity. Methods Humans restore balance after a perturbation by adjusting activity of the muscles actuating the ankle in proportion to deviations from steady-state center of mass kinematics. We designed a controller that mimics the neural control of steady-state walking and the balance recovery responses to perturbations. This controller uses both feedback from ankle kinematics in accordance with an existing model and feedback from the center of mass velocity. Control parameters were estimated by fitting the experimental relation between kinematics and ankle moments observed in humans that were walking while being perturbed by push and pull perturbations. This identified model was implemented on a bilateral ankle exoskeleton. Results Across twelve subjects, exoskeleton support reduced calf muscle activity in steady-state walking by 19% with respect to a minimal impedance controller (p < 0.001). Proportional feedback of the center of mass velocity improved balance support after perturbation. Muscle activity is reduced in response to push and pull perturbations by 10% (p = 0.006) and 16% (p < 0.001) and center of mass deviations by 9% (p = 0.026) and 18% (p = 0.002) with respect to the same controller without center of mass feedback. Conclusion Our control approach implemented on bilateral ankle exoskeletons can thus effectively support steady-state walking and balance control and therefore has the potential to improve mobility in balance-impaired individuals. Supplementary Information The online version contains supplementary material available at 10.1186/s12984-023-01205-9.

with α the pennation angle of the muscle. The pennation angle was computed under the assumption of a constant width l w of the muscle fiber that is equal to l w = l m,opt · sin(α 0 ). With α 0 the pennation angle at optimal fiber length. The force in the muscle tendon unit (F T ) is a nonlinear function of the tendon strain.
withl T the length of the tendon divided by the tendon slack length l T s , and F iso is the maximal isometric force of the muscle. The parameters k T and c 1 where chosen to represent the forcelength relation observed in the Achilles tendon and in the tendon of the tibialis anterior with k T = 20 and c 1 = −0.2328 for the soleus and k T = 35 and c 1 = −0.25 for the tibialis anterior. Contractile element velocity was calculated from muscle force-length and force-velocity and activation of the muscles (equation 4)as described in (16). The parameters of the plantarflexors and tibialis anterior were based on the OpenSim gait23dof92musc model that was slightly adjusted to combine the force generating capacity of the soleus and gastrocnemius (table 1). A forward euler integration scheme was used to calculate the length of the contractile element in the next time step (i+1).l m = f (l m , a) (4) l m,i+1 = l m,i +l m,i · ∆t (5) with ∆t = 0.001 s.

Neural control policy
The muscle model was driven by baseline muscle excitation and local feedback of muscle states in accordance with an existing model (12) and feedback of center of mass velocity (equation 6 and 7). Control parameters were optimized to track inverse dynamic ankle joint moments, which resulted in a set of control parameters used to control a bilateral ankle foot exoskeleton (table  2). We implemented a gradual transition between stance and swing phase feedback gains, based on the vertical ground reaction forces (equation 8), to improve the fit between inverse dynamic and simulated joint moments.

Soleus
Tibialis anterior e sol,0 0.027 e ta,0 0.02 G sol 1.28 G ta 1.49 K sol 0.32 l m,of f 0.9 G sol,ta 0.45 K ta -1.65 Table S 2: control parameters for the soleus and gastrocnemius muscle with K Fz the scale factor between 0 and 1 causes a smooth transition between stance and swing reflexes, F z the vertical component of the ground reaction force, m the subject mass, g = 9.81 m/s 2 the gravitational acceleration.

Supplementary information parameter identification 2.1 Overview optimization problem
We estimated the control parameters of each model to track the inverse dynamic joint moment in eight steady-state gait cycles and 16 perturbation trials (two perturbation directions, four perturbation magnitudes and two repetitions of each perturbation) in one optimization problem per subject. The difference between the inverse dynamic joint moment and simulated moment is visualized in Fig. S1 for the neuromuscular controller with and without COM velocity feedback.

Predicted muscle activity
Although we did not optimize the fit between measured and simulated muscle activity, the model with additional COM feedback also predicts an increase in calf muscle activity in response to push perturbations and an increase in tibialis anterior activity in response to pull perturbations ( Fig. S 2). The initial muscle response to the perturbations (i.e. first 200 ms) was however larger in the experiment compared to the simulations, which indicates that additional feedback of center of mass accelerations (as proposed by (22)) might improve the model further.   in response to pelvis pull (blue) and pelvis push (red) perturbations of different magnitudes (shades of the color) in one typical subject. We compared the default neuromuscular controller (left column) with neuromuscular controller with additional feedback of COM velocity (middle column) and electromyography data (right column). The model with COM feedback predicts the increase in calf muscle activity in response to push perturbation and increase in tibialis anterior activity in response to pull perturbations, which is not simulated with the model without COM feedback. The model with COM feedback also predicts the decrease in Soleus activity in response to pull perturbations. Note that there is no electromyography data on soleus muscle activity and was therefore compared to electromyography data of the gastrocnemius.

Smooth transition between stance and swing control
We found that the fit with experimental data could be improved when implementing a gradual change between the stance and swing phase feedback gains. The original neuromuscular controller proposed by (12) has a discrete transition between the stance and swing phases with G sol = 0 and G ta = 0 during the swing phase. We observed that a gradual transition between stance and swing reflex gains reduced the RMSE in ankle torque 2.23 Nm (Fig. S3).

Direction dependent reflex gains
We investigated whether separate reflex gains (K sol and K tib ) for push and pull perturbations improves the fit between measured and simulated ankle joint moments. We found that separate reflex gains decreased the RMSE between measured and simulated joint moment with only 1.26 Nm for pelvis pull and with 0.33 Nm for pelvis push perturbations ( Fig S4). Hence, we decided to use the simplest model with one set of gains independent of perturbation direction.

Interdependence of control parameters
The covariance matrix of the control parameters (P) was derived to determine the interdependence of the control parameters of the neuromuscular model (equation 9).
With N the number of time steps, J the Jacobian of the cost to the parameters and e is a vector with the difference between measured and simulated joint moments and each time step. The Jacobian is a N x np matrix, with np is the number of estimated parameters (i.e. np = 6 for the default neuromechanical model and np = 8 for the neuromechanical model with COM feedback). The interdependence of the parameters was evaluated by comparing the auto-covariance (diagonal terms of P) to the cross-covariance (off-diagonal terms of P).If the auto-covariance was higher than all cross-covariances, the corresponding parameter was estimated/assumed independently and its estimated value was assumed to be reliable. Similar as in (45), we normalized the covariance matrix such that all diagonal terms (auto-covariance) equal one (P plot i,j = | to facilitate graphical interpretation of the results (Fig S 5).

Figure S 5:
Covariance between control parameters. We observed a strong (negative) correlation between the baseline soleus activity (e sol,0 ) and the soleus force feedback gain (G sol ). Therefore, we decided to keep the baseline soleus activity constant (e 0,sol = 0.027) during the final parameter estimation process.

Adaptation to treadmill walking with exoskeleton
The experiment started with 20 minutes steady-state walking with the exoskeleton controlled with the default neuromuscular model to adapt to treadmill walking, with the heavy exoskeleton and the exoskeleton assistance. In addition to the decrease in muscle activity, we observed a gradual decrease in stride frequency and increase in mechanical work done by the exoskeleton during the adaptation (Fig. S reffig:Adaptation StrideFreq and S 7). It is however unclear if these changes during adaptation are mainly related to the adaptation to treadmill waking with the heavy exoskeleton or related to adaptation to the assistance.

Muscle activity in steady-state walking
We used the unperturbed gait cycles during the time period in between perturbations to evaluate if both neuromuscular controllers reduce soleus activity during the perturbation session. Similar to the 19% reduction in at the end of the adaptation session, we observed a 20% reduction in soleus activity for both neuromuscular controllers during the unperturbed gait cycles between perturbations. This confirms that both neuromuscular controllers cause a similar reduction in soleus activity.

Relation between COM kinematics and ankle moment in perturbed walking with exoskeleton
As an exploratory analysis we evaluated if the increase in muscle activity after perturbation is related the deviations in center of mass movement. This analysis is based on the observation that changes in muscle activity and the deviations in center of mass position and velocity are strongly correlated during perturbed walking without an exoskeleton (21). We performed a very rudimentary analysis by correlating the COM displacement (displacement of the pelvis during the perturbed stance phase) and the average muscle activity during the first 500 ms after perturbations. Similar as in perturbed walking without an exoskeleton, we found that an increase in tibialis anterior activity is related to the backward movement of the center of mass and the increase in soleus activity is related to a forward movement of the center of mass (Fig S 9). This exploratory analysis provides a first indication that subjects used a similar balance control strategy when walking with the exoskeleton compared to walking without an exoskeleton. We believe that a new experiment with a range of perturbation magnitudes and longer adaptation times are needed to evaluate if human sensorimotor processing is altered by the exoskeleton support. Nevertheless, there is a reasonable variability in muscle activity and center of mass movement after the perturbation in our dataset, which enabled this exploratory analysis ( Fig  S 9. Note that this variability is similar as perturbed in walking without an exoskeleton. We believe that this variability is most likely related to variations state of the subjects when the perturbation was applied as we observed no clear adaptation effect during the perturbation session (Fig S 10). We highlighted one subjects with larger dots and a linear regression to highlight in a particular subject the relation between COM displacement and reactive muscle activity.

Exoskeleton energetics in perturbed walking
As an exploratory analysis we evaluated whether the exoskeleton compensated for the positive (push) or negative (pull) work done by the perturbation device on the subjects. The work done by the exoskeleton during the perturbed right stance phase was computed as the time integral of exoskeleton ankle joint power. The work done by the perturbation devices was computed as the time integral of the power of the external perturbation force (computed as the dot product of the force acting on the subject and the velocity of the point of application expressed in a coordinate system with origin fixed to the treadmill belt). During steady-state walking the exoskeleton delivers on average approximately 1 J mechanical work per stance phase with the neuromuscular controller and dissipates approximately 1 J with the minimal impedance controller (Figure S 11). In response to the pull perturbations, the exoskeleton performed on average 3 J positive work during the perturbed stance phase, which compensated approximately 50% of the 6 J negative mechanical work done by the perturbation device when using the neuromuscular controller with COM velocity feedback ( Figure S 11). In contrast, the exoskeleton dissipated energy in both the default neuromuscular controller and the minimal impedance controller after pull perturbations. Hence, for the pull perturbations only the neuromuscular controller with COM feedback compensated for the mechanical work done by the perturbation device.
In response to the push perturbations, the mechanical work performed by the exoskeleton did not differ from the work performed during an unperturbed step and hence did not compensate for the positive mechanical work (16J) performed by the perturbation device. The neuromuscular controllers with and without COM feedback delivered on average approximately 1J during the pelvis push perturbed stance phase, which is similar to an unperturbed step. Similarly, the minimal impedance controller dissipated approximately 1 J in the pelvis push perturbed stance phase, which is similar to an unperturbed step.

Additional balance related metrics in perturbed walking
In the main manuscript, we reported COM displacements as our previous work demonstrates that there is a relation between COM displacement and reactive ankle torques in walking humans and we where using this insight to assist balance. However, human balance has been described by many other outcomes as well that also reflect other strategies to control balance such as adjustment of foot placement. As an exploratory analysis we evaluated several balance related metrics. We computed step length as the distance between the right and left coordinates of the markers on the lateral malleolus at the first heel strike after perturbation (left heel strike). We computed the position of the leading foot (left foot) with respect to the pelvis position and with respect to the extrapolated center of mass position at the first heel strike after perturbation (left heel strike) and report relative positions along the x-axis (walking direction). As we are mainly interested in deviations from unperturbed values we computed the changes in these balance related metrics during the perturbed step in each controller condition compared to the values during unperturbed walking with the exoskeleton in minimal impedance mode. In all controller conditions, subjects increase step length after push perturbations and decrease step length after pull perturbations. The distance between the pelvis (proxy for COM position) and the foot position did not change after the perturbations compared to steady-state walking ( Figure S 12). The position of the extrapolated center of mass with respect to the leading foot was slightly more posterior for the pull perturbations. The observation that the distance between the pelvis and the leading foot did not change after perturbation indicates that subjects mainly restored balance with an ankle strategy and not by adjusting foot placement. m m Step length position pelvis with respect to leading foot position extrapolated COM with respect to leading foot Step length position pelvis with respect to leading foot position extrapolated COM with respect to leading foot Figure S 12: Changes in balance-related outcomes in perturbed compared to unperturbed walking.

Muscle activity of the contralateral leg
As an additional exploratory analysis we evaluated muscle activity of the contralateral limb during the first stance phase after perturbation. We observed mainly a reduction in soleus activity after both pull and push perturbations with both neuromuscular controllers compared to the minimal impedance controller (Fig S 13). After pull perturbations the soleus activity was on average slightly smaller in the neuromuscular controller with COM feedback compared to the default neuromuscular controller. After pull perturbations the tibialis anterior activity is on average smaller in the minimal impedance controller compared to both neuromuscular controllers.

COM position during first steps after perturbation
As an additional exploratory analysis we evaluated the COM position during the first 4 steps after the perturbation (Figure S 14). The subjects moved approximately 10 cm on the treadmill during the perturbed stance phase and stayed at this position during the second stance phase for both pull and push perturbations. During the third and fourth stance phase after perturbation they returned approximately to their original position on the treadmill.