https://orcid.org/0000-0001-6605-1204
Figures
Abstract
The human postural control system can maintain our balance in an upright stance. A simplified control model that can mimic the mechanisms of this complex system and adapt to the changes due to aging and injuries is a significant problem that can be used in clinical applications. While the Intermittent Proportional Derivative (IPD) is commonly used as a postural sway model in the upright stance, it does not consider the predictability and adaptability behavior of the human postural control system and the physical limitations of the human musculoskeletal system. In this article, we studied the methods based on optimization algorithms that can mimic the performance of the postural sway controller in the upright stance. First, we compared three optimal methods (Model Predictive Control (MPC), COP-Based Controller (COP-BC) and Momentum-Based Controller (MBC)) in simulation by considering a feedback structure of the dynamic of the skeletal body as a double link inverted pendulum while taking into account sensory noise and neurological time delay. Second, we evaluated the validity of these methods by the postural sway data of ten subjects in quiet stance trials. The results revealed that the optimal methods could mimic the postural sway with higher accuracy and less energy consumption in the joints compared to the IPD method. Among optimal approaches, COP-BC and MPC show promising results to mimic the human postural sway. The choice of controller weights and parameters is a trade-off between the consumption of energy in the joints and the prediction accuracy. Therefore, the capability and (dis)advantage of each method reviewed in this article can navigate the usage of each controller in different applications of postural sway, from clinical assessments to robotic applications.
Citation: Jafari H, Gustafsson T (2023) Optimal controllers resembling postural sway during upright stance. PLoS ONE 18(5): e0285098. https://doi.org/10.1371/journal.pone.0285098
Editor: Rajesh Kumar, National Institute of Technology, India (Institute of National Importance), INDIA
Received: November 8, 2022; Accepted: April 14, 2023; Published: May 2, 2023
Copyright: © 2023 Jafari, Gustafsson. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data cannot be shared publicly because it contains sensitive data including health status, anthropometrics, age and location (recruited from a relatively small community). Data are available from the LTU Institutional Data Access officer: Johan Lundberg Karlsson (contact via dataskydd@ltu.se) for researchers who meet the criteria for access to confidential data.
Funding: This work was funded by the Swedish Research Council Office under the Grant Agreement No.K2015-99X-22756-01-4. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Maintaining balance involves the cooperation of three systems in the human body in a feedback loop: the musculoskeletal system, the sensory system, and the Central Nervous System (CNS). The CNS as the main controller of the balancing loop collects information from the sensory organs in the body such as vision, vestibular, and proprioception. Integrating and processing this information generates the proper motor commands to the musculoskeletal system [1, 2]. Aging or injuries can decline the performance of the human balance system. Therefore, studying human balance motor control is crucial from three perspectives. First, it can develop an understanding and diagnosis of balance impairment conditions in health care such as Parkinson’s disease. Second, it can lead to the design of new assistive devices for rehabilitation to aid individuals in regaining their mobility. Last, it can advance biped or humanoid robots in maintaining stability and movement tasks in complex environments [3].
Many studies have been done to understand the human balance structure and its complex subsystem to capture its complexity and represent simplified models. Generally, the skeletal system is modeled in the sagittal plane as a single link inverted pendulum rotating around the ankle joint [4–7] or double link inverted pendulum rotating around the ankle and hip joints [8, 9]. The multisensory system in an upright stance is the integration of three primary sensors, vision, proprioception, and vestibular. There are few studies about qualitative modeling of this system [10]. However, many studies consider the integration of data provided by these organs as inputs to the CNS affected by neurological time delay and perturbations [11, 12]. Several studies implemented the Extended Kalman Filter (EKF) for estimating the delayed time and compensating the perturbations (measurement noise from sensors) [13, 14].
Despite all efforts to understand the mechanism of the neuromuscular system to maintain balance in an upright stance, the complexity of CNS has made the model for postural control controversial. Many researchers have tried to find a simplified control model to simulate the performance of the postural controller in maintaining the human balance in an upright stance [2, 10, 15–18]. Generally, the postural controller is considered as a Proportional-Derivative (PD) or Proportional-Integral-Derivative (PID) controller that provides joints torque for the skeletal system in a feedback loop [1, 5, 16, 19–22]. Despite the simplicity of this representation, it is an unsophisticated presentation of the postural control, which does not consider the predictability and adaptability of the CNS to the internal and environmental changes. For example, in the presence of the biological transmission time delay, selecting PD’s parameters in a stable region is quite a challenge [23]. To solve this issue, modeling the neural controller in postural sway leads to different methodologies that are more robust to the parameter changes.
Intermittent Proportional Derivative (IPD) controller is utilized in many studies [24–28] in order to improve the robustness issue of the PD controller in the balance feedback loop. The paradigm is based on the idea that the feedback controller can be switched on close to the stable manifold and switched off near an unstable manifold in the phase plane. In other words, instead of equilibrium configuration in the classical feedback PD controller, the stable manifold is the goal that leads to bounded stability [26]. The method is based on an active intermittent control torque applied to the ankle joint and a passive stiffness control to the hip joint. In [29], the author has extended the IPD model by applying independent parameter gains for both the ankle and the hip joint that can explain the mixed ankle-hip strategy. While IPD is more robust to the time delay than the PD controller, its adaptability and predictability to parameter changes are not widely understood.
Another potential approach to mimic the neuromuscular controller is using optimization techniques. This approach’s main advantage is considering the body’s limitations and constraints (e.g., joint torque and angle limit) in formulating the postural sway model. The ability to predict the future is another benefit of this approach that leads to more accurate results. There are increasing number of studies that used optimization techniques [30–38]. In general, the postural controller is formulated as an optimization problem to minimize the postural sway. Other cost functions have been studied, minimizing energy consumption in joints or tracking the body’s momentum. However, to what degree they can mimic the postural sway in an upright stance in case of time delay and parameter changes and their validity to replicate experimental data is a question that needs to be answered.
Validation and justification of the hypothetical simplified postural controllers with the experimental human data is yet an open research direction. In a few articles, the postural controller is defined by system identification techniques based on the experimental data [39–43]. Data-driven controllers are practical approaches to identify the relationship between the inputs to the neural controller and the generated outputs from that in the balancing loop and can be a tool to develop the parameters of the model-based approaches [44, 45]. However, the main drawback is that they can not explicitly describe the dynamics and pathophysiology behind the neuromuscular controller. On the other hand, validating and adjusting the simplified postural control model with human data is a technique that can contribute to a reliable understanding of the postural control model.
This paper aims to evaluate the optimal controllers that can resemble the postural control in an upright stance. Our goal is to find a structure that can mimic the human balance strategy and can be implemented in balance applications. Therefore, we have studied the human-likeliness of three commonly used formulations of the optimal controller (Model Predictive Control (MPC), COP-Based Controller (COP-BC) and Momentum-Based Controller (MBC)) in the literature. Specifically, we evaluated these methods in a human balance feedback structure loop while considering the sensory system’s noise and transmittal time delays. First, in simulation studies, we compared the robustness to different perturbations, the consumption of energy, and the postural strategy of each technique to reveal the human-likeliness of each method. Second, we validated and compared the efficiency of these methods in predicting postural sway through multiple human experimental data in upright stance trials with changes in the vision and proprioception sensory input. Finally, we examined the choice of parameters in the performance of the methods mentioned above and compared the results with IPD controllers. We believe our findings aid the postural sway analysis used in clinical and assistive robotic applications.
Methods
Human data collection
The experimental data is part of a larger project collected in the Human Health and Performance Lab at Luleå the University of Technology, Luleå, Sweden [43, 46, 47]. Written informed consent was obtained from all individual participants included in the study. The study design was approved by the Regional Ethical Review Board in Umeå, Sweden (ref no. 2015-182-31) and was organized according to the 1964 Helsinki declaration.
Ten subjects were chosen from a larger study [46] where community-dwelling elderly who have a critical balance posture were selected with a mean age of 75.2(±4.5) years. The Center Of Pressure (COP) data was collected by a force plate with a sampling frequency of 3000Hz, while the subjects were asked to stand still and look straight at the fixed point. In order to measure the total time delay subjects did a reaction time test where they had to press the bottom when they received the visual and audio signals as fast as possible. The experiment was repeated for five trials. The angular position of the joints was collected by a Qualisys Motion Capture System with eight cameras and a 200 Hz sampling rate. The experiment to measure reaction time is explained in detail in [47].
Human balance control loop
Fig 1 presents the block diagram of the closed-loop control structure to stabilize the human body in a quiet stance by the hip and ankle joints.
The body’s dynamical system is activated by control input u(t) and affected by disturbances w(t). The sensory system receives the motion dynamic x(t) and transfers it to neuromuscular contorted by noise v(t) and time delay td. The delayed sensory information and a buffer of the control input up to the current time u(t) are used in the neuromuscular by EKF method to estimate delayed states. The optimal controller block refers to all optimal methods explained in this work. The reference angular position xref(t) = 0 is considered zero degrees in an upright stance.
Body dynamics described in (2) is activated by control input u(k) = [τa, τh]⊤ and exposed to unknown disturbances ψk. The states of the system are which can be derived by solving the nonlinear discrete form at time instance
as:
(1a)
(1b)
where
is the applied disturbance to the system.
is a nonlinear function describing the dynamics of the skeletal system.
is the vector of measured states affected by noise in neural sensing
and
is the vector of measured outputs by sensors. The number of states, inputs and measurements are shown by ns, nu, nm respectively.
Skeletal modeling and body dynamics
The human body in quiet standing can be modeled as a two-link inverted pendulum rotating around the ankle and hip joints, as illustrated in Fig 2 [48]. In this representation, it is assumed that for small perturbations, the knee joint in the lower body and body segments from the hip joint to the head are neglected. However, for more significant perturbation, the human body bends the knee and takes a step to increase the base of support, which this scenario is beyond the scope of this article.
qa and qh represent the angular position of the ankle and hip joints respectively. COM is the location of the center of the system’s total mass, while m1 and m2 are the mass of the lower body and upper body respectively. Lf is the length of the foot.
The equation of motion can be written in the following form:
(2)
where q = [qa, qh]⊤ is the angular rotation around ankle (qa) and hip (qh),
represents the mass matrix with:
(3a)
(3b)
(3c)
(3d)
where L1 and m1 represent the length and mass of the lower body, while L2 and m2 show the length and mass of the torso, respectively. I1 and I2 represent the inertia around the ankle and the hip joint, respectively. It is assumed the center of mass of each segment is in the middle of the segment.
is the vector of Coriolis and Centrifugal terms with:
(4)
The vector with gravitational force F(q) is depicted as:
(5)
where
.
H represents the applied torques vector as:
(6)
where τa, τh are the generated torque at the ankle and hip joint respectively.
Neural time delays and noise in the sensory system
Balancing requires different sensory systems such as proprioception, vestibular, and vision [5]. Due to the different lengths of the neural path, there are time latencies in transmitting information to the CNS. Besides, less delays exist in processing at the CNS and transducing the information to the musculoskeletal system by applying the executed forces and torques on muscles and joints. In general, the total time is called the neural time delay, and for the body, sway is in the range of ∼ 100 − 300 (ms). However, the primary time delays happen in observation and perception of sensory data ∼ 200 − 300 (ms) [49]. Aging and impaired functions in any part of the balance control loop can increase the neural time delay, resulting in difficulty of balance and falls [50].
Therefore, the neuromuscular controller receives sensory information as:
(7)
where kd is the total measurement delay from sensors, called observation delay.
The neuromuscular controller has to reduce the noise from the received delayed measurement and predict and estimate the current states of the system for further prediction of control actions. Besides, it is assumed that the Neuromuscular controller has a memory of the previous control actions .
Here, we have used EKF to resemble the prediction and estimation of noisy delayed sensory measurements. The algorithm works in two steps. First, the EKF generates the estimate of the delayed states by finding the Kalman gain
and updating the states error covariance
. Next, the states and the state error covariance matrix will be predicted for the next sampling time. The procedure is repeated until it reaches the estimate of the current time step
which is explained in more detail in [51].
Neuromuscular controllers
Model predictive controller.
The MPC solves an optimization problem at each sampling time for a finite prediction horizon N to find the sequence of optimal control inputs. The cost function to be minimized is introduced as:
(8)
where
is the prediction horizon,
is the estimated states vector, xr presents the reference states, uk and ur are the controller action and desired control action, respectfully. The first term, weighted by
, minimizes the error between the current states and the desired one. The second term, weighted by
, tracks the error between the produced control action with the desired one.
The cost function (8) is minimized subject to the following constraints:
- The dynamic of the body in discrete form as in Eq (1).
- The joint torques (τa, τh) are bounded:
(9a)
(9b)
The MPC solves the mentioned above optimal control problem for a finite horizon N at each sampling time. The optimum results for the predicted horizon are u* = {u(0)opt, u(1)opt, …, u(N − 1)opt}. Then the first optimal value u(0)opt is applied to the body to obtain the angular position and velocity.
Center of pressure based controller.
The structure of this controller is similar to the MPC controller with the difference in using COP in the objective function. Theoretically the COP is calculated from the torque applied at the toe τt and the vertical component of ground reaction force fv applied on the foot [49]:
(10)
The control torque at the toe and the ground reaction are defined from the Euler-Lagrange equations from the potential and kinetic energy as described in [49]: The cost function can be written as follows:
(11)
where
is the desired distance for ground pressure to keep the balance which is zero in this case.
and
are the weights of error of COP and control action respectively. In addition to the constraints in MPC, the horizontal position of COP should always remain in the base of support, which in quiet standing with two joints is equivalent to the length of the foot Lf:
(12)
Momentum based controller.
MBC is widely used in controlling the humanoid robots [52–54]. In [55], the usage of this controller on the mediolateral balance is studied. Here, we implemented this controller based on [34]. The main goal is to find the joint torques by the whole body momentum. The whole body momentum h can be derived as follows:
(13)
where
is the vector of angular velocities and AG is the centoridal momentum matrix calculated based on the description in [56]. The momentum rate change can be calculated by taking the derivative of Eq 13 and through the contact forces by the Newton-Euler equations.
(14)
(15)
where λ is the contact wrench and fcy is the contact force applied to the foot in the vertical axis.
The MBC optimization can be written in the following form by minimizing the momentum rate and angular acceleration error:
(16)
(17)
(18)
(19)
where
is the desired momentum rate,
is the reference angular acceleration, Φ and Ω are the weight in the minimization term of the momentum rate change and the angular acceleration, respectively. The weights are chosen according to to [34] to guarantee stability. By finding the optimal value for
and λ, the control torque τ can be computed by solving the inverse dynamics.
Intermittent PD controller.
In this method the generated torque for each joint τ = [τa, τh] from the neuromuscular controller consists of three parts:
(20)
where τb is the bias torque apply to compensate for the gravity force:
τs is the torque related to the stiffness and damping in the muscles as follows:
with Ka, Kh, Ba, Bh represent the stiffness and damping parameters of ankle and hip joints respectively. The last term τi in the Eq 20 shows the intermittent control torque. Here, we implemented this torque as in [25] contrary to other literature that considers the intermittent control on a single inverted pendulum around the ankle, which has considered two degrees of freedom around the ankle and hip. In their representation, the Virtual Inverted Pendulum (VIP) is considered to calculate the rotation around the ankle by a virtual link to the whole body Center Of Mass (COM) as:
(21)
where
and
. Then the error between the estimated value of the angular location of COM by EKF and the reference value (
) is used to generate the control action only on the ankle joint as follows:
(22)
where P, D are the proportional and derivative parameters, respectively, and α is the switching parameter. To have a wider stable region based on the value of P and D, we chose α = 0.4 as discussed in [23], and the controller becomes active in the first and third quadrants of the phase plane.
Results
Simulation results
The human body parameters used for the simulation are summarized in Table 1. The simulation results were run on MATLAB and ‘ode23t’ function was used to solve the numerical integration. For an accurate comparison, noise perturbation is simulated by a white noise signal with a standard deviation of 0.005 for all the trails. The neural delay of 200(ms) is chosen based on the literature [49]. The EKF is run with a time step the same as the sampling time and fixed covariance matrices of measurement noise and disturbances are chosen empirically as Q = [diag(0.01, 0.01, 0.01, 0.01)] and R = [diag(0.1, 0.1, 0.01, 0.01)].
The stiffness and damping parameter in the IPD controller are chosen as Ka = 500 (Nm/rad), Kh = 380 (Nm/rad), Ba = 30 (Nms/rad), Bh = 30 (Nms/rad) according the measurements and equations provided in [24, 57]. The optimization is done by MATLAB’s ‘fmincon’ function. The objective function gains in the simulation are chosen empirically so that the outcomes remain stable and sway in the base of the support region. The prediction horizon of N = 10 with a nominal sampling time of 0.01 s was assumed for the prediction. Table 2 summarized the bounded parameters used in the simulation of optimal controllers methods. The bounds are approximated based on [31, 58].
We simulated each controller starting at two different initial states to compare the performance to perturbations such as external force applied to the body. We assumed an initial state of q = [0.0873 0]T (rad) for a small perturbation initial state of q = [0.2618 0]T (rad) as a higher perturbation. Figs 3 and 4 shows the simulation results with the best-tuned parameter for each method for both ankle and hip joints, respectively. The body stabilizes around zero in all methods for both small and higher perturbations. However, it can be seen that IPD andCOP-BC controllers converge slightly faster to the stable position, while on the other hand, MBC and MPC show more oscillation around zero in ankle and hip joint respectively. Expectedly, increasing the perturbation results in higher amplitude in both angular position and angular velocity in all methods. This analysis is important, especially in the case of push recovery in robotic applications such as humanoids. Considering fast convergence in push recovery and the ability to handle constraints, COP-BC controller can outperform other controllers. It should be highlighted that the results are displayed based on the best-tuned parameter. The choice of parameters can generally affect the controller outcomes’ convergence and settling time. More investigation on the controller parameters is discussed in the discussion section.
The solid black line indicates the small perturbation and the dashed red line illustrates the higher perturbation.
The solid black line indicates the small perturbation and the dashed red line illustrates the higher perturbation.
Other vital factors for comparing the controller are the strategy of balance and the energy consumption in joints. The joint energy is calculated from joint torque τ and angular velocity as
. Fig 5 compares the energy consumed in each joint by each method for different perturbed sway. It can be seen that IPD controller uses more energy at each joint than optimal methods in both small and high perturbation. It is also clear that with higher perturbed sway, the efforts to maintain balance in all the joints will increase and with higher sway, the hip joint consumes more energy than the ankle joint. In the optimal controller methods MBC controller uses significantly lower energy at the ankle compared to the hip joint even with smaller sway, indicating higher efforts of the upper body to stabilize the body in this method. COP-BC controller consumes almost equal energy in both joints to maintain balance in both small and high perturbations.
In [59] shows that the changes in control strategy from ankle to hip strategy can be seen by changes in joints torque trajectories. [58] mentions that in hip strategy, the ankle plantar flexion torque is smaller than the hip. Considering these studies, each method’s value of joint torque trajectories is plotted in Fig 6. While, as expected, higher perturbation cause shifting from ankle to hip strategy to maintain balance, it is apparent that for small sway IPD and MPC use both ankle and hip joint (ankle-hip strategy). Surprisingly, MBC mainly utilizes a hip strategy to stabilize the body. Although this result confirms the assumption in [60] that in MBC controllers in robotic applications only the hip is actuated to keep the balance, it does not simulate the human postural sway behavior to maintain balance since the human data shows ankle and ankle-hip strategy even in perturbed sway [61]. COP-BC controller mainly uses ankle strategy which shows the robustness of this method to the perturbed postural sway.
The solid black line indicates the small perturbation and the red dashed line indicates the higher perturbation.
Experimental results
To compare the controllers with the human data, first, we used the measured angular position of the joints as reference data for all the controllers to obtain the cost function gains. An optimization problem was solved to find the gains of the controller as
(23)
where T is the total time samples (30 s) and q is the vector of the angular position of each joint. Table 3 summarizes the obtained gains for each controller.
Later on, to have a fair resemblance, we simulated all controllers with the initial point of human data measurement, the related body characteristics, and the obtained gains. Fig 7 shows the result for the COP prediction of a random subject in the data set for one step ahead prediction where the human measurement is known at each sampling time as well as Fig 8 that shows the result for prediction for the total measured time of (30 s) where only the initial point is known from the human data.
The subject body parameters are M = 67 kg, L = 1.68 m. The noise is estimated as white noise with a standard deviation of 0.005. The reaction time of the subject is 0.310 s.
The subject body parameters are M = 67 kg, L = 1.68 m. The noise is estimated as white noise with a standard deviation of 0.005. The reaction time of the subject is 0.310 s.
The results are validated by three criteria: Root Mean Square Error (RMSE) and Variance accounted for (VAF) and Power Spectral Density (PSD) in the frequency domain. VAF represents how much the estimated output is similar to the measured data. A higher percentage shows more similarity.
(24)
Table 4 presents the RMSE and VAF calculated for both prediction for a step ahead and prediction for the total duration of the experiment. According to the table, optimal approaches generally have more accurate predictions than the IPD approach. While theCOP-BC approach has the slightest error and more accurate prediction than other methods. It should be highlighted that while the VAF metric provides a rough approximation of the accuracy of a prediction, it is not a valid indicator for assessing one-step-ahead prediction. This is because the VAF primarily measures the ability of the controller to track the measured COP, whereas the objective is to validate the controllers’ ability to stabilize the position and track the zero angular position.
However, since approximating individual noise and disturbance may not be accurate, validation in the frequency domain is more reliable. PSD is a common method to analyse the COP in studying postural sway. Here, we used the Lomb-Scargle function in MatLab to calculate the PSD that is preferable in case of short time series to detect low frequencies. Fig 9 compares the PSD calculated for the COP obtained with different methods compared with measured COP of a random subject. It can be seen that only MPC andCOP-BC approaches have similar PSD in small frequencies (below 0.1 Hz), compared to the human data.
Discussion
The choice of weights and feedback parameters in the controller can affect the performance of the controllers. Here, we studied the robustness of each controller to the parameter changes as well as the energy consumption in the joints. For the IPD controller and the optimal controller (MPC andCOP-BC), we changed the parameters so that the controller’s results stay stable and in the range of body joints movements. However, as described in the methodology section, the weight in MBC is chosen as the null space projection of AG matrix and the stability is sensitive to this weight. Therefore, there is limited freedom to change this weight. Although this can be a limitation, further works need to be done to investigate the stable region of this controller.
Fig 10 study the effect of changing the proportional Kp and derivative Kd gain in IPD controller. The energy versus error plot shows that the error between the angular and reference positions decreases as the controller gain increases. At the same time, ankle and hip joints need more effort to maintain balance. Interestingly, the same trend can be observed by changing weights in the objective function for both MPC andCOP-BC controller methods as shown in Figs 11 and 12. This indicates that the change of the controller parameters in the stable region is a trade-off between the energy consumption in the joints and the accuracy of the prediction. This is an essential factor to consider in designing robotic applications. Finding the optimal parameter based on energy consumption and accuracy is in the form of the Pareto front optimization, which is deferred to our future work.
The upper plot shows the effect of gain change on the total energy consumption in the joints and the RMSE. The lower plot represents the effect on the joints’ torques and standing strategy.
The upper plot shows the effect of gain change on the total energy consumption in the joints and the RMSE. The lower plot represents the effect on the joints’ torques and standing strategy.
The upper plot shows the effect of gain change on the total energy consumption in the joints and the RMSE. The lower plot represents the effect on the joints’ torques and standing strategy.
Moreover, we investigated whether parameter changes will influence the balance strategy. As shown in Fig 10 IPD controller uses an ankle-hip strategy despite changes in the controller gains. This is almost the same case for the MPC controller (Fig 11). However, increasing the gain of the error reduction term in theCOP-BC objective function results in favoring the ankle strategy to stabilize the body (Fig 12). When designing robot applications, robustness to parameter changes may be desirable. Therefore, IPD and MPC controllers are more attractive. On the other hand, in the clinical application of the postural sway model, the relation between parameters or weights and postural strategy is an essential factor. Since, by aging or postural related disease, the human body uses more hip, hip-ankle strategy rather than ankle strategy to maintain balance [62, 63]. The results about theCOP-BC, confirm the finding in the [63] where they found the COP information valuable and plausible to generate postural sway. They also found COP metrics sensitive to the parameter changes of the active controller, where the COP obtained by their method can show the difference between younger and older adults’ balance control. Besides, the presence of tactile and proprioception sensors under the feet [64] makes consideration of COP information in the postural sway controllers essential in posturography and clinical applications.
We compared the three most common optimization methods that can mimic postural sway in quiet standing with IPD controller. We chose IPD controller since it solves the stability issue of the PD controller specifically in case of time delays [24]. However, future work should consider other methodologies, such as delayed PD or Proportional-Derivative Acceleration (PDA) controller [65–67], to compare the optimal controllers. PDa feedback controller, where control action is switched on and off based on the sensory dead zone, is a potential methodology to consider specifically in case of larger time delays [68–70]. Furthermore, the intermittency here is based on activating the PD controller based on the state-dependent mechanism. However, other intermittent controller paradigms introduced in [71, 72] and the human likeliness of each paradigm can be studied as future work.
In this work, we utilize EKF to estimate the neural time delay and the sensory noise. However, in spite of the simplicity of EKF in predicting the time delays and reducing the measurement noise, further estimation methods can be studied. Finally, on a broader level, research should also determine the three joints model without locking the knee to study the impact of time delays on the control strategies and utilizing the stepping strategy to maintain balance in a quiet stance.
Conclusion
In this paper, we investigated optimal control structures mimicking the postural sway by considering the physical body dynamic as two links inverted pendulum and comparing it with the widely used IPD controller. In addition to benefiting from using joint and body constraints, our results show that optimal control techniques consume less energy in the joints and have a more accurate prediction of the control input to the joints with higher accuracy compared to the IPD controller. However, the straightforward implementation and information about the muscles’ damping and stiffness is still an advantage of the IPD method in clinical applications.
In optimal techniques, our results revealed that although the MBC can somewhat predict the control output to the joints, the balance strategy differs from what the human body uses, especially for significant perturbation. Besides, the choice of weight to maintain the controller in a stable region needs more effort than other approaches. Improving the performance of this controller to be more similar to human behavior in postural sway; future work is required to study the additional term to the cost function (Eq 16) that consider the changes of COP.
Both MPC andCOP-BC optimal controllers showed promising results in both simulation and experimental comparison. However, COP-BC controller uses less energy in the hip and mainly uses ankle strategy in postural sway, which is preferable in robotic applications.
We also observed that tuning the parameter is a trade-off between the consumed energy in the joints and the accuracy of the prediction. These findings suggest that, while IPD controller is not superior to other methods, it benefits from simplicity and can be used in clinical and robotic applications. While for more accuracy and less energy consumption, one can benefit from optimal controllers.
Although our finding can be a helpful aid in choosing the balance controller structure for the balance applications, it has some limitations. First, to carefully compare the human likeliness of the different approaches, it is desirable to consider the stepping strategy in higher perturbation. Therefore, future research should study the capacity of different methodologies with a three-link inverted pendulum considering the knee joint. Second, more efforts are needed to investigate the methods considering the model-based structure of the muscular system in the human balance loop.
Acknowledgments
We gratefully acknowledge Mascha Pauelsen for collecting the data in the Human Health and Performance Lab at Luleå University of Technology, Luleå, Sweden.
References
- 1. Chiba R, Takakusaki K, Ota J, Yozu A, Haga N. Human upright posture control models based on multisensory inputs; in fast and slow dynamics. Neuroscience research. 2016 Mar 1;104:96–104. pmid:26746115
- 2. Horak FB. Postural orientation and equilibrium: what do we need to know about neural control of balance to prevent falls?. Age and ageing. 2006 Sep 1;35(suppl2):ii7–11. pmid:16926210
- 3. Mergner T, Lippi V. Posture control—human-inspired approaches for humanoid robot benchmarking: Conceptualizing tests, protocols and analyses. Frontiers in neurorobotics. 2018 May 7;12:21. pmid:29867428
- 4. Winter DA. Human balance and posture control during standing and walking. Gait and posture. 1995 Dec 1;3(4):193–214.
- 5. Peterka RJ. Sensorimotor integration in human postural control. Journal of neurophysiology. 2002 Sep 1;88(3):1097–118. pmid:12205132
- 6. Maurer C, Mergner T, Peterka RJ. Multisensory control of human upright stance. Experimental brain research. 2006 May;171(2):231–50. pmid:16307252
- 7.
Jafari H, Nikolakopoulos G, Gustafsson T. Stabilization of an inverted pendulum via human brain inspired controller design. In2019 IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids) 2019 Oct 15 (pp. 433-438). IEEE.
- 8. Kilby MC, Molenaar PC, Newell KM. Models of postural control: shared variance in joint and COM motions. PloS ONE. 2015 May 14;10(5):e0126379. pmid:25973896
- 9. Chumacero E, Yang J, Chagdes JR. Effect of sensory-motor latencies and active muscular stiffness on stability for an ankle-hip model of balance on a balance board. Journal of Biomechanics. 2018 Jun 25;75:77–88. pmid:29861093
- 10. Jiang P, Chiba R, Takakusaki K, Ota J. A postural control model incorporating multisensory inputs for maintaining a musculoskeletal model in a stance posture. Advanced Robotics. 2017 Jan 17;31(1-2):55–67.
- 11. Kaminishi K, Jiang P, Chiba R, Takakusaki K, Ota J. Postural control of a musculoskeletal model against multidirectional support surface translations. PloS ONE. 2019 Mar 6;14(3):e0212613. pmid:30840650
- 12.
Zebenay M, Lippi V, Mergener T. Human-like humanoid robot posture control. In2015 12th International conference on informatics in control, automation and robotics (ICINCO) 2015 Jul 21 (Vol. 2, pp. 304-309). IEEE.
- 13. van der Kooij H, Jacobs R, Koopman B, van der Helm F. An adaptive model of sensory integration in a dynamic environment applied to human stance control. Biological Cybernetics. 2001 Jan;84(2):103–15. pmid:11205347
- 14. Kiemel T, Oie KS, Jeka JJ. Multisensory fusion and the stochastic structure of postural sway. Biological Cybernetics. 2002 Oct;87(4):262–77. pmid:12386742
- 15. Kuo AD. An optimal control model for analyzing human postural balance. IEEE transactions on biomedical engineering. 1995 Jan;42(1):87–101. pmid:7851935
- 16. Tahboub K, Mergner T. Biological and engineering approaches to human postural control. Integrated Computer-Aided Engineering. 2007 Jan 1;14(1):15–31.
- 17. Ersal T, Sienko KH. A mathematical model for incorporating biofeedback into human postural control. Journal of neuroengineering and rehabilitation. 2013 Dec;10(1):1–3. pmid:23374173
- 18. Peterka RJ. Simplifying the complexities of maintaining balance. IEEE Engineering in Medicine and Biology Magazine. 2003 Apr 22;22(2):63–8. pmid:12733461
- 19. Ting LH, van Antwerp KW, Scrivens JE, McKay JL, Welch TD, Bingham JT, et al. Neuromechanical tuning of nonlinear postural control dynamics. Chaos: An Interdisciplinary Journal of Nonlinear Science. 2009 Jun 29;19(2):026111.
- 20. King GW, Abreu EL, Kelly PJ, Brotto M. Neural control of postural sway: relationship to strength measures in young and elderly adults. Experimental gerontology. 2019 Apr 1;118:39–44. pmid:30630036
- 21. Jiang P, Chiba R, Takakusaki K, Ota J. Generation of the human biped stance by a neural controller able to compensate neurological time delay. PloS ONE. 2016 Sep 21;11(9):e0163212. pmid:27655271
- 22. Iqbal K. Optimal time-varying postural control in a single-link neuromechanical model with feedback latencies. Biological Cybernetics. 2020 Oct;114(4):485–97. pmid:32865604
- 23. Asai Y, Tasaka Y, Nomura K, Nomura T, Casadio M, Morasso P. A model of postural control in quiet standing: robust compensation of delay-induced instability using intermittent activation of feedback control. PLoS ONE. 2009 Jul 8;4(7):e6169. pmid:19584944
- 24. Morasso P, Cherif A, Zenzeri J. Quiet standing: The single inverted pendulum model is not so bad after all. PloS ONE. 2019 Mar 21;14(3):e0213870. pmid:30897124
- 25. Bottaro A, Yasutake Y, Nomura T, Casadio M, Morasso P. Bounded stability of the quiet standing posture: an intermittent control model. Human movement science. 2008 Jun 1;27(3):473–95. pmid:18342382
- 26. Asai Y, Tateyama S, Nomura T. Learning an intermittent control strategy for postural balancing using an EMG-based human-computer interface. PLoS ONE. 2013 May 22;8(5):e62956. pmid:23717398
- 27. Nomura T, Oshikawa S, Suzuki Y, Kiyono K, Morasso P. Modeling human postural sway using an intermittent control and hemodynamic perturbations. Mathematical biosciences. 2013 Sep 1;245(1):86–95. pmid:23435118
- 28. McKee KL, Neale MC. Direct estimation of the parameters of a delayed, intermittent activation feedback model of postural sway during quiet standing. PLoS ONE. 2019 Sep 17;14(9):e0222664. pmid:31527893
- 29. Morasso P. Integrating ankle and hip strategies for the stabilization of upright standing: An intermittent control model. Frontiers in Computational Neuroscience. 2022;16. pmid:36465968
- 30.
Li Y, Levine WS. An optimal control model for human postural regulation. In2009 American Control Conference 2009 Jun 10 (pp. 4705-4710). IEEE.
- 31. Shen K, Chemori A, Hayashibe M. Human-like balance recovery based on numerical model predictive control strategy. IEEE Access. 2020 May 15;8:92050–60.
- 32. Qu X, Nussbaum MA. Modelling 3D control of upright stance using an optimal control strategy. Computer methods in biomechanics and biomedical engineering. 2012 Oct 1;15(10):1053–63. pmid:21598131
- 33.
Kuzmych O, Awrejcewicz J, Aitouche A, Bahniuk N. Robust control for human postural balance: design and simulation. In2018 7th International Conference on Systems and Control (ICSC) 2018 Oct 24 (pp. 454-461). IEEE.
- 34. Bayon C, Emmens AR, Afschrift M, Van Wouwe T, Keemink AQ, Van Der Kooij H, et al. Can momentum-based control predict human balance recovery strategies?. IEEE transactions on neural systems and rehabilitation engineering. 2020 Jun 29;28(9):2015–24. pmid:32746307
- 35.
Sultan N, ul Islam MN, Mughal AM. Postural Control During Standing Posture For Small Perturbations With Feedback Linearization. In2018 18th International Conference on Mechatronics-Mechatronika (ME) 2018 Dec 5 (pp. 1-6). IEEE.
- 36. Aftab Z, Robert T, Wieber PB. Balance recovery prediction with multiple strategies for standing humans. PloS ONE. 2016 Mar 14;11(3):e0151166. pmid:26974820
- 37. van der Kooij H, Jacobs R, Koopman B, Grootenboer H. A multisensory integration model of human stance control. Biological Cybernetics. 1999 May;80(5):299–308. pmid:10365423
- 38.
Atkeson CG, Stephens B. Multiple balance strategies from one optimization criterion. In2007 7th IEEE-RAS International Conference on Humanoid Robots 2007 Dec (pp. 57-64). IEEE.
- 39. Rahmati Z, Schouten AC, Behzadipour S, Taghizadeh G, Firoozbakhsh K. Disentangling stability and flexibility degrees in Parkinson’s disease using a computational postural control model. Journal of neuroengineering and rehabilitation. 2019 Dec;16(1):1–4. pmid:31412926
- 40. Masani K, Vette AH, Popovic MR. Controlling balance during quiet standing: proportional and derivative controller generates preceding motor command to body sway position observed in experiments. Gait and posture. 2006 Feb 1;23(2):164–72. pmid:16399512
- 41. Wang H, van den Bogert AJ. Identification of the human postural control system through stochastic trajectory optimization. Journal of neuroscience methods. 2020 Mar 15;334:108580. pmid:31926202
- 42. Engelhart D, Boonstra TA, Aarts RG, Schouten AC, van der Kooij H. Comparison of closed-loop system identification techniques to quantify multi-joint human balance control. Annual reviews in control. 2016 Jan 1;41:58–70.
- 43.
Jafari H, Pauelsen M, Röijezon U, Nyberg L, Nikolakopoulos G, Gustafsson T. On internal modeling of the upright postural control in elderly. In2018 IEEE International Conference on Robotics and Biomimetics (ROBIO) 2018 Dec 12 (pp. 231-236). IEEE.
- 44.
Lippi V, Maurer C, Mergner T. The Importance of Models in Data Analysis with Small Human Movement Datasets–Inspirations from Neurorobotics Applied to Posture Control of Humanoids and Humans. arXiv preprint arXiv:2102.02543. 2021 Feb 4.
- 45. Hou ZS, Wang Z. From model-based control to data-driven control: Survey, classification and perspective. Information Sciences. 2013 Jun 20;235:3–5.
- 46. Pauelsen M, Jafari H, Strandkvist V, Nyberg L, Gustafsson T, Vikman I, et al. Frequency domain shows: Fall-related concerns and sensorimotor decline explain inability to adjust postural control strategy in older adults. PLoS ONE. 2020 Nov 20;15(11):e0242608. pmid:33216812
- 47. Pauelsen M, Vikman I, Strandkvist V, Larsson A, Röijezon U. Decline in sensorimotor systems explains reduced falls self-efficacy. Journal of electromyography and kinesiology. 2018 Oct 1;42:104–10. pmid:30015133
- 48.
Kiemel S. Balance maintenance of a humanoid robot using the hip-ankle strategy. M.S. thesis, Dept. Mechanical, Maritime and Materials Eng., Delft University of Technology. Delft, Netherlands, 2012.
- 49. Li Y, Levine WS, Loeb GE. A two-joint human posture control model with realistic neural delays. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2012 Jun 6;20(5):738–48. pmid:22692939
- 50. Davidson BS, Madigan ML, Southward SC, Nussbaum MA. Neural control of posture during small magnitude perturbations: effects of aging and localized muscle fatigue. IEEE transactions on biomedical engineering. 2010 Nov 29;58(6):1546–54. pmid:21118760
- 51.
Li Q, Li R, Ji K, Dai W. Kalman filter and its application. In2015 8th International Conference on Intelligent Networks and Intelligent Systems (ICINIS) 2015 Nov 1 (pp. 74-77). IEEE.
- 52.
Nava G, Romano F, Nori F, Pucci D. Stability analysis and design of momentum-based controllers for humanoid robots. In2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2016 Oct 9 (pp. 680-687). IEEE.
- 53. Koolen T, Bertrand S, Thomas G, De Boer T, Wu T, Smith J, et al. Design of a momentum-based control framework and application to the humanoid robot atlas. International Journal of Humanoid Robotics. 2016 Mar 7;13(01):1650007.
- 54. Herzog A, Rotella N, Mason S, Grimminger F, Schaal S, Righetti L. Momentum control with hierarchical inverse dynamics on a torque-controlled humanoid. Autonomous Robots. 2016 Mar;40(3):473–91.
- 55.
Lee J, Huber ME, Sternad D, Hogan N. Robot controllers compatible with human beam balancing behavior. In2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2018 Oct 1 (pp. 3335-3341). IEEE.
- 56.
Orin DE, Goswami A. Centroidal momentum matrix of a humanoid robot: Structure and properties. In2008 IEEE/RSJ International Conference on Intelligent Robots and Systems 2008 Sep 22 (pp. 653-659). IEEE.
- 57. Casadio M, Morasso PG, Sanguineti V. Direct measurement of ankle stiffness during quiet standing: implications for control modelling and clinical application. Gait and posture. 2005 Jun 1;21(4):410–24. pmid:15886131
- 58. Runge CF, Shupert CL, Horak FB, Zajac FE. Ankle and hip postural strategies defined by joint torques. Gait and posture. 1999 Oct 1;10(2):161–70. pmid:10502650
- 59. Park S, Horak FB, Kuo AD. Postural feedback responses scale with biomechanical constraints in human standing. Experimental brain research. 2004 Feb;154(4):417–27. pmid:14618285
- 60. Featherstone R. A simple model of balancing in the plane and a simple preview balance controller. The International Journal of Robotics Research. 2017 Dec;36(13-14):1489–507.
- 61. Olchowik G, Czwalik A, Kowalczyk B. The changes in postural stability of women in early old age. The journal of nutrition, health and aging. 2020 Jul;24(7):739–44. pmid:32744570
- 62. Baston C, Mancini M, Rocchi L, Horak F. Effects of Levodopa on Postural Strategies in Parkinson’s disease. Gait and posture. 2016 May 1;46:26–9. pmid:27131172
- 63. Tigrini A, Verdini F, Fioretti S, Mengarelli A. Center of pressure plausibility for the double-link human stance model under the intermittent control paradigm. Journal of Biomechanics. 2021 Nov 9;128:110725. pmid:34509911
- 64. Viseux FJ. The sensory role of the sole of the foot: review and update on clinical perspectives. Neurophysiologie Clinique. 2020 Feb 1;50(1):55–68. pmid:32007381
- 65. Le Mouel C, Brette R. Anticipatory coadaptation of ankle stiffness and sensorimotor gain for standing balance. PLoS computational biology. 2019 Nov 22;15(11):e1007463. pmid:31756199
- 66. Pasma JH, Boonstra TA, Van Kordelaar J, Spyropoulou VV, Schouten AC. A sensitivity analysis of an inverted pendulum balance control model. Frontiers in computational neuroscience. 2017 Oct 27;11:99. pmid:29163116
- 67. Zelei A, Milton J, Stepan G, Insperger T. Response to perturbation during quiet standing resembles delayed state feedback optimized for performance and robustness. Scientific Reports. 2021 May 31;11(1):11392. pmid:34059718
- 68. Milton J, Insperger T. Acting together, destabilizing influences can stabilize human balance. Philosophical Transactions of the Royal Society A. 2019 Sep 9;377(2153):20180126. pmid:31329069
- 69.
Insperger T, Stépán G. Semi-discretization for time-delay systems: stability and engineering applications. Springer Science and Business Media; 2011 Jul 15.
- 70. Insperger T, Milton J, Stépán G. Acceleration feedback improves balancing against reflex delay. Journal of the Royal Society Interface. 2013 Feb 6;10(79):20120763. pmid:23173196
- 71. Loram ID, Gollee H, van de Kamp C, Gawthrop PJ. Is Intermittent Control the Source of the Non-Linear Oscillatory Component (0.2–2Hz) in Human Balance Control?. IEEE Transactions on Biomedical Engineering. 2022 May 13;69(12):3623–34. pmid:35560085
- 72.
Gawthrop P, Gollee H, Lakie M, Loram ID. Intermittent control of movement and balance. In Encyclopedia of Computational Neuroscience 2022 Jun 12 (pp. 1689–1694). New York, NY: Springer New York.