International Journal of Advanced Robotic Systems Planning and Control of Stable Walking for a 3 D Bipedal Robot Regular Paper

This  paper  presents  a  time‐invariant  feedback controller  that  simultaneously  regulates  the  ZMP (zero‐moment point) position and the  joint configuration of  a  3D  biped  in  order  to  achieve  an  asymptotically, periodic walking gait for a 3D bipedal robot with feet. The cyclic  walking  gait  is  composed  of  a  successive single‐support phase  and  an  impulsive  impact with  full plane‐contact between the feet and the ground. The biped robot  has  10  DOFs  (degrees  of  freedom)  in  the single‐support phase and 10 actuators.  In order  to avoid the  unexpected  rotation  of  the  supporting  foot,  the position  of  the  ZMP  in  the  horizontal  plane  has  to  be controlled.  It  is also desired  that  the  feedback  controller tracks  a  parameterized  reference  trajectory  to  achieve walking stability. We use the method of virtual constraints previously implemented for controlling point‐feet bipedal robots to create a set of parameterized reference walking trajectories.  By  creating  the  hybrid  zero  dynamics,  an orbital stability study with Poincaré map is evaluated in a reduced  space.  We  then  design  a  supplemental event‐based  feedback  controller  to  enhance  walking stability. The walking gait has an average walking speed of  0.76m/sec  (or  0.72  body  lengths  per  second)  in  the simulation study.


Introduction
One of the greatest difficulties in the control of a walking robot is respecting the ZMP (zero-moment point) constraints, which must remain inside the convex hull of the foot support region [1].In order to avoid the unexpected rotation of the supporting foot, the position of the ZMP in the horizontal plane must be controlled.Most existing on-line bipedal walking controllers are designed to track a desired ZMP trajectory and/or to compensate for the ZMP error.However, the general weakness of the ZMP-based method is that it is usually tested through walking experiments, but not by rigorous stability analysis.
A very famous example of ZMP-based control is the ASIMO robot [2,3].That approach defines the desired trajectory of the ZMP and closes successive inner control loops on the basis of the ZMP.The work of Kajita et al. [5] performed predictive control on the basis of the position of the centre of mass and produced a simplified model of the robot in order to achieve the desired ZMP trajectory.The on-line adjustment of the ZMP has been added and this control method is implemented on the robot HPR2 [6].Biped robot periodic motion generation and walking control based on various intelligent control approaches have also been studied [7][8][9][10].
The existence and stability of a periodic orbit depend on much more than just the position of the ZMP.It is possible that the ZMP is situated within the convex hull of the foot support region and where the robot remains upright, but the gait is not periodic or it is not asymptotically stable [11].Therefore, the control of the ZMP ensures that the supporting foot will not rotate about its foot, but this does not guarantee stability in the sense of convergence toward a periodic motion.The literature has proposed a path-following control strategy to simultaneously regulate the position of the ZMP and the joints of a planar biped robot [11].
Research studies exist on bipedal walking based on under-actuation robots, which are either passive dynamic walking bipeds [12,13] or robots with a point foot contact [14,15].The main purpose of this paper is to propose a new control strategy for simultaneously regulating the position of the ZMP and the joint configuration of a fully-actuated 3D biped robot in order to achieve an asymptotically, periodic walking gait.This study is motivated by extending guidelines and a detailed theory as suggested by an asymptotically stable feedback control of a 3D point-feet bipedal robot [16,17].We investigate a fully-actuated 10 DOFs (degrees of freedom) 3D bipedal robot with the objective of developing a time-invariant feedback control law that achieves periodic walking stability, while at the same time regulating the ZMP trajectory in the supporting region implicitly.The virtual constraints are holonomic constraints on the robot's configuration that are asymptotically stable and achieved through the action of a feedback controller.These virtual constraints are the keys to coordinating the evolution of the various links of the robot throughout a stride and to regulate the evolution of the ZMP within the supporting region.

Biped model
This study considers a 10 DOF fully-actuated 3D biped with feet as shown in Figure 1.The 3D biped robot consists of seven links: a torso, two legs and two feet.Each leg has a knee, a shin and a thigh of equal length.The 10 actuators are installed as follows: two at each hip joint, one at each knee and two at each ankle.Let the joint coordinate in the single-support phase be 1 2 10 . The following assumptions are made in this work.
(1) Each link is represented by a point mass located at its centre of mass.(2) Walking consists of two alternating motion phases: the single-support phase and the double-support phase.
(3) The double-support phase is instantaneous and involves the flat-foot impact of the swing leg with the ground.(4) At impact, the swing leg neither slips nor rebounds.(5) The swing and stance legs exchange their roles at each impact.( 6) Walking takes place on a flat surface.
Biped coordinate system when the left foot is the stance foot.
Let the x-axis point forward, z-axis point upward and the y-axis be determined by the right-hand rule.Figure 1 and Figure 2 respectively show the coordinate system for the bipedal robot during the left foot stance and right foot stance.Table 1 lists the rotational axes for the biped during the left foot stance phase and the right foot stance phase.All joints' rotational axes in the y-axis are in the same direction for both the left foot stance and right foot stance phases, and all joints' rotational axes in both x-axis and z-axis are different by 180 degrees.One unique feature of the above coordinate systems assignment is that for both the left foot stance and right foot stance phases the robot has the same set of dynamic equations in the joint variables q .Therefore, this coordinate system assignment makes the analysis and design of the biped walking motion less complicated.
Defining + and -as the moments just before and after impact respectively, the condition for periodic walking is as follows: 11 ( ) (0 ), 1, 2, , 10 where T is step time.In the single-support phase, the biped robot's joint dynamic equations can be expressed as ( ) ( , ) where ( ) D q is the positive-definite 10 10  mass-inertia matrix, ( , ) H q q  is a 10 1  vector of Coriolis and gravity terms, and u is a 10 1  vector of control input torques.This dynamic equation is valid for both the left foot stance and right foot stance phases.
Biped coordinate system when the right foot is the stance foot.
The biped's GRF (ground reaction force) and ZMP during the single-support phase can be repressed as ) and where g is the gravity acceleration, , ,  is the centre of mass of the i-th link and ( ) ,( ) ,( )  is the moment of the i-th link.After a series of algebra manipulations, the ZMP in Eq. ( 4) can be rewritten as where ( ) X q , ( ) Y q and ( ) Z q are 10 1  vectors and ( , ) b x q q  , ( , ) b y q q  , and ( , ) b z q q  are scales.In this way, the ZMP can be controlled through the q  term.In other words, the angular acceleration q  must satisfy two constrained equations to produce the desired ZMP.
A passive impact exists at the end of the single-support phase and this research considers the flat-foot strike.Following standard practice in the literature, we assume the double-support phase to be instantaneous.During the impact, the biped's configuration variables do not change, but the generalized velocities undergo a jump.
Let   ( ) ,( ) ,( ) be the orientation angles of the stance foot and ( , , ) x y z is its toe position, and let

 
( ) ,( ) ,( ) be the orientation angles of the swing foot and ) , , ( sw sw sw z y x its heel position.We define the augmented generalized coordinate in the double-support phase to be ( ) ( ) ( ) and their rotational axes are defined in Table 2. Thus, the biped robot's dynamic equation in the double-support phase can be expressed as

( ) ( , )
T T e e e e e e sw sw st st D q q H q q B u J J where e D is the extend mass-inertia matrix, e H is the extend vector of Coriolis and gravity terms, and e B is a full-rank constant matrix indicating whether or not a joint is actuated.Here, st J and sw J are respectively denoted as ( ) ( ) They are the corresponding Jacobian matrices for the position of the stance foot and its orientation in yaw and pitch angles.Here, st  and sw  are the vectors of the Lagrange multipliers.
During the impact, the biped's joint variables do not change, but the extended generalized velocities undergo a jump.Analogous to Chevallereau et al. [17], this yields where st F is the reaction force of the stance foot and sw F is the reaction force of the swing foot.The overall impact model is written as ( ) ( , ) is obtained from Eq. ( 7) and projects down to the joint coordinates for the supporting leg.

Controller design using the constrained approach
In order to design a time-invariant controller for biped walking, we choose a strictly monotonic configuration variable along a typical walking gait to replace time in parameterizing a biped's periodic motion.In a forward walking motion, the x-coordinate of the hip is monotonically increasing.Hence, if the virtual stance leg is defined by the line that connects the stance foot to the stance hip, then the angle in the sagittal plane is monotonic.When the shin and the thigh have the same length, the angle of the virtual leg in the sagittal lane can be selected as the minus sign here is used to make  strictly increase over a step.
The first objective in the controller design is to control the ZMP within the region of the stance foot.Let ( ( ), ( ) zmp zmp x y   be the desired ZMP position during the single-support phase and from Eq. ( 4) we obtain two constrained equations (11) which need to be taken into consideration in the controller design.
Since the biped has 10 actuators, eight holonomic constraints on the robot's configuration can be added into the consideration in the controller design.The eight virtual constraints, 0 y  , are expressed by the output function ( ) ( ( )) where y is a , , , q q q  as a function of the robot's configuration variable of  .
One of the control objective is to regulate the output function to achieve 0 y  .This can be done by using a PD control Combining Eq. ( 11) and Eq. ( 13), we have Therefore, the control input u is designed as ) zmp zmp zmp zmp u D q A q x y b q q x y H q q which is an input-output linearizing controller and results in 0 In summary, determining the desired ZMP evolution ( ( ), ( )) ) and it reduces the dimension of the biped's dynamics to a 2-dimensional dynamic system.We obtain the 2-dimensional zero dynamics below.Let the reduced variable be At this new joint coordinate Eq. ( 14) can be re-written as where 11 12 , and the first two rows of Eq. ( 19) are The biped's dynamics in the stance phase are now reduced to a 4-dimensional autonomous system, called the single-support swing phase zero dynamics, which can describe the behaviour of the original system with 20 dimensions.
The procedure to determine ( , ) Step 2: From the impact equation in Eq. ( 8) and Eq. ( 9) for a biped's configuration defined in Step 1, find the biped's beginning configuration of the next single-support phase, Step 3: The desired ( ) d h  is a vector of cubic function of configuration variable  , which satisfies the initial and the end joint positions, as well as angular velocities described in Step 1 and Step 2.
Step 4: Integrate the zero dynamics in Eq. ( 20).The ODE solver will terminate when the swing leg touches the ground.Next, evaluate the integral-squared torque per step length, , where T is the walking period and L is the step length.
Step 5: Check the following equality and inequality constraints.
Equality constraints that impose periodicity: (e1) * ( ) that minimize the integral-squared torque per step length defined in Step 4, and subject to all the constraints in Step 5.
The above optimization problem can be solved in MATLAB with the FMINCON function of the optimization toolbox.A solution minimizing J defines the desired periodic walking gait.Note that this optimization technique is only a local minimum and thus the obtained periodic motion depends on the initial set of optimization parameters.Moreover, the obtained periodic walking gait may not be periodically stable, even though the evolution of the ZMP is within the stance foot.

Restricted Poincaré Map
The stability of the searched periodic walking can be tested numerically using a linearized Poincaré map in a reduced low-dimensional space.Following the method used in Chevallereau et al. [17], it is shown that in S Z  the state of the robot can be represented using only three independent variables 1 1 , where   ( , ) ( ) 0, ( , ) 0 , if the output for the feedback control design is modified as ( , , ) ( ) ( , , ) The correction term is taken to be a three-times ( , , ) 0, 2 where i y is the initial values of output y , and i  and f  are the initial and final value respectively of  for the current step.The restricted map , gives rise to a linearized system The 3 3  square matrix z A is the Jacobian of the Poincaré map and is computed as and i d is a 3 1  column vector, having 1 in the i-th row and 0 elsewhere.A fixed-point of the restricted Poincaré map z A is locally exponentially stable if, and only if, the eigenvalues have a magnitude strictly less than one.

Event-Based Feedback Stabilization
If a desired periodic gait is not exponentially stable or the region of attraction is too small, then an event-based control can be designed and integrated with the continuous, stance-phase controller.The idea is to introduce a vector of parameters held constant during the stance phase and updated at each impact.Here, it will be updated on the basis of the state of the hybrid zero dynamics.We augment the output with an additional term, ( s h   depends on a vector of parameters 0    , where 0  is an open neighbourhood of the origin of 8 R and where ( , ) 0, 0.1 0.9  is taken to be a fifth-order polynomial for 0.1 0.9 The restricted Poincaré map is now viewed as a nonlinear control system on 2 S Z  with input k  namely: 1 ( , ) where k  is the value of  during step k.Linearizing this nonlinear system about the fixed point and the where F is the Jacobian of the map z P with respect to  .We next design a feedback matrix K such that will exponentially stabilize the fixed point.The  Table 3. Parameters of the 3D biped robot (in MKS).

Simulation results
Table 3 presents the physical parameters of the 3D biped robot used in this study, where the height is 1.05m and weight 9.9kg.These parameters result in the centre of gravity of the biped being located near the midpoint of the hips.
We now find a nominal periodic walking motion that minimizes the energy consumed per step.The search procedure is carried out in MATLAB with the FMINCON function of the optimization toolbox.We first generate the equations of motion for the 3D biped by using the functions of the symbolic toolbox and then convert to C functions in order to shorten the simulation time.
For the given biped parameters, we obtain and define a periodic solution by * q , * q  , ( , ) m and an average walking speed of 0.7582 m/sec (or 0.7221 body lengths per second).For the proposed biped walking control method, both the desired joint trajectories (Figure 4) and the desired ZMP profile (Figure 5) are controlled to satisfy the condition on the ZMP.During walking, the stance foot remains flat on the ground and the swing foot keeps above the ground (see Figure 5).
We use the within stride control for the full model of the 3D biped with virtual constraint ) , , ( ) ( can be seen that all the eigenvalues have a magnitude less than 1.0 and one eigenvalue is very close to 1.0.Therefore, the periodic walking stability is near stable.Figure 6 shows the output function and its derivative for the biped's walking control with only the within-stride control at an initial state from the above periodic solution.
To improve the relative walking stability, a stride-to-stride control can be designed and integrated with the continuous, stance phase controller.We modify the virtual constraint to 0.0761 0.7417 0.4808 j  Now all the eigenvalues have a magnitude much less than 1.0 and hence, the walking stability is much improved.Figure 7 shows the output function and its derivative for the biped's stride-to-stride walking control with an initial state from the above periodic solution.It should be noted that the output y and its derivative rapidly converge to zero.

Conclusion
This study looks at a 10 DOF 3D bipedal robot with feet under the objective of developing a time-invariant feedback control law that achieves periodic walking stability, while at the same time regulating the ZMP trajectory in the supporting region.We first use the method of virtual constraints to design a time-invariant, within-stride feedback controller that stabilizes the walking gait and the ZMP.We then design a supplemental event-based feedback controller to enhance the walking stability.
The main features and contributions of the paper are as follows: (1) An efficient optimization process is developed to obtain periodic 3D bipedal walking, satisfying the ZMP constraints implicitly.The reference motion for the biped walking is a parameterized trajectory and only the kinematic evolution of the robot state is needed, but not its temporal evolution.Therefore, the design of a time-invariant feedback controller becomes feasible.Two positions of ZMP in the horizontal plane are also regulated through the virtual constraints on the joint angles.(2) For a robot with 10 DOF, the study of the complete model required the analysis of dynamics in 20 dimensions, including 10 positions and 10 velocities.We are able to conduct the numerical study of stability using the Poincaré method on a lower-dimensional subsystem corresponding to the hybrid zero-dynamics.The Poincaré map used to analyse the stability of the walking gait is in a 3-dimensional space.(3) We use a stride-to-stride controller to enhance the stability of the walking motion.The nominal hybrid controller consists of a time-invariant within-stride controller that creates a periodic walking motion and a supplemental stride-to-stride (or event-based) controller that stabilizes the walking gait.The event-based feedback law automatically distributes small changes to all of the actuated joints in order to achieve a stable walking motion.
There are several ways in which the results can be extended in the future.They include offering a turning control, adding a double-support phase to emulate human-like walking and presenting other interesting motions.
evolution of the robot configuration as a function of  , which is chosen in the next section.This study picks the matrix M

3 8 
gain matrix K is calculated via the discrete linear quadratic regulator (DLQR) theory.

Figure 3 .
Figure 3. Stick-diagram of the searched periodic motion that minimizes the energy per length criterion.

Figure 4 .
Figure 4.The desired joint profiles of the obtained periodic walking motion over one step.

Figure 5 .Figure 6 .
Figure 5.The desired ZMP profiles and the height of the toe and the heel of the swing foot of the obtained periodic walking motion over one step.The ZMP is within the region of the stance foot, 0 15 .0    x and 05 .0 05 .0    y .

Figure 7 .
Figure 7.The output function and its derivative for the biped's walking control when an event-based stride-to-stride DLQR control is applied.

Table 1 .
Rotational axes of the biped robot in the single-support phase, where "x" mean x-axis, "y" means y-axis and "z" means z-axis.

Table 2 .
Rotational axes of the biped robot in the double-support phase.