Hybrid Model for Passive Locomotion Control of a Biped Humanoid: The Artificial Neural Network Approach

Developing a correct model for a biped robot locomotion is extremely challenging due to its inherently unstable structure because of the passive joint located at the unilateral foot-ground contact and varying configurations throughout the gait cycle, resulting variation of dynamic descriptions and control laws from phase to phase. The present research describes the development of a hybrid biped model using an Open Dynamics Engine (ODE) based analytical three link leg model as a base model and, on top of it, an Artificial Neural Network based learning model which ensures better adaptability, better limits cycle behaviors and better generalization while negotiating along a down slope. The base model has been configured according to the individual subjects and data have been collected using a novel technique through an android app from those subjects while walking down a slope. The pattern between the deviation of the actual trajectories and the base model generated trajectories has been found using a back propagation based artificial neural network architecture. It has been observed that this base model with learning based compensation enables the biped to better adapt in a real walking environment, showing better limit cycle behaviors. We also observed the bounded nature of deviation which led us to conclude that the strategy for biped locomotion control is generic in nature and largely dominated by learning.


I. Introduction
B iped locomotion has inherent complexity due to high degrees of non-linearity, high dimensionality, under actuation (in swing phase), over actuation (in stance phase).Consequently, correct analytical model and intuitive solutions are difficult to conceive and hard to find [1].Researchers have tried to make kinematics based biped locomotion models and formulate control strategy corresponding to several subphases of walking gait and tried to develop a controller for entire gait cycle by superimposing the control strategy of the control sub-phases [2].However, due to non-linearity, the condition of superposition is not appropriate and hence the overall control strategy based on the model failed.To mitigate this, a number of heuristics were incorporated in the control strategy [3], [14], [15].Appreciating this inherent complexity, here we propose to develop a computational model based on the captured data of human bipeds.The analytical model has been customized with the corresponding humanoid morphological parameters and its output has been stored [4], [5], [16].Now the deviation of trajectories between the actual gait pattern and the output of the analytical base model has been calculated [6], [7].To capture and subsequently predict this error pattern a neural network has been trained by using back-propagation algorithm.With this trained neural network, we are in a position to correct the ananalytical base model by compensating the difference.Through this compensation, we are incorporating the non-linearity of the actual gait in the output of the analytical model.We also observed the bounded nature of the difference which led us to conclude that the strategy for human locomotion control is generic in nature [8], [9], [13].
This paper has been arranged in the following manner, in the next section different types of biped models have been described followed by their critical analysis.In the subsequent section, we have developed an ODE based base model, which will serve the purpose of the analytical part of our hybrid model.Subsequently, we have described the details of our experimental data and the consequent error analysis.For the paper, a model was developed and section fifth explains its main features.Then the last section describes our results and discussions.

II. The Analytical Base Model
Due to computational simplicity, we prefer the analytical model.Here we have developed three link ODE based analytical model.The Analytical model provides the real world trajectories.These trajectories will be the input to our neural net which will then give error according to the training set provided to neural net.The generated error will be added to the real world trajectories to give stable limit cycle.Real world trajectories are parameterized according to various morphological parameters like mass of hip, shank, and thigh, length of the shank and thigh part of the human subject.Analytical model provides a stable limit cycle for almost all parameters, but starts falling for angles less than 4 degrees of inclination.So for those low degrees of inclination, we have to provide stable trajectories using artificial neural network.Use of analytical model in our project can be seen in the Fig 1 .Since analytical model plays such a vital role in our project we need to do the critical analysis of analytical model that includes (a) description of model, (b) input and output to analytical model, (c) error control method used in the model, and (d) description of mathematical equations used in the model.Due to high requirements and increasing density of robots in our daily lives, to make them move more efficiently through their environments is very important.Robots with more than two legs are more stable than the ones with two legs.Even then, due to their deficiency of incorporating in real world we have to move towards two legged robots or bipedal locomotion [10].This is the reason there have been many models regarding that and ours is one of them.

A. The Analytical Model Development
We have considered a three-link biped as shown in Fig. 1.In our model, the stance phase leg is considered to be a single link, whereas, the swing leg has a knee joint.The system is represented by 3 variable angles, 1. q1: Angle between the normal to the ground the fixedleg (corresponding to stance phase) 2. q2 : Angle between the two legs 3. q3 : Knee angle of the swing leg

B. Static Properties of Model
Model consists of five parts that are for each leg two parts, shank the lower leg and thigh the upper leg, and the additional mass mH i.e., the mass of hip which is considered as point mass.All the joints are considered friction less but ground friction is not zero, in fact it is high enough so that no sliding can occur.All the three links have their mass and their corresponding moment of inertia which is located at the centre of these respective links.The leg that stands firmly on ground is called stance leg and the other leg that will move freely will be called swing leg.Gamma g is the slope angle which is considered positive on downward slopes.Hip will be the intersection of both the legs, which we consider as one link.All these static properties will be adjusted according to the individual subject parameters in our model shown in Fig. 1.All these properties or terms or parameters related to our model are shown in Table 1.

C. Dynamic Properties
The dynamic properties in our model will be the angular velocities and the joint angles of the knee, hip and the ankle joints because all the joints specified in the static properties are rotational joints.The degree of freedom (d.o.f) for our model will be three.One leg will always be in stance position so, its knee d.o.f will be omitted.We will consider the knee d.o.f for only the swing legwhereas the third is for the angle between the knees and the hip.So we need to consider only three angles and their corresponding velocities.The various angles can be described in two ways based on their reference to world reference frame or relative to their parent frame.We have selected the second option that is relative to their parent frame.By applying the forward dynamic calculations on these dynamic properties, we fined out our limit cycle that will provide the criteria for stability.

D. The Mathematical Model
The analytical model models the bipedal walking by dividing the entire gait cycle into the following phases and computes the dynamics of each phase separately: • Three link phase: It is the most important phase in which the swing leg moves freely and knee is bent.
• Knee strike: It is the time when the knee of swing leg gets straight and the knee gets locked.At this point of time the system loses one of its degree of freedom because the swing leg becomes one single link instead of two separate parts (shank, thigh).And so shank part loses its angular momentum but conserving the angular momentum of whole system.This phase is considered to be the discrete phase of our step cycle.Now the three link model has converted into the compass gait model which is the predecessor of our three link model.
• Heel Strike: It is the phase which marks the end of one step cycle i.e. one gait.At this point of time the heel just strikes the ground after which the stance leg will become the swing leg and previous swing leg will become the stance leg and it again comes in continuous three link phase.The heel strike is assumed to be the inelastic collision.
The dynamic equations of the model while negotiating with the down slope under the force of gravity can be written in the coupled form as shown in (1).(1) Where d denotes the inertia matrix, denotes Coriolis and Centrifugal vector and will be the gravityvector, and all these matrices use avariant which is the angle with respect to the parent reference frame.

III. Simulation of the Model
Since the equations of motion in the three link model are nonlinear, we need to solve it by integrating the equations numerically.Dynamics of this model includes four phases that are categorized in two domains continuous dynamics that can be defined by the set of differential equations and discrete phases.Continuous dynamics include three link phase and two link phase.Whereas discrete phase includes knee strike and heel strike.Continuous phase are the passive elements of analytical model which means that during these phases analytical model performs passive walking.All these four phases are shown in Fig 2 .Output of this model will be the expected trajectories that are based on the various parameters provided to analytical model like all static properties: mass of hip, shank and thigh and length of shank ,thigh and the slope angle which is one of the most important factor of input.On the basis of output provided by model we will find our error or the deviation of the trajectories from the analytical model to that of the real human subject, that will then be used to train the neural net which will then predict the error pattern for the test subjects, provided all parameters are same for both the connectionist and analytical model, so that after compensating that error we can get stable limit cycle.One example of output can be seen in the set of Fig. 3.We have shown the limit cycle analysis of knee and hip joints as well as the output of the walker for 7 degree.Fig. 4 is the Limit Cycle for analysis of knee and hip joints for 4 degree walker.However, the model has several limitations such as reduced degrees of freedom, simplified contact constraint, which are not realistic.That inspired us to improve the model using actual walking data [17], [18].

IV. Experimental Data Collection
We have collected data of 24 students for the purpose of training the connectionist part of our hybrid model.The data is basically again composed of static and dynamicparameters.The static parameters consist of height, weight, thigh length and shank length of each person.The rest of the parameters can be calculated from the above 4 by simple arithmetic and some heuristics [8] regarding the human morphological structure.The dynamic parameters consisted of the hip and knee angles of the subject captured during walking down a slope.The passive gait was captured by using a plank and making it inclined at 4 degree angle and making the subject to walk on it down the inclined.We used an android application named Physics Toolbox Accelerometer developed by Vieyra Software, which will give us the acceleration along x and y axis direction [11], [12].The smartphone was attached to the subject's hip and knees and the data was captured shown in Fig. 5.The output of the accelerometer is the acceleration along x and y axis direction so we transformed it to q1 and q2 by inverse kinematic equation.Fig. 6  Next, we have applied convolution based filters on the data to smoothen it and have it zero-corrected and also the frequency of the accelerometer of the mobile phone was noted for the necessary sampling which was required for finding out the deviation from the base model shown in Fig. 7.The morphological parameters of the humansubject have been passed on to the analytical model developed in Section II and the output has been stored.This stored data have been compared with the experimental data so collected.To improve upon our analytical base model in the subsequent section, we would like to find an error pattern through detailed error analysis.

V. Error Analysis
We have the output of Analytical Base Model (unstable for 4 degree) and the Human data (stable for 4 degree).We are now going to calculate the error in Analytical model with respect to Human data.But before Error analysis, we do the following thing: Zero correction: We have to take the corresponding deviation of the analytical model's output with the actual gait captured of the human subject.So, we have done the zero correction of the captured data, since we are dealing with time-series data.
Filtering: We have filtered the curve, which we are getting by the Human data using convolution based filters.
Time sampling: Since we are comparing the corresponding data from both the sources, we had to match the frequency of the data.As we could not change the frequency of accelerometer (207 Hz), because it was the inherent property of the smart phone that we were using, so we changed the time step in the Analytical model to be 0.00483s as the time gap between two consecutive data.
Normalization: We made the output of both the data sources to be set between [0, 1] by using the following formula: x -x -x (2) After this we calculate the errors.We calculate the following 4 types of error and then compared them to select one of them.

RMS (root mean square) Error:
It represents the standard deviation of the difference between the data.The RMS servesto combine the magnitude of the errors in predictions forvarious times into a single measure of predictive power.Relative Error: It is the mean of the relative difference between the data.(6) After calculating the 4 types of errors we have checkedwhich type of error is more closely related to our dataset by calculating the Spearman's rank correlation coefficient of our data.This correlation gives us the idea about therank of similarity between the gait captured from the actualhuman subject and the output of the analytical model.Itcan be calculated as: Where d i = Difference between the corresponding datapoints, N = number of data, r = Spearman's rank ofcorrelation.From this, we calculate the correlation of each type of error with the Spearman's rank as: The error with the highest correlation with the Spearman's rank is chosen from all the 4 errors, because it has strong affinity to the signal strength of similarity between the data from the two sources.We have then plotted this output in Fig. 8. From the figure, it is clearly visible that the mean percentage error has the maximum correlation for both knee and hip.Hence, this error is suitable for training the neural network for finding the error pattern recognition.

VI. The Artificial Neural Network Approach
With the mean percentage error identified as the suitable output, we are now in a position to create a back-propagation based neural network architecture which can be used to compensate for the instability in the analytical model.This learning ability of our hybrid model can enhance the adapt-ability of the model and will also extend the operating zoneof the entire model [19], [20], [21], [22], [23].The most well-known methodologies to machine learning are simulated neural systems, also knownas hereditary calculations.
In Artificial Neural Network, a Perceptron is a classification of the input of a data into one of a few conceivable outputs i.e., a characterization calculation that makes its expectations focused around a direct indicator capacity consolidating a set of weights with the feature vector.The perceptron algorithm has been used since a long time and has its roots in the 1950s.A multilayer perceptron is a feed forward neural system with one or more concealed layers.Regularly, the system comprises of an Input layer of source neurons, no less than one or more Hidden or shrouded layers of computational neurons, and an Output layer of computational neurons.The data signs are engendered in a forward bearing on a layer-by-layer premise.The implementation of multilayer artificial neural network in the project for the biped locomotive simulation is done via 4 layers: Input layer, Two hidden layers, and Output layer.

A. Use of Neural Network in our Model
In this proposed model, we have used a fully connected neural network with 2 hidden layers, 1 input layer and 1 output layer as shown in Fig. 9.We are training the neural net using back-propagation algorithm [1].The following shows the various input and output parameters and algorithm of the neural network.

VII. Development of Hybrid Model
After finding the error pattern, we have coupled the base model with a neural net based extension so as to generate the passive walker's stable joint trajectories as accurate as possible in a computationally faster manner as base trajectories and then it is compensated with the error pattern as developed in the earlier section.This integrated system is being called as a hybrid model.
Using the neural network based extension, the error patterns corresponding to the single individual has been obtained (after normalization) as shown in the following chart described in Fig. 11.

VIII. Results
In this section we have compared the limit cycle behavior between base model alone and the hybrid model (base model together with the neural net) for the passive walking along different angles of inclination as shown in Fig. 12 and Fig. 13.We have identified that the base model is unstable below 4 degrees and above 10 degrees of inclination.For comparison, we have used walking along 4 degrees of inclination.As mentioned in the earlier section, the limit cycle behaviour clearly shows that the system is unstable at 4 degrees of inclination.Now as seen from Fig. 12(a) and 13(a), the limit cycle behavior shows that the same system is stable at 4 degrees angle of inclination.This comparison clearly shows while the base model fails to justify stable passive walking by a real subject under certain conditions, the hybrid model fully supports the same.The confusion matrix obtained by the experiment isalso given in Fig. 14

IX. Conclusions
Modelling passive biped walking along a slope analytically is extremely difficult due to high degrees of non-linearity, high dimensionality, under actuation (in swing phase), overactuation (in stance phase), etc.That is why we hardly see any biped humanoid robot capable of exploiting passive walking, which we human being do it naturally through learning.However passive walking capability is extremely needed so as to reduce the energy spent for the robot for walking.In this research we have shown how to exploit the limited capability of analytical model and extend it through a fusion between the analytical model and the neural network based connectionist model, which may help a biped walker to walk like human exploiting passivity thereby, reducing the energy spent from walking.We have shown that the trajectories obtained as an output of the hybrid model can be effectively used as a reference trajectory for execution level control.

Fig. 6 .
Fig. 6.Output of the application in X-Y plane.

( 3 )
Where Di = Desired data at a particular time, Ai = Actual data at a particular time and N = number of data.Mean Percentage Error:It is the mean of the percentage error by which desire value differ by actual value.

( 4 )
Error: It is the average of the absolute difference between the desired data and the actual one.

Algorithm 1 :Fig. 9 .
Fig. 9. Architecture of the neural network used in the model.The neural network has been used to capture the pattern of the

Fig. 10 .
Fig. 10.Convergence of the sum of squared errors.

Fig. 11 .
Fig. 11.Data used for training the ANN for error pattern matching. .