A Neuromechanical Controller of a Hexapod Robot for Walking on Sponge, Gravel and Snow Surfaces

Physiological studies suggest that the integration of neural circuits and biomechanics (e.g., muscles) is a key for animals to achieve robust and efficient locomotion over challenging surfaces. Inspired by these studies, we present a neurome-chanical controller of a hexapod robot for walking on soft elastic and loose surfaces. It consists of a modular neural network (MNN) and virtual agonist-antagonist mechanisms (VAAM, i.e., a muscle model). The MNN coordinates 18 joints and generates basic locomotion while variable joint compliance for walking on different surfaces is achieved by the VAAM. The changeable compliance of each joint does not depend on physical compliant mechanisms or joint torque sensing. Instead, the compliance is altered by two internal parameters of the VAAM. The performance of the controller is tested on a physical hexapod robot for walking on soft elastic (e.g., sponge) and loose (e.g., gravel and snow) surfaces. The experimental results show that the controller enables the hexapod robot to achieve variably compliant leg behaviors, thereby leading to more energy-efficient locomotion on different surfaces. In addition, a finding of the experiments complies with the finding of physiological experiments on cockroach locomotion on soft elastic surfaces.


Introduction
There are increasing demands for robots to walk on a series of diverse terrains (Ozcan et al., 2010;Qian et al., 2012).However, few robots can walk on soft elastic (e.g., sponge) and loose (e.g., gravel and snow) surfaces.This is because traversing these surfaces efficiently requires variable compliance of legs (Spence, 2011;Bermudez et al., 2012).Traditionally, the variable compliance of legged robots can be achieved by passive compliance mechanisms (Ham et al., 2009) and/or active compliance control (Görner and Hirzinger, 2010).For example, by using active compliance control with joint torque feedback, a hydraulically actuated quadruped robot (i.e., HyQ, 90 kg) has been developed for moving over terrains (Boaventura et al., 2012).Nevertheless, the complex mechanical and sensing components of the HyQ robot greatly increase its size and mass, thereby not fitting for developing small legged robots.Yet a small six-legged robot (i.e., EduBot, 3 kg) has been designed by using physically passive variable compliant legs (Galloway et al., 2011).The experimental results show that stiffer legs allow its faster locomotion on soft surfaces.
In contrast to the robot experimental results, owing to energy efficiency, biological study has shown that cockroaches (i.e., Blaberus discoidalis) use their softer legs on soft surfaces (Spence et al., 2010;Spence, 2011).This finding reveals a neuromehcanical control strategy of hexapod locomotion on soft surfaces.In fact, the strategy is not the result of a single component rather interactions between a nervous system, a musculoskeletal system and the environment.Inspired by this, the work here proposes a novel neuromechanical controller of a hexapod robot for walking on soft elastic and loose surfaces.The neuromechanical controller consists of a modular neural network (MNN) coordinating leg movement and virtual agonist-antagonist mechanisms (VAAM) changing the compliance of legs.The changeable compliance is simply achieved by altering two internal parameters of the VAAM without physical passive compliant mechanisms (Ham et al., 2009) or joint torque sensing (Görner and Hirzinger, 2010).Employing this controller allows the robot to walk on different surfaces with energy efficiency.Besides, a finding of robot walking complies with the finding of physiological experiments on cockroach locomotion on soft elastic surfaces (Spence et al., 2010;Spence, 2011).

Neuromechanical Controller of a Hexapod Robot
The experimental robot is a hexapod robot (5.4 kg) (see Fig. 1 (a)).Each three-jointed leg has a TC (Thoraco Coxal) joint allowing the motions of forward and backward, a CTr (Coxa Trochanteral) joint allowing the motions of elevation and depression, and a FTi (Femur Tibia) joint allowing the motions of extension and flexion (see Fig. 1 (b)).Each joint is physically driven by a standard servo motor.There is a force sensor used for detecting the analog signal at each leg (see f c 1−6 in Fig. 1 (a)).A current sensor installed inside the body of the hexapod robot is used to detect electrical current used for all motors and sensors of the hexapod robot.
For more details of the hexapod robot, we refer to (Manoonpong et al., 2013).
where m denotes the number of units, B i is an internal bias term (i.e., stationary input) to neuron i, W ij is the synaptic strength of the connection from neuron j to neuron i.
where S ∈ R [0,0.18] is the input of the modular neural network, which determines walking patterns of the hexapod robot.The speed of its leg motion increases with increasing S. Here, we set S = 0.04 resulting in slow walking behavior, which leads to stable and energy-efficient locomotion on non-flat surfaces (Manoonpong et al., 2013).
The PSM is a generic feed-forward network consisting of three hierarchical layers with ten hidden neurons (i.e., H 3 − H 12 ).The outputs of the PSM are projected to the FTi (i.e., F (R, L) (1,2,3) ) and CTr (i.e., C(R, L) (1,2,3) ) motor neurons (see Fig. 2 (d)), as well as the neurons H 13 and H 14 of the two velocity regulating modules (VRMs, see Fig. 2 (c)).The two VRMs are feed-forward networks projecting their outputs to the TC motor neurons T (R, L) (1,2,3) (see Fig. 2 (d)).In the neuromechanical controller, the outputs N 1−18 of the motor neurons are the neural activations of 18 joints of the hexapod robot.N 1−18 enable its legs to perform fast swing and slow stance phases (see Fig. 3).Delays λ L and λ between the outputs of motor neurons are fixed (see Fig. 2 (d)).For more details of the MNN, we refer to our previous work (Manoonpong et al., 2013).However, the previous work did not consist of muscle-like mechanisms (e.g., virtual agonist-antagonist mechanism (VAAM)).Including the VAAM allows the hexapod robot to achieve more energy-efficient locomotion (described below).

Virtual Agonist-antagonist Mechanism (VAAM)
The virtual agonist-antagonist mechanism (VAAM) consists of a pair of agonist and antagonist mechanisms (see Fig. 4(a)).They produce active and passive forces by its contractile and parallel elements (CEs and PEs, see Fig. 4(b)).In Fig. 4(a), the physical joint is driven by a pair of the virtual agonist-antagonist mechanism (VAAM, i.e., M 1 and M 2).'Virtual' means that the physical joint, physically driven by a standard servo motor, imitates muscle-like behaviors as if it were driven by a pair of agonist and antagonist muscles.The joint actuation relies on the CEs while the PEs govern joint compliance.
The parallel elements (i.e., PEs) are modelled as spring- The matrix [f P 1 , f P 2 ] T of passive forces created by P E (1,2) is the sum of two Hadamard products: where • L 2×1 is the matrix of displacements of P E (1,2) , i.e., L 2×1 = [l P 1 − l 0 , l P 2 − l 0 ] T .l 0 is the initial length of P E (1,2) , which is set to: l 0 = 0.085; The active forces produced by the CEs are approximated by the product of the neural activation N j and the activation intensities i (1,2) .The matrix T of active forces generated by CE (1,2) are represented by (see Fig. 4 (b)) : where It is one of the outputs N 1−18 of the MNN (see Fig. 2 (d)); The total forces f T 1 and f T 2 are the sum of the active and passive forces produced by M and M 2. They are given by (derived from Eqs.( 3) and ( 4)): The VAAM consists of contractile (i.e., CE (1,2) ) and parallel (i.e., P E (1,2) ) elements for producing active and passive forces.
The antagonist mechanism M 2 (see Fig. 4 (a)) resists the extension of the joint angle θ when receiving the force f ext , which is sensed by a force sensor.Simultaneously, the agonist mechanism M 1 (see Fig. 4 (a)) produces opposing force against M 2. Therefore, the directions of f T 1 and f ext are counter-clockwise when the direction of f T 2 is clockwise.
Their torques acting on the joint P (see Fig. 4 (a)) are represented by: where r is the radius of the joint P .f ⊥ is the amount of f ext directed perpendicularly to the position of the joint P .L is the length of the shank of the joint P .Note that the direction of torque τ (f T 2 ) is opposite to those of τ (f T 1 ) and τ (f ext ).We consider the torque pointing outward from the page as the positive torque (e.g., τ (f T 1 ) and τ (f ext )).We apply Euler's laws of motion to the rotation of the joint P (see Fig. 4(a)).The net torque τ acting on the joint P is equal to the product of its moment of inertia I and angular acceleration θ.It is given by: Derived by Eq.( 10) (see details in Appendix A), the motion equation of the joint P is given by: ]. (11) Equation ( 11) governs θ of the joint P driven by the VAAM that is activated by the output N j (j ∈ Z [1,18] ) of the MNN.

Neuromechanical Control Strategies for a Hexapod Robot
The outputs O 1−18 ∈ R [−1,1] (see Fig. 5) of the neuromechanical controller are linearly scaled and transmitted to control the position of the standard servo motors driving the 18 joints of the hexapod robot.Different control strategies are applied in swing and stance phases.

Swing phase
When a leg is in swing phase (i.e., , see Fig. 5 (a)), the outputs O (i,i+6,i+12) of its TC, CTr and FTi joints receive motor neuron signals N (i,i+6,i+12) of the MNN as their inputs.They satisfy:

Stance phase
Since there is only detection for vertical foot force in the leg, the TC joint allowing only horizontal motions is not effected by a pair of the PEs of the VAAM.Moreover,  Therefore, the control strategy of its three-jointed legs during stance phase is as follows: each TC joint (i.e., proximal joint) is purely controlled by a pair of the CEs of the VAAM (i.e., pure actuation), each CTr joint (i.e., intermediate joint) is governed by a pair of the CEs and PEs of the VAAM (the combination of actuation and compliance), and each FTi joint (i.e., distal joint) is driven by a pair of the PEs (i.e., P E 1 and P E 2 ) of the VAAM (i.e., pure compliance) (see Fig. 7).The control strategy is also comparable to the findings revealed by three-jointed leg locomotion of the BigDog-inspired study (Lee et al., 2008;Raibert, 2008).FTi joints : Each FTi joint is only driven by P E (1,2) of the VAAM (see Fig. 7 (a)).Therefore, their neural activations N 6×1 are equal to zero, where N 6×1 = [N 13 , N 14 , . . ., N 18 ] T .In addition, the forces f ext 1−6 directly result in the extension and flexion of the FTi joints.Therefore, the matrix τ F T i 6×1 of torques acting on the FTi joints is given by (derived from Eq.( 9)): where Substituting Eqs.( 13) and ( 14) into Eq.(11), θ1 6×1 is the sum of the Hadamard products: The angles θ1 m,1 (m ∈ Z [1,6] ) of the FTi joints can be linearly transformed into their outputs O j (see Fig. 5).O j are given by (j ∈ Z [13,18] ): O j = 0.92θ1 j−12,1 + 0.12.1−6 needs to be approximated, since there are no torque sensors at the CTr joint.Therefore, the matrix τ CT r 6×1 of the torques acting on the CTr joints is given by: where Substituting Eq.( 17) to Eq.( 11), the matrix θ2 6×1 of the CTr angles is the sum of the Hadamard products: where ) of the CTr joints are linearly transformed into their outputs O j (see Fig. 5).O j are given by (j ∈ Z [7,12] ):

Experimental Results
The proposed neuromechanical and pure neural controllers were implemented on the hexapod robot for walking on soft elastic (i.e., sponge) and loose (i.e., gravel and snow) surfaces.Changing the matrices of the stiffness coefficients of the FTi (i.e., K1 6×1 in Eq.( 15)) and CTr (i.e., K2 6×1 in Eq.( 18)) joints enables the legs of the hexapod robot to show variable compliance (see notations in Appendix B).
The pure neural controller (PNC) uses the outputs of the motor neurons of the MNN to directly drive the 18 joints of the robot.The computations of its outputs follow Eq.( 12) for stance and swing phases.The free parameters of the proposed neuromechanical controller are chosen based on trial and error.The parameters of three setups allow the hexapod robot to achieve coordinated and stable locomotion, which have been tested in a physical simulator (i.e., lpzrobots simulator (Der and Martius, 2012)).For each setup, the runs over each surface were repeated until ten successful runs1 were obtained.For a successful run, the power consumption P i and forward velocity v i are given by: where 5 Volts is the input voltage of the electrical board and motors of the hexpod robot.A i is an average electrical current measured using a current sensor.∆DIS i is the forward displacement during a time interval ∆t.The performance of the runs was measured by "specific resistance" ε i (Gregorio et al., 1997;Saranli et al., 2001).ε i is determined by power consumption P i and forward velocity v i : where mg is the weight of the hexapod robot, i.e., mg = 52.974N. Lower ε avg corresponds to more energy-efficient walking, which is desirable.

Sponge surface
The interval ∆t over one run was 27 s.A 1.5 m long sponge (i.e., three pieces of sponge glued together) was used as a soft elastic surface.The experiment result is shown in Table 1 and Fig. 8.The hexapod robot that was controlled by the neuromechanical controllers with the low (i.e., LSC) or high (i.e., HSC) stiffness consumed less energy than controlled by the pure neural controller (i.e., PNC).This is because LSC and HSC allow for variable joint compliance of the hexapod robot resulting in leg adaptations to sponge deformations (see Fig. 8 (a)).The experimental video can be seen at the link http://www.youtube.com/watch?v=vEqylwMXfJE .Interestingly, LSC shows the lowest average specific resistance with 19.7.This experimental result shows that softer legs (i.e., LSC setup) allow the hexapod robot to achieve more energy-efficient locomotion, compared to stiffer legs (i.e., HSC setup).The finding complies with a finding of physiological experiments on cockroach locomotion.Owing to energy efficiency, cockroaches (i.e., Blaberus discoidalis) also use their softer legs on soft elastic surfaces (Spence et al., 2010;Spence, 2011).

Gravel and snow surfaces
The interval ∆t over one run was 60 s.Gravel surface is a bed (i.e., the length is 2.4 m) of loosely packed gravels (i.e., gravel diameter φ : 5 mm -60 mm).The experimental result is shown in Table 1 and Fig. 9. HSC and LSC enable the hexapod robot to adapt its joint motions to different sizes of gravels (see Joint motion I and II in Fig. 9 (a)), while PNC does not adapt the joint motions leading to difficulty of locomotion.In addition, the average specific resistance was lowest for HSC (i.e., ε avg = 17.2), thereby leading to more energy-efficient locomotion.This is because HSC allows the legs of the hexapod robot to penetrate more deeply into gravel surface (see control signals O 7 in Fig. 9 (a)).The experimental video of walking on gravel surface can be seen at http://www.youtube.com/watch?v=f2G4UzUQ6Iw .In addition to gravel surface, we also tested HSC, LSC and PNC for walking on another loose surface (i.e., snow), which has a thickness of 8cm.The experimental result also shows that HSC allows the hexapod robot to achieve more energy-efficient locomotion (see average specific resistance Table 1), compared to LSC.Note that we did not calculate average specific resistance of the hexapod robot controlled by PNC, since it got stuck in the snow.The experimental video of walking on snow surface can be seen at https://www.youtube.com/watch?v=OkZiVNeQdCA .

Conclusion and Future Work
We implemented a neuromechanical controller on a hexapod robot for walking on sponge, gravel and snow surfaces.The controller coordinates 18 joints, generates basic locomotion, and allows for simply changing compliance of its legs for walking on the different surfaces.Due to the changeable compliance, the robot can achieve more energy-efficient locomotion (i.e., lower specific resistance) on different surfaces.Softer legs (i.e., LSC setup) do better in locomotion on a soft elastic surface (i.e., sponge), while stiffer legs (i.e., HSC setup) are better for locomotion on loose surfaces (i.e., gravels and snow).In addition, on gravel surface, the specific resistance of the robot is 17.2 when it is controlled by the neuromechanical controller with HSC presented here.In contrast, its specific resistance increases to 56.63 when it is controlled by an adaptive neural locomotion controller presented in our previous work (Manoonpong et al., 2013), which does not have muscle-like mechanisms (i.e., virtual agonist-antagonist mechanism (VAAM)).
Central properties of the VAAM of our neuromechanical controller are: (1) it enables robot legs to simply change their compliance without the requirement of additional physically compliant mechanisms (Ham et al., 2009) or joint torque sensing (Görner and Hirzinger, 2010) and (2) it allows a hexapod robot to adapt its legs to deal with challenging surfaces (i.e., sponge, gravel and snow).In future work, we plan to compare the proposed neuromechanical controller with other adaptive leg controllers (e.g., forward model (Manoonpong et al., 2013)) in different surfaces (e.g., snow).And we will also implement an adaptive mechanism for automatically adjusting stiffness coefficients of the VAAM with respect to different walking speeds or gaits.

Figure 4 :
Figure 4: Virtual agonist-antagonist mechanism (VAAM) for joint control interacting with the ground surface.(a) A physical joint is driven by a pair of the VAAM with the lengths L 1 and L 2 (i.e., M 1 and M 2).The interaction results in the force f ext , which drives the joint P with the radius r via the shank with the length L. f ext is sensed by a force sensor (i.e., O), and f ⊥ is the amount of f ext directed perpendicularly to the position of the joint P .(b)The VAAM consists of contractile (i.e., CE (1,2) ) and parallel (i.e., P E (1,2) ) elements for producing active and passive forces.

Figure 6 :
Figure 6: Two control setups for the FTi joint tested in a physical simulator.(a) Snapshot of stable walking of the hexapod robot at 15 s.(b) Vertical position of its body.(c) Snapshot of unstable walking of the hexapod robot at 15 s.(d) Vertical position of its body.

Figure 7 :
Figure 7: Control framework for a three-jointed leg of the hexapod robot in stance phase.(a) The three-jointed legs take the strategy of directional actuation and compliance (see text for details).(b) The control strategy for the leg.The function of compliance intensifies from the TC to FTi joints.

Figure 8 :
Figure 8: Comparisons of HSC, LSC, and PNC for walking on sponge surface.(a) Control signals O 7 and O 13 for the CTr and FTi joints of the right front leg.There are seven stance and six swing phases between 5 s and 15 s.(b) A series of photos shows hexapod robot walking controlled by LSC.

Figure 9 :
Figure 9: Comparisons of HSC, LSC and PNC for walking on gravel surface.(a) Control signals O 7 and O 13 for the CTr and FTi joints of the right front leg.There are six stance and five swing phases between 15 s and 25 s.(b) A series of photos shows hexapod robot walking controlled by HSC.

Table 1 :
Average specific resistances ε avg with standard deviations of the hexapod robot walking on sponge, gravel and