Human-like compliant locomotion: state of the art of robotic implementations

This review paper provides a synthetic yet critical overview of the key biomechanical principles of human bipedal walking and their current implementation in robotic platforms. We describe the functional role of human joints, addressing in particular the relevance of the compliant properties of the different degrees of freedom throughout the gait cycle. We focused on three basic functional units involved in locomotion, i.e. the ankle–foot complex, the knee, and the hip–pelvis complex, and their relevance to whole-body performance. We present an extensive review of the current implementations of these mechanisms into robotic platforms, discussing their potentialities and limitations from the functional and energetic perspectives. We specifically targeted humanoid robots, but also revised evidence from the field of lower-limb prosthetics, which presents innovative solutions still unexploited in the current humanoids. Finally, we identified the main critical aspects of the process of translating human principles into actual machines, providing a number of relevant challenges that should be addressed in future research.


Introduction
Achieving stable and efficient walking is a crucial goal in the design of humanoid robots. In humans, walking emerges from the combination of several mechanisms, which include neural, mechanical and morphological aspects. As a result, humans show very robust, versatile and energy efficient functional abilities in a vast range of locomotion conditions. The process of transferring such principles into robotic platforms is not trivial, since the complex interplay between the sensorimotor mechanisms involved in human walking is still far from being fully understood (Alexander 1992, Iida et al 2008, Ijspeert 2014. One of the key properties of human and biological systems is biomechanical compliance. In mammals, compliance results from the visco-elastic properties of muscle fibers and the series-elastic tendon structures, and can be modulated at the muscle and joint level through the activation of the agonist and/or antagonistic muscles (Sartori et al 2013. It is known that compliant behavior is particularly relevant in locomotion in order to achieve natural patterns, adapt to different terrains, and lower energetic costs (Farley and González 1996, Dickinson et al 2000, Endo and Herr 2014. Two major and currently unclear points are how compliance is modulated along the different joints of the human body, and how this modulation affects the global stability in different environmental conditions (Qiao and Jindrich 2015). The answers to these questions are crucial for their effective implementation in humanoid robotic systems, which could be particularly beneficial for the robots operating in unstructured and unknown environments.
This paper aims to provide a complete, yet synthetic, review of the key biomechanical mechanisms of human bipedal locomotion. It follows a translational approach that integrates evidence from robotics with the experimental analysis as well as biomechanical modeling and simulations of human locomotion. Section 2 describes the functional role of human joints throughout a gait cycle, focusing especially on the relevance of the different degrees of freedom (DoFs) and the compliant properties of muscle actions. Section 3 presents the current implementations of these mechanisms into robotic platforms. We specifically targeted whole-body humanoids, but also included evidence from the prosthetic systems, when relevant. In section 4 we discuss and compare the evidence collected from both human and robotic scenarios, with the aim of providing useful key principles for humanlike design. We conclude this review with a number of relevant challenges that should be addressed in future research in the field of compliant humanoids.

Human locomotion
In human walking, the gait cycle can be divided in two main phases, i.e. the stance phase, in which the foot is in contact with the ground, and the swing phase, during which the foot is airborne (see figure 1). Each of these phases has a different functional goal. Stance ensures body progression while maintaining upright posture, whereas swing is used to advance the leg and to prepare for the next step. The temporal distribution of the two phases is approximately 60% for stance and 40% for swing. Considering the relative contact of the two feet with the ground, the stance phase can be further segmented into more specific sub-phases of walking, which are: loading response, mid stance, terminal stance, pre swing, initial swing, mid swing, and terminal swing, defined based on specific kinematic events, as shown in figure 1.
Due to their intrinsic visco-elastic properties, muscles produce a compliant behavior at the joint level. Compliance, i.e. the flexibility of physical structures in response to an external force, is beneficial under different conditions. Typical advantages of compliance are the possibility of rapidly and efficiently storing and releasing energy (e.g. bouncing motion), or reacting instantaneously to sudden impacts, with positive effects on stability and thereby safety. A recent modeling study by (Geyer et al 2006) demonstrated the importance of leg compliance for reproducing typical human-like walking patterns and its corresponding ground reaction forces. However, the extent of the functional benefits of compliance during locomotion is still an open question.
In this section we analyze the compliant properties of human joints during locomotion, in order to identify how compliance is implemented across the different DoFs, and how it affects the whole-body performance. This issue is particularly relevant during humanoid design, due to the direct implications to the complexity of mechatronic hardware and control paradigms. In order to facilitate the identification of the relevant aspects, we have grouped the human joints in three main 'functional units': the ankle-foot complex, the knee, and the hip-pelvis complex, and classified the biomechanical states of the joints into four categories, characterized by the actions of the internal forces produced by muscle activations : • Motive state, when the internal (muscle-tendon) forces are used to accelerate the joint motion, thus generating positive work. This is achieved in human joints by concentric activity of agonist muscles; • Resistive state, when the internal forces are used to decelerate the joint motion against external forces or inertia, thus producing negative work. In Figure 1. Phases of human locomotion and related biomechanical events. According to the reciprocal contact of one leg and the ground, a gait cycle is composed by stance, where the foot is on the ground, and swing. Stance can be subdivided into four sub-phases: (i) loading response, in which both limbs are in contact with the ground, (ii) mid stance, starting at contralateral toe off, (iii) terminal stance, starting at heel rise and (iii) pre swing, in which, again, both limb are in contact with the ground. With respect to the timing, the gross normal distribution of the floor contact periods is 60% for stance and 40% for swing. Timing for the phases of stance is 10% for each double stance interval (loading response and pre swing) and 40% for single limb support. The precise duration of these gait cycle intervals varies with the person's walking velocity: increasing velocity increases total stance and swing times, while lengthens single stance and shortens the two double stance intervals. Information based on (Perry 1992). humans, this is achieved by eccentric contraction of antagonist muscles; • Stabilizing state, when the internal forces are used to counteract external forces in order to maintain the joint in a fixed position, producing low levels of work. This is achieved in humans by the isometric contraction of agonist and/or antagonist muscles; • Passive state, when no internal forces are produced as a result of neural recruitment, leaving the joint free to move under the effect of inertial and gravitational forces as well as intrinsic muscle fiber, tendon, and ligament constraints. In this condition, muscles are not activated.

The ankle-foot complex
The action of the ankle and foot, taken as a whole, has been compared to the rolling mechanism of a wheel, due to the relative motion of the center of pressure (CoP) between the foot and the ground, which follows a circular trajectory (see figure 2). This wheel-like mechanism has been defined geometrically with the so-called roll-over shapes (ROSs) (Hansen et al 2004). Several efforts have been devoted in the literature to determine the effective geometry (e.g. shape, length) of the ROS. It was shown that the ROS is circular, with an equivalent radius of 0.3 times the leg length (McGeer 1990), and this is maintained across different walking speeds and biomechanical conditions (Hansen et al 2004, Hansen and Childress 2005). It is generally believed that the ROS reflects the mechanisms humans use in order to: (i) transform the passive pendulum dynamics of lower limbs into body progression (McGeer 1990, Gard andChildress 2001); (ii) smoothen the transitions between opposite stance legs (Adamczyk et al 2006, Adamczyk andKuo 2013), thus having direct implications on energy expenditure. The energetically optimal values of both length and radius found in experimental trials were comparable to the ROS parameters characterizing the human foot, confirming that a human is the golden standard for bipedal walking efficiency (Hansen et al 2004).
This wheel-like mechanism results from the compliant interaction between the foot-ankle complex and the ground during the stance phase. This compliant behavior can be explained by three pivotal mechanisms: the heel rocker, the ankle rocker and the forefoot rocker (see figure 3(B)). The heel rocker is the mechanism by which the foot rolls around the heel due to the momentum generated by the body falling. This movement happens in the first 10% of the gait cycle (loading response), just after the heel contact and until the foot is in full plantar contact with the ground. The activation of dorsiflexor muscles (e.g. tibialis anterior, see figure 3(A)) creates a resistive momentum that transforms the energy of falling into forward progression of the lower leg around the heel, at the same time decelerating the plantar flexion to control the lowering of the forefoot to the floor (Hansen et al 2004). After plantar contact, the ankle rocker is the mechanism by which the shank rotates around the ankle, driven by the forward momentum, while the plantar flexor muscles (soleus and gastrocnemius, see figure 3(A)) act eccentrically to decelerate this movement. This mechanism (figure 3) also contributes to energy storing, which results from the stretching of the Achilles tendon (Lichtwark and Wilson 2005). The forefoot rocker is the mechanism by which the toes flex under the forward momentum of the body (figures 3(A) and (E)), occurring during terminal stance. During this period, the body moves forward, beyond the base of support, entering a brief free-falling phase. In this phase, the plantar flexors are activated isometrically to restrain the ankle joint against the dorsiflexor torque of the ground reaction forces, making the shank follow the body progression. At the same time, the Achilles tendon is shortened, releasing the energy previously stored. This energy storingreleasing mechanism has been shown to improve the energetic costs, during walking (Zelik et al 2014) and running (Lichtwark and Wilson 2005). During the pre-swing phase, the plantarflexors are activated in order to accelerate the tibia forward (push off) while the body weight is transferred to the contralateral side (Perry 1992), triggering the start of the swing movement. Figure 2. The wheel-like mechanism of the foot and the resulting roll-over shapes (ROS). The figure shows three snapshots of the stance phase, as seen from an observer placed on the foot. The relative motion of the ground and the foot follows a circular trajectory, as it would be if the foot was substituted by a wheel. (Data from Hansen andChildress 2005, Hansen et al 2004.) During stance, the rocker mechanisms previously described are accompanied by the compliant action of the foot (see figure 3(E)), which contributes to shock absorption and arch stabilization during foot-ground interaction. The human foot is a very complex structure composed of numerous bones, muscles, and ligaments. As a simplification, three main joints can be identified: the subtalar, midtarsal and metatarsophalangeal joints (see figure 3(A)). During the loading response phase, the transfer of the body weight onto the heel produces subtalar pronation (McPoil and Knecht 1985), which is decelerated by the activation of the dorsiflexors (tibialis anterior). This actions represents the first shock absorption mechanisms in the gait cycle (Perry 1992). During mid-stance, the internal foot muscles implement another shock absorption mechanism represented by the flattening and recovery of the foot arch, through the midtarsal joint (Ker et al 1987). During terminal stance, the forefoot enters in contact with the ground by means of the metatarsophalangeal joint, whose motion is then decelerated by toe flexors in order to produce a stable weight-bearing area during the forefoot rocker (Hughes et al 1990). In this phase, the toe flexion also triggers and (E) Schematic view of the functional role of the ankle and foot. Black dotted arrows show the direction of movement, while colored arrows indicate the direction of net joint torques (red, with a '+' symbol, for motive; blue, with a '−' symbol, for resistive torques). Joints of interest are indicated as well, where the red color indicates joint stabilization. (C) Joint velocity profile and muscle activations. The joint velocity (black line) gives an immediate indication on the direction of movement, i.e. dorsiflexing (positive velocity) or plantarflexing (negative velocity). Activity of dorsiflexor and plantarflexor muscles is depicted in the same graph (data taken from Winter 1991). This information gives a qualitative indication on the internal joint torques that may not be visible from external power (e.g. if co-contraction is present), therefore helping to distinguish between stabilizing and passive role of the joint. (D) Joint power, obtained by the product between net torque (not shown) and velocity. It does not take into account neither internal opposing force, or forces generated in absence of motion. The plots for velocity, power and muscle activations are given in arbitrary units. the windlass mechanism, which increases the rigidity of the arch by locking the midtarsal joint (Caravaggi et al 2009).
During the swing phase, the main goal of the footankle complex is to ensure foot clearance, avoiding collision with the ground. To this aim, the tibialis anterior muscles are activated to generate a dorsiflexion during initial and mid-swing, opposing the plantarflexion moment due to gravity. The internal foot joints do not participate actively during swing phase, since no relevant forces are applied to the foot.

The knee
Among all the joints involved in locomotion, the knee exhibits the highest range of motion (approx. 70°) (Winter 2009). It represents a crucial site of action of most biarticular leg muscles (see figure 4), and plays a central role for the energy transfer between the knee and hip and ankle. Sagittal knee motion is essential for achieving the major bipedal functions, including weight acceptance, weight support and forward propulsion. Furthermore, it is the main responsible for foot clearance during leg swing and knee stability during weight bearing (Winter 2009). Apart from sagittal motion, also transversal and frontal knee movements are present, but they have a minor role in locomotion, compared to the sagittal motion (Perry 1992). For this reason, in this review, we will only focus on sagittal knee motion.
The human knee assumes different states throughout the gait cycle. A motive state is present during mid stance (see figures 4(C) and (D)), when the quadriceps (B) Schematic view of the functional role of the knee. Black dotted arrows show the direction of movement, whereas colored arrows indicate the direction of net joint torques (red, with a '+' symbol, for motive; blue, with a '−' symbol, for resistive torques). Joints of interest are indicated as well, where the red color indicates joint stabilization. (C) Joint velocity profile and muscle activations. The joint velocity (black line) gives an immediate indication on the direction of movement, i.e. extending (positive velocity) or flexing (negative velocity). Activity of knee flexors and extensors muscles is given in the same graph (data taken from Winter 1991). This information gives a qualitative indication on the internal joint torques that may not be visible from external power (e.g. if cocontraction is present), therefore helping to distinguish between stabilizing and passive role of the joint. (D) Joint power, obtained by the product between net torque (not shown) and velocity. It does not take into account neither internal opposing forces, or forces generated in absence of motion. The plots for velocity, power and muscle activations are given in arbitrary units. muscles (biceps femoris and vasti) are activated to extend the knee against gravity. The resistive action of the knee muscles is present in several phases of the gait cycle, confirming the important role of the knee as energy dissipater (Perry 1992): (i) in the loading response phase, the eccentric activity of knee extensors decelerates the knee flexion generated by the ground reaction forces, (ii) in terminal stance, the knee flexors decelerate the knee extension produced by the previous motive mid-stance, (iii) in the pre-swing phase, the knee extensors (rectus femoris) decelerate the otherwise uncontrolled knee flexion generated indirectly by the residual activity of the ankle plantarflexors combined with the loss of weight bearing, (iv) in initial swing, the knee extensor are used to counteract the rapid knee flexion caused indirectly by hip flexion (see also figure 5(B)), and (v) in terminal swing, the knee flexors decelerate the extension of the knee. A passive role of the knee is observed during mid-swing. In this phase, especially in the first half, the muscles spanning the knee reduce the joint stiffness to very low values, enabling the leg to accelerate under the effect of gravity and inertia.

The hip-pelvis complex
The role of the pelvis is twofold, i.e. maintaining the trunk (more precisely the HAT, i.e. head-arms-trunk complex) in the upright position, and ensuring the movement of the lower limbs for body progression (Perry 1992). The pelvis is connected with the lower limbs through the hip joints, and with the HAT through the spine, comprising multiple vertebral joints (see figure 5(A)). During walking the pelvis shows a complex three-dimensional continuous motion (Saunders et al 2005).
In order to maintain the HAT in the upright position, the pelvis contributes with two actions. During the loading response phase, a resistive action of hip abductors during the downward movement of approx. 4°, called hip drop, is used to dampen the impact of weight loading (Perry 1992) (see figure 5(E)). During the stance phase, the pelvis shows an anterior rotation of 4°. In order to facilitate the lower limb motion, the pelvis contributes with a transversal rotation of 10°e nhancing the leg swing movements. Pelvic rotation and frontal plane tilt contribute to the control of the trajectory of the center of mass (COM) of the whole body. According to the 'six determinants of gaits' theory (Saunders et al 1953), these pelvic movements, together with other mechanisms at knee, foot and hip, appear to be the main responsible for reducing the vertical excursions of the COM. Whether this flattening of the COM trajectory is adopted to reduce metabolic costs is still controversial (Kuo and Donelan 2010).
Hip sagittal movement is characterized by the second largest range of motion across human joints during walking (approx. 40°) (Eng and Winter 1995). In this plane, the hip assumes a motive state during pre-and initial swing (see figures 5(B) and (C)), when the hip flexors contract concentrically to pull the swinging limb upwards and forwards (Winter 1991). A resistive action is observed during terminal swing, when the hip extensors decelerate the limb forward momentum to prepare for ground contact (Kepple et al 1997). A passive role is prevalent during mid stance and midswing, in which the hip is passively extended under the gravitational and inertial forces, accelerating the thigh, with no relevant muscle contribution (Perry 1992). The hip assumes a stabilizing role during loading response, when the co-activation of extensors and flexors stiffen the joint to ensure postural stability during body weight transfer (Perry 1992).
The hip frontal movement has been related to balance and posture stability during walking, which is obtained through the control of the lateral foot placement by hip abductor/adductor muscles during swing (MacKinnon and . This action is important to maintain the COM within the lateral borders of the base of support (Shimba 1984, Winter 1995. The frontal movement of the hip is also used to control the movements of the pelvis, representing a relevant mechanisms for balancing the HAT in response to perturbations (Winter 1995).

Bipedal robots locomotion
The design of bipedal robots that can mimic human locomotion has been the focus of scientific research for nearly half a century. The 2015 DARPA robotics challenge showed the current capability of the most advanced humanoid robots in natural disaster situations (DeDonato et al 2015, Murphy 2015), but other platforms have been developed to take into account more biomimetic approaches. Two different approaches can be distinguished. Traditional methods have focused on mimicking the human body morphology by realizing complex humanoid structures that include high number of joints and DoF. These robots have been usually controlled by classic paradigms such as the zero-moment point (ZMP), a well-known method for the synthesis of prescribed joint trajectories by means of continuous tracking of stable footground contact (Vukobratović1975, Vukobratović and Borovac 2004). This approach demonstrated good stability in vast range of condition (Kaneko et al 2008, Galdeano and Chemori 2013, Yu et al 2014. Nevertheless, several functional and topological shortcomings can be observed, such as unnatural motions, high-energy costs, high computational demands, dependence on dynamic model, and rigidity. As opposed to this approach, there are the socalled 'passive dynamic walkers', exhibiting humanlike behavior without any actuator or control strategy. These approaches have focused mainly on exploiting the inherent dynamics of the limbs and the gravity pull in order to obtain very efficient and natural locomotion (McGeer 1990, Hobbelen and Wisse 2005, Wisse et al 2006. Contrary to ZMP-based humanoids, these bipeds maintained a very simple kinematic complexity, resulting in a low number of DoFs. Despite their efficiency, passive walkers are not versatile and of little practical use. Most of them are constrained to walk along an inclined sagittal plane and they are rather sensitive to external perturbations. Collins et al (2005) added simple actuation mechanisms to the passive dynamic walkers to overcome their dependency to gravity pull.
Recently, variable compliant actuation has been used to improve passive walking, showing good potential for generating stable and efficient locomotion . Activity of hip flexors and extensors muscles is given in the same graph (data taken from Winter 1991). This information gives a qualitative indication on the internal joint torques that may not be visible from external power (e.g. if co-contraction is present), therefore helping to distinguish between stabilizing and passive role of the joint. (D) Joint power, obtained by the product between net torque (not shown) and velocity. It does not take into account neither internal opposing forces, or forces generated in absence of motion. The plots for velocity, power and muscle activations are given in arbitrary units. The basic principle is to include spring or damping elements in series with stiff actuators (for a recent review see . These elements can include fixed intrinsic stiffness (series elastic actuators, SEA), or variable stiffness (variable stiffness actuators, VSA). Compliant actuators can store and release energy with no bandwidth limits, improving energy efficiency and stability during contacts with the environment. A similar approach is the use of parallel compliance, which allows for a less complex control design compared to series compliance while maintaining good energetic efficiency during walking (Yang et al 2007(Yang et al , 2008). An alternative solution to these mechanisms is the use of pneumatic (Hosoda et al 2008, Vanderborght et al 2008a, Narioka et al 2012 or hydraulic , Nelson et al 2012 actuation, since they have the advantage of having an intrinsic compliance. In addition, such actuators can be used in an antagonistic setup, mimicking biological muscles. Table 1 shows a summary of the most relevant robotic solutions that have included human-inspired compliant properties into their actuation systems. In order to facilitate a direct comparison with humans, we will refer to the three functional units presented in the previous section, namely ankle-foot complex, knee, and hip-pelvis complex.

The ankle-foot complex
The foot structure of humanoid robots has been a major topic of research in the past years, resulting in different approaches. The best performing and robust humanoid robots nowadays, e.g. Honda's Asimo (Hirose and Ogawa 2007) or some of the robots that participated at the DARPA challenge (DeDonato et al 2015, Murphy 2015), use a flat foot. Beside their good general performance, these robots produce walking patterns that deviate considerably from humans. In order to improve stability, adaptability to different surfaces, and obtain natural walking patterns, researchers have been trying to implement more human-like ankle-foot systems. In humanoids, the ankle joint is mostly actively controlled using a stiff actuator, and does not constitute a major issue in the ankle-foot complex. However, new robots have been exploring the usage of SEA actuators at the ankle. makes use of two parallel linear SEA at the ankles to cope with unstructured and unstable terrain and improve stability. Contrarily, in the prosthetic scenario, compliant ankle is gaining relevance (Svensson and Holmberg 2006, Eilenberg et al 2010, Mancinelli et al 2011, Cherelle et al 2013. In these systems, the ankle compliance modulation is realized by active actuation in combination with quasi-passive mechanisms. Quasi-passive mechanisms do not apply a direct motive force (Herr and Wilkenfeld 2003) but use controllable compliant elements such as variable dampers, springs, and clutches to change the stiffness of the joint. The use of quasi-passive devices in leg prostheses have led to lightweight and efficient designs (Endo et al 2006, Lapre 2014. As discussed in section 2.1, the human ankle-foot mechanism follows circular ROS. This is a characteristic that has been exploited in passive and limit cycle walkers (Hobbelen and Wisse 2008, Bhounsule et al 2014. These bipeds usually have arc-shaped feet attached to the shank since this allows the COP to evolve from the heel to the ankle, resulting in improved energy efficiency and more human-like walking characteristics (Wisse et al 2006, Yamane andTrutoiu 2009). However, the main disadvantage of arc-shaped feet is the instability on uneven terrain, the reduced friction in the transversal plane, and the fact that the biped cannot stand still in an upright position (Wisse et al 2006).
In order to combine the stability of the flat foot with the efficiency of the circular ROS, new foot structures including movable parts, such as toes, have been investigated. The presence of toes is a critical issue in most of ZMP-based robots, since during the forefoot rocker the contact point is reduced substantially, resulting in increased complexity in the control. . These implementations have demonstrated that actuation at toe-joint can reduce the maximum speed of knee joints, and increase walking speed and step length. Toes also contributes to more natural human-like walking and might also contribute to reductions in the energy consumption (Takahashi andKawamura 2002, Ouezdou et al 2005).  A study comparing different types of foot structure was performed by Ouezdou et al (2005). In this study they calculated the total energy consumption and normal contact force of four different simulated foot structures: (i) flat, (ii) non-actuated toe-joint, (iii) actuated toe-joint, and (iv) a hybrid design between non-actuated and actuated toe-joint (figure 6), called the 'flexible active foot'. It included an actuated toejoint and two non-actuated compliant joints, one at the heel and one at the tip of the toe. The results of the simulation showed that a non-actuated toe-joint was characterized by a human-like 'M' profile of ground reaction forces, which was not present during flat and rigidly actuated toe-joint configurations. On the other hand, the active toe-joint and the hybrid design structure reduced the energy consumption 25% compared to the flat structure, whereas the passive toe-joint only reduced the energy consumption 10%. Finally, the flexible active configuration was able to exploit the advantages of both the non-actuated and actuated toejoints, by reducing the energy consumption and improving the shape of the ground reaction force.
Some studies have integrated actuated toe-motion solutions into more complex foot systems. Davis and Caldwell (2010) developed a foot with under-actuated toes, hind, mid and fore foot sections. Also, silicone was used to mimic the properties of the biological tissue. With this structure they were able to adapt to different ground configurations, to absorb impact and store and release energy. However, this foot was tested in isolation, without considering how the rest of the body would affect its operation. Narioka et al (2012) developed a novel robotic foot with deformable arch for the pneumatic-based biped Pneumat-BB. This foot structure includes two actuated joints, one controlling the deformation of the foot arch and one for the toejoint. It also contains pressure sensors on the hind, mid and the fore foot. They were able to replicate to some extent the truss and windlass mechanism found in human walking (see section 2.1). However the robot was constrained to move only in the sagittal plane. Hashimoto et al (2010) improved the feet design of the WABIAN-2R. By mimicking the change in the elasticity of the medial longitudinal arch they were able to realize shock absorption function, windlass mechanism, and push-off function.
Other research focused on the compliant behavior of the entire foot, proposing more complex designs.
Most of them have been tested only in simulation, showing encouraging results. Kwon and Park (2012) proposed a foot based on 5 pillars (3 on the toes and 2 on the heel) that use springs to absorb the initial impact and to improve stability in all directions. Owaki et al (2011) proposed a non-actuated deformable foot using torsional springs on the toe, the foot arch and the ankle. Seo and Yi (2009) modeled a biomimetic foot using springs, mimicking the intrinsic foot muscles and tendons in order to automatically adapt to uneven terrain. Minakata et al (2008) proposed a shoe-like foot containing a simple array of springs at the bottom in order to save kinetic energy by allowing lateral motions. Li et al (2008) fabricated a sensor-integrated flexible foot with rubber pads and brushes to absorb the ground impact force.

The knee
During normal walking humans extend the knee joint during the mid and terminal stance phase. However, this is difficult to achieve on humanoids that use the ZMP method to control locomotion, since knee extension leads to a singular inverse kinematic solution (Okada et al 2003, Kurazume et al 2005, Ogura et al 2006, Handharu 2008. This is one of the main reasons why most humanoid robots walk with their knees flexed. Also, by maintaining the flexed knee the center of gravity (CoG) is easier to control, and thus the overall stability of the biped while walking is improved. The problem is that this strategy requires high-energy consumption and results in a gait pattern that is not human-like.
Researchers have tested several solutions in order to overcome the singularity problem. Adding extra pelvis movement has been proposed by Ogura et al (2006aOgura et al ( , 2006b, who included a 2-DoF waist joint to the WABIAN-2 robot, and by Miura et al (2011) who changed the height of the waist within the HRP-4C robot. Kurazume et al (2005) proposed a method to realize walking with proper knee extension by controlling the CoG according to the state of the ZMP controller. The knee was extended only when the ZMP was almost achieved and there was no need to control the ZMP position precisely. This approach has been tested in simulations and on the HOAP-1 robot platform. Handharu (2008) and Yoon et al (2010) demonstrated that adding a heel joint to the robot enabled a walking pattern with extended knees. They argued that this is a simpler solution compared to using the waist motion. With their method they were able to increase the support area during the double support phase, and achieve better performance than other robots in terms of joint torque and energy consumption.
For most of the above-mentioned robots stiff actuation was the preferred control choice. This requires high-energy consumption, since the actuators have to be actively energized for moving. An interesting approach was presented by Okada et al (2003), who developed a backlash clutch that is integrated in the knee of the humanoid robot. This allowed the robot to block the output of the motor and swing the leg passively due to the leg's inertia and gravity pull.
Compliant mechanisms have been also built for the knee. For example in the JenaWalker II biped (Seyfarth et al 2009) each leg consists of three segments with a passive knee and ankle, and four linear tension springs, representing muscles. Three of the four muscles are bi-articular and span the knee joint. However, due to limitations of the servomotor's torque, extending the knee could be realized only during jogging. Another example of a compliant knee mechanism is the COMAN robot . Using this robotic platform researchers have tested the stabilization of the robot exploiting the intrinsic and controlled compliance . Also this platform was used to investigate the reduction of energy consumption during walking (Kormushev et al 2011).

The hip-pelvis complex
Research in robotics has also investigated the advantages of increasing the mobility of the pelvis, as observed in humans. By adding an additional DoF at the waist joint, the pelvis could move in the frontal and transversal planes independently from trunk position. One clear advantage of implementing this movement is that it avoids the singularity due to the extended knee, as mentioned before. This strategy has been successfully applied in the humanoid WABIAN-2 (Ogura et al 2006a(Ogura et al , 2006b) and HRP-4C, improving the human likeness of the whole-body motion (Kaneko et al 2009. The robot Lola was designed with 3 DoFs at the hip and 2 DoFs at the waist. This setting represents a redundant structure that contributes to more agility and reduces joint loads while walking (Buschmann et al 2009). The robot BHR (Xu et al 2010) was developed with 2 DoFs at the waist in order to keep balance by regulating the ZMP using waist motions. They show that the margin of stability for the ZMP trajectory was larger when using a waist joint, producing a smoother movement. Okada et al (2003) developed a double spherical joint mechanism that realizes the functions of the hip and waist joints without increasing the number of actuators. Ellenberg et al (2013) proposed a skewed-rotation-plane waist joint for the humanoid robot HUBO2 (Park et al 2007. Their objective was to increase the range of motion of the waist, motivated by the fact that most of the humanoids designs so far used only orthogonal joints, limiting the range of motion. However, it was not clarified how this configuration affects locomotion. In a different approach Liang and Ceccarelli in (2012) proposed a novel waist-trunk system using 2 parallel structures connected together in a serial chain. This system allows for a full 6 DoFs at the trunk and 3 DoFs at the pelvis. Simulations show that, besides high DoFs, this structure offers flexibility and high payload capacity. Okada et al (2003) developed a double spherical joint for the hip joint so that the robot does not have to flex the knee joint to balance the upper body.
All the aforementioned solutions use stiff actuation and do not consider the advantages of having compliant mechanisms at the hip-pelvis complex. Simulated and experimental results showed that the basic hip motor oscillation signals induce the human-like whole body dynamics, and that the system stabilized itself into periodic gait cycles, for both walking and running. From the experience obtained with the Jena-Walkers, a new robot called BioBiped was developed (Radkhah et al 2012, Sharbafi et al 2014. This biped has three-segmented legs with three biarticular structures and five monoarticular structures. The hip's roll and pitch action are driven by bidirectional SEA's or bionic drives, with fixed elastic elements, but adjustable quasi-stiffness through active compliance. This mechanism was tested for hopping and running (Sharbafi et al 2014). Figure 7 reports a summary of the analysis performed in the human and humanoid scenarios. In addition to the schematic functions of human joints, we have highlighted in yellow the phases where humans exhibit compliant behavior. This is mostly observed when the joints assume 'resistive' and 'passive' roles. In both cases, compliance of the joint is expressed with deviations of joint position under the action of external forces or inertia. In the 'stabilizing' state, instead, the joint acts to minimize such deviations, maintaining the joint in a constant position in presence of external forces. In the 'motive' state, compliance is not visibly expressed, because muscle activations are used to generate motive torques rather than reacting to an external force or inertia.

Discussion
Within each of the compliant phases highlighted in yellow, figure 7 reports the current state of the art of robotic implementations, using an hourglass icon, which represents the 'human-to-robot' transference process. We have identified three status levels in this process: (i) the human-like compliant principle is well represented and implemented in a real-life humanoid (indicated with a completed hourglass), (ii) the compliant principle has been implemented in a standalone prototype, but still not included and tested in a whole biped (hourglass in halfway position), and (iii) Figure 7. Role of the joints in human locomotion, and corresponding compliant implementation in robots. The phases in which compliant actuation is needed are highlighted in yellow, and correspond to the resistive and passive states of the joint. An hourglass icon is used to identify the current state of the 'human-to-robot' process, whose goal is to transfer the human-like compliant mechanisms into humanoid robots. An empty hourglass indicates that no robotic implementation has been proposed yet. A halfway hourglass indicates that robotic prototypes exist, but they have not been included in full humanoids yet. A completed hourglass indicates that the compliant principle has been already implemented in a humanoid platform. the compliant principle has still not been implemented in any real-life robotic solution (indicated with an empty hourglass).
Our analysis presents some limitations. First, excluding the motive and stabilizing states from the compliant group represents a simplification of the actual biological actuation system, which is instead always characterized by intrinsic viscoelastic properties of muscle-tendon structures (see section 4.4 for details). Nevertheless, we consider that, from a robotic design perspective, focusing on resistive and passive states may help identifying where and when compliance is more relevant and useful in a bipedal machine. Second, the four-level classification of joint states categories is based on single-joint perspective, and therefore has limited ability to classify how the energy transfer mechanisms occurs across gait phases or joints. For instance, the positive work produced in the motive state can result from the energy previously stored in resistive state (e.g. in stance through the Achilles tendon stretching-shortening mechanisms) or be generated entirely from metabolic energy.
Third, our analysis is limited to the unperturbed scenario, and therefore not able to report the relevance of compliance in perturbed conditions. Actually, compliance of biological structures is thought to be particularly beneficial in an unconstrained environment (Ijspeert 2014), such as unstable terrains, pushes or viscous mediums. Nevertheless, the limited literature on human walking in such conditions does not permit to provide an extensive review on this topic, which we consider a promising area of research in both human and robotics fields.
In the following subsections we report a critical analysis on how human mechanisms have been transferred into actual robotic implementations, following the joint-to-joint analysis approach so far used. Within each section, we also propose a number of key aspects and challenges that may be relevant for future research. We conclude the discussion by presenting some key principles based on a more in-depth analysis on how compliance is achieved physiologically at muscle-tendon level, and how this can be translated into truly biomimetic actuator designs.

Ankle-foot complex
Based on the compliant properties of human footankle complex, some key principles for biologically motivated humanoid design can be identified, in particular when efficiency and robustness under perturbations are targeted. The desirable robotic foot should: (i) have a flexible heel to facilitate shock absorption during heel rocker (loading response); (ii) have metatarsophalangeal motion (toes flexion) to allow the forefoot rocker (terminal stance); (iii) have a flexible arch to ensure shock absorption and energy harvesting during single stance phase, and also possess the possibility of being stiffened in terminal stance phase, mimicking the windlass mechanism of human foot. As for the ankle, the desirable solution should store energy in the first part of the stance phase (loading response and mid stance) and then release it during terminal stance to generate motive plantar flexion, therefore improving the energy efficiency of the forefoot rocker.
Due to their technical complexity, all these human-like properties are still not available in one single ankle-foot solution. As for the ankle, most of current humanoids still implement a very basic stiff ankle actuation, with no compliant properties. The most interesting implementations lie in the prosthetic field, in which the inclusion of quasi-passive elements with variable behavior permits to mimic the modulation of joint stiffness observed in humans (Svensson and Holmberg 2006, Eilenberg et al 2010, Mancinelli et al 2011, Cherelle et al 2013, Herr and Wilkenfeld 2003, Endo et al 2006, Lapre 2014. These solutions demonstrated good performance and efficiency in amputee walking, but their implementation into humanoids is still not available. We also noticed that the mechanism of heel rocker has not been explicitly considered in the reviewed solutions. This will likely be a challenging step since it requires controlled rotation about a small area of support (heel). With respect to the foot, in section 3.1 we reported several interesting ideas and implementations that have been proposed in the humanoid field. Most of them focused on actuated (Ahn et al 2003, Guihard and Gorce 2004, Wang et al 2006, Nishiwaki et al 2007, Buschmann et al 2009, Tajima et al 2009) and compliant, non-actuated (Ogura et al 2006, Sellaouti et al 2006, Kondo et al 2008, Buss et al 2014 toe motion. Others are based on more complex solutions mimicking the compliant structure of the whole foot (Ouezdou et al 2005, Li et al 2008, Minakata et al 2008, Seo and Yi 2009, Davis and Caldwell 2010, Hashimoto et al 2010, Owaki et al 2011, Kwon and Park 2012, Narioka et al 2012. The strong interest in the field is a clear indication that a properly designed foot is expected to produce improvement in robotic performance. On the other side, the foot structure complexity should be carefully considered, since making the foot closer to the human's counterpart does not guarantee a better human-like gait (Yamane andTrutoiu 2009, Hashimoto et al 2010). Different combinations of actuated and non-actuated mechanisms for ankle and foot can result in similar overall compliant behavior of the foot-ankle complex. For this reason, foot and ankle should not be designed separately, but as one functional entity. One aspect that should be taken into account is the tight relation between the structure of the foot and the control algorithm used to control the gait. For example, ZMP based algorithms have difficulties to handle the rolling of the foot since the contact surface is very small, and thus they have better performance with flat-foot structures (Erbatur andKurt 2009, Miura et al 2011). Algorithms based on limit cycle or reflexive CPG have better performance with non-flat foot shapes, but are less robust to disturbances and during still standing (Yamane and Trutoiu 2009, Owaki et al 2011, Narioka et al 2012.

The knee
The knee entails several crucial biomechanical properties for the achievement of human-like behavior, which can be summarized as follows: (i) dissipating the energy of the shocks due to heel-strike at the start of loading response, (ii) assuming a passive role during the swing phase to improve efficiency, (iii) being able to extend during the stance phase to allow energy efficient inverted-pendulum dynamics, and (iv) assuming a resistive role for energy storage during the loading response and terminal swing phase. The correct implementation of all these mechanisms in one bipedal robot is very challenging. Specific solutions addressing some of the above properties have been proposed. As for the shock absorption and energy storage mechanisms, research on prosthetics has generated the most interesting implementations using elastic mechanisms to store energy, improving efficiency, and reducing motor power requirements (Martinez-Villalpando et al 2008) (Geeroms et al 2013(Geeroms et al , 2014. With respect to the passive behavior during swing, the most effective implementation has been presented in passive dynamic walkers, which however lack the stability properties of current humanoids. One of the key mechanisms that may allow including all these human-like properties in one solution is the complex action of the biarticular muscles. A valuable implementation in this respect has been done in the JenaWalker and the BioBiped (Iida et al 2008, Radkhah et al 2012, Sharbafi et al 2014 robots. These bipeds implement most of the biarticular mechanisms of the knee, showing interesting and promising results. Nevertheless, stable walking has not been achieved yet.

The hip-pelvis complex
Experiments in healthy and pathological humans demonstrate that pelvis movements, even if not easily perceivable, might be crucial for human-like locomotion (Saunders et al 1953, Perry 1992. The inclusion of pelvis motion in a robotic biped should produce the following functional benefits: (i) additional forward progression in the terminal stance and pre-swing phases by means of transverse pelvis rotation, which should also enhance the natural look of motion; (ii) improved shock absorption during loading response phase by means of lateral pelvis drop. This movement is also expected to produce smoother weight shift from one side to the other during double support phase; (iii) minimized vertical excursions of the CoM by means of the sagittal tilt and frontal rotation of the pelvis, which will also result in reduced energy consumption and shocks.
In the humanoid implementations realized so far, the motion of the waist allows the robot to achieve a straight leg configuration by changing the height of the waist and improve the cadence by moving the waist in the transversal plane. This improves the human-likeness of the walking pattern even when using the classical ZMP control approach. However, most of the mechanisms used are based on stiff motors and little attention has been paid on how the waist joint would impact on the other control approaches and the use of compliant mechanisms.

Key aspects for truly biomimetic actuation
Compliant actuator technology is still at an early stage of research and development . Current technology is still far from emulating the impressive performance of biological actuation system. Nevertheless, recent studies in human kinesiology and modeling are shading some light on the basic mechanisms of biological compliant behavior, which may be used in future implementations.
In humans, joint compliance is achieved through the visco-elastic properties of muscles and series-elastic tendons , which are flexibly modulated during motion according to the task demands (Zajac 2002, Sartori et al 2016. The resulting resistive forces produced by these mechanisms can be quantitatively expressed by means of stiffness and damping characteristics. In human locomotion studies, the term 'quasi-stiffness' is referred to as the relationship between joint angle and torque (Shamaei et al 2013). This relation has been used to design compliant actuators that replicate the net human joint torque by the action of a spring across specific sub-phases of walking (Vanderborght et al 2008b, Eilenberg et al 2010. Nevertheless, quasi-stiffness does not properly reflect actual stiffness of biological joints, i.e. the 'position-dependent component that stores (and releases) mechanical energy' (Latash andZatsiorsky 1993, Sartori et al 2015). The distinction between the two is crucial for the development of truly biomimetic actuators (Rouse et al 2013).
Human joint stiffness is composed of two components, one represented by the series elasticity of muscle fibers and tendons, and the other resulting from muscle activation and contraction velocity . In passive conditions, when no muscle activity is present, intrinsic stiffness equals quasi-stiffness. During the motive state, the muscle activation component becomes predominant, making the net joint stiffness to differ considerably from the quasistiffness. A similar distinction between stiffness and quasi-stiffness can be drawn for compliant actuators, as demonstrated by (Rouse et al 2013). In a system composed of a motor attached to the equilibrium point of a series spring, multiple combinations of stiffness and equilibrium positions can produce a given quasi-stiffness profile, with two boundary conditions: (i) stiff actuation, where no spring, or spring with infinite stiffness, is present, and (ii) passive actuation, where no motor, or a motor blocked in a fixed position, is present. In this last case, the quasi-stiffness equals the stiffness of the spring. To solve this redundancy problem, the value of stiffness should be chosen carefully based on the application goals, e.g. improved efficiency, movement accuracy, or a particular compliant response to external disturbance . In humans, this is naturally accomplished by the interplay of several mechanisms, as reported in the section 2.
Therefore, when the goal is human-like walking behavior, biomimetic actuators should replicate not only the human quasi-stiffness profiles, but also reproduce physiological joint stiffness. While quasistiffness can be easily measured in humans through inverse dynamics, physiological stiffness profiles are more difficult to obtain experimentally, due to the complex system identification techniques needed (Andersen and Sinkjaer 1995, Ludvig and Perreault 2011, 2012, Plocharski and Plocharski 2013. Model-based methods have been proposed to allow estimating stiffness across joints and locomotion modes that are difficult to approach experimentally. These methods provide quantitative information on how the stiffness is modulated from the muscle-level all the way up to the joint-level . In our opinion, the estimation of physiological joint stiffness gathered with these approaches and its correct combination with quasi-stiffness profiles will be a crucial point for future biomimetic robotic designs. This will be an important step towards truly mimicking the functions of their biological counterparts (Eilenberg et al 2010, Markowitz et al 2011, Pfeifer et al 2015.

Conclusions
The human body includes a number of compliant mechanisms distributed throughout the musculoskeletal system, whose complex and dynamical interplay is at the basis of human-like behavior (Iida et al 2008). Strong efforts have been devoted to understand these mechanisms and include them in humanoid design. In this review we focused on the role of three functional units of human locomotion, i.e. foot-ankle complex, knee, and hip-pelvis complex. We reported the key principles of human biomechanics and the most relevant implementations of these principles into walking machines.
Among the reviewed works, we have found that many relevant human mechanisms have a corresponding implementation in wearable assistive devices such as prostheses. However, in the humanoid field, only a few solutions specifically addressed and tested human-like mechanisms into whole-body platforms. Below we summarize the major human-like mechanisms currently missing in state of the art bipedal humanoids and future research directions for filling the current gap between human and humanoids.
Concerning the ankle-foot complex, one of the aspects that should be clarified is how the different foot structures (i.e., rigid, flexible, actuated or nonactuated) affect the global walking performance of a biped robot in real-life conditions. This analysis should go beyond the unperturbed scenario, i.e., walking over a flat and smooth surface, and consider perturbations that affect foot-ground interaction, such as uneven surfaces, slopes, and soft terrain. As for the ankle and knee mechanisms, the most effective compliant implementations have been proposed in the prosthetic field, where the quasi-passive solutions have produced promising results. The implementation of these solutions into whole-body humanoids would be particularly interesting in order to test the potential advantages and stability under biomechanical and control perspectives. The role of biarticular actuation around the knee is another very relevant topic that has been largely disregarded in the humanoids community, probably due to lack of knowledge on the human side, but which is now gaining more relevance. As for the pelvis motion, future research should cope with the comparison between stiff and compliant actuation, and in particular how each of the DoFs in the waist-i.e. pitch, roll and jaw-affect the whole-body performance of the robot. Also here, it will be essential to consider perturbed conditions, in particular those affecting trunk-pelvis coordination, e.g. pushes or variable walking speeds.
A relevant further step in the direction of improving human likeness of biped robot locomotion would be to investigate the effective stiffness of the human joints, and make this information available for its replication in robotic control. In this respect, crucial research challenges include the development of neuromuscular modeling methods for the real-time prediction of physiological instantaneous joint stiffness that can be translated to the technology level as well as the design and development of fast and efficient variable compliant actuators that can mimic human-like compliant joints.
In general, most of the research has focused on the sagittal plane only. We consider that the frontal and the transversal planes should also be considered, in particular in those DoFs related the lateral balance, which appears to be a fundamental component of walking, in unperturbed and perturbed conditions (Tang et al 1998, Bauby andKuo 2000).
As a concluding remark, there is a lack of established set of quantitative benchmarks to evaluate and compare the human-like performance, and in particular compliant behavior, of current robotic solutions (Torricelli et al 2014. A well-established benchmarking framework, in which humans and robots are directly compared on quantitative basis, would be beneficial not only to improve the robotic performance, but also to gain new insights on human behavior.