Highly-Integrated Hydraulic Smart Actuators and Smart Manifolds for High-Bandwidth Force Control

Hydraulic actuation is the most widely used alternative to electric motors for legged robots and manipulators. It is often selected for its high power density, robustness and high-bandwidth control performance that allows the implementation of force/impedance control. Force control is crucial for robots that are in contact with the environment, since it enables the implementation of active impedance and whole body control that can lead to a better performance in known and unknown environments. This paper presents the hydraulic Integrated Smart Actuator (ISA) developed by Moog in collaboration with IIT, as well as smart manifolds for rotary hydraulic actuators. The ISA consists of an additive-manufactured body containing a hydraulic cylinder, servo valve, pressure/position/load/temperature sensing, overload protection and electronics for control and communication. The ISA v2 and ISA v5 have been specifically designed to fit into the legs of IIT’s hydraulic quadruped robots HyQ and HyQ-REAL, respectively. The key features of these components tackle 3 of today’s main challenges of hydraulic actuation for legged robots through: (1) built-in controllers running inside integrated electronics for high-performance control, (2) low-leakage servo valves for reduced energy losses, and (3) compactness thanks to metal additive manufacturing. The main contributions of this paper are the derivation of the representative dynamic models of these highly integrated hydraulic servo actuators, a control architecture that allows for high-bandwidth force control and their experimental validation with application-specific trajectories and tests. We believe that this is the first work that presents additive-manufactured, highly integrated hydraulic smart actuators for robotics.


intrOductiOn
Hydraulic actuation is the most widely used alternative to electric motors for legged robots and manipulators. It is not only the high power-to-weight ratio and the high control bandwidth (Mavroidis et al., 1999;Siciliano and Khatib, 2007) that make hydraulic actuators interesting. Other major advantages are related to the inherent properties of hydraulic oil that acts as a lubricant as well as cooling liquid. Despite these advantages, hydraulic actuation suffers from a number of short-comings. The recently published Survey on Control of Hydraulic Robotic Manipulators with Projection to Future Trends (Mattila et al., 2017) mentions two of them: First, the difficult controller design due to the nonlinearity of the hydraulic system dynamics and second, the low energy efficiency of traditional closed-loop hydraulic systems. Additionally, compact hydraulic actuation components are rare on today's market and their compact integration into articulated robots is challenging (Semini et al., 2011). Hydraulic servo actuators have been used for several decades in legged robots. Marc Raibert's early hopping robots (e.g., the 3D one-legged hopping machine) were driven by hydraulic actuators that combined a low friction cylinder, position sensor, velocity sensor and pressure control servo valve (Raibert, 1986). Raibert continued using similar custom actuators for the robots developed by his company Boston Dynamics, Inc. (BDI). The legs of BigDog, for example, are powered by a custom hydraulic actuator with a Moog Series 30 servo valve 1 , cylinder, load cell and potentiometer (Buehler et al., 2005). The servo valves allowed the actuators to be controlled in force as well as in position. Subsequently developed BDI robots like LS3, Cheetah, Wildcat, ATLAS and Spot use hydraulics 2 , but no detailed information about the servo actuators is available. IIT's HyQ robot uses a custom hydraulic servo actuator that consists of a Moog E024 servo valve 3 , a Hoerbiger cylinder, a custom hydraulic manifold, 2 pipes, a load cell, an absolute and relative joint encoder, and electronics for sensors and valve amplifiers (Semini et al., 2011). Boaventura et al. presented highperformance force control on these actuators . Another force-controlled hydraulic actuation unit was developed by Hyon et al. for a light-weight hydraulic leg (Hyon et al., 2013) that was later used as the basis for the actuation of the joints of a hydraulic humanoid robot called TaeMu (Hyon et al., 2017). The actuator units use PSC AS110 servo valves 4 . The BabyElephant robot  is powered by custommade hydraulic actuators called Hy-Mo. These actuators consist of a hydraulic cylinder, an electric motor to move the spool of the valve and pressure sensors to measure the two chamber pressures (Wang et al., 2014). Another example is the hydraulic force control unit of the hydraulic quadruped robot BITDOG (Lu et al., 2015). The servo actuator consists of a hydraulic servo valve, hydraulic actuating cylinder, displacement sensors, pressure sensors and shock absorber. The actuator's active compliance controller was presented in Lu et al. (Lu et al., 2015). The ROBOCLIMBER is a 4,000 kg quadruped machine with force-controlled prismatic 1 http://www.moog.com/content/dam/moog/literature/ICD/Moog-Valves-30series-datasheet-en.pdf 2 The most recently presented SpotMini is BDI's smallest and lightest (30 kg) quadruped and it uses electric actuation. 3 http://www.moog.com/literature/ICD/e024seriesmicrovalves-ds.pdf 4 http://www.psc-net.co.jp/category/1359255.html legs. Montes et al. present various force control strategies for the hydraulic cylinders (Montes and Armada, 2016).
Besides academic prototypes, there are also a few hydraulic servo actuators available on the market, such as the Moog A085 Series Servo Actuators 5 that combine high performance cylinders, linear feedback devices and servo valves in one assembly. These actuators were recently installed on a Menzi Muck walking excavator to implement active chassis balancing with force controlled cylinders (Hutter et al., 2017). The force was estimated with hydraulic pressure sensors inside the two cylinder chambers.
Other commercial examples are the linear and rotary actuators developed by KNR systems (Kim et al., 2014) that feature KNR series KSV070 servo valves 6 . A combination of KNR actuators and Star Hydraulics series 200 servo valves 7 are used for the hydraulic quadruped robot Jinpoong developed by the Korea Institute of Industrial Technology (KITECH) (Cho et al., 2016).
A related class of integrated hydraulic actuators are electrohydrostatic actuators (EHA) that combine an electric motor, hydraulic pump, small tank, and rotary/linear actuator into one unit. Two examples from academia are the following: Alfayad et al. have recently presented the IEHA -Integrated Electro Hydraulic Actuator for a hydraulic humanoid called Hydroid (Alfayad et al., 2011). Kaminaga et al. have been developing EHAs for robotic hands and recently for the humanoid robot called Hydra (Kaminaga et al., 2014). The remainder of this paper will exclusively focus on actuators driven by servo valves.
Most high-performance, hydraulic legged robots (see list above for examples) rely on miniature servo valves to control the actuators in their legs. The fast response and high control bandwidth of servo valves, when compared to other types of valves, allow the implementation of force/torque control, which is an important building block to achieve robust locomotion . The high bandwidth of servo valves, however, has its price: The pilot stage has a continuous internal leakage (the so-called tare leakage) that routes part of the flow directly from the pressure port back to the return port. This wasted flow leads to a generally low energy efficiency of servo hydraulics. Note that the tare leakage does not include the spool null leakage. Table 1 compares the most important specifications of the abovementioned servo valves. The last column show the specifications of the ISA v5 that will be introduced next. 5 http://www.moog.com/products/actuators-servoactuators/industrial/hydraulic/ a085-series-hydraulic-servo-actuators.html 6 http://rnd.knrsys.com/english/view.html?id_no=48 7 http://www.star-hydraulics.co.uk/assets/data-docs/product%20pdfs/ST-200-2017-2-En.pdf This paper presents two versions of the new Integrated Smart Actuators (ISA) developed by Moog in collaboration with IIT, as well as a smart manifold on a rotary actuator. The ISA consists of a hydraulic cylinder, servo valve, various sensors, overload protection and electronics for control and communication. Its body is additively manufactured (AM) in a titanium alloy, allowing a very compact design with integrated flow paths and wire channels. The two presented versions ISA v2 and ISA v5 (Figure 1A,B) have been specifically designed to fit into the legs of IIT's hydraulic quadruped robots HyQ (Semini et al., 2011) and the newest version HyQ-REAL (under construction). The last column of Table 1 shows the key specifications of the new ultra-low leakage valve inside the ISA v5 and smart manifolds (see Section 2).
The main contributions of this paper are the derivation of the representative dynamic models of these highly integrated hydraulic servo actuators, a control architecture that allows for high-bandwidth force control and their experimental validation with applicationspecific trajectories and tests. The key features of these components tackle the disadvantages of hydraulic actuation for legged robots through: (1) built-in controllers running inside integrated electronics, (2) low-leakage servo valves, and (3) compactness thanks to metal additive manufacturing. We believe that this is the first work that presents additive-manufactured, highly integrated hydraulic smart actuators for robotics. The ISA has been mentioned in a paper on additive manufacturing by Moog's Guerrier et al. presented at the 2016 conference on Recent Advances in Aerospace Actuation Systems and Components (Guerrier et al., 2016). A short overview of the ISA has been presented during the IROS 2016 workshop on The Mechatronics behind Force/Torque Controlled Robot Actuation: Secrets and Challenges (Semini et al., 2016).
This paper is structured as follows: Section 2 presents the main features and specifications of the ISA and smart manifolds. Section 3 describes the control architecture running on the actuator's embedded ARM processor, and Section 4 explains the mathematical model of the actuator dynamics. Section 5 presents the results of simulation and experiments, followed by the conclusions in Section 6.

isa and sMart ManiFOld Features
The linear actuator ISA v2 was developed with the goal of integrating standard hydraulic components by means of additive manufacturing to create an optimized actuator unit. The ISA v2 has a high response valve and the most important sensors for position/force control and self-protection (temperature and mechanical impacts). Figure 2A shows a CAD rendering of the ISA v2 with a section to illustrate the main features of the actuator and integrated components.
The use of ISAs in a legged robot has several advantages on system level: First, it reduces the overall complexity of the machine, since the various actuator components are combined into one device. Sensor wires are routed inside the AM body and several components are merged into the same electronic board (e.g., microcontroller, valve amplifier, Inertial Measurement Unit (IMU), temperature sensor). Fewer and shorter wires result in higher reliability and less signal noise. Second, it reduces the total robot weight, and increases its ruggedness. For an illustrative comparison, Figure 3 shows the linear actuator units of a HyQ leg with all the components that belong to one unit. Note that the shown A/D converter and communication electronic boards -shown on the left of the bottom-right picture -are connecting  to 3 actuators in total. The black box to its right hosts the electronics for 6 valve amplifiers. The total estimated weight of the components of 1 actuator unit (excluding the electronics) is 1.3 kg. The corresponding ISA v2 on the other hand weighs only 0.92 kg and additionally includes pressure relief valves for overload protection, as well as all electronics to close local control loops.
The need for more efficient actuators in autonomous robots triggered the redesign of the linear actuator ISA v2 to achieve the version ISA v5, with a compromise between high performance and energy wasting.
The combination of additive manufacturing and standard parts allows for customisation of actuators and retain all of the functionalities that of the ISA. This idea has been used to create the smart manifolds to provide most of the ISA technology to custom made rotary actuators as shown in Figure 2B.
In this section we present the most important features and specifications, regarding the mechanical and electronic components, of the integrated smart actuators and smart manifolds.

integrated servo Valve for high Performance (isa V2)
The integrated servo valve is a derivative of the high performance version of the Moog E024 series sub-miniature servo-valves (MOOG Inc, 2015) (valve used in the HyQ robot). The high bandwidth of around 320 Hz allows high performance force and position control, as previously demonstrated in . The frequency response for the high response valve (HRV) used in the ISA v2 is shown in Figure 4.

integrated servo Valve for high
Performance with improved efficiency (isa V5) ISA v5 incorporates an ultra-low leakage valve (ULLV) technology to considerably reduce throttling losses and improve the efficiency of the overall unit. The pilot stage of the valve is modified to improve tare leakage and still have high bandwidth of greater than 100 Hz as shown in Figure 4. The second stage peak leakage is reduced with valve overlap and dual gain flow slots. Figure 5 compares the flow gain of the valve configuration to that of the high response valve used in ISA v2. Here the v5 has low-flow gain near null and high-gain stroke greater than 25%. The low gain flow slots near null considerably reduces losses and reduces nonlinearity in the flow curve as opposed to purely overlapped valves. Figure 6 compares the leakage flow of ISA v5 (ULLV) to that of ISA v2 (HRV). When the actuator is stationary the leakage of ISA v5 is approximately 36% of ISA v2. This results in a power saving of approximately 112W per actuator.

unequal area Flow slots (isa V5)
ISA v5 has unequal area flow slots which match the area ratio of the actuator. This results in constant actuator velocity during retraction and extension and removes the need for gain scheduling based on the piston motion direction.

integrated sensors for Position, Pressure and Force Measurement
A 1Mbps absolute position sensor is used to measure the position of the piston rod. The two cylinder chamber pressures of the actuator are measured using strain gauge based miniature pressure sensors. Actuator force is measured using a strain gauge based load cell integrated in the actuator tail stock. The electronic board additionally includes an IMU and ports for optional sensors.

integrated electronics to control actuator Position/Force
The design of the Remote Electronics Unit (REU) follows Moog's commercial aircraft flight control standards. It closes several control loops (see Section 3) on an ARM processor and offers various communication options such as, e.g., EtherCAT, CAN and Serial bus.

Valve spool Feedback
The smart manifold has a mechanical feedback valve (MFB valves), i.e., the spool position is indirectly controlled from the valve current command. The ISA v2 and v5, instead, have electric feedback valves (EFB valve) and the spool position can be fed back in a specific control loop. Electric feedback technology allows for a valve response less sensitive to the null bias (which is dependent on the return and supply pressures and on the oil temperature).

integrated Overload Protection
Pressure relief valves limit the pressure inside the two cylinder chambers. These valves vent to return if the chamber pressures reach 22 MPa thus resulting in an effective and repeatable overload protection. Tables 2, 3 show the most relevant mechanical and electrical features of the ISAs and smart manifolds, respectively.

cOntrOl architecture
This section describes the standard control loops available on the Remote Electronics Unit (REU) that is integrated in the smart actuators (i.e., ISA v2, ISA v5 and smart manifolds). The REU, also called thumb-REU for its small board size, has standard nested control loops that can be configured according to the actuator hardware. For example, for the more innovative lines (the ISA v2 and v5 with electric feedback valves), it is available a control block with a set of functionalities to control the spool position. For the case of smart manifolds, which have mechanic feedback valves and no spool position measurements, such control block can be reconfigured to control another state of the actuator. By available control block we mean a block with a set of control functionalities that can be used by the control designer. Such functionalities are, e.g.,: PID controllers, feedforward terms, feedback terms, filters, selection of control loop frequency, saturation functions, anti wind-up for integral actions, gain scheduling, offsets, and logic blocks for nonlinear compensations.
The standard firmware of the thumb-REU provides four control blocks where each one can be configured to be associated to one of the following actuator variables: piston/rotor position, pressure difference between the actuator chambers (i.e., ∆P ), actuator force/  The HRV presents a constant flow increasing of about 2.3 l/min/mA until near 2 mA, when the valve gain start decreasing. The ULLV, instead, presents a low flow gain of 0.4 l/min/mA and a high flow gain of ± 2.5 l/min/mA inside and outside the current range of ± 1.2 mA, respectively. Note: these curves were obtained from experimental measurements considering the valve redundant coils (2 coils) in series. The ISA and smart manifolds integrate these valves with the coils connected in parallel to increase the operation safety. In this case, the scale of the current is doubled and the current command ranges from ± 10 mA. torque, spool position and valve current. Each control block, though, can be activated or deactivated at will, to achieve different control strategies. For example, to perform pure actuator position control, force/torque control, or even activate only the valve current loop to use the thumb-REU as a simple valve amplifier. Figure 7 shows a simplified view of each control architecture (arrengement of the available control blocks) that was used to test the ISA v5 (shown in Figure 1B) and the smart manifold with the custom made rotary actuator (shown in Figure 2B). The frequency of the control loops range from ~1 kHz for the piston/rotor position to ~10 kHz for the valve current. Both control diagrams show a force loop with an outer position loop that leads to an impedance control, which is an essential strategy for legged systems and will be used in the HyQ-REAL robot.
Both control architectures shown in Figure 7 have a force/torque control loop with feedback of the force/torque measured at the load level (instead of performing hydraulic force/torque control), what minimizes tracking issues due to the internal hydraulic actuator friction. To compensate for the pressure dropping due to the piston/ rotor motion, a velocity compensation term is used to inject extra flow into the chambers. The extra current command from the velocity compensation is modulated according to the dual-gain feature of the ULLV to avoid a flow over compensation, at the valve region of high gain, that can make the overall control loop to be unstable.
For the case where the velocity compensation is introduced at the level of the spool control loop (see Figure 7B), Kv is tuned to take into consideration the relationship between the spool opening and the valve flow. Details on the PID actions, of each block, and the additional terms considered for the test of the actuators are described in Sec. 5.

current loop tuning
The current control loop of the smart actuators is the innermost loop and it is the loop that has negligible coupling with the load dynamics. Moreover, such decoupling allows for a control tuning only dependent on the internal dynamics of the valve, i.e., the valve coil dynamics.
Given the importance of the current loop dynamics for the outer loops, we design a current controller that aims for a small phase lag between the desired and actual valve current. We propose a control structure in a 2-DOF (degrees of freedom) configuration where: (1) the closed-loop has the forward path composed of a proportional action, in series with a lead-lag compensation of second-order; and (2) a gain K pf applied on the demand to reduce steady-state errors. The gain K pf applied on the demand compensates for the steady-state error due to the absence of integral action (avoided to reduce the phase lag in the response). The lead-lag compensation is implemented to supress high-frequency resonance modes of the valve coil that are excited as the proportional action is increased. Figure 8A shows a step input response test performed in openloop to obtain the static gain, here denoted as Ksg between the desired current and the actual current. Figure 8B, instead, shows the apearance of a high-frequency resonance mode, around 1.8 kHz, for a step input response test when the current loop is closed with a certain proportinal action. The transfer function of the current controller, named as PID + LLC in Figure 7, is described in the Laplace domain as: where Kp is the proportional gain of the current loop, w lead and w lag are the natural frequencies for the lead and lag compensations, respectively, and ζ lead and ζ lag are the damping ratios for the lead and lag compensations, respectively. The gain K pf can be computed from Kp and Ksg as: The values used for the current loops of both the ISA and the Smart Manifold are described in (Table 4). With such tuning one obtains a current control loop with response time of less than 2 ms.

actuatOr MOdeling
In this section the dynamics of the hydraulic actuators are described 8 . The main goal is to provide a representative mathematical model, 8 An extended version of this section will be published in the Springer book Humanoid Robotics: A Reference in the chapter Actuator Modeling and Simulation by Jorn Malzahn, Victor Barasuol, and Klaus Janschek.
from the valve to the load dynamics, in order to: (1) complement the simulation of the rigid body dynamics of a legged system making it more realistic; (2) help designing new control strategies to improve force/torque tracking performance and robustness; and (3) to serve as a tool to understand the impact of the mechanical design of parts on the controller performance (e.g., friction and backlash). In Section 5 the model is used to simulate the experiments where the actuator is tested under representative conditions of legged systems. The power conversion in hydraulic actuation relies on the transmission of fluid by means of a pump to a hydraulic actuator. The role of the actuator is to transform back the hydraulic energy into mechanical energy, which is then transmitted to a mechanical device (e.g., a robot leg).
In the following paragraphs a description of the pressure, the flow, the valve spool and the load dynamics of hydraulic actuators is given. The rate of change of the pressure with respect to time in a given chamber is represented by the so-called Continuity Equation, which can be used to obtain the pressure dynamics for a linear hydraulic actuator as where P A and P B are the pressures inside the chambers, V 0A and V 0B are the dead volumes coming from the valve inside the chambers, A A and A B the piston/vane areas where pressure is exerted in each of the chambers, Q A and Q B are the flows inside the chambers, x p is the piston position, Lp represents the cylinder stroke length   (3) and (4), the pressures in each of the chambers can be obtained and the hydraulic force delivered by the piston is given by The pressure dynamics equations for a rotary actuator can be obtained in a similar fashion with respect to the linear case by replacing both piston areas A A and A B with the volumetric displacement of the motor Dm , the piston position x p with the angular position of the rotor θa and the cylinder stroke length Lp with the range of the motor Lm . The hydraulic torque delivered by the rotor can also be obtained from the pressure dynamics as It can be noticed that the pressure dynamics not only depend on the position of the piston or the rotor, but they also depend on the flows going into each of the chambers of the actuators. These flows ( Q A and Q B ) can be obtained from the dynamics of the valve. However, the valve is one of the elements that renders the modeling and control of hydraulic systems more complex, due to its nonlinear behavior. Figure 9 shows a schematic drawing of a valve which controls the flows Q A and Q B . The spool position determines the amount of flow that goes through the orifices of the valve. In the case of our experimental setups, a novel dual-gain Ultra Low Leakage Valve (ULLV) developed by Moog and IIT is being used. The spool position is controlled by means of an input current. The relation between input current and spool position is usually modeled as a second order system [see ] of the form where Xv ( s ) represents the spool position in the Laplace domain, is the valve current in the Laplace domain, K spool relates the current input with the spool position in steady state, ωn is the natural frequency of the spool and ζ is the damping coefficient of the spool. However, for the ULLV a frequency domain analysis (depicted in Figure 4) showed higher order, which have been fit to a third-order transfer function given by where x r v is the spool position reference, K loop = 540 rad/s, ωn = 6132 rad/s and ζ = 1.5 . The relationship between the spool position reference X r v ( s ) and the spool current command Ir ( s ) , in Laplace domain, are approximated by the following equations: where K spool = 0.0356 mm/mA and pc = 2,000 rad/s is the pole of the first order systems that approximates the current loop response of the current controller described in Sec. 3. With the definition of the spool dynamics, one can obtain the flow going through an orifice using the following expression: where kv ( xv ) is the so-called valve gain, and it is a factor that depends on the discharge coefficient, orifice area gradient and the density of the fluid (Merritt, 1967). Normally, kv is computed based on experiments with nominal input current and nominal flow, making use of a flow-meter. In the case of the ULLV, plots relating the flow going through the valve with respect to the input current were provided. These plots are depicted in Figure 5. The plots indeed show clearly the zones were the gain value changes, in the case of the ULLV. The low and high values of kv were estimated from these plots.
The flows Q A and Q B obtained from the differences between Q 1 , Q 2 , Q 3 and Q 4 in Figure 9 are computed according to the following equations (considering a critically centered valve) where the function sg ( x ) is defined as: Figure 9 | Schematic drawing of a valve that controls the flows Q A and Q B in the actuator chambers. P T and P S represent the return and supply pressure, respectively, Q T and Q S are the return and supply flows, respectively, and Q i for i = 1, 2, 3, 4 are valve port flows.
and P T and P S are the return and supply pressures, respectively. The variable valve gains k vi ( xv ) , with i = 1..4 , are equally modeled according to the ULLV flow gain curve shown in Figure 5 and Eq. 9 as: 6.667 × 10 −6 m 3 /s for |xv| < x dg 4.167 × 10 −5 m 3 /s for |xv| ≥ x dg , where x dg is equal to 42.7 × 10 −3 mm. It is possible to consider fluid compressibility, elasticity of the fluid container and fluid resistance in the model, but these effects are highly nonlinear and are out of the scope of this paper.
Hydraulic force (in the case of linear actuators) or hydraulic torque (in the case of rotary actuators) is applied onto a mechanical device. In the case of this study, an experimental setup was built in order to test a custom-made rotary actuator integrated with a smart manifold. Additionally, the ISA v5 linear servo actuator was mounted on the Knee Flexion/Extension (KFE) joint of the hydraulically actuated leg of HyQ-REAL. These experiments are explained in Section 5.

siMulatiOn and exPeriMental results
To evaluate the performance of the ISA v5 and the smart manifold, two experimental setups were devised. The first setup consists of the smart manifold mounted on a custom made rotary actuator, which drives a metal wheel. This setup is shown in Figure 10A. For the second experimental setup, the ISA v5 was mounted on a leg of the HyQ-REAL to drive the KFE joint. This setup is depicted in Figure 10B. This section includes the simulation and experimental results on the rotary actuator setup ( Figure 10A) and the experimental results on the KFE joint driven by the ISA v5 ( Figure 10B).
The experiments were performed using the control strategy depicted in Figure 7 and Table 4 shows the specific values of the controller gains for each of the loops (unless it is indicated differently). It is worth mentioning that in the rotary actuator there is no spool loop. This is due to the fact that the connection of the first and second stage of the valve is done mechanically. On the other hand, the stages of the valve in the leg experimental setup is done through electric feedback, which gives rise to the spool control loop.

integrated smart Manifold and rotary actuator
The core of the test rig is made of three main components: the smart manifold, a custom-made rotary actuator and a set of sensors useful for control and analysis (position, torque, pressure, among others). All of these elements are shown in Figure 2B. This configuration of integrated electronics along with a rotary hydraulic actuator is suitable for actuating a legged system, for example, a Hip For the experiments, a metal wheel was mechanically connected to the rotary actuator, as it can be seen in Figure 10A. The weight, shape and dimensions of this wheel were designed in order to approximate the rotary inertia of the leg of HyQ-REAL, which is known from CAD data. The mathematical model of the test setup (along with the hydraulics model of Section 4) was obtained for simulation and control purposes. A schematic drawing of the linear equivalent of the model is shown in Figure 11 and its mathematical description is given by where Ia and I l represent the rotational inertia of the actuator rotor and the load, respectively, θa and θ l correspond to the angular position of the actuator and the load, respectively, τ H is the hydraulic torque coming from the actuator, τ f is the friction torque acting on the mechanical rotor, Ks stands for the torque sensor stiffness and Ds is the damping present in the torque sensor. In this model, the inertia of the torque sensor is being neglected since it is considered to be very small (approximately 2 × 10 3 times smaller with respect to the load inertia). We consider as well that the sensor and the rotor are rigidly connected to each other (sensor position is equal to actuator position). We use a Lund-Grenoble Friction Model for the friction torque acting on the actuator rotor (Aberger and Otter, 2002), with a stiction level of Ts = 2 and a Coulumb friction level of Tc = 1.9 , identified from experiments. Nonlinear effects such as backlash between the load and the sensor (actuator) position are considered using a similar approach as in (Merzouki et al., 2003). We also model the hard-stops of the system as spring-damper systems with no pulling effects. The proposed model is considered accurate enough for simulation and controller design. The specific values of the parameters of the model, such as stiffness and inertias, were obtained from CAD models and experiments. The experiments were designed to identify damping coefficients of the involved dynamic elements.
In addition, the inherent characteristics of the sensors (e.g., resolution and sampling frequency) and the frequencies implemented in the various loops are also included in the simulation. This information is taken from the datasheets provided by the manufacturers of the sensors and electronic boards. In further studies, variations on the system due to quantization errors or sampling frequencies can be analyzed using the proposed simulation. The motor parameters used in simulation are the ones of the custom made actuator provided by the manufacturer and are shown in Table 5. To obtain V 0A and V 0B the following expressions using the pipe volume Vp are used: For the first experiment, a reference signal for a rotary joint of a leg was obtained from simulation. The simulation corresponds to the robot HyQ-REAL performing a trotting gait with a forward velocity of 1 m/s. Figure 12 shows the position and torque tracking performance achieved, both during simulations and experiments. It is worth noting the resemblance between the generated signals both in simulation and experiments. We consider that the achieved performance is sufficient to show the capabilities of the designed actuators in robot locomotion applications. Performance can be further improved by including the nonlinearities related to the hydraulic force and the valve dynamics in the controller design in a similar fashion as in .   For the second experiment, the rotor actuator position was blocked, and a step torque reference was given in order to evaluate the torque tracking performance of the system. Figure 13 shows an example of the torque tracking during simulation and the experiments. The actuator eliminates the steady state error with a rise time of approximately 8 ms. These results match between simulation and experimental data. This step response suggests that the achievable bandwidth goes from 50 Hz to 100 Hz during blocked condition.
A difference between simulation and experiments can be noted in Figure 13, where low-amplitude oscillations are present before reaching steady state during experiments. One cause of this oscillatory behavior can be attributed to the contact model between the setup and the mechanical end-stop when the actuator is blocked. This assumption is supported by the fact that in the case of the rotary actuator, a stiff metal to metal contact was used to keep the motor in a constant position. This contact shows low-amplitude oscillations. In the case of the linear actuator experiments (shown in Figure 14) the contact was kept between metal and rubber. The compliance displayed by the rubber and its possible nonlinear stiffness, potentially increase the amplitude of the oscillation. This effect was also tested in simulation and the result coincide with this assumption. Added to the contact dynamics, a hunting effect originated due to the overlapping of the valve around the null position may also accentuate this oscillatory behavior.
To evaluate the performance of the rotary actuator against impacts, we provided a periodic reference trajectory that results in continuous impacts against the mechanical end-stop. An example of these tests in simulation and experiments is shown in Figure 15. It can be seen that the magnitude of the applied torque in order to cope with this impulse-like disturbances remains within the actuator limits (i.e., 170 Nm). These results show that the hydraulic servo actuators are able to deal with this kind of perturbation remaining stable and avoiding saturation. It is also important to point out that the reference signal in the case of the experiment shows some spikes after the impact. These sudden increments in the reference are mainly originated from the velocity measurements and the rotor velocity error at the beginning of the contact.
The final experiment for the rotary actuator is designed to emulate the presence of external disturbances into the system. The perturbation is introduced by attaching a weight to the edge of the load wheel of the experimental setup. A sequence of snapshots of this experiment is shown in Figure 16. The weight and rope slack used to apply the external disturbance could lead to a torque disturbance impulse with an amplitude larger than 250 Nm if the wheel was perfectly blocked or its joint had infinite joint stiffness. Figure 17 shows such large impacts considering the case where the system has load position control without impedance torque control. For the test shown in Figure 17 we consider a pure position control where the output of the  position PID controller block Fr becames the direct command signal Uv to the hydraulic valve (see Figure 7A). As it can be seen, when torque control is not implemented, the time that the system takes to stabilize is much larger. On the other hand, when torque control is applied (as depicted in Figure 18), steady-state is reached much faster, with a lower level of oscillations. Additionally, we tested our control scheme for different values of controller gains. This set of gains is considered to contain a range of impedances suitable for quadruped robot locomotion. Based on our experience with the HyQ-series robots, depending on the application, these values of impedances could be implemented in systems ranging from 80 kg to 120 kg. It can be seen that even with a compliant set of gains, the actuator is able to cope with a disturbance. These experiments show the relevance of implementing torque and impedance control when dealing with impacts.
Studying the comparison between the experimental data and the simulation, the proposed setup proved to be useful for controller design, since the same strategy with the same control parameters (controller gains and frequencies of the control loops) were chosen with similar results in performance. Added to the fact that the designed smart actuators presented in this paper fulfill the requirements of legged systems, it has been shown that the integration of the electronics and hydraulic actuation aids greatly in the design of new control strategies for hydraulic systems.

isa V5 driving Knee Joint of hyQreal
For the linear actuator experiments, the ISA v5 was mounted on the KFE joint of the experimental setup for the leg of the hydraulically actuated robot HyQ-REAL. Position control and force tracking were also tested. Due to time constraints, a detailed simulation study of the implementation of the linear smart actuator was not carried out along with the experiments, mainly because additional experiments are required in order to properly identify the parameters of the leg. Nevertheless, the tests presented in this section contributed not only in the evaluation of the performance of the ISA v5, but also gave important insight about key parameters, such as the bandwidth of the system. It remains as future work to develop a simulation environment for behavior analysis and controller design, such as the one developed for the smart manifold integrated with the custom made rotary actuator.
Similarly to the position tracking experiment performed with the rotary actuator, a sine wave signal was set as reference for the enconder position of the KFE. Figure 19 shows the tracking performance of the system. Implementing the control strategy described in Figure 7, it can be seen that the tracking is similar to the one achieved with the rotary actuator.   The second experiment for the linear actuator is similar to the force control against the end-stop for the rotary actuator. In the case of the leg, the foot was put in contact with a wooden pallet (as it can be seen in the lower part of the picture displayed in Figure 10B) and a step reference for force was given, for low and high force values. The results of these experiments can be seen in Figure 14.
For the last experiment, the leg was dropped from a height of 10 cm in order to evaluate the performance of the ISA when subjected to critical impacts, generally present while executing locomotion tasks. The total weight of the leg is approximately of 10.2 kg and the moving part from the slider that is attached to the leg (see Figure 10B) weighs approximately 5 kg. For this experiment the impedance values were Kp = 100000 N/m and K d = 4000 Ns/m. Figure 20 shows the force tracking during this experiment. The impact of the leg takes place around 0.1, at which the reference signal sent by the position control loop starts being tracked by the force loop. It can be noticed, that the actuator is able to stabilize the leg, considerably below its maximum output force (7500 N). The size of the peak appearing right after the time of the impact is due to the time the system takes to perceive and respond against the external disturbance (the ground reaction force).
The results of the experiments show a similar behavior as the ones displayed by the rotary actuator for similar tests, after tuning the controllers in order to account for the dynamics of the leg. This represents a positive result towards the implementation of a systematic method to design, test and implement controllers for legged systems in the future. Regarding this last point, one of essential task for further research, is to take advantage of the possibility to provide a detailed model of the integrated actuators shown in this paper, and develop control strategies that could improve the performance based on these models such as the feedback linearization .

discussiOn and cOnclusiOns
This paper presented a detailed description of the Integrated Servo Actuators ISA v2, ISA v5 and the smart manifold integrated with a custom made rotary actuator, developed by Moog and IIT. The main features and specifications of the components that build up the system were conceived in order to satisfy the requirements of legged systems and to overcome the current shortcomings of hydraulically actuated robots. The devices here described successfully integrate the electronics for controller implementation along with the components that comprise a high-performance hydraulic actuation system. A detailed dynamic model of the actuator is given and was verified to be representative after comparing the simulation and the experimental results. The main goals of providing such dynamic model and a detailed parametrization of the actuator features are: to help on the elaboration of new control strategies; to make the simulation of multi-legged systems more realistic by integrating the actuator dynamics; to help designing mechanical parts by foreseeing the impact of their geometric tolerances (e.g., that lead to backlash) and properties (e.g., that lead to friction or structure stiffness).
Differences in the results between experimental and simulated tests are expected due to the difficulty to model the fast and nonlinear dynamics present in the systems. Sources of mismatching dynamics are likely to be due to, e.g.: inaccuracies in the environment stiffness model; the asssunption of a test bench stiffness infinitly rigid; the nonlinearities of the valve around the spool null position that can not be precisely modeled; and the innacuracies in the friction modeling. Most of these modeling mismatching and assumptions tend to cause a difference between simulation and experimental results when the actuator performs velocities close to zero (condition where friction modeling is critical) or goes into a steady state where the valve spool tends to be positioned around null. At this critical conditions, high controller gains might lead to limit cycles as a steady state.
A control strategy implemented on the integrated electronic boards of the servo actuators is explained in detail. This control strategy was tested on two experimental setups and the experimental results for the smart manifold integrated with the rotary actuator and the ISA v5 mounted on the knee joint of HyQ-REAL leg were presented. The force/torque control performance, shown in Sec. 5, suggests a control bandwidth between 50 Hz and 100 Hz for low amplitude signals (about 10% of the maximum output force/  torque). Considering the control bandwidth found in state-of-theart papers for related applications (Paine et al., 2014;Boaventura et al., 2015;Hutter et al., 2016), that ranges from 30 Hz to 60 Hz for small signals around 10%, the control performance obtained with the smart actuarors is relevant, promissing and are part of the current state-of-the-art.
In its current state, the main limitations noticed in the smart actuators are at firmware level. Future work includes implementing new functionalities to test different control strategies (e.g., nonlinear control approaches) as well as tools to run system identification algorithms and to evaluate the control performance of the various control loops (e.g., frequency response analysis).
Future work also comprises new steps towards the system modeling. A detailed characterization for simulation and controller design of the experimental setup of the ISA v5 along with the HyQ-REAL leg will be obtained. Moreover, different control strategies that might improve the performance of the overall system (such as model based strategies like feedback linearization) will be tested both in simulation and experiments. Finally, the novel servo actuators here described, will be implemented on HyQ-REAL, the newest version of the hydraulically actuated quadrupeds developed in IIT.