A Method to Guide Local Physical Adaptations in a Robot Based on Phase Portraits

In this paper, we propose a method that shows how phase portraits rendered by a controller can inform the development of a physical adaptation at a single degree of freedom (DoF) for a given control task. This approach has the advantage of having physical adaptations sharing the responsibility of control to accomplish a task. We use an inverted pendulum which is reminiscent of the trunk of a biped walker to conduct numerical simulations and hardware experiments to show how our method can innovate a physical adaptation at the pivot joint to reduce the control effort. Our method discovered that a torsional spring at the pivot joint would lead to a lower input effort by the regulator type feedback controller. The method can tune the spring to minimize the total cost of control up to about 32.81%. This physical adaptation framework allows multiple degrees of freedom robotic system to suggest local physical adaptations to accomplish a given control objective.


I. INTRODUCTION
T HERE is an increasing interest among the robotics community to co-design the physical embodiment and the feedback controllers due to its merit in stable and efficient behaviour [1], [2].Very often these approaches are based on biological inspirations as the dynamic synergy between biomechanical structures and neural control in animals play a pivotal role to exhibit efficient, robust, and compliant locomotion [3]- [5].For instance, human walking energetics cost about 75% less than chimpanzees' quadrupedal and bipedal locomotion [6].Studies suggest the aforementioned efficiency in locomotion in human is due to developing longer legs (hind limbs) and extended hip compared to their earlier ape-like primates [6], [7].
In many robotic examples and bioinspirations [8]- [10], the sharing of control responsibility between the physical structures and the sensory-motor control loops is such that there is no clear boundary or distinction between the two biomechanical and the neural controller [11], [12].In other words, the "controller" and "controlled" are not explicitly detached from one another [13].This opens the possibility that controllers and physical adaptations can co-evolve [11], [12], [14], [15].
We adopt an embodied and embedded view of a robotic system [3], [5], [16] where the robotic system's behaviour emerges from the interplay between physical structure/hardware, the controller and the environment.[12], [16] refer to the distribution of control task between the controller and the physical body/morphology as morphological computation.[9] addresses this integrated/embodied approach by use of "templates and anchors" where some of the control task is delegated to a distributed mechanical feedback.
Suitable changes in physical/hardware parameters assist the closed loop controller to minimize energy consumption.Related examples such as the RHex robot [10], the pure passive dynamic model of a goat's hoof, with no sensors or actuators [17], and various models of passive dynamic walkers [1], [14], [18] all clearly demonstrate how the physical mechanisms can simplify control and control algorithms to deal with unstructured environments.On the other hand, VOLUME 4, 2016 studies in control theory and automation have made advances in co-design by taking into account the interplay between the physical body and controller through adaptive control schemes [19], [20].
One challenge in developing methods to share the control responsibility between physical mechanisms and feedback controllers is the difficulty to model dynamics of complex systems with multiple degrees of freedom.To tackle this, studies have provided evolutionary algorithms for system design [1], [2], self-assembly, and reproduction of robotic systems where the robotic systems are evolved using an algorithm based on some predefined elementary segments/building blocks to be used as physical solutions [11], [21].Such modular systems [22] are currently more easily adapted than non modular systems [23], [24] because modular systems evolve by changing a sub-section of the system without disturbing other parts.We propose a method to identify possible physical adaptations for a non-modular robotic system which results in a reduction of feedback control effort.The focus is on implementing physical adaptations in just one DoF/joint at a time without disrupting the performance of other joints in the system.The method attempts to bypass the complexities of analytical dynamic modelling by considering the projection of the dynamics on the phase portrait of a single joint.
Another challenge in apportioning the numerical control task to physical system is quantifying [25] how much of control task should be allocated to the physical structure and what shape or size [13] should the additional physical components take.Here we propose a method which focuses on adapting one DoF of the system at a time with the goal of reducing the controller's effort.An optimal range for size and shape of the physical adaptation emerges naturally from the information obtained from the behaviour of the embodied robotic agent.
Here, we propose a methodical approach to identify a range of possible local physical adaptations with the goal of reducing the control effort in second order systems.Given that the proposed method considers one DoF at a time, it can easily be extended to higher order systems with multiple degrees of freedom.This method attempts to resolve the aforementioned challenges to some extent or at least to simplify them to a degree.We use the uniqueness property of phase portraits and categorize the dynamic behavior of a robotic system based on the phase portrait at a single DoF.This will give us an insight into the behavior of the controller, the robot's physical structure and the interactions with the environment.This is done by focusing on the phase portrait of one DoF and proposing a physical solution by looking at local information without the need for a detailed knowledge of the global dynamics or kinematics of the robot.In the following section we refer to suggested physical components/hardware by the method as a physical adaptations of the system which reduce the control input.In the following sections, this paper first demonstrates theoretically, how to interpret the shape or form of energy needed to reach stability in a given system using phase portraits.Our method will then identify a physical solution that will complement required form of phase portrait.We use an inverted pendulum to conduct simulations and experiments.

II. ANALYTICAL MODELLING A. ANALYTICAL INSIGHTS INTO A PHYSICAL CONTRIBUTION IN CLOSED LOOP CONTROL
A nonlinear dynamical system q = f (q, u) with n linearly independent Degrees of Freedom (DoF), generalized coordinates q, can be linearized to obtain the state space form ẋ = Ax + Bu.The state vector is described by x = q 1 q1 q 2 ... q n qn and u = u 1 u 2 ... u r is the control vector.
In the case of a linear feedback controller, the control input u can be written in terms of state vector; u = −Kx where K ∈ R r×2n .Therefore the closed loop model of the system takes the form [26] Matrix A holds the physical system's information, relating to the physical system parameters, while BK contains information relating to the numerical controller.We introduce matrix M ∈ R 2n×2n to re-write BK as a summation of two matrices; BK = BK − M .M represents a physical addition to the original dynamical system to lower the control effort contributed by BK.Then, the closed loop state space system becomes Since matrix addition is associative and commutative, M can be subsumed in either matrices A or BK or both.
The linear feedback controller becomes where Ã = A + M .On that account, one can infer that the control effort can be decomposed into two sources; an internal and an external effort.The former comes from the internal dynamics; we refer to addition of a physical change in the hardware of the robot as physical adaptation represented by matrix M and the external effort is the controller input BK.This can be demonstrated by Adding M to matrix A means changing the physical system in the following two ways: 1) by varying the existing physical parameters, i.e. mass, length to centre of mass, and/or size of components.This involves changing the magnitude of these parameters which appear in both state matrix A and input matrix B. 2) Adding or tuning parameters such as mass, damping or anything in general that affect the kinetic energy parameters ( kinetic part T of Lagrangian L = T −V ) can affect multiple degrees of freedom.Moreover, these parameters appear in control input matrix B and changing them results in changing the control gains or controller.
The idea here is to find a new physical solution or component at one local joint such that it reduces the control effort.Therefore, the choice of new components should cause minimum changes to the overall dynamics (macro-dynamics) of the system.
Additional component should only introduce a change in the potential energy V , and have minimal effect on kinetic energy.for example, negligible mass, momentum, viscosity, damping, etc.Thus, it is essential to identify the dynamic behaviour of the states at the chosen DoF and to find physical equivalents (new components) to be added to the robotic system.

B. IDENTIFYING PHYSICAL SYSTEM ADAPTATION BASED ON THE LOCAL INFORMATION AT A SINGLE DOF
Modeling the dynamics becomes increasingly complex with the increase in DoF.Therefore, we use a method that focuses on the phase portrait of one single DoF.Phase portraits reflect the qualitative behaviour of the solutions to a dynamic system [27], [28].The method is based on a pattern match of the experimental phase portrait of the real dynamical system with predefined templates to identify a closest known class of dynamics for each DoF [28], [29].The phase portraits are obtained from the states of the system at one single DoF and it encompasses the physical structure of the robot and how it interacts with the environment.
In summary, the following four steps should be carried out: Step-1: Obtain phase portrait From experiment/simulation obtain the state trajectories of the chosen i th DoF, x i (t) = [q i (t), qi (t)] for i = 1, . . ., n and time t ∈ [0, t f ] where the final time t f is a user defined time window.Plot the phase portrait.
Step-2: Find closest model at single DoF Fit data from step 1 to each template solution [q (j) , q(j) ], where q(j) and q(j) are states from template solutions j = 1, . . ., 6 in table 1. Compute cost function [26] Choose the best fit model from table 1 using arg min j C (j) .
Step-3: Find physical parameter equivalence Once the optimal model from step 2 is obtained, we find a range of equivalent physical components that can render the chosen phase portrait behavior.Depending on the existing physical system and available resources, the human user or manufacturer can choose a component such as a magnet, a spring, or a light pulse.
Step-4: Determine magnitude of physical parameter In successive trials, adapt the magnitude/size of the chosen physical parameter at the i th DoF to maximize the user defined reward function.
Step-5: Maintain macro-dynamics Compare the phase portrait in i th DoF after physical adaptation to the phase portrait of the original system.They should both follow the same category from table 1.

C. A REFLECTION ON METHODOLOGY
Step-2 uses the uniqueness property of phase portraits [29].The set of solutions to (1) can be classified into a finite number of groups based on the nature of the eigenvalues of A as shown in table 1.The table contains 6 classes of phase portraits [28], [29].For the purpose of the dynamic system discussed in (1) and model in section III, i.e. the inverted pendulum, we focus on the non-singular case of matrix A. The classes of solutions represent both the original system and the control parameters.Therefore, the physical solution derived from the template is specific to the original system and the control objective.This leads us to Step-3.The phase portraits are solutions to differential equations of motion.The phase portrait of a dynamic system is formed by a collection of representative integral curves and is a geometric representation of the trajectories of the system [27], [30] and therefore indicate the form of energy a system follows at a particular joint.For example, the model solutions in table 1 exhibit a harmonic motion.Such attracting and repelling behavior can be obtained by adding a spring or a magnet at the corresponding degree of freedom.In this work, the user will choose an appropriate physical component, which follows the suggested energy form, depending on the existing physical system and available resources.In future research, it is possible to build a library including some information reflecting the type of hardware and link it to possible physical components.For example, a capacitor for an electrical system with current and voltage, a spring for a mechanical system or a magnet depending on the hardware material.Such a library will enable the system to be more autonomous in the process of selecting the physical component.Table 1 is only an example of solution phase portraits for systems with a dynamic behaviour described by equation (1).Therefore the physical solutions from this table follow a visco-elastic force field giving rise to physical solutions such as spring and magnet.The physical parameter will only adapt the part corresponding to the potential energy for example, stiffness with negligible damping as it is not physically plausible to have zero damping/friction.Just to name a few examples of dynamic systems with a different category of phase portraits, we can refer to a duffing oscillators, subharmonic oscillator and van der poll portraits [31].In these categories, this can be interpreted as physical components such as nonlinear springs, capacitors, light pulses, semiconductors and magnetic nanodots.
It is essential for the new system to maintain the same macro-dynamics as the original system.Hence, it should maintain the shape and category of original phase portrait after addition of the suggested physical component.Therefore, the magnitude of parameters in the suggested component should be chosen such that 1) After identifying the physical components, the parameter's magnitude should be determined (Step-4).For example the size of spring stiffness or magnetic flux density in the case of the visco-elastic force field, both analogous to impedance [32].Although the primary role of the new component is to reduce the control input, it should not interfere with the performance of other DoF.Therefore, an initial estimated magnitude of new physical component is optimized by minimizing the following cost function such that the control input, the error in states and settling time is minimized.
where e(x) ≡ x − x * is the error vector, x * is the desired states vector, t 0 is the settling time and u is the control/input vector as defined in II.λ, γ and η are the normalization factors for state vector errors, control vector and settling time respectively.There is a range of possible values for the magnitude of physical parameters which satisfy the above cost function.The number of solutions for the physical parameter magnitude satisfying equation ( 6) is too high and therefore there is no proof of optimality.It is sufficient to choose one optimal solution that minimizes equation ( 6) to suggest possible modifications.The human user can then choose the physical component based on what is available to them within a range of parameters satisfying equation ( 6). 2) The system stays within the same category of phase portrait after physical adaptations (Step-5).
The practical implementation of the physical component is heavily user dependant for the purpose of this research and achieving system autonomy in this aspect is out of the scope of this study.There are much research in the field of robotics to realize autonomy in physical adaptation and solution such as use of thermally tunable composites [33] and electroadhesive materials [34].

III. INVERTED PENDULUM EXAMPLE
Inverted pendulum is an inherently unstable, non linear, under-actuated system.Therefore, it is a good candidate to test the proposed method of using the phase portrait of a single DoF under any given controller for discovering a physical solution which lowers the burden of the controller.An inverted pendulum, modeled in Fig. 1 was used in experiments.The pendulum rod can swing around a passive pivot joint mounted on a motor-driven cart.A regulator type controller has been chosen to stabilize the pendulum in the vertical upright position.The governing equations of motion are as follows; Where all the symbols are defined as per table 2. Linearization around the equilibrium point x e = x e 0 0 0 obtains the state-apace equation; The goal is to identify suitable physical components that can be added to a chosen joint in the system in order to reduce the size of control input.In this study we consider the passive joint of the pendulum.The method was carried out as follows: 1) The phase portraits of pendulum joint were obtained from simulation (section IV) and experimental data (section V).
2) The phase portraits are then compared to the given models in table 1 and the best fit solution is found.Numerous methods can be chosen for this pattern matching such as different pattern recognition algorithms and regression.For the experiments in this paper, we fitted the simulation/experimental data to a set of solutions to the system described in (1) [27], [28].In accordance with table 1, six categories of solutions to (1) emerge depending on the form of eigenvalues of the system.Given that the control objective is stability of inverted pendulum, only stable solutions from table 1 were considered here; Complex Conjugate Eigenvalues (CCE) (λ = χ ± iψ) with nonzero real part and ψ < 0, CCE with χ = 0, Repeated Real Eigenvalues (RRE) with linearly independent eigen vectors κ1 = κ2, λ < 0, RRE with one linearly independent eigen vector κ1 = κ2 = κ and Distinct Real Eigenvalues (DRE) with negative eigen values λ 1,2 < 0. The simulation/experimental time data was substituted in these solutions and their derivatives to obtain q(t) and q(t) referred to in equation ( 5) respectively.A matlab fmincon function was used to find the optimal parameter values λ and κ to minimize cost C (j) for each template solution.The template with the least cost function is the best fit template.3) Each phase portrait represents a range of possible physical solutions that follow the same behaviour.Depending on the physical system, the human user can then choose a component such as a magnet, a spring, or an elastic band/tension belt.4) The magnitude of the physical parameter is optimized to minimize the cost function given by ( 6).This ensures that the Physical adaptation reduces the control input while it does not have an adverse effect on the performance of other DoF.e.g.Translational position and velocity errors should be minimized.5) The phase portrait of the system after physical adaptation was compared to the original portrait from step 1 to ensure they both fall under the same category in table 1.

Pendulum Back drivable servo motor Belt
Servo motor Figure 1: Experimental Setup.The inverted pendulum can swing in the plane of the belt.The belt is driven by a servo motor to move the base of the pendulum back and forth.The pendulum is hinged on the axis of a back drivable servo motor used to simulate the suggested physical system adaptations.

IV. SIMULATION
The simulation of modeling and control of inverted pendulum has been developed using MATLAB-R2016a (ODE45 function).A Linear Quadratic Regulator (LQR) controller was used for simulations.Table 3 gives the parameters of the inverted pendulum set-up used for experiments.After Linearization, the system matrices are computed as; Where state matrix A and input matrix B are as defined in section II.C and D are the output and feedforward matrices respectively such that output vector y = Cx + Du.Using the control law u = −Kx and with the choice of LQR weighting parameters Q = diag 1 1 1 1 and R = 1, the computed LQR gain vector is K = −3.1623−6.5988 66.4903 14.8010 .The simulation was started at an small initial angle deviation; x initial = 0 0 0.2 0 .
Fig. 2(a) shows the phase portrait of the simulated pendulum joint in solid black line (Methodology: Step-1).The dotted curves are the phase portraits obtained by substituting simulation time data t = [0, 2000] in equation (10) for different eigenvalue solutions as per table 1. Sampling interval of T = 0.002s was used.A Matlab fmincon function was used to minimize the cost function given in (5) for finding the best phase portrait match between the simulated data and model solutions in table 1 (Methodology: Step-2).Table 4 includes the cost values associated with each solution.The CCE solution with nonzero real parts (spiral portrait) has the least cost and is therefore the best match.Fig. 2(b) shows how different solutions in table 1 compare with the angular velocity trajectory of the simulated system.The velocity profile has been shown to offer clarity.It can be seen that the model with Complex Conjugate Eigenvalues (CCE) with non-zero real part is the best fit.This solution describes a damped harmonic motion, suggesting that a visco-elastic component is a good candidate as a new physical component at the pivot joint (Methodology: Step-3).For the specific case of the pendulum, we chose to add a simulated spring.A spring with a small spring constant was initially introduced to the system and the cost J associated with this stiffness was obtained using (6).The stiffness value was increased in steps while computing cost J for each stiffness value (Methodology: Step-4).Fig. 3 shows the effect of adding a spring at the pendulum joint and how control effort and cost J vary with stiffness.The control effort, Root Mean Square (RMS) of input force, decreases with the addition of physical adaptation.However, this does not imply an infinite stiffness maximizes the system's performance as the effect of the physical adaptation on other DoF should not be ignored.Computing cost J given by ( 6) ensures that besides the control input effort, the performance of other DoF and settling time is also taken into account.An optimal range of solutions for the spring stiffness can be found by comparing cost J as pendulum pivot joint stiffness is varied.In the case of the inverted pendulum, cost J starts to decrease as stiffness is introduced at the pendulum joint.however, a very high stiffness increases the state vector errors e(x) in (6) causing cost J to increase after a threshold.Fig. 2(d) compares the original controller to several iterations of adapting the magnitude of physical parameter (spring constant adaptations).It is important to choose the magnitude of the new component parameters such that the macro-dynamics of the system do not change.I.e. the phase portrait should stay within the same category of solutions in table 1.A very high spring stiffness, k = 18N/m, changes the shape of the phase portrait.Fig. 2(c) is the phase portrait of the system after physical adaptation with spring stiffness k = 9N/m (solid black line).The dotted curves are the phase portraits  6) between stiffness variations.The cost decreases as a spring is introduced to the system at the pendulum joint.After a stiffness threshold, the state errors start to increase.(c) Shows the numerical controller's effort, root mean square of input force, tends to decrease as stiffness is increased.Despite the reduction in the input effort, a more stiff joint is not favourable as cost J increases after a threshold.The simulation is repeated for different spring constants k = 0, 3, 6, 9, 12, 15, 18 N/m. of different templates (stable cases) from table 1.The blue dotted curve corresponds to the CCE solution with nonzero real parts.Table 4 confirms that this solution is the best fit to the simulated model after physical adaptation with the least value of cost C. Therefore, the requirement for the system to follow the same phase portrait prior and after physical adaptations is satisfied (Methodology: Step-5).

V. EXPERIMENT
Maxon EC60 brushless motor (part number 167132) was used with Maxon epos 50/5 controller to drive the belt which moved the base of the inverted pendulum back and forth to balance the pendulum.A T2.5 pulley with diameter of 23.87 mm was used to connect a 6mm belt and the servo motor shaft.The pendulum was fabricated using a hollow cardboard cylinder(1.1 m long with diameter of 0.024 m) mounted on the motor shaft.A Maxon EC-i52 brushless motor (part number: 574741) was used with Maxon epos 70/10 controller to simulate the behaviour of any suggested physical adaptations.Both motors had Maxon HEDL5540 joint encoders with resolution of 500 counts per turn (CPT).

A. SCENARIO 1: REGULATOR TYPE CONTROLLER WITH LINEAR QUADRATIC OPTIMAL FEEDBACK GAINS
We used a state feedback regulator with feedback gains K = −10 −30 100 25 to balance the pendulum.The experimental gains were tuned around the optimal gain suggested by LQR simulation.These gains were higher than those of the simulation due to the higher friction in the system, the nonlinearities and disturbances.Initially, the pendulum was balanced at the upright position and a current disturbance was applied for 200milliseconds.Four different current disturbances (I = 1.0, 1.5, 2.0 and 2.5A equivalent to 5.68, 8.51, 11.35 and 14.15N respectively) were applied to the cart to initiate the experiment.An "fmincon" Matlab function was used to find the best fit between experimental data and model solutions in table 1.An impedance controller was used to simulate the stiffness [32] of the suggested physical solution.The P gain relates to the stiffness and the D gain relates to damping.It can be noticed that when P = 0 and D = 0, the joint behaves as a pure revolute joint and the numerical controller is fully responsible for balancing the pendulum.

B. SCENARIO 2: A WEAK REGULATOR TYPE CONTROLLER
This scenario was tested to demonstrate that the method can identify the intention of the controller even when the phase portrait provides sub-optimal steady state behavior.In this scenario, the controller is only capable of balancing the pendulum for very small disturbances I ≈ 0.8A and therefore the system is not capable of reaching the control goal for larger disturbances.A state feedback regulator with feedback gains K = 0 0 100 25 was used to balance the pendulum.Similar to the first experiment, pendulum was balanced at the upright position.Four different current disturbances (I = 0.8, 1.1, 1.4 and 1.7A equivalent to 4.54, 6.25, 7.95 and 9.66N respectively) were applied to the cart to initiate the experiment.
A small current pulse of I = 0.8A was applies to the pendulum cart for a duration of 200ms.The proposed method in section II-B was carried out.The phase portrait of the pendulum joint state trajectories was obtained and a best fit to VOLUME 4, 2016 the models in table 1 was computed.Any larger disturbance would not bring the pendulum to the upright equilibrium position within the available range of cart position x ∈ [−0.35, 0.35]m.The phase portrait match was the same as the one in scenario 1; CCE model with nonzero real parts.Therefore, we used an impedance controller at pendulum joint.The experiment was carried out for five different values of P gain (stiffness-0 = 0, stiffness-1 = 200, stiffness-2 = 300, stiffness-3 = 350, stiffness-4 = 400) at the pendulum joint.A small D gain of 10N s/rad was used.Each experiment was repeated for 20 trials.

A. SCENARIO 1
Fig. 4 shows phase portraits of the original system and the portrait of the system after physical adaptation.Fig. 4(a) demonstrates the phase portrait of one representative trial of the original system (top), with no stiffness at pendulum joint in solid black line (Methodology: Step-1).The dotted lines are phase portraits of model solutions as referred to in table 1.We set the maximum velocity reached after the initial current disturbance as the starting point for the pattern match.The experimental time data was substituted in equation (10) for each of the five stable solutions described in section III (Methodology: Step-2).Table 6 includes the cost values C (j) , computed using ( 5), associated to each solution.The best fit model with the least cost is CCE with nonzero real parts.This solution is in agreement with simulation and suggests an addition of a component that would follow a visco-elastic force field (Methodology: Step-3).In this case, a mechanical impedance at the pivot joint best suits this particular dynamic solution.Therefore, we used a back-drivable motor at the pendulum joint to simulate a spring.A small stiffness was initially applied at the pendulum joint and the control cost associated with this stiffness was obtained using equation (6).The stiffness value was increased gradually while computing cost J (Methodology: Step-4).The phase portraits in fig.4(b) were obtained from the average angle deviation and average velocities over 20 trials for each category of stiffness and for four different initial current disturbances.A comparison of average (over 20 trials) input force by the state feedback controller at each particular stiffness and the corresponding behavior of average angle and position deviation is presented in fig.5(a).In this figure, only the case of I = 2.0A has been shown as a representative.5(b) demonstrates the optimal cost obtained from equation (6).Similar to simulation results, Cost J decreases with the addition of physical solution up to a threshold and therefore, an optimal stiffness range for this system is about 10 − 20N/rad.As discussed in section II-C, there is no necessity to be limited to one single optimal stiffness value but a range of possible stiffness values are available as long as they decrease cost J and controller input u.The user can choose one value depending on the design requirements.Fig. 4(c) shows the evolution of RMS values of input force as stiffness is increased.Table 5 includes the average (over 20 trials) values of cost J and average RMS values of control input for each stiffness and current disturbance.Fig. 4(a) shows the portraits after physical adaptation (bottom).This figure along with table 6 show that the inverted pendulum system after physical adaptation follows the same form of solution as the original system with no added stiffness (Methodology: Step-5).

B. SCENARIO 2
Fig. 6(a) demonstrates the phase portraits for one representative experimental trial under the weak feedback controller.Steps 1-5 of the method were followed similar to scenario-1.In summary, Step-1 was to obtain the phase portrait at the pendulum joint as shown in the top fig.6(a).
Step-2 was fitting the experimental data (angle and angular velocity) to the stable solutions in table 1 using equation (10) while minimizing cost C (j) given in (5).This is shown in fig.6 and table 7.For Step-3, we adopt the same solution as scenario-1; simulated spring using a back-drivable motor.Step-4 found an optimum range for the magnitude of physical parameters by minimizing cost J given in equation ( 6); fig. 4. Step-5 ensured the inverted pendulum system after physical adaptation follows the same phase portrait model as the original system; fig.6(a):bottom.Phase portraits in fig.6(b) were obtained from the average angle deviation and average angular velocities of all the trials for each category of stiffness and initial current pulse.The stiffness (P gain) values in the case of the weaker controller in this section are much higher than the values in scenario1 and the phase portraits do not necessarily follow the exact shape as the portrait of the original system.This is due to the high stiffness.In this scenario, the role of the physical system in achieving stability is more dominant than the numerical controller in presence of high disturbances.Therefore the phase portrait is more similar to the case of k = 18N/m in simulations shown in fig. 2

(b).
A comparison of average input torque by the state feedback controller at each particular stiffness and the corresponding behavior of average angle deviation and average position error is presented for the case of I = 1.4A in fig.7(a).Cost J computed as in equation ( 6) decreases with the addition of an optimal physical adaptation (stiffness-3) while the pendulum is balanced within the available translational position range.Fig. 7(b) shows a comparison of cost J computed from equation (6) at different current disturbances for a range of stiffness.Stiffness-3 (k = 350) is one optimal candidate in presence of the chosen disturbance range for this experiment however it is not the optimal value for current disturbances lower than I = 0.8A.Fig. 7(c) shows the evolution of average RMS values of input force as stiffness is increased.It is noteworthy that the controller could not balance the pendulum within the available translational distance for the cart movement for initial current disturbances higher than 1.1A.The physical adaptation enables the inverted pendulum system to reach under LQR control.The solid black curve represents the experimental data from the original system, prior to physical system adaptation(top) and the phase portrait of the system after physical adaptation, at optimal stiffness(bottom).The dotted lines demonstrate how five stable model solutions fit to experimental data.CCE, RRE and DRE denote complex conjugate eigenvalues, repeated real eigenvalues and distinct real eigenvalues respectively.The blue dotted line shows the best fit model to this data; Complex Conjugate Eigenvalue (CCE) with nonzero real part.The phase portrait of the system after physical adaptation follows the same template solution as the original system.(b) Average phase portraits (over 20 trials) for the original system and after physical adaptations for different stiffness levels.The chosen ranges are k = 0, 5, 10, 20, 30, 50N/rad.Each plot corresponds to a particular initial disturbance given at balance.The chosen current disturbances are I = 1.0, 1.5, 2.0, 2.5A (equivalent to 5.68, 8.51, 11.35 and 14.15 N respectively).Each disturbance was applied for 200ms.In the case of I = 2.5A the original system could not balance the pendulum.stability at high disturbances.In the case of experiment in scenario 2, we set a requirement for the system to operate in an environment with higher disturbance ranges.I.e.I = 1.1, 1.4, 1.8A therefore, we choose the physical adaptation accordingly.This implies that the phase portrait gives the format of physical adaptation and the optimal solution should be chosen based on the environmental factors, such as the user defined cost function and the statistics of environment.(c) Numerical controller's effort, root mean square of input force, averaged over 20 trials for each stiffness and initial current disturbance.The numerical controller's effort tends to decrease as stiffness is increased.Despite the reduction in the input effort, a more stiff joint is not favourable as cost J increases after a threshold.This experiment was repeated for current disturbances of I = 1.0, 1.5, 2.0, 2.5 A.
current disturbance range.

VII. DISCUSSION
This paper gives an analytic insight and experimental evidence for a new method whereby local physical adaptations are suggested in order to reduce the input effort required from a feedback controller to achieve a given task.We propose using the statistical average phase portrait at a single DoF and compare the pattern obtained from simulation/experimental data to a set of model solutions to find the phase portrait which best matches and describes the dynamics at the chosen DoF.
The phase portrait is a reflection of both the control objective and the state trajectories of the chosen DoF/joint in a specific environment.In other words, the shape of the phase portrait gives us an indication of the behavior of the chosen DoF in order to achieve the global control goal of the embodied robotic system.In this approach, the physical adaptations are derived based on existing physical resources.
In the inverted pendulum case considered in this paper, the physical adaptation was a visco-elastic force field on the pendulum angle.In a generic case, this can be any physical solution depending on the phase portrait of states of the relevant DoF and the physical resources available at the DoF concerned.For instance, there can be other physical solutions like a electromagnetic solution for a simple harmonic motion like in a fin of a fish, or a dominant spring solution like in a frog or a cricket who would often store and release energy.We used numerical simulations and hardware experiments to show that this physical adaptation leads to a reduction in a user defined cost function based on the input effort by the external control algorithm and external disturbances on the system from the environment.These findings based on a mechanical system agree with recent findings on biological systems.Studies on disturbance rejection during locomotion show how the mechanical impedance at the muscular level can relieve the feedback controller from micromanaging the trajectory of each joint [9], [18].
The extent of adaptation and evolution of controller and morphology design are more far-reaching and not limited to locomotion.Reducing the cost of information processing [8], [35]- [37] and decreasing the response time of a controller's action [9], [10], [38] are other advantages of an embodied view to control.The proposed method can be  66N respectively).Each disturbance was applied for 200ms.The original system with no stiffness could only balance the pendulum at I = 0.8A.Implementing physical adaptation allows the system to reach stability at higher current disturbances.The pendulum can stabilize at higher angle deviations compared to the original system.extended to autonomous robots in challenging environments after introducing several more layers of data processing.We envisage that a challenging environment will lead to diverse autonomous control actions that could be clustered using a set of user defined performance criteria.Then, statistics of phase portraits in each of these clusters can be used to generate physical adaptations suitable for each environmental context.These physical solutions may provide a useful basis for the next generation hardware iteration.However, developing such a comprehensive methodology is beyond the scope of this paper.This paper focused on a methodical approach to identify possible physical adaptations.In order to analyze the contribution of a physical adaptation in isolation, the controller remained unchanged through out the experiments in each scenario.The future research directions will involve a method for co-adaptation of the controller and the physical system as a duo and further development of the analytical basis for autonomous hardware evolution of robotic systems based on distributed sensor feedback of state variations under a given control policy to survive in a given set of environments.

VIII. CONCLUSIONS
In this paper, we present a method to use the phase portrait of states rendered by an external controller at a given degree of freedom (DoF) to innovate a physical adaptation that gradually reduces the burden on the external control algorithm.
Experiments were conducted for 2 scenarios.In the first scenario, a regulator type controller with optimum linear quadratic solutions of feedback gains was used to balance the pendulum.Implementing the suggested physical adaptation at the pendulum joint helps reduce the control input effort without interrupting the performance of neither the controller nor the other joints.As seen in table 5, an average cost reduction of up to 32.81%, 27.13% and 29.31% was achieved for initial current disturbance of 1.0, 1.5 and 2.0A respectively.The corresponding average RMS value of the input force for the aforementioned currents was reduced upto 28.52%, 30.52% and 15.58%.In the case of the weaker controller in scenario 2, the pendulum could not reach the upright unstable position.We used the shape of phase portraits of rotational DoF to have an insight into the intention of the control objective.The physical adaptation helped the controller achieve stability.This amounts to a gradual phasing out of a closed loop controller with a supportive layer added by the physical adaptation which is informed by the interplay of the closed loop controller, the dynamical system and its environment.

Figure 2 :
Figure 2: (a) Phase portraits of simulated controller at an initial angle of 0.2 radians.The dotted lines demonstrate how five stable model solutions fit to simulation data.CCE, RRE and DRE denote complex conjugate eigenvalues, repeated real eigenvalues and distinct real eigenvalues respectively.The blue dotted line shows the best fit model to this data; Complex Conjugate Eigenvalue (CCE) with nonzero real part.(b) Compares the velocity trajectory of the simulated data (solid black line) against the velocity trajectories of the model solutions.(c) Demonstrates the phase portrait of the system after physical adaptation.The closest template is the same as that of the original system; CCE with nonzero real part.(d) Compares phase portraits of the controller in its original form, stiffness k = 0 with the evolved system at six different stiffness values.

Figure 3 :
Figure3: (a) The Input force (top), angle error (middle), and position error(bottom) decreases as a spring is introduced to the system at the pendulum joint.After a stiffness threshold, the state errors start to increase (only angle and position errors shown as representatives).(b) Shows a comparison of cost J given in equation (6) between stiffness variations.The cost decreases as a spring is introduced to the system at the pendulum joint.After a stiffness threshold, the state errors start to increase.(c) Shows the numerical controller's effort, root mean square of input force, tends to decrease as stiffness is increased.Despite the reduction in the input effort, a more stiff joint is not favourable as cost J increases after a threshold.The simulation is repeated for different spring constants k = 0, 3, 6, 9, 12, 15, 18 N/m.

Figure 4 :
Figure 4: (a)  The phase portrait for one representative experimental trial with initial current disturbance of I = 2.0A under LQR control.The solid black curve represents the experimental data from the original system, prior to physical system adaptation(top) and the phase portrait of the system after physical adaptation, at optimal stiffness(bottom).The dotted lines demonstrate how five stable model solutions fit to experimental data.CCE, RRE and DRE denote complex conjugate eigenvalues, repeated real eigenvalues and distinct real eigenvalues respectively.The blue dotted line shows the best fit model to this data; Complex Conjugate Eigenvalue (CCE) with nonzero real part.The phase portrait of the system after physical adaptation follows the same template solution as the original system.(b) Average phase portraits (over 20 trials) for the original system and after physical adaptations for different stiffness levels.The chosen ranges are k = 0, 5, 10, 20, 30, 50N/rad.Each plot corresponds to a particular initial disturbance given at balance.The chosen current disturbances are I = 1.0, 1.5, 2.0, 2.5A (equivalent to 5.68, 8.51, 11.35 and 14.15 N respectively).Each disturbance was applied for 200ms.In the case of I = 2.5A the original system could not balance the pendulum.

Figure 5 :
Figure 5: (a) Average force, average angle and average position behaviour of the inverted pendulum, over 20 trials, with stiffness variations for the case of current disturbance I = 2.0A.The state errors initially decrease as stiffness is increased and starts to increase after a stiffness threshold.Only angle and position errors are shown as representatives.The experiment was repeated for different P gains k = 0, 5, 10, 20, 30, 50N/rad.(b) Average cost J given in (6) over 20 trials for each stiffness and across 4 different initial current disturbance.Initially cost J decreases however, after a threshold the cost value starts to increases.(c) Numerical controller's effort, root mean square of input force, averaged over 20 trials for each stiffness and initial current disturbance.The numerical controller's effort tends to decrease as stiffness is increased.Despite the reduction in the input effort, a more stiff joint is not favourable as cost J increases after a threshold.This experiment was repeated for current disturbances of I = 1.0, 1.5, 2.0, 2.5 A.

Figure 6 :
Figure 6: (a)  The phase portrait for one representative experimental trial of the system with weak controller.The solid black curve represents the experimental data from the original system, with small initial current disturbance (top) and the phase portrait of the system after physical adaptation, at optimal stiffness(bottom) under larger current disturbance.The dotted lines demonstrate how five stable model solutions fit to experimental data.CCE, RRE and DRE denote complex conjugate eigenvalues, repeated real eigenvalues and distinct real eigenvalues respectively.The blue dotted line shows the best fit model to this data; Complex Conjugate Eigenvalue (CCE) with nonzero real part.The phase portrait of the system after physical adaptation follows the same template solution as the original system.(b) Average phase portraits (over 20 trials) for the original system and after physical adaptations for different stiffness values.The chosen stiffness values are k = 0, 100, 200, 300, 350N/rad.Each plot corresponds to a particular initial disturbance given at balance.The chosen current disturbances are I = 0.8, 1.1, 1.4, 1.7A (equivalent to 4.54, 6.25, 7.95 and 9.66N respectively).Each disturbance was applied for 200ms.The original system with no stiffness could only balance the pendulum at I = 0.8A.Implementing physical adaptation allows the system to reach stability at higher current disturbances.The pendulum can stabilize at higher angle deviations compared to the original system.

Figure 7 :
Figure7: (a) Average (over 20 trials) force, angle and position behaviour of the inverted pendulum with stiffness variations for the case of current disturbance I = 1.4A.The state errors initially decrease as stiffness is increased however error starts to increase after a stiffness threshold.The experiment was repeated for different P gains k = 0, 100, 200, 300, 350N/rad.(b)Average cost J given in (6) over 20 trials for each stiffness and each initial current disturbance.Initially the cost J decreases, after a threshold the cost value starts to increases.(c) Numerical controller's effort, root mean square of input force, averaged over 20 trials for each stiffness and across 4 different initial current disturbance.The numerical controller's effort tends to decrease as stiffness is increased.Despite the reduction in the input effort, a more stiff joint is not favourable as cost J increases after a threshold.This test was repeated for current disturbances (I = 0.8, 1.1, 1.4, 1.7A).

Table 1 :
CLASSIFICATION OF PHASE PORTRAITS

Table 2 :
PARAMETER DESCRIPTION OF MODEL I Moment of inertia of the pendulum link computed at the CoM around z axis x Position of cart βx Coefficient of friction of cart θ Pendulum's deflection from the vertical axis g Acceleration due to gravity fn.c non conservative forces

Table 3 :
LIST OF EXPERIMENTAL PARAMETERS

Table 4 :
COST C OF FITTING SIMULATION PHASE PORTRAITS

Table 5 :
SCENARIO 1-COST J AND RMS VALUE OF CONTROL INPUT FOR EACH SPRING STIFFNESS The pendulum in balance was given a disturbance by giving a torque pulse to the motor driving the base of the pendulum.The torque pulse was generated by commanding different magnitudes of current pulses of 200ms duration.

Table 8
includes the average(over 20 trials) values of cost J and average RMS values of input force for each stiffness and

Table 6 :
COST C OF FITTING SCENARIO-1 PHASE POR-TRAITS

Table 7 :
COST C OF FITTING SCENARIO-2 PHASE POR-TRAITS

Table 8 :
SCENARIO 2-COST J AND RMS VALUE OF CONTROL INPUT FOR EACH SPRING STIFFNESS