The Inverted Pendulum Benchmark in Nonlinear Control Theory: A Survey

Abstract For at least fifty years, the inverted pendulum has been the most popular benchmark, among others, in nonlinear control theory. The fundamental focus of this work is to enhance the wealth of this robotic benchmark and provide an overall picture of historical and current trend developments in nonlinear control theory, based on its simple structure and its rich nonlinear model. In this review, we will try to explain the high popularity of such a robotic benchmark, which is frequently used to realize experimental models, validate the efficiency of emerging control techniques and verify their implementation. We also attempt to provide details on how many standard techniques in control theory fail when tested on such a benchmark. More than 100 references in the open literature, dating back to 1960, are compiled to provide a survey of emerging ideas and challenging problems in nonlinear control theory accomplished and verified using this robotic system. Possible future trends that we can envision based on the review of this area are also presented.


Introduction
Control theory is a field dealing with disciplines and methods that lead to an automatic decision process in order to improve the performance of a control system. The evolution of control theory is related to research advances in technology, theoretical controller design methods and their real-time implementation [1][2][3]. Control theory progress is also associated with education and control systems engineering [4][5][6].
Concerning engineering systems, many robotic benchmarks of high interest exist in the literature. They are frequently used to realize experimental models, validate the efficiency of emerging control techniques and verify their implementation. The most common robotic benchmarks are Acrobot [7], Pendubot [8], the Furuta Pendulum [9], the inverted pendulum [10], the Reaction Wheel Pendulum [11], the bicycle [12], the VTOL aircraft [13], the Beam-and-Ball system [14] and TORA [15].
In spite of its simple structure, the inverted pendulum is considered, among the last examples, the most fundamental benchmark. For this system, different versions exist, offering a variety of interesting control challenges.
The most familiar types of the inverted pendulum are the rotational single-arm pendulum [16], the cart inverted pendulum [17] and the double inverted pendulum [18]. The less common versions are the rotational two-link pendulum [19], the parallel type dual inverted pendulum [20], the triple inverted pendulum [21], the quadruple inverted pendulum [22] and the 3D or spherical pendulum [23].
This paper proposes to enhance the wealth of the pendulum benchmark and attempt to provide an overall picture of historical, current and trend developments in nonlinear control theory based on its simple structure and its rich nonlinear model. The paper is organized as follows. The wealth of the inverted pendulum model will be explained in the next section. In Section 3, different control design techniques will be surveyed through applications of this benchmark. Trends and challenges will be addressed in the last section.
2. Why the inverted pendulum is the most popular benchmark?
The inverted pendulum benchmark (see Figure 1) can be considered as the simplest robotic system, with only one rigid body and only one rotational joint. For this robotic system, let us denote by  the angle between vertical and the pendulum, which is positive in the clock-wise direction and by m, J and  , the mass of the pendulum, the moment of inertia with respect to the pivot point and the distance from the pivot to the centre of mass, respectively. The acceleration of gravity is g and the acceleration of the pivot is u. Using the Newton Euler approach and under some predefined assumptions, the equations of the motion of the pendulum can described by [16]: The energy of the system can be deduced as: where: The normalized state space system is then described by: As can be deduced from system (4), the inverted pendulum system is a typical nonlinear dynamic system including a stable equilibrium point when the pendulum is in a pending position and an unstable equilibrium point when the pendulum is in an upright position. When the system is moved up from the pending position to the upright position, the model is strongly nonlinear with the pendulum angle.
However, though the simplicity of its structure, many standard techniques in control theory are ineffective when tested on the inverted pendulum benchmark. The study of the system is much more difficult than it appears to be for many reasons, e.g., many geometric properties of the system are lost when the pendulum moves through horizontal positions [33]. Indeed, the output-zeroing manifold does not contain any equilibrium points, the relative degree of the system is not constant and the controllability distribution of the system does not have a constant rank. The system is therefore not linearizable [34].
The control problem to be solved is generally composed of two specific tasks: the first one is the upswing control of the pendulum and the second one is the stabilization of the pendulum around the unstable equilibrium point [35][36]. Finding a continuous feedback that can render the vertical upward position of the pendulum globally asymptotically stable has been, until recently, considered a major problem.

Major accomplishments in control theory
Since the 1950s, the inverted pendulum benchmark has been used for teaching linear feedback control theory [37] to stabilize open-loop unstable systems. The first solution to this problem was described in 1960 by Roberge [38] and then by Schaefer and Cannon in 1966 [39]. This benchmark was considered in [40] as a typical model for root-locus analysis and was used in [41] to solve the linear optimal control problem.
The inverted pendulum benchmark has in addition maintained its usefulness in nonlinear control theory. When robustness performances are required, a sliding mode control approach is applied. In such cases, stability conditions are well proven and robustness performances are often guaranteed. Time optimal control and predictive control techniques can give very satisfactory results by solving optimization problems. However, the predictive control approach suffers in many cases of a lack of stability condition proofs.

Future trends and challenges
In this section, possible future trends that we can envision based on the review of this area are presented. One of the more complex problems is the global asymptotic stabilization of the origin by a single and smooth continuous feedback for hybrid systems. For the inverted pendulum case, the problem can be approached by designing a single controller to realize both the upswing and the stabilization control. A complete solution to this crucial problem was recently proposed in [65,[109][110]. For non linear systems, handling delays, instable internal dynamics and actuator saturation are also considered as challenging problems. For the rotational single-arm pendulum, the output regulation problem under uncertain conditions in the presence of nonlinear backlash effects is solved in [111] using hybrid architecture, which combines Type-1 or Type-2 fuzzy logic systems and genetic algorithms. A solution to the same crucial problem for the cart inverted pendulum is proposed in [109] using an input-output linearization, energy control and singular perturbation theory.
Alternatively, friction is a very complicated phenomenon that occurs in all control systems. When friction effects are not modelled, feedback controllers may fail. Indeed, frictions are usually associated with oscillatory behaviours or in limiting cycles. In this framework, very few results exist, see for example [112] and references therein.
Regarding the question of the lack of a mathematical method proving the stability and robustness of the fuzzy control approach, a solution to this problem was recently proposed in [113], where type-1 and type-2 fuzzy logic controllers are based on a Lyapunov theory. First, the control problem is solved for the rotational single-arm pendulum without a gravity effect to prove stability conditions and then robustness conditions are proved for the nonlinear system with a gravity effect. An extension of this approach for non-smooth mechanical systems is found in [114].
On one hand, based on the inverted pendulum stabilization principle, many robotic control strategies are emerged. Recent major accomplishments include control design of under actuated robotic systems [115][116][117], mobile wheeled inverted pendulums [118][119][120][121] and gait pattern generation for bipedal and humanoid robots [122][123][124][125][126][127][128][129][130][131][132]. In most cases, the non linear control problem is based on finding a suitable reference trajectory obeying a certain criteria formulated as a constrained optimization problem (see [133] and [134] and references therein). For such complex control problems, new hybrid control approaches, such as fuzzy-neural control approaches [135,136,137], are emerging. The use of neural networks has increased in recent years in control theory, since they do not involve any mathematical theories. They reduce the development cost of controllers for complex systems, particularly nonlinear ones [138]. For tuning controller parameters and solving constrained control optimization problems of robotic systems, bio-inspired methods, such as genetic algorithms [111], particle swarm optimization [139,140] and ant colony optimization [141], are promising optimization algorithms. The recent concise review in [142] explains and discusses, for example, the design of type-2 fuzzy systems using such optimization methods. It is expected that these approaches and similar ones could emerge in the near future in the area of nonlinear control theory and especially for robotic applications.
On the other hand, chaos theory is an open research area extensively investigated nowadays in nonlinear control theory including very promising research fields such as chaos analysis, chaos control and chaos synchronization. In this framework, the damped driven pendulum can be considered as a basic example. Indeed, it is recently shown, that if the pendulum is placed in certain spots, the corresponding motion will become chaotic [143,144]. The coexistence of uncountable non-periodic motions and countable periodic motions of the pendulum is proved. New results on chaos synchronization of the pendulum motion with other systems can be also found in [145].

Conclusion
In this paper, it has been shown that the inverted pendulum system is a fundamental benchmark in nonlinear control theory. The particular interest in this application lies in its simple structure and the wealth of its nonlinear model. The simple structure allows real and virtual experiments to be carried out. The richness of the model has shown its usefulness in illustrating all emerging ideas in nonlinear control theory.
At the moment, energy based control, energy-shaping techniques, hybrid control approaches and variable structure control approaches can be considered as the most mathematical rigorous approaches to solve the upswing control and the stabilization of the pendulum around the unstable equilibrium point since global stability is based on Lyapunov theory. However, new hybrid control approaches, such as fuzzy-neural control approaches and bio-inspired methods like genetic algorithms, particle swarm optimization and ant colony optimization, have attracted more attention because they reduce the complexity of controllers.
Finally, based on this review, we can confirm that handling for nonlinear systems, delays, instable internal dynamics, uncertainty conditions, actuator saturation and chaos dynamics are the future trends and the most challenging problems to be solved in control theory.