Hybrid Functions Direct Approach and State Feedback Optimal Solutions for a Class of Nonlinear Polynomial Time Delay Systems

The aim of this paper is to determine the optimal open loop solution and a nonlinear delay-dependent state feedback suboptimal control for a class of nonlinear polynomial time delay systems. The proposed method uses a hybrid of block pulse functions and Legendre polynomials as an orthogonal base for system’s states and input expansion. Hence, the complex dynamic optimization problem is then reduced, with the help of operational properties of the hybrid basis and Kronecker tensor product lemmas, to a nonlinear programming problem that could be solved with available NLP solvers. A practical nonlinear feedback controller gains are deduced with respect to a least square formalism based on the optimal open loop control results. Simulation results show efficiency of the proposed numerical optimal approach.


Introduction
Time delays affect systems dynamics in many engineering applications like chemical control systems, biology, and medicine [1,2].Delays are also encountered in communication and information technologies like high-speed communication networks [3].It should be noted that time delay may be, in some applications like communication lines, a source of instability and performance degradation [4].Time delay system is therefore a very important class of processes whose stabilization [5] and optimization [6,7] have been of interest to many researchers.
Particularly, many attempts have been made in literature to solve optimal control problems for many classes of linear [8][9][10][11] and nonlinear [6,12,13] time delay systems.Among them, we recall the application of Pontryagins maximum principle to the optimization of control systems with time delays which was firstly proposed by [14].It had been shown that it results in a system of coupled two-point boundaryvalue (TPBV) problem involving both delay and advance terms whose exact solution, except in very special cases, is very difficult to determine (see [15]).Perhaps one of the most effective techniques is dynamic programming approaches (see [2]) for overcoming the complexity of the nonlinear time delay systems in optimal control problems.Of course, application of dynamic programming methods has some difficulties due to the need to provide an appropriate level model and also to define recursive relationships for each case problem.Also a computational algorithm considering a linear approximation of the original system which is defined about a nominal trajectory is offered by [16].Clearly, using the linear approximation is not reliable and may lead to large errors.Reference [7] proposed an approach based on discretization techniques and necessary conditions to obtain approximate optimal control and the state for optimal control problems with nonlinear delay systems.Despite the good performance of this method, achieving the necessary conditions in some problems and the implementation of approach may be faced with difficulties.So different numerical methods have been proposed to avoid the problems arising from the applications of analytical methods.It is then straightforward that many of the numerical methods dedicated to solving 2 Complexity classical optimal control problems have been extended to handle optimal control problems governed by time delay systems.
Typically, direct methods are based on converting the dynamical optimal control problem into static optimization problem.Among direct methods, parametrization technique [17][18][19] is known to minimize decision variables compared to the discretization of the problem [7].It is worth noting that parametrization relies basically on orthogonal functions or wavelets [20][21][22]; however that tool have been used to solve various other problems of dynamic systems like identification (see [8]), tracking control (see [23]), observer based control (see [24]), or minimum time control (see [25]).The main characteristic of this pseudo-spectral technique is that it allows transforming complex dynamic optimization problems to solving a set of algebraic equations in the least square sense in the linear systems case [26,27] or permits formulating an equivalent nonlinear static programming problem for problems related to nonlinear systems [13,28,29].
In recent years, a growing interest has been appeared toward the application of hybrid functions, which is a combination of block pulse and an orthogonal polynomials basis [26].In the nonlinear time delay optimal control problems context, an approach using hybrid functions which consist of block pulse functions and orthonormal Taylor series (see [15,29]) had been proposed, where authors propose to solve the necessary and sufficient condition equations for stationary emanating from the Hamiltonian based on state and control coefficients over the basis.Similarly, [28] propose a direct approach based on a hybrid of block pulse functions and Lagrange interpolating polynomials in order to convert the original optimal problem containing multiple delay into a mathematical programming one, where the resulting optimization problem is solved numerically by the Lagrange multipliers method.Reference [27] proposed similar approach based on hybrid functions of block pulse and Bernouilli polynomials, while [30] uses biorthogonal cubic Hermite spline multiwavelets in addition to block pulse functions to constitute the hybrid basis.Although above contributions treat some nonlinear delayed optimal control problem, they do not propose any general nonlinear programming problem that could handle all examples depicted in their works.In fact, for each considered nonlinear system, a nonlinear optimization problem is formulated and then solved with an NLP solver.Furthermore, only open loop control solutions are investigated therein, which is not of great interest in practice.
In the present paper, we introduce a direct method to solve forwardly the finite time quadratic optimal control problem of polynomial systems with delayed state, by the use of hybrid functions of block pulse and Legendre polynomials.The operational matrices of delay and Kronecker product specific to that basis are recalled.At first, the open loop solution of the nonlinear time delay optimal control problem is investigated.Secondly, a suboptimal nonlinear state feedback is determined based on the first part results.Hence, the main contributions in this work could be summarized as follows: (a) expressing the constraint of the formulated NLP problem properly for the class of polynomial systems; thus the proposed formulation could handle a wide range of nonlinear analytic nonlinear systems.Then, a unified development is carried for that class of systems, (b) deriving a nonlinear polynomial delay-dependent nonlinear suboptimal state feedback that reproduce the optimal state trajectories determined in the open loop framework, (c) using an hybrid basis with reduced number of elementary functions, which makes open loop synthesis faster, with a good enough accuracy compared to other approaches, and closed loop solution within a simpler formulation and resolution.
The remainder of the paper is organised as follows.In the second section, hybrid functions and their properties are introduced.In the third section, the open loop numerical solution of the nonlinear time delay optimal control problem is detailed.The suboptimal closed loop framework is presented in the fourth section.In the fifth section, computational results are depicted.Finally, concluding remarks and future works are presented.

Hybrid Functions
Hybrid functions ℎ  (),  = 1, 2, ⋅ ⋅ ⋅ ,  = 0, 1 ⋅ ⋅ ⋅  − 1, have three arguments;  and  are the order of block pulse functions and Legendre polynomials, respectively, and  is the normalized time.They are defined on the interval  = [0,   ] as [26] ℎ  () Here,   () are the well-known Legendre polynomials of order  which constitute the base L() and satisfy the following recursive formula: We define Φ() the vector of  block pulse functions   (),  = 0, 1, ⋅ ⋅ ⋅  − 1, as follows: Since ℎ  () is the combination of Legendre polynomials and block pulse functions which are both complete and orthogonal, then the set of hybrid functions is a complete orthogonal system.
In the rest of the paper we notate  =  the dimension of the hybrid basis.

Operational Matrix of Integration. The integration of h(𝑡)
can be approximated by [26] where  is the integration operational matrix of order × where and 2.2.Delay Modeling with Hybrid Functions.A vector function () of r dimensional components which are square integrable in [0,   ] can be approximated by a block pulse series as where  = [ 1 ,  2 , . . .,   ]  .For an  component delay vector variable ( − ) with the block pulse series approximation of ( − ) may be defined as [31] where with and  is the number of block pulse functions considered over 0 ≤  ≤ , and Then, it comes [31] V (, ) and (, ) are called the shift operational matrices, given by and It is worth noticing that the Shift operational matrices of hybrid functions could be derived forwardly from those of block pulse functions by where ⊗ stands for the Kronecker product.However, it should be noticed that block pulse functions are fundamental for delay modeling.The choice of  depends on  and   , which issue had been addressed in [26].In this framework, we propose to choose N as follows: where  is a nonnegative integer, to be chosen bigger than one if possible in order to improve approximation and (.)denotes the nearest integer function [22] (implemented by  routine in MATLAB).

The Integration of the Cross Product. The integration of the cross product of two hybrid functions vectors h(𝑡) can be obtained as [26]
where  is an × matrix.0  stands for the zero  ×  matrix and  is an  ×  diagonal matrix that is given by 2.4.Kronecker Product Operational Matrix.It would be interesting, for bilinear systems, as it will be proven later, to investigate the Kronecker product operational matrix for hybrid functions.This particular matrix operator derivation, as it is the case for the integration, cross product, and delay operators, is highly inspired of the Kronecker operational matrix of both Legendre polynomials and block pulse functions.
For the block pulse functions, we can state with where  Φ ∈ R  2 × is the Kronecker product operational matrix of block pulse functions and the matrix  × , is defined in Appendix.
On the other hand, the product of two Shifted Legendre Polynomials   () and   () can be expressed by with A practical implementation of the latter scalar products is given in [19].
Then, we may write where    is a  ×  square matrix.Then it comes where  L ∈ R  2 × is the Kronecker product operational matrix of Legendre polynomials.
Based on relations ( 26) and ( 23), we define where  h ∈ R  2 × is the Kronecker product operational matrix of hybrid functions.

Numerical Solution of the Nonlinear Time Delay Optimal Control Problem
3.1.Description of the Studied System.We consider the nonlinear continuous system which can be represented by the following state space representation: where () ∈ R  is the state vector, ( − ) ∈ R  is the delayed state where  denotes the time delay, () ∈ R  is the control vector, and ((, )) from R  into R  and ((, )) from R  into R × are nonlinear analytic functions of () and ( − ).
The direct approach presented in this paper is based on expanding system equations ( 28), (29), and (33) as well as objective function (34) to be minimized over an hybrid functions basis.Hence, the main purpose is to transform the optimal control problem under dynamic constraints to a nonlinear programming problem.To this end, each of the state and control variables is approximated by a finite length of unknown parameters as follows: where  and  are unknown state and control parameters, respectively.Applying the V operator (see Appendix) and related Kronecker product property [32] yields where   and   are  ×  and  ×  identity matrices.Moreover, at the initial time,  0 = 0, the initial state could be written where is an  constant vector.
For clarity purpose, let us denote  as the whole unknown parameters vector.  = V(  ) and   = V(  ) are, respectively, the state parameters and the control ones, such that and  0 = V(  0 ).According to ( 14) and ( 17), the delayed state coefficients are given by V ( *  ()) =  ℎ  0 +  ℎ   (40)

Optimal Control Problem Reformulation Using Hybrid
Functions.The cost function (33) is composed of two parts.
The first is the terminal penalty of the state, while the second is known to be the running cost.

Cost of the Final State Approximation.
At the final time,   , the state approximation could be written It is important to mention here that hybrid functions inherit an important property from Legendre polynomials(  (  ) = 1, ∀ = 0, . . .,  − 1).In fact, the subset ℎ  () verifies The rest of hybrid functions are null at  =   .The cross product of two hybrid functions at the final time is given by where   is  ×  matrix. 1  stands for the all-ones  ×  matrix.
Hence, the terminal penalty of the state could be approximated as follows: which is equivalent to Using the integral of the cross operational matrix , it reduces to    ( ⊗ )   +    ( ⊗ )   (47)

System Path Approximation.
The expansion of the system state over a hybrid basis requires the development of functions ((, )) and ((, )) over that basis.To this end, several preliminary lemmas need to be introduced.

Lemma 1. The development of the 𝑖 𝑡ℎ Kronecker power of the state vector over a hybrid basis h(𝑡)
gives where with and We recall that   [] denotes the  ℎ Kronecker power of the state coefficients   , with K ℎ being the operational matrix of the Kronecker product.
Proof.The proof of this lemma needs only a few manipulations.
Notice that results of Lemma 1 could be applied to the  ℎ Kronecker power of the delayed state coefficients (i.e.,  [] ( − ) =  * T [] .h())and express it in terms of decision variable   by the mean of relation (32): Expansion of ((, )) over the Hybrid Basis.The third term of ( 29) could be developed over the hybrid basis as follows: where we notate . Now, ((, )) could be approached as follows: where Expansion of ((, ))() over the Hybrid Basis.Notice that the first term of ((, ))() under the sum could be expanded over the hybrid basis as follows: while the second one could be derived similarly.
The third term could be approached as where  and  belong to N * .Then it comes with Expansion of System Equation over the Hybrid Basis.The integration of the system equation by introducing the operational matrix of integration  with respect to notations (58) and ( 54) gives Our objective is to express the constraint (60) in terms of decision variables   and   ; to this end we apply the V operator to (60).That allows us to state Using the linearity property of the V operator it comes which is equivalent to Lemma 2.
Proof.The proof of this lemma needs only a few manipulations.
Notice that results of Lemma 2 could be applied to the  ℎ Kronecker power of the delayed state coefficients (i.e.,  *  [] ) and express it in terms of decision variable   by the mean of relation (42).We note Applying the V operator to (  [] ⊗  *  [] ) yields Similarly Finally, the system path constraint could be implemented using the following equation: Now, it could be noticed that the system path constraint is expressed properly in terms of unknown parameters   and   .

The Nonlinear Programming
Problem.The optimal control problem has been approximated by a nonlinear programming problem and is given by the following: find the optimal vector  of the unknown parameters   and   that minimizes subject to (68).
One has The mathematical programming problem can be solved by using available nonlinear programming solvers like IPOPT or the routine  of the MATLAB Toolbox.
After solving the latter nonlinear programming problem and determining the optimal value of the unknown parameters vector , these parameters are substituted back into (28) to determine the optimal state vector and the optimal control.

Suboptimal Feedback Control
Once the optimal open loop results are obtained by solving the nonlinear programming problem given by ( 69)-(68), let us note the optimal state and control coefficients.We are interested now, based on previous results, to synthesize the following nonlinear state feedback control law: The idea is to find control matrices   and   such that the optimal vector (71) verifies the control equation (72).
Expanding (72) over the hybrid basis yields Substituting the control and state coefficients with their optimal values and applying the V operator give with Finding control parameters could be then reduced to solving, in the least square sense, the following problem: where V ( 2 ) . . .

Example 1. Consider the system [33]
When  = 0, the above system has two equilibria, one which is at the origin and the other one at (1/3, 1/3).
In this example, we aim to minimize the following criterion ∫  Nonlinear system (77) could be written under a polynomial form (28), (29), and (32) with The time delay is considered as  = 2.

Open Loop Study.
The development presented above is implemented in this subsection by using both hybrid functions (HFs) and block pulse functions (BPFs).Table 1 summarizes considered parameters for simulations below and obtained performances indexes with the different bases.It is then clear that hybrid basis is superior over the piecewise constant one, both with the same number of elementary functions.
Simulation results for the above open loop controlled system initialized with (0.5, 0, 5) are given in Figures 1 and  2 based on hybrid and block pulse functions.
Controlled state trajectories, obtained with determined gains, are depicted in Figure 3.It is shown that the hybrid functions technique is also better in closed loop.
Figure 4 shows the optimal states trajectories obtained by minimizing the formulated NLP problem, by using the hybrid of block pulse and Legendre polynomials basis, over a finite horizon   = 10.Controlled states with obtained suboptimal feedback are drawn on the same figure over a simulation time 20.It could be seen that system states converge to the origin equilibrium with respect to imposed criterion.Figure 5 exposes optimal control and suboptimal state feedback control signals using HFs.

Example 2:
A Two-Stage Chemical Reactor.In this section, we consider a cascade chemical system with two reactors [34] where   ,  = 1, 2, are the compositions,  is a known time delay,   are the recycle flow rates,   are the reactor residence times,   are the reaction constants,  is the feed rate,   are reactor volumes, and   are nonlinear functions for describing the system uncertainties and external disturbances.Note that (80) is the transformed reactor model given by [34].The original one may have a nonzero equilibrium point and the compositions   actually denote the deviations from the equilibrium point.
Nonlinear system (80) could be written under a polynomial form (28), (29), and ( 32 Table 2 gives simulations parameters and obtained Performances indexes with the two bases utilized in Example 1. Also, state feedback gains are included.
It is worth noting that for this particular system, control gains determined by the mean of block pulse functions are not    stabilizing.While hybrid functions results are illustrated on Figures 6 and 7, controlled states with obtained suboptimal feedback designed over a finite horizon   = 3 are drawn on the same figure over a simulation time 5.It could be seen that suboptimal system states coincide perfectly with optimal solution.Figure 7 exposes optimal control and suggested suboptimal nonlinear state feedback control signals.The proposed Complexity nonlinear feedback reproduces sharply the optimal open loop control.

Conclusion
In this paper a practical approach is developed to solve the problem of finite time quadratic optimal control for polynomial time delay systems.The proposed method is based on the expansion of the system model on a complete set of orthogonal hybrid of block pulse and Legendre polynomials.Two types of optimal control laws have been investigated.In the first step, the method focuses on the determination of the open loop optimal control law.Thus, by defining a general NLP problem for the considered system class, in the second step, a nonlinear delay-depending state feedback control law has been derived in order to meet the optimal states trajectories.The developed results have been illustrated on different examples of nonlinear time delay systems; namely, a two-stage chemical reactor and the obtained results are significant.Note that the proposed method may be enhanced, by decreasing the polynomial system matrices order when introducing the nonredundant form.In addition, if the state information is not completely available, the output feedback is a good choice.The method is feasible for the output feedback control and could be even extended to handle dynamic nonlinear state/output feedback synthesis.Moreover, the presented development is limited to some class of analytical nonlinear systems.In future work, we intend to apply the presented method to the class of switched systems, where the used hybrid functions seems to be convenient to treat that models which are a mixture of smooth functions and piecewise constant signals.

Figure 6 :
Figure 6: Optimal and suboptimal states trajectories of the chemical reactor.

Figure 7 :
Figure 7: Optimal and suboptimal control signals for the chemical reactor.

Table 1 :
Hybrid functions and block pulse functions direct approach performance analysis for Example 1.

Table 2 :
Hybrid functions and block pulse functions open loop and closed loop results for Example 2.