A hybrid direction algorithm for solving optimal control problems

Abstract: In this paper, we present an algorithm for finding an approximate numerical solution for linear optimal control problems. This algorithm is based on the hybrid direction algorithm developed by Bibi and Bentobache [A hybrid direction algorithm for solving linear programs, International Journal of Computer Mathematics, vol. 92, no.1, pp. 201–216, 2015]. We define an optimality estimate and give a necessary and sufficient condition to characterize the optimality of a certain admissible control of the discretized problem, then we give a numerical example to illustrate the proposed approach. Finally, we present some numerical results which show the convergence of the proposed algorithm to the optimal solution of the presented continuous optimal control problem.


PUBLIC INTEREST STATEMENT
The optimal control theory consists in finding a control which optimizes a functional on a domain described by a system of differential equations, with box and terminal constraints on the control. This theory is applied in various fields of the engineering sciences: aeronautics, physics, finance, etc. For example, finding the minimal time necessary for moving a missile from one starting point to a destination point can be modeled as an optimal control problem, where the constraints are given by the motion equations of the missile. In this work, we have proposed a method which finds a numerical solution for the linear optimal control problem. Our method can be used for the simulation of optimal trajectories of control problems which arise in military applications, finance, etc.

Introduction
The optimal control theory consists in finding a control which optimizes a functional on a domain described by a system of differential equations, with box and terminal constraints on the control. This theory is applied in various fields of the engineering sciences: aeronautics, physics, finance, etc. Because of the importance of this theory, several researchers have been interested in the development of effective numerical methods for solving this type of problems. In (Gabasov & Kirillova, 1980;Gabasov, Kirillova, & Prischepova, 1995), the authors developed the adaptive method for solving linear optimal control problems. This method is then generalized for solving general quadratic optimization problems (Bibi, 1994(Bibi, , 1996Brahmi & Bibi, 2010;Khimoum & Bibi, 2019;Kostina & Kostyukova, 2001).
In (Bentobache, 2013;Bentobache & Bibi, 2016;Bibi & Bentobache, 2011, 2015, the authors proposed a new improvement direction for the adaptive method in order to solve linear programming problems with bounded variables. This direction is called hybrid direction because some of its components take extreme values and the other components take the values of the opposite gradient. In this paper, we present an algorithm based on this hybrid direction for solving linear optimal control problems. In a similar way to (Bibi & Bentobache, 2015), we define an optimality estimate and give a necessary and sufficient condition to characterize the optimality of a certain admissible control of the discretized problem. Then, we describe a numerical algorithm for finding an approximate solution and we present some numerical results in order to show its convergence.
The paper is organized as follows: In Section 2, we present the problem and give some definitions. In Section 3, we present the details of the proposed algorithm and we give a numerical example to illustrate our approach in Section 4. Finally, we conclude this paper and give some perspectives.

State of the problem and definitions
Consider the following terminal optimal control problem: where JðuÞ is the quality criterion, A 2 M nÂn R ð Þ is the dynamic matrix of the system, xðtÞ 2 R n is the state vector of the system, b 2 R n , H 2 M mÂn R ð Þ is a matrix of rank m n, g 2 R m , uðtÞ 2 R is a piecewise constant control bounded by f Ã ; f Ã 2 R and c 2 R n . The symbol (') designates the transposition operation.
An admissible control u 0 ¼ ðu 0 ðtÞ; t 2 TÞ is said to be optimal if Jðu 0 Þ ¼ max u JðuÞ: An admissible control u ε is said to be ε À optimal or suboptimal if where ε ! 0 is an accuracy chosen in advance.
The solution of the problem consists in the determination of an admissible control u 0 which, with the trajectory x 0 , maximizes the quality criterion JðuÞ: The solution of the system ð1:2Þ is given by where FðtÞ; t 2 T; is the solution of the system _ FðtÞ ¼ AFðtÞ; Fðt Ã Þ ¼ I n ; t 2 T; and the matrix I n represents the identity matrix of order n.

Support control
is called a support if the corresponding matrix P B ¼ ðdðτÞ; τ 2 T B Þ 2 M mÂm ðRÞ is nonsingular.
The pair fu; T B g formed by the admissible control u and the support T B is called a support control of the problem ð4Þ. The latter is said to be nondegenerate if f Ã < uðτÞ < f Ã ; τ 2 T B :

Increment formula of the functional
Let fu; T B g be a support control and xðtÞ; t 2 T; its corresponding trajectory. Using the support T B , we construct the vector of the potentials ν 2 R m and the cocontrol vector EðτÞ; τ 2 T h , as follows: Consider another control uðtÞ ¼ uðtÞ þ ΔuðtÞ; t 2 T; and the corresponding trajectory Then, the increment of the functional ð4:1Þ is given by The following theorem gives a necessary and sufficient condition of optimality for an admissible control u of the problem ð4Þ.
Theorem 1 (Gabasov et al., 1995) The following relationships: are sufficient, and in the case of the nondegeneracy of the support control fu; T B g also necessary, for the optimality of the admissible control u: 3. An iteration of the hybrid direction algorithm Let fu; T B g be a support control for the problem ð4Þ and η 2 ½0; 1. Define the following sets: Recall that the suboptimality estimate βðu; T B Þ is given by the following formula (Gabasov et al., 1995): We call optimality estimate, the quantity γðη; u; T B Þ defined by: Theorem 2 (Necessary and sufficient condition of optimality (Bibi & Bentobache, 2015)) Let fu; T B g be a support control for the problem ð4Þ and η > 0. Then the condition γðη; u; T B Þ ¼ 0 is sufficient and, in the case of the nondegeneracy of the support control fu; T B g also necessary, for the optimality of the admissible control u.
Let fu; T B g be a starting support control of the problem ð4Þ, for which the optimality criterion is not satisfied. An iteration of the hybrid direction algorithm consists in moving from fu; T B g to fu; T B g, where u ¼ u þ θ 0 Δu: This passage is done in two steps: 1. Change of control: u ! u: 2. Change of support: T B ! T B :

The increment of the objective function is then
So Jð uÞ > JðuÞ, for θ 0 > 0.

Scheme of the hybrid direction algorithm
Let fu; T B g be a support control for the problem ð4Þ and η a real number such that η 2 ½0; 1. In order to take into account the specificity of the studied linear optimal control problem, we present in this section a slightly modified version of the algorithm presented in (Bibi & Bentobache, 2015). Indeed, if θ 0 ¼ 1 and T P þ N [ T P À N Þ;, then we reduce the value of the parameter η by setting η ¼ η=2 and we start a new iteration with the new control u. The scheme of the hybrid direction algorithm for solving the linear optimal control problem is described in the following steps: Algorithm 1 (1) Compute dðτÞ; qðτÞ; ν; EðτÞ with relationships (5)-(6); (2) Determine the sets T þ N , T À N , T P þ N and T P À N ; (3) Compute γðη; u; T B Þ with the formula (7); (4) If γðη; u; T B Þ ¼ 0, then the algorithm stops with fu; T B g, an optimal support control for the discretized problem; (5) Compute the improvement direction ΔuðτÞ using the relationship (8); (6) Compute θðτ 1 Þ ¼ min τ2T B θðτÞ; where θðτÞ is determined by (10); (7) Compute θ 0 ¼ minf1; θðτ 1 Þg; uðτÞ ¼ uðτÞ þ θ 0 ΔuðτÞ; τ 2 T h ; and Jð uÞ ¼ JðuÞ þ θ 0 γðη; u; T B Þ; (8) If θ 0 ¼ 1, then (8.1)-If T P þ N [ T P À N ¼ ;, then uðτÞ is optimal. Stop.

Consider the following problem
JðuÞ ¼ c 0 xð2Þ ! max u ; _ xðtÞ ¼ AxðtÞ þ buðtÞ; xð0Þ ¼ 0; with We have Consider the admissible control The corresponding trajectory is We choose h ¼ 0:5 and η ¼ 1: We take the support control fu; T B g for the problem (14), with T B ¼ f1g. Iteration 1.
In order to find a good approximate solution for the original continuous problem (14), we have implemented the discretization technique using the Cauchy formula and the hybrid direction algorithm with MTALB2018a. The developed solver was tested on a computer surface pro 2, with 4GO of memory and processor Intel(R) Core(TM) i5-4300U CPU 1.90GHz 2.50GHz, running under Microsoft Window 10 operating system.
The initialization approach proposed in (Bentobache & Bibi, 2012) can be used to compute an initial admissible support control, however we have initialized the hybrid direction algorithm with the following obvious admissible control: if t 2 ½0; 1½ À 1 2 ; if t 2 ½1; 2: In Table 1, we report numerical results for different values of N, where CPU 1 , CPU 2 , IT and J 0 represent respectively the cpu time of the discretization phase, the execution time, the number of  Figure 1. Graph of the optimal control in terms of t for N ¼ 5; 000.
iterations of the hybrid direction algorithm and the optimal value of the quality criterion of (14). We plot the optimal control in terms of t for N ¼ 5000 ( Figure 1) and we plot the optimal objective values of the linear program (4) corresponding to the problem (14) in terms of N ( Figure 2).
Note that for large values of N, our method converges to the optimal value of the continuous original problem J Ã ¼ 0:4495. Furthermore, we can see from Graph of Figure 1, that the commutation time is approximately equal to 1.23 sec.

Conclusion
In this paper, we applied the hybrid direction algorithm developed in (Bibi & Bentobache, 2015) to find an approximate optimal solution to a linear optimal control problem. A numerical example was given to illustrate the described algorithm, and some numerical simulation results were presented in order to show the convergence of our algorithm to the optimal solution of the continuous problem. In a future work, we will compare the presented approach with classical approaches on practical optimal control problems.