Nearly Optimal Patchy Feedbacks for Minimization Problems with Free Terminal Time

The paper is concerned with a general optimization problem for a nonlinear control system, in the presence of a running cost and a terminal cost, with free terminal time. We prove the existence of a patchy feedback whose trajectories are all nearly optimal solutions, with pre-assigned accuracy.


-Introduction
for a nonlinear control system of the form ẋ = f (x, u) u(t) ∈ U . (1.2) Here x ∈ IR n describes the state of the system, the upper dot denotes a derivative w.r.t.time, and U ⊂ IR m is the set of admissible control values.The minimum is sought over all times T ≥ 0 and all measurable control functions u : [0, T ] → U.
In the literature, several results are available, which provide the existence of an optimal control t → u opt (t) in open-loop form [14,16,23], for any fixed initial condition x(0) = y ∈ IR n . (1.3) On the other hand, the existence and regularity of an optimal control in feedback form is a far more difficult issue.In an ideal situation, one would like to construct a (sufficiently regular) feedback u = U (x) such that all trajectories of the corresponding O.D.E.ẋ = f x, U (x) (1.4) are optimal w.r.t. the cost criterion (1.1).Only few general results are presently known in this direction [7,16,20,26].In general, the optimal feedback can be discontinuous, with an extremely complicated structure [8,17].Moreover, its performance may not be robust: an arbitrarily small external perturbation may produce trajectories which are far from being optimal [24].An alternative strategy, pursued in [3,18,19], is to construct sub-optimal feedbacks, trading off the full optimality in favor of a simpler structure of the control and the robustness of the resulting system.This approach also faces difficulties.In some cases, because of topological obstructions it is not possible to construct any continuous asymptotically stabilizing feedback [10,13,14,25], or any continuous near-optimal feedback [9].Therefore, one needs to work with discontinuous feedback controls [11,12,21,22].For discontinuous O.D.E's, however, no general result about existence and uniqueness of solutions is available.Carathéodory solutions can be constructed only under additional assumptions on the structure of discontinuities [15].
Following the approach developed in [1,2,3], asymptotic stabilization and optimal control problems can be solved using patchy feedbacks as discontinuous controls.We recall that a patchy feedback has a particularly simple structure, since it is a function u = U (x) that is piecewise constant on the state space IR n .For patchy vector fields, one can prove that Carathéodory solutions forward in time always exist [1].Moreover, the set of forward solutions is stable w.r.t.small perturbations [2].The analysis in [3] showed that any minimum time problem can be approximately solved using these patchy feedbacks.
Aim of the present paper is to extend the results in [3] to the general optimization problem (1.1).In addition, we present a construction which greatly simplifies the previous approach, thus clarifying the main lines of the proof.
For convenience, we list here all the basic assumptions.
(A) The set of admissible control values U ⊂ IR m is a compact, the function f : IR n × U → IR n is continuous w.r.t.both variables, and twice continuously differentiable w.r.t.x.In addition, f satisfies the sub-linear growth condition for some constant C f .Both the terminal cost ψ : IR n → IR and the running cost L : IR n × U → IR are continuous and non-negative.Moreover, L is strictly positive: Throughout this paper, V denotes the value function for the optimization problem (1.1)-(1.2),namely where the minimization is taken over all T ≥ 0 and all solutions of t → x(t, u), corresponding to a measurable control u : [0, T ] → U. Our main result can be stated as follows.
Theorem 1.Let the functions ψ, L, f in (1.1)-(1.2) satisfy the assumptions (A).Let ε > 0 and a compact set K ⊂ IR n be given.Then there exist a closed terminal set S ⊆ IR n and a patchy feedback u = U (x) defined on the complement IR n \ S such that the following holds.For each y ∈ K, every Carathéodory solution of reaches the set S within finite time.Calling τ .= inf t ; x(t) ∈ S the first time where the trajectory reaches S, we have (1.9) We recall that, by well known properties of patchy vector fields, for every initial point y ∈ IR n \ S the O.D.E.(1.8) has at least one forward Carathéodory solution.According to (1.9), all of the solutions starting from the compact set K are nearly optimal, for the cost (1.1).
In the remainder of the paper, Section 2 contains a brief review of the main definitions and properties of patchy feedbacks and patchy vector fields.The proof of Theorem 1 is then worked out in Section 3.

-Review of patchy feedbacks
The following definitions were introduced in [1].
Definition 1.By a patch we mean a pair Ω, g where Ω ⊂ IR n is an open domain with smooth boundary ∂Ω, and g is a Lipschitz continuous vector field defined on a neighborhood of the closure Ω of Ω, which points strictly inward at each boundary point x ∈ ∂Ω.
Calling n(x) the outer normal at the boundary point x, and denoting the inner product by a dot, we thus require n(x) • g(x) < 0 for all x ∈ ∂Ω.
(2.1) Definition 2. We say that g : Ω → IR n is a patchy vector field on the open domain Ω if there exists a family of patches (Ω α , g α ); α ∈ A such that -A is a totally ordered set of indices, -the open sets Ω α form a locally finite covering of Ω, -the vector field g can be written in the form We shall occasionally adopt the longer notation Ω, g, (Ω α , g α ) α∈A to indicate a patchy vector field, specifying both the domain and the single patches.
By setting we can write (2.2) in the equivalent form It is important to observe that the patches (Ω α , g α ) are not uniquely determined by a patchy vector field (Ω, g).Indeed, whenever α < β, by (2.2) the values of g α on the set Ω α ∩ Ω β are irrelevant.Of course, the values of g α for x outside the domain Ω don't matter either.Therefore, if the open sets Ω α form a locally finite covering of Ω and if for each α ∈ A the vector field g α satisfies then the vector field g in (2.2) is still a patchy vector field.Indeed, without changing the function g, one can suitably redefine the values of each g α on the set β>α Ω β , or outside Ω, and achieve the strict inequality Remark 2. For convenience, we are always assuming that the single patches Ω α are open, while the vector fields g α are defined on the closure Ω α .In certain situations, it would be natural to choose patches of the form for some unit vector n.In this way, however, the union Ω 1 ∪ Ω 2 does not cover all of Ω, because it does not contain points where n • x = c.This situation is easily fixed, replacing Ω 1 by a slightly larger open set which contains also these boundary points.The resulting vector field can still be written in patchy form.
If g is a patchy vector field, the differential equation has several useful properties.There are collected in the following theorem, proved in [1].
Theorem 2. Let g be a patchy vector field.Then the set of Carathéodory solutions of (2.6) is closed (in the topology of uniform convergence) but possibly not connected.For each Carathéodory solution t → x(t) , the map t → α * (x(t)) defined by (2.3) is left-continuous and non-decreasing.Given an initial condition the Cauchy problem (2.6)-(2.7)has at least one forward solution and at most one backward solution, in the Carathéodory sense.
Remark 3. In some situations it is convenient to adopt a more general definition of patchy vector field than the one formulated above.Indeed, one can consider patches (Ω α , g α ) where the boundary of the domain Ω α is only piecewise smooth.For example, Ω α could be a polytope, or the intersection between a ball and finitely many half-spaces.In this more general case, the inward-pointing condition (2.1) can be reformulated by asking that, for each boundary point x ∈ ∂Ω α , the vector g α (x) lies in the the interior of the tangent cone to Ω α at the point x.Namely g α (x) ∈ int T Ωα (x) . (2.8) As in [4], this tangent cone is defined by One can easily check that all the results concerning patchy vector fields stated in Theorem 2 remain valid with this more general formulation.
Definition 3. Let Ω, g, (Ω α , g α ) α∈A be a patchy vector field.Assume that there exist control values v α ∈ U such that, for each α ∈ A, there holds Then the piecewise constant map is called a patchy feedback control on Ω .
Recalling (2.3), the patchy feedback control can thus be written on the form

-Proof of the theorem
The proof of Theorem 1 will be given in several steps.
1. Various reductions can be performed.By a smooth approximation, we can assume that ψ ∈ C ∞ .Moreover, approximating the cost function L by a more regular function, it is not restrictive to assume that L is twice continuously differentiable w.r.t.x.Recalling that L(x, u) ≥ α 0 > 0, we can now replace f (x, u) by and consider the equivalent problem inf with dynamics ẋ = g(x, u) , x(0) = y .
Notice that the function g in (3.1) is continuous w.r.t.both variables x, u, and twice continuously differentiable w.r.t.x.Moreover it satisfies the growth condition In the following, we thus assume without loss of generality that the running cost is simply L(x, u) ≡ 1, so that the minimization problem (1.1) reduces to (3.2).

2.
Choose a constant M such that To fix the ideas, throughout the following we assume that 0 < ε < 1/8 and that the compact set K is contained in the open ball B ρ centered at the origin with radius ρ.Because of the sub-linear growth condition (1.5), for τ ≤ 2M , every trajectory of the system (1.2) starting at a point y ∈ K ⊂ B ρ will satisfy the a priori bound where ρ .= e C f •2M (ρ + 1) . (3.5) 3. Let V = V (y) be the value function for the optimization problem (3.2), with dynamics (1.2).We claim that V is semi-concave.More precisely, there exists a constant κ such that, for any y, y ′ ∈ B ρ, one has for some vector w ∈ D + V (y) in the upper gradient of V at the point y.Indeed, from the theory of optimal control [6] it is well known that the optimization problem (3.2), (1.2) with initial data x(0) = y has at least one solution, within the class of chattering controls.Let t → x(t) = x(t ; y, ũ, θ) be an optimal chattering trajectory, with for some measurable functions (ũ, θ) = (u 0 , . . ., u n , θ 0 , . . ., θ n ) satisfying For any other initial data y ′ , we can consider the same chattering control (ũ, θ), always stopping at the same terminal time t = τ .This yields the cost V ũ, θ,τ (y ′ ) = τ + ψ x(τ ; y ′ , ũ, θ) . (3.9) The regularity assumptions on f, ψ w.r.t. the variable x imply that, as y ′ varies in the ball B ρ , the map y ′ → V ũ, θ,τ (y ′ ) is twice continuously differentiable.Moreover, its C 2 norm remains bounded: Since τ ∈ [0, T max ] while both ũ and θ in (3.8) range over compact sets, this bound is uniform, i.e. in (3.10) we can take a constant κ > 1 which does not depend on the particular chattering control, or on the time τ .Observing that the inequality (3.6) follows from (3.10), choosing w = ∇V ũ, θ,τ (y).

4.
As shown in the previous step, the value function is Lipschitz continuous on the ball B ρ.In fact, the constant κ > 1 in (3.10) also provides a Lipschitz constant for V , namely By Rademacher's theorem, V is differentiable almost everywhere.At each point x ∈ B ρ where the gradient ∇V (x) exists, if V (x) < ψ(x) then one has the well known relation [5,10,16] Consider the open set Given δ > 0, we can choose finitely many points y 1 , . . .y m ∈ B ρ ∩ D such that ∇V (y i ) is well defined for each i = 1, . . ., m, and moreover Define the approximate value function where We claim that, by choosing δ > 0 sufficiently small, for all x ∈ B ρ the following relations hold. ) Indeed, the first inequality in (3.16) follows from (3.6).Next, since f is continuous and U is compact, we can find δ 1 ∈ ]0, 1] such that the following conditions hold.If x ∈ B ρ , w = ∇V (y) exists and We now choose δ > 0 such that 2κδ + κδ 2 ≤ min ε , κ δ 2 1 2 .
Given any x ∈ B ρ, if j is an index such that |x − y j | ≤ δ, recalling the Lipschitz condition (3.11) we find Hence from (3.19) it follows from (3.18) we deduce the inequality (3.17).This establishes our claim.

5
. By the definition of W i , it is clear that all level sets where W i is constant are spheres.Indeed, for any given constant c we can write with x i = y i − ∇V (y i )/2κ and a suitable radius r.
For each i = 1, . . ., m, consider the set In this step we show that there exists a minimum radius r min > 0 and a maximum radius r max such that, fixed x ∈ D i , the level set where W i = W i (x) is a sphere of center x i and radius r with Indeed, since ε < 1/2, by (3.17) it follows Therefore, for any ξ such that On the other hand, by (3.15) and (3.20) we have Hence, for any ξ such that 6.We are now ready to construct the near-optimal patchy feedback.We will define U (x) on the open set and the required terminal set S will be defined as S .= IR n \ Ω.Given η > 0 small, for each point x ∈ D i consider the point (see Figure 1) . By (3.17), there exists a nearly-optimal control value u = u Consider the lens-shaped region Its upper boundary will be denoted as Moreover, for z ∈ ∂ + Γ x i , we write n i (z) for the outer unit normal at the point z.
x W =W (x) We claim that, by choosing η > 0 sufficiently small, the following holds: Moreover, the constant η > 0 can be chosen uniformly valid for all i = 1, . . ., m and all x ∈ D i .For fixed i, x this is clear because, as η → 0, the diameter of the set Γ x i approaches zero.Moreover, as z varies on the upper boundary ∂ + Γ x i , all the unit normals n i (z) approach the vector ∇W i (x)/ ∇W i (x) .Therefore, both inequalities (3.29)-(3.30)follow from (3.26).
We now observe that f = f (x, u) is uniformly continuous on the compact domain B ρ × U.Moreover, on each set D i , the gradient ∇W i (x) is uniformly Lipschitz continuous and bounded away from zero.Finally, the radius of each level set, where W i is constant, by (3.22) is uniformly bounded above and below.This allows us to choose a constant η > 0 uniformly valid for all i, x.

7.
To achieve a nearly optimal feedback, we would need the inequality , this is a trivial consequence of (3.29).However, we must also consider the case where some of the points z ∈ Γ x i lie in a region where W (z) = W j (z) < W i (z), for some different index j.For this purpose, we observe that the set where W i = W j is always a hyperplane, say for a suitable constant c ij and a unit normal vector n ij .The orientation of n ij will be chosen so that We claim that, by choosing η > 0 sufficiently small, uniformly w.r.t.i, x, one of the following two cases occurs (see Figure 2).CASE 1: At every point z ∈ Γ x i ∩ H ij one has Indeed, assume that (3.33) fails.Then there exists a point z * ∈ Γ x i ∩ H ij such that By (3.32) and the orientation of the unit vector n ij , we can write for some constant β > 0. Together, (3.29) and (3.35) now imply provided that we choose η > 0 sufficiently small.Since f and ∇W j are uniformly Lipschitz continuous, from (3.37) it follows that (3.34) is valid for all z sufficiently close to z * .By reducing the size of η > 0, we can i intersects the half-space where W j < W i , two cases must be considered.Left: in Case 1, the vector field f (•, u x i ) points toward the set where W i < W j .As a patch we then take the shaded region Ω x i ⊂ Γ x i .Right: In Case 2, f (•, u x i ) points toward the set where W j < W i .We can now take Ω x i = Γ x i , because the control u x i is nearly optimal on this whole region.make the diameter of the lens-shaped domain Γ x i as small as we like.Hence the inequality (3.34) will hold for all z ∈ Γ x i .To prove our claim, it remains to observe that the functions f and ∇W i are uniformly continuous, and that the constant β in (3.36) remains uniformly bounded.Hence the constant η > 0 can be chosen uniformly valid for all i, j, x .
We now define the smaller domain where I i ⊂ {1, . . ., m} is the set of indices j = i for which CASE 1 applies.By the previous analysis, for each j such that W (z) = W j (z) for some z ∈ Γ x i , two cases can occur.If CASE 1 applies, then the vector field f (•, u x i ) is strictly inward-pointing along the portion of the boundary ∂Ω x i where W i = W j .On the other hand, if CASE 2 applies, then (3.34) holds on the entire domain Γ x i .
8. Consider the family of all domains Ω x i , as i ∈ {1, . . ., m} and x ranges over the closure of the set Ω .= x ∈ B ρ ; W (x) < ψ(x) .It now remains to select finitely many domains Ω x i which cover the compact set Ω.This last step, however, must be done with some care because on the lower portion of the boundary the vector field f (•, u x i ) may not be inward-pointing.To cope with this problem, we first observe that there exists a uniform constant h > 0 such that for every i, x and every z ∈ ∂ − Ω x i .We now set M * .
= max W (x) ; x ∈ B ρ , and split the domain Ω in sub-domains of the form For each ℓ, we cover the compact set Ω ℓ with finitely many domains Ω x i , constructed as in step 7, choosing x ∈ Ω ℓ .After a relabelling of both the domains and the correspondent vector fields from (3.26), this yields the patches (see Figure 3) We claim that the above construction yields a patchy vector field: Indeed, according to Remark 1, it suffices to check that, for each patch Ω ℓ,α = Ω x i , the vector field f (•, u ℓ,α ) = f (•, u x i ) is inward pointing at every point of the set In the present case, this is clear, because the only boundary points where f (•, u Therefore, given any point z ∈ ∂ − Ω x i , either W (z) = ψ(z) and z / ∈ Ω, or else z is contained in a patch Ω ℓ ′ ,α ′ with ℓ ′ > ℓ, as required in Remark 1. 9. To complete the proof, we now check that the patchy feedback that we have constructed is nearly optimal.We recall that, by the analysis in step 7, for every i, x we have By (3.4), it follows that x(τ ) cannot be on the boundary of B ρ.We thus conclude that W x(τ ) = ψ x(τ ) .Stopping at time τ , since W (x) ≥ 0 and V (y) ≤ M , the total cost can be estimated as Since ε > 0 was arbitrary, this completes the proof.

Figure 3 .
Figure 3.The domain Ω = ∪ Ω ℓ is covered by a family of patches Ω ℓ,α , ordered like tiles on a roof.
.44) Now take any initial point y ∈ K and let t → x(t) be any Carathéodory solution of the Cauchy problem ẋ = g(x)x(0) = y ,with g defined at (3.43).If y ∈ K \ Ω, Ω = x ∈ B ρ ; W (x) < ψ(x)as in (3.25), we are in the terminal set S and there will be no evolution, since it is more convenient to stay in y than to move along a trajectory.Otherwise, call τ ≥ 0 the first time at which x(t) reaches the boundary of the set Ω.By (3.44) we haveW x(τ ) − W (y) = τ 0 d dt W x(t) dt ≤ (−1 + 4ε)τ , hence τ ≤ W (y) − W x(τ ) 1 − 4ε ≤ 2W (y) ≤ 2ψ(y) ≤ 2M .