Elsevier

Systems & Control Letters

Volume 109, November 2017, Pages 30-36
Systems & Control Letters

Hamilton–Jacobi–Bellman equations for optimal control processes with convex state constraints

https://doi.org/10.1016/j.sysconle.2017.09.004Get rights and content

Highlights

  • We provide a characterization of the Value Function of an Optimal Control problem with state constraint sets.

  • A NFT theorem is proved without requiring any of the so-called Inward/Outward Pointing Conditions.

  • The technique relies on the convexity of the state constraint set and the graph of the dynamics.

  • We generalize the notion of constrained viscosity solutions to some situations where the state constraint set has an empty interior.

Abstract

This work aims at studying some optimal control problems with convex state constraint sets. It is known that for state constrained problems, and when the state constraint set coincides with the closure of its interior, the value function satisfies a Hamilton–Jacobi equation in the constrained viscosity sense. This notion of solution has been introduced by H.M. Soner (1986) and provides a characterization of the value functions in many situations where an inward pointing condition (IPC) is satisfied. Here, we first identify a class of control problems where the constrained viscosity notion is still suitable to characterize the value function without requiring the IPC. Moreover, we generalize the notion of constrained viscosity solutions to some situations where the state constraint set has an empty interior.

Introduction

This work is concerned with optimal control problems governed by a linear system and subject to state constraints. For a given finite horizon T>0, we consider the state constrained linear system: ẏ(t)=A(t)y(t)+B(t)u(t),a.e. t[0,T],u(t)U(t)Rm,a.e. t[0,T],y(t)KRN,for any t[0,T]where A(t) and B(t) are time-dependent matrices of dimension N×N and N×m, respectively. Here K and U(t) are nonempty closed sets. We center our attention on the Bolza problem

Minimize Ψ(y(T))+0TL(t,y(t),u(t))dt over all y:[0,T]RNsatisfying  (1).

It is known that when the interior of K is nonempty, the value function associated with the Bolza problem is a constrained viscosity solution [[1], [2], [3], [4]] to the Hamilton–Jacobi–Bellman (HJB) equation: tϑ(t,x)+H(t,x,xϑ(t,x))=0,on (0,T)×K,with ϑ(T,x)=Ψ(x) on K, and where H(t,x,p)sup{p,A(t)x+B(t)uL(t,x,u)uU}.

In this paper we show that the notion of constrained viscosity solution is a convenient notion for characterizing the value function of Bolza problems governed by state constrained linear systems, which satisfy an interior equilibrium condition of the following type: u0:[0,T]U measurable such that A(t)x0+B(t)u0(t)=0 for a.e. t[0,T],for some x0 in the relative interior of K. This assumption says that there is an interior point in K that is in addition an equilibrium for the linear system. This result provides a new qualification condition for the characterization of the value function as a unique constrained viscosity solution to the HJB equation.

As pointed out earlier, the notion of constrained viscosity solution was introduced by Soner [[5], [6]]. It turns out to be suitable for characterizing the value function when some controllability conditions are satisfied. In particular, the so-called inward pointing condition (IPC), or related outward pointing condition (OPC) condition turn out to be a sufficient requirement to guarantee Lipschitz continuity of the value function and its characterization as the constrained viscosity solution to the HJB equation, see [[2], [3], [7], [8]] and the references therein. From the point of view of the dynamical system, either the IPC or the OPC insure the existence of the so-called neighboring feasible trajectories (NFT) which makes it possible to approximate any trajectory hitting the boundary by a sequence of arcs which remain in the interior of K; see for instance [[1], [9], [10]]. We refer to [[3], [11], [12], [13], [14]] for weaker inward pointing assumptions. Let us point out that [14] appears in the context of non-degenerate and normal forms of the Maximum principle.

When the IPC is not satisfied, the HJB equation may admit several solutions and then it needs to be completed by additional boundary conditions in order to single out the value function as the unique solution, see for example [[4], [7], [15], [16], [17]].

In the general case where K is assumed to be any closed subset of RN, and under some convexity assumptions on the dynamics, the value function is lower semicontinuous (l.s.c.) and it can be characterized as the smallest supersolution to the HJB equation; see [18] for more details. In [19], it has been shown that the epigraph of the value function can always be described by an auxiliary unconstrained optimal control problem for which the value function is Lipschitz continuous and characterized, with no further assumptions, as the unique viscosity solution to a HJB equation. This approach leads to a constructive way for determining the epigraph of the value function and to its numerical approximation. It can also be extended to more general situations of time-dependent state constraint sets [20].

In this work, we extend the notion of constrained viscosity solution in an appropriate way to deal with situations where K may have an empty interior. Furthermore, the analysis we provide is slightly more general and allows to treat cases beyond linear systems. The proofs of these results are based on the observation that the value function can be characterized as constrained viscosity solution if any admissible trajectory can be approximated by a sequence of admissible trajectories that are lying in the relative interior of the set K (and when the latter is dense on K). This property, required by any IPC approach, turns out to be also satisfied for a large class of problems with some convex properties.

Throughout this paper, R denotes the sets of real numbers, || is the Euclidean norm and , is the Euclidean inner product on RN, B the unit open ball {xRN:|x|<1} and B(x,r)=x+rB. For a set SRN, intS and S¯ denote its interior and closure, respectively. Also for S convex we denote by ri(S) its relative interior. The distance function to S is distS(x)=inf{|xy|:yS}. Let S1 and S2 be two compact sets. Then the Hausdorff distance is given by dH(S1,S2)=maxsupxS2distS1(x),supxS1distS2(x).We adopt the convention that dH(,)=0 and dH(,S)=+ if S. For a given function v:RNR{+}, the effective domain of v is given by dom(v)={xRNv(x)R}.

If Γ is a set-valued map, then dom(Γ) is the set of points for which Γ(x). For an embedded manifold of RN, the tangent space to M at x is TM(x).

Let AC[a,b] stand for the set of absolutely continuous arcs y:[a,b]RN and Mn×m(R) the space of matrices of dimension n×m. For any matrix-valued map tA(t) defined on [0,T] we set Asupt[0,T]maxi=1,,nmaxj=1,,m|Ai,j(t)|.

Section snippets

General controlled systems

Consider a differential inclusion in RN for a given initial time τ[0,T]: ẏ(t)F(t,y(t)),for a.e. t(τ,T)y(τ)=x,where F:[0,T]×RNRN is a set-valued map satisfying (i)F is a continuous multifunction,with nonempty compact and convex images.(ii)There is LF>0 such that dH(F(t,x),F(t,y))LF|xy|,x,yRN,t[0,T].(iii)There is cF>0 such that max{|v|vF(t,x)}cF(1+|x|),xRN,t[0,T].

Remark 2.1

Let us point out that the linear system (1) can be reformulated as a differential

Convex problems

In this section we construct a particular suboptimal collection of trajectories, well suited for treating Bolza problems governed by linear systems with a convex state constraint set. We assume that K is a convex set of RN whose interior may be empty, that is, K is a nonempty, closed andconvexsubset of RN.

We recall that ri(K), the relative interior of K, is always a non-empty set. Furthermore, we have that this set is always an embedded manifold of RN.

Proposition 3.1

Suppose that (H

Conclusion

In this paper we have provided a characterization of the value function for a Bolza problem governed by linear systems with convex state constraint sets. We have seen that our framework covers situations involving an accumulative cost that is convex in the velocity. This follows from Remark 2.1, Remark 2.4, Remark 3.1.

The preceding analysis is based on a hypothesis (HQ) (concerning properties of the data of the problem), that does not impose any a priori conditions on the behavior of the

Acknowledgments

This work was supported by the European Union under the 7th Framework Programme FP7-PEOPLE-2010-ITN Grant agreement number 264735-SADCO. C. Hermosilla was supported by CONICYT-Chile through FONDECYT grant number 3170485.

References (24)

  • IshiiH. et al.

    A new formulation of state constraint problems for first-order pdes

    SIAM J. Control Optim.

    (1996)
  • BettiolP. et al.

    Regularity of solution maps of differential inclusions under state constraints

    Set-Valued Anal.

    (2007)
  • Cited by (21)

    • Stochastic optimal control in infinite dimensions with state constraints

      2022, Nonlinear Analysis, Theory, Methods and Applications
    • Optimal trajectory tracking solution: Fractional order viewpoint

      2019, Journal of the Franklin Institute
      Citation Excerpt :

      A classical approach to this problem is the receding horizon control technique [23–25]. However, in this paper, we solve the problem directly based on the HJB (Hamilton–Jacobi–Bellman) theory [26,27] and derive the necessary and sufficient conditions for the optimal pairs (x*, u*). In this paper, a generalized tracking problem was formulated in the optimal control context.

    View all citing articles on Scopus
    View full text