Hamilton–Jacobi–Bellman equations for optimal control processes with convex state constraints
Introduction
This work is concerned with optimal control problems governed by a linear system and subject to state constraints. For a given finite horizon , we consider the state constrained linear system: where and are time-dependent matrices of dimension and , respectively. Here and are nonempty closed sets. We center our attention on the Bolza problem
It is known that when the interior of is nonempty, the value function associated with the Bolza problem is a constrained viscosity solution [[1], [2], [3], [4]] to the Hamilton–Jacobi–Bellman (HJB) equation: with on , and where .
In this paper we show that the notion of constrained viscosity solution is a convenient notion for characterizing the value function of Bolza problems governed by state constrained linear systems, which satisfy an interior equilibrium condition of the following type: for some in the relative interior of . This assumption says that there is an interior point in that is in addition an equilibrium for the linear system. This result provides a new qualification condition for the characterization of the value function as a unique constrained viscosity solution to the HJB equation.
As pointed out earlier, the notion of constrained viscosity solution was introduced by Soner [[5], [6]]. It turns out to be suitable for characterizing the value function when some controllability conditions are satisfied. In particular, the so-called inward pointing condition (IPC), or related outward pointing condition (OPC) condition turn out to be a sufficient requirement to guarantee Lipschitz continuity of the value function and its characterization as the constrained viscosity solution to the HJB equation, see [[2], [3], [7], [8]] and the references therein. From the point of view of the dynamical system, either the IPC or the OPC insure the existence of the so-called neighboring feasible trajectories (NFT) which makes it possible to approximate any trajectory hitting the boundary by a sequence of arcs which remain in the interior of ; see for instance [[1], [9], [10]]. We refer to [[3], [11], [12], [13], [14]] for weaker inward pointing assumptions. Let us point out that [14] appears in the context of non-degenerate and normal forms of the Maximum principle.
When the IPC is not satisfied, the HJB equation may admit several solutions and then it needs to be completed by additional boundary conditions in order to single out the value function as the unique solution, see for example [[4], [7], [15], [16], [17]].
In the general case where is assumed to be any closed subset of , and under some convexity assumptions on the dynamics, the value function is lower semicontinuous (l.s.c.) and it can be characterized as the smallest supersolution to the HJB equation; see [18] for more details. In [19], it has been shown that the epigraph of the value function can always be described by an auxiliary unconstrained optimal control problem for which the value function is Lipschitz continuous and characterized, with no further assumptions, as the unique viscosity solution to a HJB equation. This approach leads to a constructive way for determining the epigraph of the value function and to its numerical approximation. It can also be extended to more general situations of time-dependent state constraint sets [20].
In this work, we extend the notion of constrained viscosity solution in an appropriate way to deal with situations where may have an empty interior. Furthermore, the analysis we provide is slightly more general and allows to treat cases beyond linear systems. The proofs of these results are based on the observation that the value function can be characterized as constrained viscosity solution if any admissible trajectory can be approximated by a sequence of admissible trajectories that are lying in the relative interior of the set (and when the latter is dense on ). This property, required by any IPC approach, turns out to be also satisfied for a large class of problems with some convex properties.
Throughout this paper, denotes the sets of real numbers, is the Euclidean norm and is the Euclidean inner product on , the unit open ball and . For a set , and denote its interior and closure, respectively. Also for convex we denote by its relative interior. The distance function to is . Let and be two compact sets. Then the Hausdorff distance is given by We adopt the convention that and if . For a given function , the effective domain of is given by .
If is a set-valued map, then is the set of points for which . For an embedded manifold of , the tangent space to at is .
Let stand for the set of absolutely continuous arcs and the space of matrices of dimension . For any matrix-valued map defined on we set
Section snippets
General controlled systems
Consider a differential inclusion in for a given initial time : where is a set-valued map satisfying
Remark 2.1 Let us point out that the linear system (1) can be reformulated as a differential
Convex problems
In this section we construct a particular suboptimal collection of trajectories, well suited for treating Bolza problems governed by linear systems with a convex state constraint set. We assume that is a convex set of whose interior may be empty, that is,
We recall that , the relative interior of , is always a non-empty set. Furthermore, we have that this set is always an embedded manifold of .
Proposition 3.1 Suppose that (
Conclusion
In this paper we have provided a characterization of the value function for a Bolza problem governed by linear systems with convex state constraint sets. We have seen that our framework covers situations involving an accumulative cost that is convex in the velocity. This follows from Remark 2.1, Remark 2.4, Remark 3.1.
The preceding analysis is based on a hypothesis () (concerning properties of the data of the problem), that does not impose any a priori conditions on the behavior of the
Acknowledgments
This work was supported by the European Union under the 7th Framework Programme FP7-PEOPLE-2010-ITN Grant agreement number 264735-SADCO. C. Hermosilla was supported by CONICYT-Chile through FONDECYT grant number 3170485.
References (24)
- et al.
Hamilton-jacobi characterization of the state constrained value
Nonlinear Anal.-Theor. Math. Appl.
(2005) - et al.
The state constrained bilateral minimal time function
Nonlinear Anal. Theor. Math. Appl.
(2008) - et al.
estimates on trajectories confined to a closed subset
J. Differential Equations
(2012) - et al.
Multivalued dynamics on a closed domain with absorbing boundary applications to optimal control problems with integral constraints
Nonlinear Anal. Theor. Math. Appl.
(2000) - et al.
Infinite horizon problems on stratifiable state constraints sets
J. Differential Equations
(2015) - et al.
Existence of neighboring feasible trajectories: Applications to dynamic programming for state-constrained optimal control problems
J. Optim. Theor. Appl.
(2000) - et al.
Discontinuous solutions of hamilton-jacobi-bellman equation under state constraints
Calc. Var. Partial Differential Equations
(2013) - et al.
The Mayer and minimum time problems with stratified state constraints
Set-Valued Var. Anal.
(2017) Optimal control with state-space constraint I
SIAM J. Control Optim.
(1986)Optimal control with state-space constraint II
SIAM J. Control Optim.
(1986)
A new formulation of state constraint problems for first-order pdes
SIAM J. Control Optim.
Regularity of solution maps of differential inclusions under state constraints
Set-Valued Anal.
Cited by (21)
On the fragility of the basis on the Hamilton–Jacobi–Bellman equation in economic dynamics
2024, Journal of Mathematical EconomicsStochastic optimal control in infinite dimensions with state constraints
2022, Nonlinear Analysis, Theory, Methods and ApplicationsOptimistic planning algorithms for state-constrained optimal control problems
2022, Computers and Mathematics with ApplicationsA fundamental solution concept for a family of state constrained regulator problems
2022, IFAC-PapersOnLineOptimal trajectory tracking solution: Fractional order viewpoint
2019, Journal of the Franklin InstituteCitation Excerpt :A classical approach to this problem is the receding horizon control technique [23–25]. However, in this paper, we solve the problem directly based on the HJB (Hamilton–Jacobi–Bellman) theory [26,27] and derive the necessary and sufficient conditions for the optimal pairs (x*, u*). In this paper, a generalized tracking problem was formulated in the optimal control context.