WHEN ARE MINIMIZING CONTROLS ALSO MINIMIZING RELAXED CONTROLS?

. Relaxation refers to the procedure of enlarging the domain of a variational problem or the search space for the solution of a set of equations, to guarantee the existence of solutions. In optimal control theory relaxation involves replacing the set of permissible velocities in the dynamic constraint by its convex hull. Usually the inﬁmum cost is the same for the original optimal control problem and its relaxation. But it is possible that the relaxed inﬁmum cost is strictly less than the inﬁmum cost. It is important to identify such situations, because then we can no longer study the inﬁmum cost by solving the relaxed problem and evaluating the cost of the relaxed minimizer. Following on from earlier work by Warga, we explore the relation between the existence of an inﬁmum gap and abnormality of necessary conditions (i.e. they are valid with the cost multiplier set to zero). Two kinds of theorems are proved. One asserts that a local minimizer, which is not also a relaxed minimizer, satisﬁes an abnormal form of the Pontryagin Maximum Principle. The other asserts that a local relaxed minimizer that is not also a minimizer satisﬁes an abnormal form of the relaxed Pontryagin Maximum Principle.

(Communicated by the associate editor name) Abstract. Relaxation refers to the procedure of enlarging the domain of a variational problem or the search space for the solution of a set of equations, to guarantee the existence of solutions. In optimal control theory relaxation involves replacing the set of permissible velocities in the dynamic constraint by its convex hull. Usually the infimum cost is the same for the original optimal control problem and its relaxation. But it is possible that the relaxed infimum cost is strictly less than the infimum cost. It is important to identify such situations, because then we can no longer study the infimum cost by solving the relaxed problem and evaluating the cost of the relaxed minimizer. Following on from earlier work by Warga, we explore the relation between the existence of an infimum gap and abnormality of necessary conditions (i.e. they are valid with the cost multiplier set to zero). Two kinds of theorems are proved. One asserts that a local minimizer, which is not also a relaxed minimizer, satisfies an abnormal form of the Pontryagin Maximum Principle. The other asserts that a local relaxed minimizer that is not also a minimizer satisfies an abnormal form of the relaxed Pontryagin Maximum Principle.
On first acquaintance with optimal control, one might expect that a minimizer for problem (P) exists, under hypotheses ensuring the existence of a unique state trajectory for every control function and initial state and the continuous dependence of this state trajectory on these quantities, the continuity of the cost function and closedness of C and the values of U (.), and the non-emptiness and boundedness of the set of admissible processes. But, as is well-known, such hypotheses are not enough, and an extra hypothesis is required. The most commonly invoked additional hypothesis to guarantee existence of minimizers is 'convexity of the velocity set', namely (C): f (t, x, U (t)) is convex for all t ∈ [0, 1], x ∈ R n .
Yet situations arise when the convexity hypothesis is violated and no minimizers exist. Here, it is still of interest to calculate inf(P ), since this number provides a tight lower bound on all possible values of the cost. The concept of 'relaxation' was introduced in the 1960's to deal with this eventuality. (See [11].) It takes inspiration from Hilbert's 20th problem 'Has not every regular variation problem a solution, provided certain assumptions regarding the given boundary conditions are satisfied . . ., and provided also if need be that the notion of a solution shall be suitably extended?
In the optimal control context, relaxation involves adding additional admissible processes ('relaxed admissible processes') to guarantee existence of minimizers, leading to the relaxed problem, which we write (R). Of course inf(R) ≤ inf(P ), because the relaxed problem involves minimizing the same cost over a larger domain. The set of relaxed processes is chosen to be close to the set of admissible processes, in some sense, and so, usually, we have inf(R) = inf(P ) .
Notice that the velocity sets for the relaxed problem are So the convexity hypothesis (C) is satisfied and the relaxed problem has a minimizer.
It is possible however that there is an 'infimum gap', i.e. inf(R) < inf(P ) .
It is important to identify such situations because, then, the above justification for studying the relaxed problem no longer applies.
The aim of this paper is to derive new conditions for an infimum gap to occur. These conditions require necessary conditions of optimality, expressed in terms of the Maximum Principle, to apply in abnormal form.
The link between the occurence of an infimum gap and abnormality is a natural one. It is well known that the minimum cost of a nonlinear programming problem can fail to be stable under perturbations of the constraints when there exists an abnormal set of Lagrange multipliers [1]. Since an optimal control problem is a special kind of infinite dimensional nonlinear programming problem, the existence of an infimum gap is a manifestation of instability of the infimum cost under perturbations to the endpoint and pathwise state constraints and the Maximum Principle is a kind of Lagrange multiplier rule, we would expect the occurence of an infimum gap to be revealed by the abnormality of the Maximum Principle conditions. Two consequences of an infimum gap are explored. In the first we focus attention on a strong local minimizer which cannot also be interpreted as a strong relaxed minimizer; in the second, on a relaxed minimizer, whose cost is strictly less than the infimum cost over admissible (non-relaxed) processes.
Type A: A strong local minimizer satisfies the Pontryagin Maximum Principle in abnormal form (i.e. with cost multiplier zero) if, when regarded as a relaxed admissible process, it is not also a relaxed strong local minimizer.
Type B: A relaxed strong local minimizer satisfies the relaxed Pontryagin Maximum Principle in abnormal form if its cost is strictly less than the infimum cost over all admissible processes, whose state trajectories are close (in the L ∞ sense) to that of the relaxed strong local minimizer.
Warga was the first to investigate the relation between the existence of an infimum gap and validity of the Maximum Principle in abnormal form. Warga announced a Type A relation for state constraint-free optimal control problems with smooth data in his early paper [10]. In his monograph [11] he proved a Type B relation for optimal control problems with state constraints. In a subsequent paper [12], Warga generalized his earlier Type B results to allow for nonsmooth data, making use of local approximations based on 'derivative containers', developed in [13].
We prove both Type A and Type B relations for a class of non-smooth stateconstrained optimal control problems which subsume those considered by Warga, and under less restrictive hypotheses on the data. (We allow, for example, the endpoint constraint set C to be a general closed set, whereas Warga requires C to have a functional representation.) Our relations also differ because they are based on the, by now, standard form of the non-smooth Maximum Principle, originally derived by Clarke [4] (and generalized to allow for state constraints in [5]), expressed in terms of subdifferentials; our results are therefore better suited as analytical tools for future developments in optimal control theory. The proofs in this paper make use of perturbation techniques. They are very different from those of Warga, which are based on the construction of approximating cones to reachable sets.
The main results of this paper can be summarized as (A): '(x(.),ū(.)) is a strong local minimizer but not a relaxed strong local minimizer' implies '(x(.),ū(.)) satisfies an 'averaged' version of the Maximum Priniciple in abnormal form', and (B): '(x(.),ū(.)) is a relaxed strong local minimizer with cost strictly less than that of any admissible (non-relaxed) process, whose state trajectory is close (in the L ∞ sense) to that of the relaxed strong local minimizer' implies '(x(.),ū(.)) satisfies the relaxed Maximum Principle in abnormal form'.
The second statement links the occurence of an infimum gap and abnormality of the Maximum Principle precisely (in relation to relaxed minimizers). But the first statement, we notice, does a little bit less, because it invokes a weaker, averaged, version of the Maximum Principle, which, for smooth data, involves the adjoint inclusion It remains an open question whether a sharper Type (A) relation is valid, involving the adjoint equation (a some related non-smooth generalization) in place of its averaged version. In the final section, however, we show that the two adjoint relations are the same when the dynamics are affine with respect to the control variable.
The conditions for existence of an infimum gap are given for optimal control problems with and without pathwise state constraints. While the state constraintfree problem is a special case of the state constraint problem, we state the state constraint-free conditions separately, to bring out the underlying relationships more clearly, without the distraction of 'measure multilpliers' and other complications in the statement of the state constrained Maximum Principle.
Type (A) and Type (B) relations have been previously derived for optimal control problems in which the dynamic constraint takes the form of a differential inclusion. See [7], [2], [3]. Here a link is established between the existence of an infimum gap and satisfaction of necessary conditions in abnormal form, when the necessary condition involved is Clarke's Hamiltonian inclusion. The theory developed in these papers for optimal control problems, in which the dynamic constraint is taken to be a differential inclusion, employs the Hamiltonian inclusion in both Type (A) and (B) relations, not some averaged version, and so provides rather more precise relations than those can currently be obtained when the dynamic constraint takes the form of a controlled differential equation. Nonetheless, the relations of this paper are of interest because most applications of optimal control theory are based on formulations involving a controlled differential equation, not a differential inclusion.
The following notation will be used throughout the paper: for vectors x ∈ R n , |x| denotes the Euclidean length. B denotes the closed unit ball in R n . Given a multifunction Γ(.) : Given a set A ⊂ R n and a point x ∈ R n , we denote by d A (x) the Euclidean distance of a point x ∈ R n from A: We denote by N BV + [0, 1] the space of increasing, real-valued functions µ(.) on [0, 1] of bounded variation, vanishing at the point 0 and right continuous on (0, 1). The total variation of a function µ(.) ∈ N BV + [0, 1] is written ||µ|| TV . As is well known, each point µ(.) ∈ N BV + [0, 1] defines a Borel measure on [0, 1]. The associated measure is also denoted µ. We write W 1,1 in place of W 1,1 ([0, 1]; R n ), N BV + in place of N BV + [0, 1], etc. when the meaning is clear.
We shall use several constructs of nonsmooth analysis. Given a closed set D ⊂ R k and a pointx ∈ D, the normal cone N D (x) of D atx is defined to be Here, the notation y i D → y is employed to indicate that all points in the convergent sequence {y i } lie in D.

Given a lower semicontinuous function
For details of definition and properties of these objects, we refer the reader to [6], [8] and [9].

2.
Conditions for Non-Coincidence of Infima. In this section we state two theorems relating the existence of a gap between the infimum costs for the optimal control problem (P ) and its relaxed counterpart (R), and the validity of a Maximum Principle in abnormal form. The following hypotheses, in whichx(.) is a given absolutely continuous function, will be invoked.
(b): Suppose that, for every > 0, there exists an admissible relaxed process is not also a local relaxed minimizer). Then conditions (i)-(iv) above are satisfied for some choice of multipliers (p(.), λ) such that λ = 0 and in which (ii) is replaced by
(3): Part (a) of the theorem is valid in a stronger form than in the theorem statement, in which the costate differential inclusion is replaced by We mention that this is a distinction that applies only to nonsmooth data; if f (t, .,ū k (t)) is continuously differentiable atx(t), for all k and a.e. t ∈ [0, 1], then (ii) and (ii) both reduce to the relation: It remains an open question, whether part (b) is also valid for the more precise version of the costate differential inclusion (1).
In Section 5 we provide a proof, not of Thm. 3.2, but of Thm. 3.3 below. Thm. 3.3 is more general but, perhaps, of lesser interest in the context of this paper, because it does not make explicit the link between the non-coincidence of the infimum costs (over admissible processes and admissible relaxed processes) and the existence of an abnormal multiplier set. Notice that Thm. 3.3 part (b) makes no reference at all to the cost function g(., .)).
4. An Example. In this section we present an example of an optimal control problem which illustrates the assertions of Thms. 3.1 and 3.2. Earlier examples of optimal control problems in which the infimum costs over admissible processes and over relaxed admissible processes do not coincide are to be found, for example, in ( [11], p. 246) This is an example of (P ), in which n = 3, m  To validate the claim, suppose there exists an admissible process (x(.), u(.)) with lower cost than that of (x(.),ū(.)). Sinceẋ 1 (t) = 0 and the cost is −x 1 (1), we must have x 1 ≡ k for some k > 0. We have It follows from this relation and the fact that x 3 (0) = x 3 (1) = 0, that x 2 (.) ≡ 0 . But thenẋ 2 (t) ≡ 0 a.e. Sinceẋ 2 (t) = ku(t) (for some k = 0) and u(t) ∈ {−1}∪{+1} a.e. we deduce thatẋ 2 (t) = 0 on a set of full measure. From this contradiction it follows that no admissible process exists with cost less than that of (x(.) ≡ (0, 0, 0),ū(.) ≡ 1), as claimed.

Illustration of
for some k = 0. Thus there exists a set of multipliers (it is, in fact, a unique set modulo scaling of the k parameter) which is abnormal, as predicted by Thm. 3.1 part (b).
Presently, we shall apply the state constrained Maximum Principle to (Q i ) (with reference to the strong local minimizer), the hypotheses for the validity of which are here satisfied. Before doing so, we note that, following extraction of an appropriate subsequence, we may restrict attention to one of the following two cases.
(v) : m i (t) ∈ ∂ > x h(t, x i (t)) µ i − a.e. and supp {µ i } ⊂ {t : h(t, x i (t)) = 0}. Notice that we have made use of the continuous differentiability of B(t, .) to write the adjoint inclusion as (ii) .
All the asserted relations have been verified. The proof is complete.