Maximum principle for a stochastic delayed system involving terminal state constraints

We investigate a stochastic optimal control problem where the controlled system is depicted as a stochastic differential delayed equation; however, at the terminal time, the state is constrained in a convex set. We firstly introduce an equivalent backward delayed system depicted as a time-delayed backward stochastic differential equation. Then a stochastic maximum principle is obtained by virtue of Ekeland’s variational principle. Finally, applications to a state constrained stochastic delayed linear-quadratic control model and a production-consumption choice problem are studied to illustrate the main obtained result.


Introduction
In , the nonlinear backward stochastic differential equation (BSDE in short) was introduced by Pardoux and Peng []. Until now, it has had applications in many fields, such as partial differential equation (see []), stochastic control (see [, ]) and mathematical finance (see []). Meanwhile, BSDE itself has been developed to many different branches, such as BSDE with jumps (see [-]), mean-field BSDE (see []), time-delayed BSDE (see [-]), anticipated BSDEs (see [, ]) and so on. A lot of works have been done for the control problem of such BSDEs. However, fewer works have been done on the control problems of stochastic delayed systems.
For a stochastic delayed system, Chen and Wu [] obtained a stochastic maximum principle by virtue of a duality between stochastic differential delayed equations (SDDEs in short) and anticipated BSDEs. Øksendal, Sulem and Zhang [] studied the optimal control problems for SDDEs with jumps. Yu [] obtained a maximum principle for SDDEs with random coefficients. A maximum principle of optimal control of SDDEs on infinite horizon was proved in Agram, Haadem and Øksendal []. Some other recent developments on stochastic delayed system can be found in Huang, Li and Shi [], Meng and Shen [], etc.
To the authors' knowledge, there has been no result concerning the control problem of a stochastic delayed system with state constraints until now. However, the state con-straints of stochastic delayed systems indeed exist in reality. In this paper, the stochastic control problem of a forward delayed system with terminal state constraint is studied. The controlled system is depicted as the following SDDE: X(t), X(tδ), u(t)) dt + σ (t, X(t), X(tδ), u(t)) dW (t), where X(T) ∈ K , a.s., K ∈ R n is a convex set. However, there are two (main) difficulties in this study. The first one is that the control system (.) is a delayed system, as stated in [], which is more complex than the classical case. Another difficulty is the terminal state constraint, which is a sample-wise constraint. As interpreted in Ji and Zhou [], the stochastic control involving sample-wise state constraints cannot be resolved by the classical theory.
Some recent developed results on state constraints (see [, -]) as well as the duality relation between time-advanced stochastic differential equations (SDEs, for short) and time-delayed BSDEs (see []) may help us to overcome the above mentioned difficulties. Firstly, an equivalent backward formulation of stochastic delayed system (.) is introduced, where X(T) is judged as a control variable. Meanwhile, the state constraint turns out to be a control constraint. However, such a treatment brings us both the advantage and the disadvantage. The advantage is that, in the classical control theory, to manage control constraint is easier than to manage state constraint. The disadvantage is that the initial condition (X() = η()) now turns into an additional constraint. To deal with the additional initial constraint, Ekeland's variational principle is used.
Note that the equivalent backward delayed system is described by a time-delayed BSDE, so the adjoint equation of the time-delayed BSDE via duality relation is an anticipated SDE. Therefore, both the delayed system and the anticipated system are needed in our study. As a routine, the variational procedure is made firstly. Then, by virtue of Ekeland's variational principle, the variational inequality is got. At last, the necessary condition is derived by applying the duality relationship between the backward delayed controlled system and the anticipated forward adjoint system. There is a good thing that the theory of BSDE and our assumption allow us to make the inverse transformation, so that the optimal control process can be solved by the obtained optimal terminal control. To make our conclusions be directly perceived, we also study two applications. One of them is the stochastic delayed linear quadratic (LQ in short) control model. Moreover, a production and consumption choice optimization problem (see []) is also adapted to our case.
We organize this article as follows. Some preliminary results about time-delayed BSDE and anticipated SDE are presented in Section . In Section , the original control problem of a forward delayed controlled system with terminal state constraint is formulated. Then an equivalent transformation is made to get a backward delayed controlled system. Moreover, a stochastic maximum principle is derived, which presents the required condition of the optimal terminal control. In Section , two applications are given.

Preliminaries
Denote by ( , F, F, P) a probability space such that F  includes all P-null elements of F and assume the filtration F = {F t , t ≥ } is generated by a d-dimensional standard Brownian motion W = {W (t), t ≥ }. Let T > . And δ >  is a given finite time delay. We denote the following notations: Similarly, we can define L  F (, T; R n×d ), L  F (-δ, T; R n ) and L  F (, T + δ; R n ). Now we recall some useful results for the study of the following sections. Consider the following SDDE: where η is a given continuous function, which represents the initial path of X, and b : [, T] × R n × R n → R n and σ : [, T] × R n × R n → R n×d are given measurable functions satisfying the following: Then, from Theorem . in [], under (H.), SDDE (.) has the unique adapted solution X(·) ∈ L  F (-δ, T; R n ). For the time-delayed BSDE, we need the following assumption.
The following is the well-posedness of time-delayed BSDE.
Proposition . Suppose ξ ∈ L  (F T ; R n ) and ϕ(·) is a given continuous function. Then, under (H.), for sufficiently small δ > , the following time-delayed BSDE has the unique adapted solution (Y (·), Z(·)) ∈ L  F (-δ, T; R n ) × L  F (, T; R n×d ), and it satisfies the following estimate: As we stated in the part of Introduction, the anticipated SDE is necessary in our study. The following is the condition for anticipated SDE.
The above results can be found in Delong and Imkeller [] and Chen and Huang []. The following is the famous Ekeland's variational principle.

Main result
We study our main result in this part, i.e., a maximum principle about the optimal control of a stochastic delayed system involving terminal state constraint. It should be pointed out that the time-delayed state of the controlled system is different from the case without delay.

Problem formulation
Let be the set of admissible controls. For every given u(·), for the control system, we consider the past-dependent state X(·) depicted as Define the cost function as follows: where l : [, T] × R n × R n×d → R n and φ : R n → R n are given measurable functions. We give the following assumptions: (H.) The functions b, σ , l, φ are all continuously differentiable in the arguments (x, x , u), and their derivatives are all bounded. (H.) Denote by C( + |x| + |u|) and C( + |x|) the bounds of derivatives of l in its arguments (x, u) and φ in its argument x, respectively.
Therefore, for every given u(·) ∈ U ad , under assumptions (H.) and (H.), Eq. (.) admits the unique adapted solution X(·) ∈ L  F (-δ, T; R n ). Denote by K ∈ R n a given nonempty convex subset. The goal of our control problem is to solve

Time-delayed backward formulation
We now show an equivalent backward system of Problem A. In order to do this, one additional assumption is needed: Note that (H.) and (H.) imply, for every (t, x, x ) ∈ [, T]×R n ×R n , that the following function is a bijection on R n×d . Hence, by letting q ≡ σ (t, x, x , u), we obtain that there is the inverse x, x , u) is a bijection, hence q(·) could be regarded as the control, which is the crucial observation that encourages this method for working out Problem A. Furthermore, by virtue of the theory of BSDE, choosing the terminal state X(T) is equal to choosing q(·). Therefore we innovate the following 'controlled' system, which essentially is a time-delayed BSDE: where now ξ becomes the 'control' and belongs to the following set: Moreover, here the equivalent cost function is Hence, the original Problem A is equivalent to the following Problem B: where X ξ () = a (we denote a = η() in the following for simplicity) is the solution to Eq. (.) at the initial time  under ξ . In control theory, it is well known that to solve the control constraint is easer than to solve the state constraint. From now on, since Problem A is equivalent to Problem B, we concentrate on dealing with Problem B. The benefit is that by virtue of ξ becoming a control variable now, a control constraint in Problem B replaces the state constraint in Problem A.
Definition . For ξ ∈ U and a ∈ R n , if the solution to (.) suits X ξ () = a, then we call the random variable ξ feasible. For any given a, the collection of every feasible ξ is denoted by N (a). Moreover, if ξ * ∈ U gets the minimum value of J(ξ ) over N (a), we call ξ * optimal.

Variational equation
In the following subsections, we denote the following notations for simplicity: where f ϕ denotes the partial derivative of f at ϕ with ϕ = x, x δ , q, respectively. In U, for ξ  , ξ  ∈ U, define a metric by Apparently, (U, d(·, ·)) becomes a complete metric space. Suppose ξ * is optimal and, associated with ξ * , the pair (X * (·), q * (·)) is the corresponding state processes of Eq. (.).
Because U is convex, for every ξ , the following variational control ξ ρ is also in U: Denote the solution to Eq. (.) associated with ξ = ξ ρ by (X ρ (·), q ρ (·)). And denote by ( X(·), q(·)) the solution of the following variational equation: Remark . Since the proof of Lemma . above is the same as that of Lemma . in Chen and Huang [], for simplicity of presentation, we only present the main result and omit the detailed proof. In fact, it is straightforward to prove Lemma . by applying Proposition ., Taylor expansion and the Lebesgue dominated convergence theorem.

Variational inequality
We solve the initial constraint X ξ () = a and obtain a variational inequality in this subsection.
Given the optimal ξ * , for a constant ε > , define Fε(·) : U → R as follows: Remark . One can test that the functions |X ξ () -a|  and φ(ξ ) are both continuous in their argument ξ . Hence, F ε , defined on U, is also a continuous function in its argument ξ .
Theorem . Under assumptions (H.)-(H.), suppose that ξ * is an optimal solution to Problem B, so we have h  ∈ R + and h  ∈ R n satisfying |h  | + |h  | = , so that for every ξ ∈ U, we have the following variational inequality:
By using similar analysis, when l(t, x, q) = , the following variational inequality can be obtained.

Maximum principle
For the sake of establishing the maximum principle, in this part, as the dual equation of Eq. (.), the following equation is introduced: Theorem . Let (H.)-(H.) hold. If ξ * is optimal to Problem B with (X * (·), q * (·)) being the corresponding state of Eq. (.), then we have h  ∈ R + and h  ∈ R n satisfying |h  | + |h  | = , so that for every η ∈ U, where m(·) is the solution of Eq. (.).
Proof By using Itô's formula to m(t), X(t) , we obtain Therefore, Paying attention to the terminal and initial conditions, one derives From the arbitrariness of ξ ∈ U, for every η ∈ U, we have Now, we let ∂K represent the boundary of K and denote  := ω ∈ |ξ * ∈ ∂K .
According to Theorem ., we directly deduce the following result.
Corollary . Assume that the assumptions in Theorem . hold, then for each η ∈ K , we have Remark . By the above study, for the optimal terminal control ξ * , we obtain the necessary condition. Note that the previous transformation process and (H.) allow us to make the inverse transformation. Therefore, the characterization of the optimal control process u * (·) can be derived by the obtained stochastic maximum principle of the optimal terminal control ξ * .

Applications of the main result
As stated in the section of Introduction, we study two applications of the main result established above in this section.

Stochastic delayed LQ control involving terminal state constraints
Stochastic delayed LQ control problem involving terminal state constraints is considered in this subsection. In order to simplify the presentation, we focus on the case d = n = . For the higher dimensional situation, one can deal with it in a similar method without substantial difficulty. Consider the following state equation: Next, we investigate the cost function independent of the running cost without loss of generality. Therefore, subject to u(·) ∈ U ad , X(T) ∈ R + , a.s., the goal is to minimize the following cost function: Now we give the backward formulation of problem (.). Denote Then Eq. (.) becomes and we can rewrite problem (.) as follows: in which m(·) is the solution to the following adjoint equation:

Production-consumption choice optimization problem
By applying the maximum principle established before, we investigate a type of productionconsumption choice optimization problem in this subsection. The shape for this issue, as in [], originates from Ivanov and Swishchuk []. For the sake of completeness, let us present the model at length. We assume an investor is going to invest his money to invent goods, and he could obtain benefits from the goods. We mark the capital of investor, the labor at time t and the rate of consumption by X(t), A(t) and c(t) ≥ , respectively. Based upon the assumption that earning of the production is a function of the total sum of the capital and labor, in order to depict this system, Ramsey [] introduced the following shape: Since in the procedure of investment, in reality, there is some risk and delay, Chen and Wu [] generalized the model in (.) to the following case: where η is a given continuous function. However, the rationality of the shape has been questioned as there is no constraint for the terminal capital X(T) on the basis of the hypothesis. In fact, in real situations, sometimes the investor will set a goal (constraint) for the terminal capital X(T) in the investment. Hence we believe that in a concordant model of the production and consumption some constraints for the terminal capital X(T) should be considered, i.e., X(T) ∈ Q, where Q ∈ R n .

Conclusions
In this content, we study a stochastic optimal control problem for stochastic differential delayed equation with terminal state constraint (at the terminal time, the state is constrained in a convex set). However, the control problem with terminal non-convex state constraint is still open. We will focus on the open problem in the future study.