Università di Milano – Bicocca Quaderni di Matematica Hamilton Jacobi Bellman equations in infinite dimensions with quadratic and superquadratic Hamiltonian

We consider Hamilton Jacobi Bellman equations in an inifinite dimensional Hilbert space, with quadratic (respectively superquadratic) hamiltonian and with continuous (respectively lipschitz continuous) final conditions. This allows to study stochastic optimal control problems for suitable controlled Ornstein Uhlenbeck process with unbounded control processes.


tein Uhlenbec
process with unbounded control processes.

Introduction

In this paper we study semilinear Kolmogorov equations in an infinite dimensional Hilbert space H, in particular Hamilton Jacobi Bellman equations.More precisely, let us consider the following equation ∂v ∂t (t, x) = −Lv (t, x) + ψ ∇v (t, x)
√ Q + l(x), t ∈ [0, T ] , x ∈ H v(T, x) = φ (x) ,(1.1)
where L is the generator of the transit stein-Uhlenbeck process
dX t = AX t dt + √ QdW t , t ∈ [0, T ] X 0 = x,(1.2)
that is, at least formally, (Lf )(x) = 1 2 (T rQ∇ 2 f )(x) + Ax, ∇f (x) .

The aim of this paper is to to consider the case where ψ has quadratic or superquadratic growth, and to apply our results to suitable stochastic optimal control problems: to this aim we make some regularizing assumptions on the Ornstein Uhlenbeck transition semigroup.At first in equation (1.1) we consider the case of final condition φ lipschitz continuous: with this assumption we can solve the Kolmogorov equation with ψ quadratic and superquadratic.In the case of quadratic hamiltonian we can solve equation (1.1) also in the case of final condiotion φ only bounded and continuous.A similar result, with ψ quadratic and superquadratic and with final condition φ lipschitz continuous, is proved in [14] by means of a detailed study on weakly continuous semigroups, and making the assumption that the transition semigroup P t is strong Feller.Here we include the degenerate case and we exploit the connection between PDEs and backward stochastic differential equations (BSDEs in the following).

Coming into more details, we assume that A and Q in equation (1.2) commute, so that, see [17], the transition semigroup P t satisfies the following re b (H), for every ξ ∈ H, the function P t φ is Gâteaux differentiable in the direction √ Qξ and for 0 < t ≤ T , ∇P t [φ] (x) Qξ ≤ c t 1/2 φ ∞ |ξ| .

(1.3)

In order to prove existence and uniqueness of a mild solution v o following):
       dX τ = AX τ dτ + QdW τ , τ ∈ [t, T ] ⊂ [0, T ], X t = x, dY τ = −ψ(Z τ ) dτ − l(X τ ) dτ + Z τ dW τ , Y T = φ(X T ),(1.4)
It is well known, see e.g.[21] for the finite dimensional case and [12] for the generalization to the infinite dimensional case, that v(t, x) = Y t,x t , so that estimates on v can be achieved by studying the BSDE
dY τ = −ψ(Z τ ) dτ − l(X τ ) dτ + Z τ dW τ , τ ∈ [0, T ], Y T = φ(X T ),(1.5)
Moreover, if ψ is quadratic, we can remove the lipschitz continuous assumption on φ and prove existence and uniqueness of a mild solution of equation (1.1) with φ continuous and bounded.The fundamental tool is an apriori estimate on Z, and the classical identification Z t,x τ = ∇v(τ, X t,x τ ) √ Q: the fact that A and Q commute is crucial in proving this estimates on ∇v(t, x) √ Q by means of backward stochastic differential equations.This estimate is obtained with techniques similar to the ones introduced in [2], and specialized in [22] in the quadratic case, to treat BSDEs with generator ψ with superquadratic growth and in a markovian framework.In [2] the Markov process X solves a finite dimensional stochastic differential equation, with constant diffusion coefficient and with drift not necessarily linear as in our case.In order to obtain an estimate on Z t,x τ , some non degeneracy assumptions on the coefficients are made.In the present paper the process X is infinite dimensional and we need the coefficient A and √ Q commute.We note that not in [2] nor in [22] the estimate on Z is used in order to solve a PDE related.We also cite the paper [3] where infinite dimensional Hamilton Jacobi Bellman equations with quadratic hamiltonian are solved: the generat d here in (1.2), and no assumptions on the coefficicent are made, but only the case of final condition φ Gâteaux different a stochastic optimal control problem.Let us consider the controlled equation
dX u τ τ + √ QdW τ , τ ∈ [t, T ] X u t = x.
(1.6 where the control u takes values in a closed subset K of H. Define the cost
J (t, x, u) = E T t
[l (X u s ) + g(u s )]ds + Eφ (X u T ) .

for real functions l, φ and g on H.The control problem in strong formulation is to minimize this functional J over all admissible controls u.We notice that we treat a control problem with unbounded controls, and, in the case of superaquadratic hamiltonian, we require weak coercivity on the cost J.Indeed, we assume that, for 1 < q ≤ 2, 0 ≤ g(u) ≤ c(1 + |u|) q , and g(u) ≥ C|u| q for every u ∈ K : |u| ≥ R so that the hamiltonian function ψ (z) = inf u∈K {g (u) + zu} , ∀z ∈ H, has quadratic growth in z if q = 2, and superquadratic growth of order p > 2, the coniugate exponent of q, if q < 2. Some example of ope satisfied by a stochastic heat equati n with coloured noise:
       ∂y ∂s (s, ξ) = ∆y(s, ξ) + ∂W Q ∂s (s, ξ), s ∈ [t, T ], ξ ∈ O, y(t, ξ) = x(ξ), y(s, ξ) = 0, ξ ∈ ∂O.
(1.7)

Here W Q (s, ξ) is a Gaussian mean zero random field, such that the operator Q characterizes the correlation in the space variable.The bounded linear operator Q is diagonal with respect to the basis {e k } k∈N of eigenfactors of the Laplace operator with Dirichlet boundary conditions.Equation (1.7) can be reformulated in H = L 2 (O) as an Ornstein-Uhlenbeck process (1.2) with A and Q commuting.The paper is organized as follows: in section 2 some results on the Ornstein-Uhlenbeck process are collected, in section 3 the Kolmogorov equation (1.1) is solved with ψ with superquadratic growth and φ lipschitz continuous, and these results are applied to optimal control, in section 4 the Kolmogorov equation (1.1) is solved with quadratic ψ and φ only continuous and again an a

lication to control is briefly presented, finally in section 5
optimal control problems for a controlled heat equation are solved.


Preliminary results on the forward equation and its semigroup l and separable Hilbert space H, that is a Markov process X solution to equation
dX τ = AX τ dτ + BdW τ , τ ∈ [t, T ] X t = x,(2.1)
where A is the generator oup in H and B is a linear bounded operator fr m Ξ to H. We define a positive and symmetric operator
Q σ = σ 0 e sA BB * e sA * ds.
Throughout the paper we assume the following.

Hypothesis 2.1 1.The linear operator A is the generator of a strongly continuous semigroup e tA , t ≥ 0 in the Hilbert space H.It is well known that here exist M > 0 and ω ∈ R such that e tA L(H,H) ≤ M e ωt , for all t ≥ 0. In the followi g, we always consider M ≥ 1 and ω ≥ 0.

2. B is a bounded linear operator from Ξ to H and Q σ is of trace class for every σ ≥ 0.

We notice that in some of the literature, in the case Ξ = Ornstein-Uhlenbeck process, a bounded, symmetric and positive operator Q is considered, and in equation 2.1 B is replaced by √ Q.

The process n that the Ornstein-Uhlenbeck semigroup can be represented as P τ −t = P t,τ , where
P τ [φ] (x) := H φ (y) N e τ A x, Q τ (dy) ,
and N e τ A x, Q τ (dy) denotes a Gaussian measure with mean e τ A x, and covariance operator Q τ .

In the following we are mainly concerned with the case Ξ = H, so we can take B = √ Q and we assume that A and √ Q commute.This happens e.g. when (e n ) n is an orthonormal and
Q = (−A) β = 0 −∞ (−s) β dE (s), for some β ∈ R. It turns out that Q t = t 0 e sA (−A) β e sA ds = 1 2 1 − e 2At (−A) β−1 .
It is proved in [17] that in this ca irectional derivative ∇ √ Q at a point x ∈ H in direction ξ ∈ H is defined as follows: ∇ √ Q f (x; ξ) = lim s→0 f x + s √ Qξ − f (x) s , s ∈ R. A continuous function f is √ Q-Gâteaux differentiable at a point x ∈ H if f admits the √ Q- directional derivative ∇ √ Q f (x; ξ)
in every directions ξ ∈ H and there exists a functional, the
√ Q−gradient ∇ √ Q f (x) ∈ Ξ * such that ∇ √ Q f (x; ξ) or every φ ∈ C b (H), the f (2.2)
In [17] hypothesis 2.2 is verified by relati e operators
A and √ Q. Namely if Im e tA Q ⊂ Im Q 1/2 t . (2.3)
and for some 0 ≤ α < 1 and c > 0 the operator norm satisfies
Q −1/2 t e tA Q ≤ ct −α , for 0 < t ≤ T.
then hy hat this can be proved with a procedure similar to the one use in [8] ly, see e.g [8], lemma 7.7.2, for every uniformly continuous function φ with bounded and uniformly continuous derivatives up to the second order, φ(X
E φ(X t,x τ ) τ t < ∇(X t,x s )h, dW s > = E τ t < Q∇P s,τ φ(X t,x s ), ∇(X t,x s )hds > (2.5) = E τ t < ∇P s,τ φ(X t,x s ), Qe (s−t)A )hds > = τ t < ∇ √ Q EP ∇ √ Q P t,τ φ(x), h >
By arguments sim ], lemma 7.7.5, we get that for eve nsition semigroup of the perturbed Ornstein Uhlenbeck process
dX τ = AX τ dτ + √ QF (τ, X τ ) + √ QdW τ , τ ∈ [t, T ] X t = x,
wi of an heat equation.Namely let O be a bounded domain in R. We denote by H the Hilbert space L 2 (Ω) and by {e k } k∈N the complete orthon y conditions in O.We consider the equation
       ∂y ∂s (s, ξ) = ∆y(s, ξ) + ∂W Q ∂s (s, ξ , s ∈ [t, T ], ξ ∈ O, y(t, ξ) = x(ξ), y(s, ξ) = 0, ξ ∈ ∂O.
(2.6)

Here W Q (s, ξ) is a Gaussian mean zero random field, such that the operator Q characterizes the correlation in the space variables.Namely the covariance of the noise is given by
E < W Q (s, •), h > H < W Q (t, •), k t so that ∂ 2 W Q ∂s∂ξ (s, ξ) in this case i the space-time white noise.More in general we think about a coloured noise and on Q we make the following assumptions:

Hypothesis 2.3 The bounded linear operator Q : H → H is pos igenvalues {λ k } k∈N .

By previous assumptions it turns out that λ k ≥ 0. Note that W Q (s, •) is formally defined by
W Q (s, •) = n k=1 Qe k (•)β k (s)
where {β k (s)} k∈N is a sequence of mutually independent standard Brownian motions, all defined on the same stochastic basis (Ω, F, F t , P).

Equation (2 ay in H as
dX τ = AX τ dτ + √ QdW τ , τ ∈ [t, T ] X t = x,(2.7)
where A is the Laplace operator with Dirirchlet boundary conditions, W is a cyl ndrical Wiener process in H and Q is its covariance opera continuous final condition

The aim of this section is to present exitence and uniqueness results for the solution of a semilinear Kolmogorov

the BSDE (3.4) admit a unique solution and b
[3], under the further assumption 3.2 setting v(t, x) := Y t,x t , it turns out that v is the unique mild solution of equation (3.1), and
∇ √ Q v(t, x) = Z t,x
t .By assuming that A and Q commute, or, more in general, by assuming that hypothesis 2.2 holds true, also imposing a mor restrive structure on the forward equation and on the backward equation, we will prove in section 4 an estimate on Z t,x τ depending on τ , t, T and φ ∞ but not on ∇φ.Thanks to this estimate we will prove that by setting
v(t, x) := Y t,x t ,(3.5)
it turns out that v is the unique m v(t, x) = Z t,x
t without assumption 3.2.We note that differentiability on l, thanks to the regularizing property of terval.We work on the PDE following [14], but the same result can be achieved by working on the BSDE with a procedure similar to ion 4.1.

We start e Λ :
C ([0, T ] × H) × C s ([0, T ] × 0 the closed ball of radius R 0 , with respect to the product norm
(f, L) C([0,T ]×H)×C s ([0,T ]×H,H * ) = f C([0,T ]×H) + L C s ([0,T ]×H,H * ) . Let us also define, for (u,

om the singularity of the Lipschitz cons
ant of G in T = t: by theorem 3.3, the local mild solution u is Lipschitz continuous according to estimate 3.15, and so also G is. Indeed, for every x, y ∈ H,
(T − t) α |G(t, x) − G(t, y)| ≤ L|x − y|
Existence, uniqueness and Lipschitz property of a mild solution of equation 3.10 follow as in [14], prop on is to find a priori estimates for the local mild solution of equati n 3.1 by using reperesentation (3.11).We notice that, the transition semigroup R t,T is a perturbed Ornstein-Uhlenbeck transition semigroup, so we could try to investigate if it satisfies regularizing properties like the ones satisfied by the Ornstein-Uhlenbeck transition semigroup contained in 2.2.Anyway there are some difficulties related to the coefficient G in equation (3.10): G is not d perturbed Ornstein-Uhlenbeck transition semigroup, such as the strong Feller property or property 2.2, are proved by means of "generalizations" of the Girsanov theorem and then by means of the Malliavin involved.


A priori estimates and global existence

In this section we investigate a priori estimates for the local mild solution of equation 3.1 that position 3.5, the equivalent representation (3.11) for the mild solution 0 ) ≤ R 0 2 . (3.16)
Next, we look for a priori estimates for the norm
• C(H * ) of ∇ G v.
In order to prove an a priori estimate for the norm • C(H * ) of ∇ G v, we exploit again the strict connection between PDEs and BSDE and X t,x • is the Ornstein-Uhlenbeck process defined by (2.1), then ((v(τ, X t,x τ )) τ , (∇v(τ, X t,x τ )) τ ) is solution to the the BSDE (3.4), that we rewrite for the reader convenience:
dY t,x τ = −ψ(Z t,x τ ) d dτ + Z t,x τ dW τ , τ ∈ [0, T ], Y t,x T = φ(X t,x T ),(3.17)
For ξ ∈ H let us define, if it exists,
F t,x τ = ∇ √ Q Y t,x τ ξ, V t,x τ = ∇ √ Q Z t,x τ ξ.
It turns out that such processes exist, (F t,x τ , V t,x τ ) ∈ K cont ([0, T ]), and that they are solution to
dF t,x τ = −∇ψ(Z t,x τ )V t ,(3.18)
which is equation 3.17 differentiated in direction Proof.We notice that again
√ Qξ, since for t ≤ τ ≤ T , ∇ √ Q X t,x τ ξ = e (τ −Z t,x τ = ∇ √ Q u(τ, X t,x τ )
where u is the local mild solution of equation 3.1.Since ∇ψ is locally lipschitz continuous, it turns out that
f 1 (τ ) :=   ∇ψ(Z t,x τ ) |Z t,x τ | 2 Z t,x τ if Z t,x τ = 0 0 otherwise ,
is bounded.Following again the method in [4] by the Girsanov theorem there exists a probability measure P, equivalent to the original one P, such that Ŵτ = − τ 0 f 1 (r)dr + W τ , τ ≥ 0 is a B

wnian motion.So in (Ω, F, P) equation (3.18) can be rewri
nerator −∇l(X t,x τ )e (τ −t)A √ Qξ is independent on F and V and it is bounded, so by classical theorems on BSDEs equation (3.19) admits a unique solution (F, V ) such that F has continuous paths and [20].Notice that the constant C depends only on A, G, l, ψ on the initial condition x given at initial time t.In particular
(F, V ) 2 Kcont := Ê sup τ ∈[0,T ] F t,x τ 2 + Ê T 0 V t,x τ 2 dτ < C, see e.g.|F t,x t | < M e ωT ( φ 1 + that hypotheses 2.1, 2.2 , 3.1, 3.2 hold true and let u be the local mild solution of equation (3.1), as stated in theorem 3.3.
v C([0,T ]×H) + ∇ √ Q v C([0,T ]×H,H * ) ≤ R 0 . (3= v(τ, X t,x τ ), Z t,x τ = ∇ √ Q v(τ, X t,x τ )) τ ∈[0,T ]
is the unique solution of the BSDE (3.4 f the mild solution v follows by the local existence (Theorem 3.3) and by the a priori estimates (Corollary 3.9).The connections between PDEs and BSDEs is classical in the literature a fferentiability assumptions on l and φ.We start by assum

g l bounded and continu
us and φ bounded and lipschitz continuous.Theorem 3.11 Assume that hypotheses 2.1, 2.2, 3.1, hold true and that l is bounded and continous and φ is bounded and lipschitz continuous.Then equation (3.4) admits a unique solution, that is a pair of processes
(Y t,x • , Z t,x • ) ∈ K cont ([0, T ]). The function v(t, x) = Y t,x
t is the unique mild solution of equation (3.1) according to definition 3.1.

Proof.We consider the inf-sup convolution of φ (see e.g.[16] and [8]) denoted by φ n and defined by
φ n rly, let us define l n the inf-sup convolution of l.It is well known that φ n ∈ U C 1,1 b (H) and as n tends to +∞, φ n converges to φ uniformly.Moreover, see also [17], let us denote by L the Lipschitz constant of φ; then |∇φ| ≤ L. Now let us denote by (Y n,t,x • , Z n,t,x


•

) the unique solution of the BSDE (3.4) with φ n and l n in the place of φ and l respectively.By standard results on BSDEs we know that as n → ∞
E sup τ ∈[0,T ] Y n,t,x τ − Y t,x τ 2 + E T nd moreover (Y n , Z n ) 2 Kcont < C
where C is a constant independent on n.We need to prove some further regularity on Z let us denote by (F n,t,x • , V n,t,x


•

) the unique solution of the BSDE in the place of φ and l respectively.It turns out that
E sup τ ∈[0,T ] F n,t,x τ 2 < C
where C is a constant depending on L and on φ ∞ , and independent on n.So the process Z n,t,x • is uniformly bounded in n, since for ξ ∈ H, Z n,t,x τ ξ = F n,t,x τ .We also get that
E sup τ ∈[0,T ] Y t,x τ 2 + E sup τ ∈[0,T ] Z t,x τ 2 < C. By settin v(t, x) = Y t,x t , ∇ G v(t, x) = Z t,
x t , we have found a (unique ) mild solut on to (3.1).


Application to control

We formulate the stochastic optimal control problem in the str pace with a filtration (F τ ) τ ≥0 satisfying the usual conditions.{W (τ ) , τ ≥ 0} is a cylindrical Wiener process on H with respect to (F τ ) τ ≥0 .The control u is an (F τ ) τ -predictable process with values in a closed set K of a ormed space U ; in the following we will make further assumptions on the control processes.Let R : ion
dX u τ = AX u τ + √ QR (u τ )) dτ + √ QdW τ , τ ∈ [t, T ] X u t = x. (3.22)
The solution of this equation will be denoted by X u,t,x τ or simply by X u τ .X is also called the state, u and T > 0, t ∈ [0, T ] are fixed.The special tructure of equation (3.22) allows to study the optimal control problem related by me inear H milton Jacobi Bellman equation which is a special case of the Kolmogorov equation (3.1) we have studied in the previous sections.The occurrence of the op

ator √ Q in
the control term is imposed by our techniques, on the contrary the presence of the oper ), define the cost
J (t, x, u) = E T t [l (X u s ) + g(u s )]ds + Eφ (X u T ) . (3.23)
for real functions l, φ on H and g on U .The control proble ver all admissible controls u.We make the following assumptions on the cost J.

Hypothesis 3.12 1.The function φ : H → R is lipschitz continuous and b for some 1 < q ≤ 2 there exists a constant c > 0 such that
0 ≤ g(u) ≤ c(1 + | C|u| q for every u ∈ K : |u| ≥ R.

(3.25)

In the following we denote b E T 0 |u t | q dt < +∞.
This summability requirement is justified by (3.25): a control process which is not q-summable would have infinite cost.We denote by J * (t, x) = if it exists, by u * the control real ptimal control.We make the following assumptions on R.
Hypothesis 3.13 ≤ C(1 + |u|) for every u ∈ U .
We have to show that equation (3.22) admits a unique mild olution, for every admissible control u.Proposition 3.14 Let u be an admissible control and assume that hypothesis 2.1 holds true.Then equation (3.22) ) τ ∈[t,T ] such that E sup τ ∈[t,T ] |X τ | q < ∞.
Proof.The proof follows the proof of proposition 2.3 in [11], with suitable changes since in that paper the finite dimensional case in considered and the current cost g has quadratic growth with respec to u, that is in (3.25) q = 2.

In order to make an approximation procedure in (3.22) we introduce the sequence of stopping times
τ n = inf t ∈ [0, T ] : E t 0 |u s | q ds > n
with the ususal convention that τ n = T if this set is empty.Following he approximation procedure used in the proof of proposition 2.3 in [11] we can prove that there exists a unique mild solution with the required q-integrability.

We define in a classical way the Hamiltonian function relative to the above problem:
ψ (z) = inf u∈K {g (u) + zR(u)} ∀z ∈ H. (3.26)
We prove that the hamiltonian function just defined satisfies the polynomial growth conditions and the local lipschitzianity required in hypothesis 3.1.


Lemma 3.15

The hamiltonian ψ :
H → R is Borel measurable, there exists a constant C > 0 such that −C(1 + |z| p ) ≤ ψ(z) ≤ g(u) + |z|(1 + |u|), ∀u ∈ K,
where p is the coniugate exponent of q.Moreover if the infimum in (3.26) is attained, it is attained in a ball of radius C(1
+ |z| p−1 ) th t is ψ(z) = inf u∈K,|u|≤C(1+|z| p−1 ) {g (u) + zR(u)} , z ∈ H,andψ(z) < g (u) + zR(u) if |u| > C(1 + |z| p−1 ).
In particular it follows that ψ is locally lipschitz continuous, namely ∀ z 1 , z 2 ∈ H, for some
C > 0, |ψ(z 1 ) − ψ(z 2 )| ≤ C(1 + |z 1 | p−1 + |z 2 | htforward.By a − C 1 |z|(1 + |u|)(3.28)
where C and R are as in (3.25) and C 1 > 0, and by t |z|(1 |u|) ≥ −C 2 |z| p − C 3
for suitable constants C 2 and C 3 .Moreover
|ψ(z)| ≤ g(u) + c|z|(1 + |u|).
Now we prove that the infimum is attained in the ball of radius C(1 + |z| p−1 ).By (3.28),
g(u) + zR(u) ≥ C|u|(|u| q−1 − C 1 C |z|) − C|R| q − C 1 |z|.
On the other hand d so there exists a constant C such that if |u| ≥ C(1 + |z| p−1 ) then
g(u) + zR(u) 7) now asily follows.

Remark 3. 16 We give an example of hamiltonian we can treat.Let g(u) = |u| q , 1 < q ≤ 2.

Then, if R(u) = u, the hamiltonian function turns out to be
ψ(z) = 1 q 1/(q−1) − 1 q p |z| p
where p ≥ 2 is the coniugate of q.With this example, for p = 2, the Hamilton Jacobi Bellman related can be solved with the ad hoc exponential transform, see e.g.[14].Our theory cover also the case of hamiltonian functions not exactly equal to |z| 2 .

We define
Γ(s, x, z) = {u ∈ U tion, i.e. there exists a measurable function γ : H → U with γ(z) ∈ Γ(z) for every z ∈ R.

In the following theorem, in order to prove the so called fundamental relation, we have to make further assumptions concerning differentiability of the hamiltonian function ψ.These assumptions allow us to say that the Hamilton Jacobi Bellman equation relative to the above problem, which is given by equation (3.1), admits a unique mild solution by theorem 3.3.Moreover this solution can be represented by means of the solution of the BSDE (3.4), namely the solution is given by v(t, x) = Y t,x t .So, adequating to our context the techniques e.g. in [11], we can prove the fundamental relation for the optimal control.Theo ies Gâteaux differentiability assumptions stated in hypothesis 3.1.For every t ∈ [0, T ], x ∈ H and for all admissible control u we have J(t, x, u(•)) ≥ v(t, x), and the equality holds if and only if
u s ∈ Γ ∇ √ Q v(s, X u,t,x s )
Proof.For every admissible control (u t ) t∈[0,T ] , we define, for every n ∈ N,
u n t = u t 1 |ut|≤n + n1 |ut , then the convergence holds for almost all t ∈ [0, T ] and P-almost surely.Moreover it follows that
T 0 |R(u n s )| 2 ds ≤ C T 0 (1 + |u n s |) 2 ds ≤ C(1 + n 2 ).
Let us define
ρ n = exp − T 0 R(u n s )dW s − 1 2 T 0 |R(u n s )| 2 ds
In the probabilit s )ds is a P n -Wiener process.In (Ω, F, P n ), X u n solves the following stochastic differential equation
dX u n τ = AX u n τ + √ QdW n τ , τ ∈ [t, T ] X u n t = x.(3.31)
Let us also denote s A, √ Q, ψ, l and φ and not on the Wiener proces 5) and it is also the solution of the related Hamilton-Jacobi-Bellman equation.This fact will be crucial in order to study the convergence of (Y n , Z n ) as n → +∞.In the probability space (Ω, F, P), we denote by X u the solution of equation (3.22), then
E sup t∈[0,T ] |X u n − X u | q ≤ E sup t∈[0,T ] | t 0 e (t−s)A Q(u s − n)1 |us|>n ds ≤ C(T, A, Q)E T 0 |u s − n| q 1 |us|>n ds
and so X u n → X u in L q (Ω, C([0, T ], H)), with probability measure P. By combining this fact with the previous arguments on the the dt × P-almost sure convergence o x t , Z n,t,x t ) = (v(t, x), ∇ √ Q v(t, x), and for all τ ∈ [t, T ], (Y n,t,x τ , Z n,t,τ t ) = (v(τ, X u n ,t,x τ ), ∇ √ Q v(τ, X u n ,t,x τ ) (3.33)
and since we work with lipschitz continuous assumptions on the final cost φ an assumptions on ψ by theorem 3.3 we get that both v and ∇ √ Q v are bounded and continuous.In onvergence, of X u n to X u , uniformly with r assing hrough the identification (3.33) of Y n and Z n .Namely, in (Ω, F, P),
(Y n,t,x τ , Z n τ ), ∇ √ Q v(τ, X u n ,t,x τ )) → (v(τ, X t,x τ ), ∇ √ Q (τ, X t,x τ
)) P-almost surely uniformly with respect to τ .Now we are ready to prove the fundamental relation: we integrate the BSDE (3.32) in [t, T ]: at first we write down the equation with rspect to the P n -Wiener process W n and then we pass to the process W , which is a standard Wiener process in the original probability space (Ω, F, P):
dY n t = φ(X u n T ) + T t ψ(Z n s ) ds + T t l(X u n s ) ds − T t Z n s dW n s = φ(X u n T ) W s
We notice that by standard arguments since Z n ∈ L 2 ((Ω, F, P n ) × [0, T ]), then it also holds that Z

∈ L 2 ((Ω, F, P) × [0, T ]).Now we integrate with respect to the original probabilit
P: by taking expectation in the previous integral equality
dY n t = Eφ(X u n T ) + E T t ψ(Z n s ) ds + E T t l(X u n s) ds − E T t Z n s dW n s = Eφ(X u n T ) + E T t ψ(Z n s ) ds + E T t l( u n s ) ds − E T t Z n s R(u n s )ds − E T t Z n s dW s = Eφ(X u n T ) + E T t ψ(∇ √ Q v(s, X u n s )) ds + E T t l(X u n s ) ds − E T t ∇ √ Q v(s, X u n s )R(u n s )ds
where in the last passage the stochastic integral has zero expectation, and we have identified Z n s with ∇ √ Q v(s, X u n s ).Next we also identify Y n t with v(t, x) and then we let n → +∞:
v(t, x) = Eφ(X u n T ) + E T t ψ(∇ √ Q v(s, X u n s )) ds + E T t l(X u n s ) ds − E T t ∇ √ Q v(s, X u n s )R(u n s )ds → Eφ(X u T ) + E T t ψ(∇ √ Q v(s, X u s )) ds + E T t l(X u s ) ds − E T t ∇ √ Q v(s, X u s )R(u s )ds.
By adding and subtracting E T t g(u s )ds we get
J(t, x, u) = v(t, x) + E T t −ψ(∇ √ Q v(s, X u s )) + ∇ √ Q v(s, X u s )R( s ) + g(u s ) ds,(3.34)
from which we deduce the desired conclusion.

Under the assumptions of Theorem 3.17, let us define the so called optimal feedback law:
u(s, x) = γ ∇ √ Q v(s, X u,t,x s ) , s ∈ [t, T ], x ∈ H. (3.35)
Assume that the closed loop equation admits a solution {X s , s ∈ [t, T ]}:
X s = e (s−t)A x 0 + s t e (s−r)A QdW r + s t e (s−r)A R(γ(∇ √ Q v(s, X r )))dr, s ∈ [t, T ]. (3.36)4
The semilinear Komogorov equation in the quadratic case: cont nuous final condition

Let v be the mild solution of the semilinear Kolmogororv equation 3.1, with the nonlinear term which is quadratic with respect to the √ Q-deriva aim of this section is to present an estimate for the √ Q-derivative of u(t, x) depending this estimate has been obtained in [22] by imposing some conditions on the coefficient of the forward equation for X.With those condit m BMO martingales, the estimate is proved.Also in [2] a similar estimate is proved with X finite dimensional, with a more restrictive s atic growth.Applications of this estimate to a related Kolmogorov equation are not exploited in [2] nor in [22].

Here we pro is an infinite dimensional Ornstein Uhlenbeck p ocess and A and Q commute.We apply this estimate to prove that there exists a mild solutio [22], solution of the related Kolmogorov equation can be achieve A and √ Q commute and that v is the unique mild solution of equation (3.1).

As already noticed entiable, then v is give solve the BSDE in the forward-backward system (3.3) with l = 0, hat we rewrite here:
       dX τ = AX τ dτ + QdW τ T = φ(X t,x T ).(4.1)
Moreover, by [15], see also 3], when ψ is quadratic in Z it turns out that (Φ(τ ) = τ +∞,
where the supremum is taken over all stopping times σ ∈ [t, T ] a.s.Moreover, as a consequence, the stochastic exponential ma p τ t Z s dW s − 1 2 τ t Z 2 s ds ,
is uniformly integrable.We are ready to prove an estimate on Z, independent on ∇φ.Then the folllowing estimate hol DE in (4.1) in the direction h ∈ H. Let us
denote F t,x τ = ∇ √ Q Y t,x τ h and V t,x τ = ∇ √ Q Z ability measure such that is a BMO martin τ = exp τ t ψ(Z t,x s )W s − 1 2 τ t |ψ(Z t,x s )| 2 ds ,
is a uniformly integrable martingale.Notice that dP dQ = E T .

In (Ω, F, Q), (F t,x , V t,x ) solve the follo ∇φ(X t,x T )e (T −t)A h,(4.4)
It turns out that (F t,x ) 2 is a Q-submartingale.

In (Ω, F, Q), (Y t,x , Z t,x ) solve the following BSDE:
dY t,x τ = −ψ(Z t,x τ )dτ + ∇ψ(Z t,x τ )Z t,x τ dτ + Z t,x τ dW Q τ , Y t,x T = φ(X t,x T ),
and by our assumptions the generator −ψ(Z t,x τ ) + ∇ψ(Z t,x τ ) s a BMO Q-martingale, with BMO norm depending only on T , t, A and φ ∞ .Moreover let us denote again Y t,x t = v(t, x).Since A and Q commute, F t,x τ :=< ∇v(τ, X t,x τ ), Qh >=< ∇ x v(τ, X t,x τ ), e (τ −t)A Qh > =< ∇ x v(τ, X t,x τ ), Qe (τ −t)A h >=< Z t,x τ , e (τ −t)A h > .

With these facts we ca BMO,√ v in C s α ([0, T ] × H, H * ), with α = 1/2.
is a uniformly integr niformly in n and k.Next we have to prove that
|Z n,t,x t − Z k,t,x t | ≤ C φ n − φ k ∞
We differentiate equation (4.7), ,k ), F n,t,x τ − F k,t, 2 ds|F t = E Q n,k T t | < Z n,t,x s − Z k,t,x s , e (s−t)A h > | 2 ds|F t ≤ C t,T E Q n,k T t |Z n here C t,T is a bounded constant depending on t, ion (4.7) we immediately get
E Q n,k T t |Z n,t,x s − Z k,t,x s | 2 ds|F t ≤ φ n − φ k ∞ and so |Z on of the Kolmogorov v n (t, x e by F (t, x).For every
n ≥ 1, v n (t, x + s √ Qh) − v n (t, x) s 1 0 ∇ √ Q v n (t, x + r Qh)h dr.
As n → +∞ we get
v(t, ], ∇ √ Q v(τ, X t,x τ )h = Z t,x
τ , where Z t,x is the limit of Z n,t,x in L 2 (Ω × [0, T ]).It turns out that by previous calculations
∇ √ Q v n (τ, X t,x τ ) → ∇ √ Q v(τ, X t,x τ ) in C s 1/2 ([0, T ] × H)). So ∇ √ Q v(τ, X

Optimal control problems for the heat equation

In this section we present some control problem related to a stochastic heat equation.As in section 2, when introducing equation (2.6), here O is a bounded domain in R, H = L 2 (O) and {e k } k∈N is the c ndary conditions bset of H, not necessarily coinciding with H.Here as in equation (2.6), W Q (s, ξ) is a Gauss es.Our aim is to minimize over all admissible controls the cost functional
J (t, x(ξ), u) = E T t O [ l (X u s (ξ)) + |u s (ξ)| q ]dξds + E O φ (X u T (ξ)) dξ.(5.2)
for real functions φ and l, and for q ≤ 2.

We make the following assumptions on the cost J. 1 q−1 1−q q |z| p .Moreover equation (5.1)

an be written in an abstract way in
as
dX τ = AX τ dτ + √ Qu τ + √ QdW τ , τ ∈ [t, T ] X t = x,(5.4)
where A is the Laplace operator with Dirirchlet boundary conditions, W is a cylindrical Wiener process in H and Q is its covariance operator.The control problem in its abstract formulation is to minimize over all admissible controls the cost functional
J (t, x, u) = E T t
[l (X u .

(5.5)

By applying results in section 3.4, we get the following Theorem 5.2 Let X u be the solution of equation (5.1), let the cost be defined as in (5.2) and let 5.1 hold true.Moreover assume that the hamiltonian function ψ satisfies Gâteaux differentiability assum sis 3.1.For every t ∈ [0, T ], x ∈ L 2 (O) and for all admissible control u we have J(t, x, u(•)) ≥ v(t, x), and the equality holds if and only if
u s ∈ Γ ∇ √ Q v(s, X u,t,x s )
Moreover assume that the set-valued map Γ is nonempty and let γ The closed loop equation admits a weak solution (Ω, F, (F t ) t≥0 , P, W, X) which is unique in law and setting
u τ = γ ∇ √ Q v(τ, X τ ) ,
we obtain an optimal admissible control system (W, u, X).

Proof.The proof follows from the abstract formulation of the problem, and by applying theorems 3.17 and 3.18.

Next we turn to an optimal control problem related to the controlled equation (5.4) with quadratic cost g and consequently quadratic hamiltonian function, and with final cost continuous.In this case, in order to perform the synthesis of the optimal control, we apply the results of section 4.1.Namely we consider equation (5.1).We have

o minimize the cost function
l J (t, x(ξ, u) = E It turns out that if φ, l and ḡ satisfy hypothesis 5.3, then φ, l and g satisfy hypothesis 3.12 with q = 2.



basis in H and A and Q have the spectral decomposition Ae n = −α n e n and Qe n = γ n e n where α n , γ n > 0 and α n ↑ +∞.If the α n are positive apart from a finite number the result is still true.More in general let Ξ = H, B = √ Q. Suppose that A is an unbounded, selfadjoint and negative defined operator, A = A * ≤ 0, A : D (A) ⊂ H → H, with inverse bo al representation of A A = 0 −∞ sdE (s) .
Theorem 3 . 3
33
Assume that hypotheses 2.1, 2.2, 3.1, 3.2 hold true.Then equation (3.1) admits a unique local mild solution 0 < δ < T according to definition 3.1.


Theorem 4 . 1
41 et (Y, Z) be the solution of the BSDE in (4.1).Let A and √ Q satisfy hypothesis 2.1, and assume that A and √ Q commute.Let φ and ψ satisfy hypotheses 3.1 and 3.2 w th p = 2.




ξ) = ∆y(s, ξ) + + k∈N λ k O u s (η) k (η)dη e k (ξ) + ∂W Q ∂s (s, ξ), s ∈ [t, T ], ξ ∈ O, y(t, ξ) = x(ξ), y(s, ξ) = 0, ξ ∈ ∂O.(5.1)where u s ∈ L 2 (O) represents the control.In the following we denote by d the set of admissible controls, that is the real valued predictable processes such thatE T 0 O |u t (ξ)| 2 dξ q/2dt < +∞.


Hypothesis 5 . 1
51
The function φ : R → R is : R → R is bounded and continuous.Let us define, forξ ∈ H φ(x) = O φ (x(ξ)) dξ, l(x) = O l (x(ξ)) dξ.(5.3)It turns out that if l and φ satisfy hypothesis 5.1, then φ and l defined in (5.3) satisfy hypothesis 3.12.Moreover by defining g(u)= O |u s (ξ)| q dξ = |u| L 2 (O), then the hamiltonian function turns out to be thesis 5 . 3
053
u s (ξ)) + ḡ(u s (ξ))]dξds + E O φ (X u T (ξ)) dξ.(5.6)over all admissible control , that is real valued predictable processes such t tisfying the following: The function φ : R → R is continuous and bounded; l : R → R is bounded and continuous; g : R → R is continuous and for every u ∈ R0 ≤ g(u) ≤ c(1 + |u| 2 )and there exist R > 0, C > 0 such that g(u) ≥ C|u| 2 for every u ∈ K : |u| ≥ R.Equation (5.1) admits the abstract formulation given by (5.4) and the cost functional can be formulated in an abstract way asJ (t, x, u) = E T t l (X u s ) + g(u s )ds + Eφ (X u T ) .with notation 5.3 and by setting moreover g(u) = O ḡ The estimate for the norm • C of u follows by(3.16), the e timate for the norm • C(H * ) of ∇
√ Q u follows by proposition 3.8.We can state a result on existence and uniq eness of a mild solution u of equation (3.1),which immediately gives a unique mild solution of equation (3.4).Theorem 3.10 Assume that hypotheses 2.1, 2.2, 3.1 and 3.2 hold true. Then equation (3.1) admits a unique mild solution u according to definition 3.1. Let X t,x • be solution of equation (2.1). The pair of processes (Y t,x τ



t,x s )ds + W τ is a Wiener proces umptions ∇ψ has linear growth with respect to Z, so
τ t ∇ψ(Z t,x s )dW sτ ∈[t,T ]



t,x τ ) = Z t,x τ P a.s.for a.a.τ ∈ [t, T ].Since (Y, Z) solve the BSDE in (4.1), with Y t,x By classical arguments we deduce that v solve equation(3.1).Moreover the solution is unique since the solution of the corresponding BSDE is unique.

t = v(t, x), by previous arguments we get Z t,x t = ∇ √ Q v(t, x).

Then the pair (u = u(s, X s ), X) s∈[t,T ] is optimal for the coNntrol problem.
e nevertheless notice that existence of a solution of the closed loop equation is not obvious, due to the lack of regularity of the feedback law u occurring in(3.36).This problem can be avoided by formulating the optimal control problem in the weak sense, following[9], see also[12]and[17].By an admissible control system we mean (Ω, F, (F t ) t≥0 , P, W, u(•), X u ),where W is an H-valued Wiener process, u is an admissible control and X u solves the controlled equation(3.22).The control problem in weak formulation is to minimize the cost functional over all the admissible control systems.Theorem 3.18 Assume hypotheses 2.1, 2.2, 3.12 and 3.13 hold true, and assume that the hamiltonian function ψ satisfies Gâ umptions stated in hypothesis 3.1.For every t ∈ [0, T ], x ∈ H and for all admissible control systems we have J(t, x, u(•)) ≥ v(t, x), and the equality holds i