Coupling and tracking of regime-switching martingales

This paper describes two explicit couplings of standard Brownian motions $B$ and $V$, which naturally extend the mirror coupling and the synchronous coupling and respectively maximise and minimise (uniformly over all time horizons) the coupling time and the tracking error of two regime-switching martingales. The generalised mirror coupling minimizes the coupling time of the two martingales while simultaneously maximising the tracking error for all time horizons. The generalised synchronous coupling maximises the coupling time and minimises the tracking error over all co-adapted couplings. The proofs are based on the Bellman principle. We give counterexamples to the conjectured optimality of the two couplings amongst a wider classes of stochastic integrals.


Introduction
Let (Ω, (F t ) t≥0 , F, P) be a filtered probability space that supports a standard (F t )-Brownian motion B = (B t ) t≥0 and let V := {V = (V t ) t≥0 : V is an (F t )-Brownian motion with V 0 = 0} be the set of all (F t )-Brownian motions on this probability space. It is well-known that for any time horizon T > 0 the Brownian motion in V which minimises the probability that the processes X = x+B and Y (V ) = y + V couple after time T (for any starting points x, y ∈ R), i.e. the Brownian motion that solves the problem where τ 0 (X − Y (V )) := inf{t ≥ 0 : X t = Y t (V )}, is given by the mirror coupling V = −B (see e.g. [5]).
Furthermore it is easy to see that the Brownian motion which minimises the tracking error of Y (V ) with respect to the target X at time T , i.e. solves is given by the synchronous coupling V = B. This paper investigates the following generalisations of these questions.
1.1. Problems. Let Z = (Z t ) t≥0 be an (F t )-Feller process, i.e. a Feller process on our probability space, which is (F t )-Markov. Let the state space E of Z be a subset of a Euclidean space R d for some d ∈ N. For real Borel measurable functions σ i : E → R, i = 1, 2, define the stochastic integrals X = (X t ) t≥0 and Y (V ) = (Y t (V )) t≥0 by where x, y ∈ R and V ∈ V. Throughout the paper we assume that for each starting point the process Z is a semimartingale (in particular, it is non-explosive and has cádlág paths) and This implies that the processes X and Y (V ) in (1.1) are well-defined true martingales (e.g. see [11, Cor IV. 1.25]). In the case the state space E of Z is embedded in a multidimensional space, a natural choice for the volatility functions σ 1 and σ 2 are the projections resulting in σ 1 (Z) and σ 2 (Z) being coordinate processes of Z in R d . Furthermore, to avoid degenerate situations, we assume throughout the paper that (|σ 1 | + |σ 2 |)(z) > 0 for all z ∈ E. The class of stochastic integrals in (1.1), with the integrand Z typically a jump-diffusion (i.e. a Feller process), arises frequently and is of interest in the theory and practice of mathematical finance in the guise of stochastic volatility models (see e.g. [3]).
We are interested in the "distance" between the two processes X and Y (V ) for any V ∈ V. In other words we seek to understand how large and small the following quantities can be for T > 0 a fixed time horizon, (1.4) φ : R → R a convex function satisfying |φ(x)| ≤ a|x| p + b for some a, b > 0, p ≥ 2 and ∀x ∈ R, and τ 0 (X − Y (V )) := inf{t ≥ 0 : X t = Y t (V )} the coupling time of the processes X and Y (V ). Since V is an arbitrary (F t )-Brownian motion, the law of the difference X − Y (V ) is in general not easy to describe. Therefore we cannot expect to be able to identify the quantities in (1.3) explicitly. Our goal is to establish sharp upper and lower bounds for the expectations in (1.3), which hold for any choice of Brownian motion V ∈ V and are based on a natural generalisations of the mirror and synchronous couplings of Brownian motions described in Section 1.2. More precisely, we are looking for Brownian motions V M , V S ∈ V such that the following inequalities hold for all V ∈ V: where the generalised mirror (resp. synchronous) coupling holds for B and V M (resp. V S ).
In Problems (T) and (C), the goal is not merely to prove the existence in an abstract sense of the integrators V M , V S ∈ V, but primarily to understand for which classes of (F t )-Feller processes Z are the generalised mirror and synchronous couplings of Brownian motions, described in Section 1.2, extremal in the inequalities of Problems (T) and (C). In particular, for the volatility processes Z with the property that the generalised mirror and synchronous couplings satisfy the inequalities above for all Brownian motions V ∈ V, the following holds: maximising the coupling time of the stochastic integrals minimises the "convex distance" of the two processes and vice versa uniformly over all time horizons T > 0.

1.2.
Results. In the setting of processes (1.1), it is natural to define generalised synchronous and mirror couplings of Brownian motions in the following way. Let the functionsĉ I ,ĉ II : E → R be given by the formulaeĉ I (z) := sgn(σ 1 (z)σ 2 (z)),ĉ II (z) := − sgn(σ 1 (z)σ 2 (z)) for any z ∈ E, and define the Brownian motions V I = (V I t ) t≥0 and V II = (V II t ) t≥0 in V by , for i = 1, 2, which in particular implies the following for all t ≥ 0: Fix a time horizon T > 0 and note that, since (1.7) implies the supports of the random variables X T − Y T (V I ) and X T − Y T (V II ) are given by any non-negative non-zero convex function φ : R → R that satisfies the assumptions in (1.4), with support (i.e. the closure of φ −1 (0, ∞)) contained in the half-line (x − y + 3z 0 , ∞), clearly yields Hence the tracking part of the conjecture fails for Z = z 0 M .

1.2.2.
The generalised mirror and synchronous couplings are optimal if Z is a continuous-time Markov chain. Unless otherwise stated, in the rest of the paper Z denotes an (F t )-Markov semimartingale with a countable state space. More precisely, we assume that (1.8) Z is a non-explosive, irreducible, cádlág (F t )-Markov process on a discrete space E ⊂ R d .
Assumption (1.8) makes E a countable set (i.e. the cardinality of E is at most that of N) and Z a continuous-time (F t )-Markov chain on E. The following assumptions on the semigroup P of the volatility chain Z implies that the expectations in the inequalities of (T) are finite (see Section 3): ∀z ∈ E : (P T (|σ 1 | p + |σ 2 | p ))(z) < ∞. (1.9) Theorem 1.1. Let a Markov chain Z satisfy (1.2), (1.8) and (1.9) and φ be as in (1.4). Then The integrability condition in (1.9) is not necessary for the solution of Problem (C Remarks. (i) The functionĉ I = −ĉ II , and hence the Brownian motions V I = −V II , that feature in the solution of Problems (T) and (C) depend neither on the maturity T nor on the precise form of the convex cost function φ. No local regularity (e.g. differentiability) of φ is required for Theorem 1.1 to hold. Note also that essentially no restriction on the volatility functions σ 1 and σ 2 in the stochastic integrals in (1.1) is necessary, for the two theorems to hold. Furthermore, the assumptions in Theorems 1.1 and 1.2 place no restrictions on the filtration (F t ) t≥0 ; in particular (F t ) t≥0 need not be generated by the processes B and Z.
(ii) Brownian motion V I (resp. V II ) is chosen to minimise (resp. maximise) at each moment in time the Radon-Nikodym derivative of the quadratic variation of the process X − Y (V ) over the set V. It is clear that V I and V II can also be defined for much more general integrands than the ones considered in (1.1) and that the generalisations will still be locally extremal.
(iii) Section 3.2 shows that local maximisation/minimisation of the Radon-Nikodym derivative mentioned in item (ii) is also globally optimal in a non-Markovian setting in the special case of the quadratic tracking (i.e. where the cost function is φ(x) = x 2 (v) The results in Theorems 1.1 and 1.2 are likely to remain valid in the generalised setting given by the filtered space (Ω, (F t ) t≥0 , F, P) supporting an additional filtration (G t ) t≥0 , such that F t ⊆ G t for t ≥ 0, with properties that every Brownian motion in V ∈ V is also a (G t )-Brownian motion and the continuous (G t )-Feller process Z is independent of any V ∈ V. These conditions are satisfied for example by G t := F t ⊗ H t , where the filtration (H t ) t≥0 is independent of (F t ) t≥0 and supports a continuous (H t )-Feller (and hence (G t )-Feller) process Z, e.g. Z is a stochastic volatility process (i.e. a solution of an SDE) driven by an (H t )-Brownian motion. The reason why such a generalisation is likely to remain true lies in the fact that the representation in (2.3) still holds in this setting and the continuity of the paths of the process Z could be used to perform the necessary localisations in the proofs of Theorems 1.1 and 1.2. Note that by Lemma 2.3 the setting of the paper is given by G t := F t and Z a continuous-time (F t )-Markov chain. 1 (vi) The volatility functions σ 1 and σ 2 are typically distinct, which makes the maximal coupling time (viii) In [1] the authors establish an inequality, analogous to the first inequality of Theorem 1.1, in the case X and Y (V ) are solutions of driftless SDEs. A related inverse question to the tracking problem is studied in [9]. A general reference on the theory of coupling is given in [8].
(ix) The seminal paper [4] introduced regime-switching models to economics and finance. Since then, regime-switching models have found a plethora of applications in areas as diverse as macroeconomics, term-structure modelling and option pricing (see e.g. [7] and the references therein). and (C) are constructed from B and Z alone, they must also be extremal in the original problem.
We shall henceforth assume that B ⊥ ∈ V exists. Any V ∈ V and the process X − Y (V ), which plays a key role in all that follows, therefore possess the following representation.
Lemma 2.1. For any V ∈ V there exist (F t )-Brownian motion W ∈ V and C = (C t ) t≥0 , such that W and B are independent, C is progressively measurable with −1 ≤ C t ≤ 1 for all t ≥ 0 P-a.s., and the following representations hold: Remarks. (i) Equality (2.1) in Lemma 2.1 is a well-known representation for a Brownian motion V ∈ V in terms of B (see e.g. [1] and the references therein). For completeness and because of the importance of the representation in (2.2), which follows directly from (2.1), the proof of Lemma 2.1 is given in the appendix (see Section A.1); it is this proof that requires the existence of B ⊥ ∈ V independent of B.
(ii) Note that W and B in Lemma 2.1 are independent, but the process C may depend on either (or both) Brownian motions B, W .
2.2. Q-matrices, related operators and martingales. Let Q denote the Q-matrix of the continuous-time (F t )-Markov chain Z. We define the action of Q on the space of bounded functions on E in the standard way: for a bounded g : E → R, let Qg : E → R be given by the formula (Qg)(z) := since the series converges absolutely for every z ∈ E.
Let the function H : E × R → R satisfy the assumptions: H(·, z) ∈ C 2 (R) and H(r, ·) : E → R is bounded for any r ∈ R. Then, for any c ∈ [−1, 1], we define L c H : E × R → R by the formula: The operator L c is closely related to a generator of the process (R(V ), Z) and will play an important role in the solution of the stochastic control problems.
The next lemma describes a class of martingales related to the chain Z.
Lemma 2.2. Let F : R + ×R×E → R be a bounded function, such that for any z ∈ E the restriction to the first two coordinates F (·, ·, z) : R + × R → R is continuous. Assume that the generator Q satisfies Let U = (U t ) t≥0 be any continuous semimartingale, adapted to the filtration (F t ) t≥0 . Then the process is a true (F t , P z )-martingale for any starting point z ∈ E.
Remarks. (i) The key point in Lemma 2.2 is that we do not assume that the process (U, Z) is Markov, since all that is required of U is that it has continuous paths and is adapted to the underlying filtration on the original probability space. This fact plays a crucial role in the solution of our optimisation problems, as it allows us to eliminate all the (suboptimal) non-Markovian couplings of the Brownian motions V and B, the laws of which are not tractable.
(ii) Assumption (2.4) on Q is equivalent to stipulating that Q is a bounded linear operator. This is clearly satisfied when the state space E is finite.
(iii) The result in Lemma 2.2 is well-known but a precise reference appears difficult to find. For this reason, and because of its importance in the proofs of Theorems  itively, the independence of the chain Z and a Brownian motion W ∈ V follows from the fact that any (F t )-martingale of the form (ψ(Z t , t)) t≥0 , where ψ is a real function defined on the product E × R + , is equal to the sum of its jumps minus an absolutely continuous compensator and therefore has constant covariation with any continuous semimartingale adapted to (F t ) t≥0 . The key fact underpinning this argument is that Z is a Markov process on the filtration (F t ) t≥0 (see Section 5.2 for counterexamples to Theorems 1.1 and 1.2 when this assumption is relaxed).
Proof. We first show that the random variables W T and Z T are independent for any T > 0. Let the functions f : R → R and g : E → R be bounded and measurable with f suitably smooth.
We need to establish the equality Note that it is sufficient to prove that the product where P W is the Brownian semigroup, and hence M f is a continuous martingale. Similarly we have N g t = (P T −t g)(Z t ), where P denotes the semigroup for Z, and hence Itô's lemma for general semimartingales [10, Sec II.7, Thm. 33] and the Kolmogorov backward equation )(Z t )dt (Q denotes the generator matrix for Z). In particular, the quadratic variation of N g is equal to the sum of its jumps, i.e. the continuous part of the process [N g , N g ] is almost surely zero. Hence the continuity of M f and [10, Sec II. 6,Thm. 28] imply that the covariation satisfies d[M f , N g ] t = 0. Therefore, by the product rule, the infinitesimal increment of the process M f N g equals the Dominated Convergence Theorem we conclude that (2.5) holds for arbitrary bounded measurable functions f and g and the independence of W T and Z T follows.
To prove independence of random vectors (W t 1 , . . . , W tn ) and (Z t 1 , . . . , Z tn ) for any n ∈ N and a sequence of times 0 = t 0 < t 1 < · · · < t n , pick any bounded measurable functions f : R n → R and g : E n → R and define recursively the functions f k : R k∨1 → R and g k : E k∨1 → R for k = n, . . . , 0, which are again bounded and measurable, by f n := f, g n := g and Note that f 0 and g 0 are constant functions. Equality (2.5) applied to the bounded measurable func- . . , W t n−1 , x) and z → g(Z t 1 , . . . , Z t n−1 , z) shows that the following conditional expectation factorises: Therefore, by iteration and the tower property, we see that the following holds Since f and g were arbitrary, the processes W and Z are independent.
It follows from Lemma 2.3 that an (F t )-adapted volatility process, given by a strong solution of an SDE, cannot be approximated pathwise by a continuous-time (F t )-Markov chain.
Corollary 2.4. Let Z ′ be an (F t )-adapted Feller semimartingale, which solves a scalar SDE with Lipschitz drift and diffusion coefficients µ, σ such that σ > c > 0. Then there exists no sequence of continuous-time (F t )-Markov chains that converges to Z ′ almost surely on compacts. Proof.
, is an (F t )-adapted continuous local martingale with [W, W ] t = t. W is therefore an (F t )-Brownian motion (by Lévy's characterisation theorem) and Z ′ is a strong solution of the SDE dZ ′ t = µ(Z ′ t )dt + σ(Z ′ t )dW t . By Lemma 2.3, any sequence of continuous-time (F t )-Markov chains is independent of W and therefore also independent of Z ′ . Therefore, since Z ′ is non-deterministic, the sequence cannot converge to Z ′ almost surely on compacts.

Tracking
In this section we consider the problem of tracking X by the process Y (V ), defined in (1.1), where the control is being exercised solely by choosing the driving Brownian motion V . Recall that the tracking criterion, stated for a convex function φ in (1.4) and a time horizon T > 0, can be equivalently expressed in terms of the following problems: Theorem 3.1. Let the Brownian motions V I and V II be as in (1.5). Assume Z satisfies (1.2), (1.8) and (1.9) and that the function φ is as in (1.4). Then for any positive T we have In this section we prove Theorem 3.1, which clearly implies Theorem 1.1, and hence solves Problem (T). The proof of Theorem 3.1 is based on Bellman's principle, a martingale verification argument and an approximation scheme. The first stage consists of "approximating" Problems (3.1)-(3.2). More precisely, we proceed in two steps: we first introduce a stopped chain Z n and, in the second step, the stopped process R K,n (V ).
To this end let U n ⊂ R d , n ∈ N, be a family of compact subsets such that ∪ n∈N U n = R d and For each n ∈ N, define a stopping time τ n and the stopped (F t )-Markov chain Z n by Hence, Z n is an (F t )-Markov chain with the state space E and a Q-matrix Q n given by where I {·} denotes the indicator function. In particular, since U n is compact and hence U n ∩ E must be finite by (1.8), Q n satisfies assumption (2.4) in Lemma 2.2. Since the chain Z has cádlág paths, the sequence of positive random variables (τ n ) n∈N is non-decreasing and the following holds Hence, we can extend the definition in (3.3) in a natural way to the case n = ∞ by Z ∞ := Z.
Fix a large K > 0 and define, for any V ∈ V, the stopping time where R(V ) is given in (2.2). The stopped process of interest R K,n (V ) = (R K,n t (V )) t≥0 can now be defined by For given φ satisfying (1.4), T > 0 and any K ∈ (0, ∞) and n ∈ N ∪ {∞}, consider the problems and ψ respectively. Note that by definition we have ψ Lemma 3.2. Assume that φ, given in (1.4), is bounded from below and φ ∈ C 2 (R). For any K ∈ (0, ∞) and n ∈ N ∪ {∞}, the functions ψ (I) K,n and ψ (II) K,n , defined in (3.8), have the following properties.
(i) For all r ∈ R, z ∈ E and t ∈ [0, T ], there exists a constant ℓ ∈ R, such that (iii) For any r ∈ R, z ∈ E and t ∈ (0, T ], the derivatives satisfy the following inequalities: Proof. Part (i) follows from (3.8) and the properties of φ. To prove that ψ and note that its distribution does not depend on the starting point of R K,n (V I ). Since φ ∈ C 2 (R), Lagrange's mean value theorem implies that, for any small h > 0, there exists a random variable ξ S,h such that Since |S| ≤ K almost surely and r is fixed, the continuity of φ ′ yields that the random variable |φ ′ (r + ξ S,h )| is bounded above by a constant. Equation (3.11), almost sure convergence of ξ S,h to S, as h → 0, and the Dominated Convergence Theorem imply that ψ Furthermore, the convexity of φ and (3.12) yield the first inequality in (3.9). An identical argument applied to the function ψ (II) K,n (·, z, t) implies its differentiability in r and yields (3.9). Since φ ′′ is continuous by assumption, we can apply an analogous argument to the one above, now using formula (3.12) instead of (3.8), to conclude that the functions ψ The convexity of φ now implies part (iii) of the lemma. Differentiability of ψ (I) K,n (r, z, ·) in t follows from the smoothness of φ and the standard properties of Itô integrals.

Pick a function
for each z ∈ E, and for each r ∈ R, t ∈ [0, T ) the restriction to the second coordinate F (r, ·, t) : E → R is bounded. Then for any constant c ∈ [−1, 1] we define the function K c F : R × E × [0, T ) → R by the formula: where the operator L c is as defined in (2.3).

Lemma 3.3 (HJB equation).
Let φ in (1.4) be bounded from below and satisfy φ ∈ C 2 (R). Let n ∈ N and K ∈ (0, ∞). Then the functions K,n and ψ (II) K,n ) satisfy the HJB equations: Furthermore, if at least one of the conditions |r| ≥ K or z ∈ E \ U n or t = T is satisfied, it holds Remark. Unlike Lemma 3.2, the proof of Lemma 3.3 depends on Lemma 2.2 and so requires the assumption n < ∞.
Proof. Note first that the definitions in (3.8) imply the boundary behaviour stated in (3.15).
We now focus on the proof of (3.13). Recall that for any starting point z ∈ E and t ∈ [0, T ), on the event {τ n ≥ t} we have Z n t = Z t . The Markov property of the process (R(V I ), Z) and the equality in (3.15) now imply The following observations are key: • the quadratic covariation [R K,n (V I ), Z n,i ] t vanishes for all t ≥ 0 and i = 1, . . . , d, where Z n,i is the i-th component of Z n (recall that we are assuming E ⊂ R d ); • the chain Z n satisfies the assumptions of Lemma 2.2 and hence the process where Q n is the generator of the chain Z n given in (3.4), is a true (F t , P z )-martingale for any starting point z ∈ E.
By Lemma 3.2, the function ψ (I) K,n possesses the necessary smoothness so that Itô's lemma for general semimartingales [10, Sec II.7, Thm. 33] can be applied to the process (ψ the pathwise representation of this bounded martingale implies that the following process N = (N t ) t∈[0,T ] , is a continuous martingale. The quadratic variation of N is clearly equal to zero and hence N t = 0 for all t ∈ [0, T ] and starting points (r, z). Since by (1.8) the process (R K,n (V I ), Z) visits a neighbourhood of any point in the product . This inequality and identity (3.16) imply (3.13). The proof of (3.14) is analogous and therefore left to the reader.
3.1. Proof of Theorem 3.1. Assume that φ satisfies condition (1.4) as well as where V I , V II are given in (1.5). In other words, for each t ≥ 0, the Brownian motions V It and V IIt are arbitrary (but fixed) up to time t and have increments equal to those of the candidate optimal Brownian motions after this time. We now consider two Bellman processes ( , associated to Problems (3.6)-(3.7), given by This equality, together with a similar argument based on the definitions of V IIt and ψ (II) K,n , yields the following representations for the Bellman processes By Lemma 3.2 we can apply Itô's formula for general semimartingales (see [ . Put differently, we have established the following inequalities for any starting points r ∈ R, z ∈ E, any K ∈ (0, ∞), n ∈ N and all Brownian motions V ∈ V: The next step in the proof of Theorem 3.1 requires two limiting arguments. First, note that for any for any starting points r ∈ R and z ∈ E. Furthermore, by Lemma 3.2 (i), the random variables φ(R K,n T (V )) are bounded in modulus by a constant uniformly in n ∈ N. Therefore, the Dominated Convergence Theorem implies that the inequalities in (3.20) hold for n = ∞.
For the second limiting argument, note that assumption (1.9) and the semigroup property imply where the second equality follows from the following facts: and Fubini's theorem for non-negative functions. This, together with assumption (1.9) and another application of Fubini's theorem, yields The following almost sure inequality is a direct consequence of the definition in (3.5) By assumptions (1.4) and (3.17) the following inequalities hold for some constants a, b > 0 and ℓ ∈ R: The Burkholder-Davis-Gundy inequality [11,Thm IV.4.1] applied to the martingale R(V ) at time T , together with inequality (3.21), implies that |S| p is an integrable random variable. The Dominated Convergence Theorem therefore yields the By (3.20) for n = ∞, we obtain the following inequalities for any V ∈ V: implying Theorem 3.1. under the additional assumption in (3.17).
In order to relax the assumption φ ∈ C 2 (R), fix a non-negative g ∈ C ∞ (R) with support in [M, 0], Note that φ n : R → R is a convex function, which satisfies both (1.4) and (3.17) (here we still assume that φ is bounded from below), and the sequence (φ n ) n∈N converges point-wise to φ as n ↑ ∞ (see e.g. [11], proof of Theorem VI.1.1 and Appendix 3). 2 Since φ satisfies (1.4), for any x ∈ R and n ∈ N we have where the constants A, B > 0 are independent of both n and x. Since the random variable |S| p is integrable (see previous paragraph), where S is defined in (3.22), so is |R T (V )| p for any V ∈ V. The inequality above and the Dominated Convergence Theorem imply for any V ∈ V, which together with the inequalities in (3.23), establishes Theorem 3.1 for φ that are bounded from below and satisfy (1.4).
Since for any V ∈ V the processes X and Y (V ) are true martingales by (1.2), we may substitute Assume (in this section only) that the stochastic integrals X and Y (V ) are given by for some progressively measurable integrands H = (H t ) t≥0 and J = (J t ) t≥0 on (Ω, (F t ) t≥0 , F, P) and any V ∈ V. As usual, we denote the difference of X and Y (V ) by R(V ) = X − Y (V ). The extremal Brownian motions V I and V II , defined in (1.5), can be generalised naturally by Hence, for any fixed V ∈ V, we can define the Brownian motions V It and V IIt as in (3.18). If the generalisation of Theorem 3.1 were to hold in this setting, the Bellman processes B I (V ) and B II (V ), defined in (3.19), would be a submartingale and a supermartingale, respectively, for any V ∈ V. We will focus on B I (V ), as the issues with B II (V ) are completely analogous. Representation (2.1) of V in Lemma 2.1 and Itô's formula yield where M I is a local martingale, which we assume to be a true martingale. The process This proposition is consistent with an argument based on Itô's isometry: the variance of a stochastic integral is equal to the expectation of its quadratic variation and hence minimising/maximising its variance is equivalent to locally minimising/maximising the Radon-Nikodym derivative of its quadratic variation. Furthermore, it is also clear from the representation above that in the absence of an underlying Markovian structure, for a general convex φ, the process B I (V ) may fail to be a submartingale and hence the strategy in Theorem 3.1 is not optimal for general non-Markovian integrands (see Section 5.2.1 for an explicit example demonstrating this phenomenon).

Coupling
In More precisely, for any fixed T > 0, we consider the following problems: Theorem 4.1. Let V I and V II be as given by (1.5) and Z satisfy (1.2) and (1.8). Then for any The localisation procedure will allow us to reduce the problem to the case where the generator of the volatility chain Z is bounded, which will in turn make it possible to establish sufficient regularity of the candidate value functions and conclude that certain processes are true martingales (see Section 4.1). The two Markov processes (R IIn , Z n ) and (R In , Z n ), which play a key role in the solution of Problems (4.1) and (4.2), are defined by for any r ≤ 0, where B and Z n are as above and the functions Σ II , Σ I : E → R are given by Σ II (z) := σ 1 (z) + sgn(σ 1 (z)σ 2 (z))σ 2 (z) ∀z ∈ E, (4.7) Σ I (z) := σ 1 (z) − sgn(σ 1 (z)σ 2 (z))σ 2 (z) ∀z ∈ E. (4.8) Note that, according to our definitions, we have R n (V II ) = R IIn and R n (V I ) = R In for any n ∈ N, since the Brownian motions V I and V II , defined in (1.5), are given in terms of Z and not Z n . However, if we define the Brownian motions V In and V IIn by (1.5) with Z replaced by Z n , then the equalities R n (V IIn ) = R IIn and R n (V In ) = R In hold.
The proof of Theorem 4.1 can now be carried out in three steps. First, we formulate a pair of "approximate" coupling problems (for each n ∈ N): for a fixed T > 0 and any starting points r ≤ 0, z ∈ E. The following probabilistic representations for the candidate value functions of Problems (4.9) and (4.10) play an important role in their solutions:  .7) and note that our standing assumption (|σ 1 | + |σ 2 |)(z) > 0 implies Σ 2 II (z) > 0 for all z ∈ E. Therefore, the stochastic time-change A II = (A II t ) t≥0 , given by is a differentiable, strictly increasing process. Furthermore, the definition of Z n and (4.13) imply that the almost sure limit lim t↑∞ A II t = ∞ holds. Hence, the inverse E II = (E II s ) s≥0 , defined as the unique solution of and is a strictly increasing process with differentiable trajectories. Since Z n is an (F t )-Markov chain, it is by Lemma 2.3 independent of the (F t )-Brownian motion B in (4.6). Therefore the laws of the processes (R IIn , Z n ) and (r + B A II , Z n ) coincide, where B A II denotes the Brownian motion B time-changed by the increasing process A II .
Let Σ I : E → R be as in (4.8) and assume further that |σ 1 |(z) = |σ 2 |(z) for all z ∈ E. This implies the inequality Σ 2 I (z) > 0 for all z ∈ E. Define, in an analogous way to (4.13), the strictly increasing continuous time-change A I = (A I t ) t≥0 and its inverse E I = (E I s ) s≥0 , and note that the processes (R In , Z n ) and (r + B A I , Z n ) have the same law. We can now state and prove Lemma 4.2.
Lemma 4.2. Fix n ∈ N and pick any r < 0. Define the stopping time τ B r := inf{s : B s = −r} (with inf ∅ = ∞) and recall that the function G(r, t) := P τ B r > t , for any t ≥ 0, takes the form where N (·) denotes the standard normal cdf.
Proof. We first establish (a). Recall the definition of the time-change process A II and its inverse E II introduced above and note that the following equalities hold almost surely by the definition of the stopping time τ B r : Therefore, since the processes (R IIn , Z n ) and (r + B A II , Z n ) are equal in law, so are the random variables τ + 0 (R IIn ) and E II τ B r . Since E II is a strictly increasing continuous inverse of A II , we have The required differentiability of ζ (II) n in r follows from (4.14), along the same lines as in the proof of Lemma 3.2. An application of the Dominated Convergence Theorem, the mean value theorem and the smoothness and boundedness of the functions ∂G ∂r and ∂ 2 G ∂r 2 on a rectangle (r − ε, r + ε) × (0, ∞) for any fixed r < 0 and small ε > 0, such that ε + r < 0, together imply the existence of ∂ζ (II) n ∂r (r, z, t) and ∂ 2 ζ (II) n ∂r 2 (r, z, t). The differentiability of ζ (II) n in t is more delicate as it is intimately related to the integrability of the chain Z n and the unboundedness of the function Σ II . We start with the following observation.
Claim. The stopping time τ n , defined in (3.3), is a continuous random variable and , the continuity of τ n follows (the definition of the sets U n is given above equation (3.3)). To prove (4.15), note first that the assumption in (1.2), the irreducibility of the chain Z assumed in (1.8) and definition (4.7) imply E z t 0 Σ 2 II (Z s )ds < ∞ for all z ∈ E and t ≥ 0. If E z Σ 2 II (Z t 0 ) = ∞ for some t 0 ≥ 0 and z ∈ E, then the Markov property, the irreducibility of the chain Z and the positivity of Σ 2 II > 0 together imply the following equalities for any u > 0: for all z ∈ E and t ≥ 0, which, together with (4.16) and the irreducibility of the chain Z, yields |(QΣ 2 II )(z)| < ∞ for all z ∈ E. We therefore get since, by definition (3.4), there are only finitely many states z ∈ E, such that (Q n Σ 2 II )(z) = 0, and for each of those states we have (Q n Σ 2 II )(z) = (QΣ 2 II )(z).

Definition (3.3) implies the following inequalities
for any s ≥ 0.
Hence, to prove (4.15), we need to show E z Σ 2 II (Z n s ) < ∞ for all states z ∈ E and times s ≥ 0. Recall, from the definition of Q n in (3.4), that Q n is a bounded operator on the Banach space ℓ ∞ (E) of bounded real functions mapping E into R. Let Q n ∞ < ∞ denote its norm and recall that the norm satisfies Q k n ∞ ≤ Q n k ∞ for all k ∈ N. We can therefore use the exponential series to define a bounded operator exp(sQ n ) and express the semigroup of Z n as follows: E z Σ 2 II (Z n s ) = exp (sQ n ) Σ 2 II (z). Hence, by (4.17), we find for all z ∈ E and s ≥ 0. This implies (4.15) and proves the claim.
In order to prove that ζ (II) n is differentiable in time, fix t > 0, r < 0, z ∈ E and, for any ∆t > 0, define the random variable s. Note also that the random variable |D ∆t (r, z, t)| is bounded by a constant uniformly in ∆t > 0. This follows from the existence of a uniform bound on ∂G ∂t (r, ·) in the second variable for any fixed r < 0 and the mean value theorem. Furthermore the following limits hold: The quotient (ζ (II) n (r, z, t + ∆t) − ζ (II) n (r, z, t))/∆t now takes the form Therefore, by the Dominated Convergence Theorem, the second expectation on the right-hand side of (4.19) converges to E z ∂G ∂t (r, A II t )I {τn>t} Σ 2 II (Z t ) as ∆t → 0. We will now prove that the third expectation on the right-hand side of (4.19) converges to 0 as ∆t → 0. By decomposing the path of Z n at τ n on the event {t < τ n < t + ∆t} and applying the arguments used in the previous two paragraphs to each of the two parts of the trajectory of Z n , there exists a constant C + > 0 such that The probability P z [t < τ n < t + ∆t] tends to zero as ∆t → 0 by the claim and Σ 2 II (Z τn )I {t<τn<t+∆t} is, for ∆t ∈ (0, 1), bounded above by the random variable Σ 2 II (Z τn )I {τn<t+1} , which is integrable by (4.15). Therefore, another application of the Dominated Convergence Theorem implies that the function ζ (II) n is right-differentiable in time. In the case ∆t < 0, analogous arguments to the ones described above yield the left-differentiability of ζ For the proof of part (b), note that, under the assumption |σ 1 |(z) = |σ 2 |(z) for all z ∈ E, we have Σ 2 I (z) = (|σ 1 | − |σ 2 |) 2 (z) > 0 for all z ∈ E. Therefore, a completely analogous argument to the one that established the equality in (4.14), based on the stochastic time-change A I and the fact that the laws of the processes (R In , Z n ) and (r + B A I , Z n ) coincide, where B A I denotes the Brownian motion   If |σ 1 |(z) = |σ 2 |(z) for all z ∈ E, then the modulus | ∂ζ (II) n ∂r | is bounded on (−∞, −ε) × E × (0, ∞), for any ε > 0, and we have If |σ 1 |(z) = |σ 2 |(z) for all z ∈ E, then for all r < 0, t ∈ [0, T ) and z ∈ E we have for all r < 0, t > 0. The derivatives ∂ i G ∂r i , i = 1, 2, are bounded on (r − ε, r + ε) × (0, ∞) for any r < 0 and small enough ε > 0 and hence, as in the proof of Lemma 4.2, the Dominated Convergence for all r < 0, z ∈ E, t > 0. Inequality (4.20) now follows from the inequality in (4.25) and the for all r < 0 and z ∈ E, since ∂ 2 ζ (II) n ∂r 2 (r, z, T − t) ≤ 0 by (4.20). This inequality, the definition of L c ζ (II) n in (2.3) and identity (4.27) imply (4.22). The boundary behaviour of the function ζ (II) n , stated in the lemma, at t = 0 and at r = 0 follows directly from the representation of ζ (II) n given in (4.11).
In the case of the function ζ (I) n , by (4.21) it follows that for any c ∈ [−1, 1] and all r < 0, z ∈ E, t ∈ [0, T ). An analogous argument to the one in the case of  n , defined in (4.11), satisfies the following: Proof. (a) Pick any Brownian motion V ∈ V and, for any t ∈ [0, T ], define the corresponding Brownian (4.28) where V IIn ∈ V is given in (1.5) with Z substituted by the stopped chain Z n . The Bellman process for any r ≤ 0, z ∈ E, t ∈ [0, T ], where the second equality follows from the Markov property and definitions (4.11) and (4.28) of the candidate value function ζ (II) n and of the Brownian motion V IIn .
Note that the stopping time τ + 0 on the right-hand side of the second equality is given by τ + 0 := τ + 0 (R n (V )), and hence does depend on the choice of V . This is not explicitly stated in the formula for brevity of notation.
To establish that the Bellman principle applies, it is sufficient to prove that the process S II is a submartingale for any starting point (r, z) of the process (R n (V ), Z n ). By Lemma 4.2 (a) and Itô's formula for general semimartingales [10, Sec II.7, Thm. 33] we obtain the following pathwise representation of the process S II : implies that the first integral on the right-hand side is almost surely non-negative and hence we have: Apply Lemma 2.2, with F (s, r, z) := ζ (II) n (r, z, T − s), U := R n (V ) and the chain Z n (with bounded generator Q n ), to conclude that the process given by the second and third lines of the last display is a martingale.
The first integral on the right-hand side of (4.30) is clearly a local martingale since the integrator R n (V ) is a martingale. Define a stopping time τ + −ε := inf{t ≥ 0 : R n t (V ) = −ε} for any small ε > 0 and note that τ + −ε < τ + 0 . The quadratic variation of the local martingale is, by Lemma 4.3 (a) and assumption (1.2), bounded above by an integrable random variable. Therefore this stochastic integral is a martingale and, by taking expectations on both sides of inequality (4.30), we obtain Since the paths of R n (V ) are continuous we have lim ε→0 τ + −ε = τ + 0 P r,z -a.s. Hence the definition of S II and its representations in (4.29) imply the following limit: Indeed, on the complement of the event {τ + 0 = T } the limit holds by the definition of S II . On {τ + 0 = T }, the limit is implied by representations (4.29) because the integrals on the right-hand side of (4.29) are continuous in the upper bound of the integration (recall that R n (V ) is a continuous martingale) and τ + −ε ↑ τ + 0 = T , while the sum in (4.29) is P r,z -a.s. constant in ε as the chain Z n does not jump P r,z -a.s. at time T . Since S II is a bounded process, this almost sure limit and the Dominated Convergence Theorem imply the inequality P r,z τ + 0 (R n (V )) > T ≥ ζ (II) n (r, z, T ) for any V ∈ V.
This inequality and the definition of ζ (II) n in (4.11) imply part (a) of the lemma.
where V In ∈ V is given in (1.5) with Z n in the place of Z. The Bellman process S I = (S I t ) t∈[0,T ] is given by for any r ≤ 0, z ∈ E, t ∈ [0, T ], where again τ + 0 := τ + 0 (R n (V )) and the second equality holds by the Markov property and (4.12). Analogous arguments to the ones used in the proof of part (a) can now be applied to conclude that the process S I , appropriately stopped is a supermartingale. Then the optimality of the Brownian motion V In ∈ V follows by a limiting argument as in part (a). This concludes the proof of the lemma. to the case where the assumption |σ 1 |(z) = |σ 2 |(z) for all z ∈ E is not satisfied. The function ζ (I) n in this expression is given in (4.12) and R n (V ) and τ + 0 (R n (V )) are defined in (4.4) and (4.5) respectively. The second step in the proof of Theorem 4.1 consists of a limiting argument that generalises Lemma 4.4 to volatility chains with possibly unbounded generator matrices.
Define the process R n,ǫ (V ) by (4.4), but with σ 1 replaced by σ ǫ 1 , and note that for any t ≥ 0 we have The chain Z has cádlág paths in a state space with discrete topology by assumption (1.8) and hence Z n , defined in (3.3), has only finitely many jumps, say Therefore identity (4.32) implies the inequality |R n,ǫ where the process R In,ǫ is defined in (4.6) with σ 1 substituted by σ ǫ 1 and the last inequality follows by Lemma 4.4 (b).
Define a strictly increasing process A I,ǫ = (A I,ǫ t ) t≥0 and a non-decreasing process A I = (A I t ) t≥0 , analogous to (4.13), by The properties of σ ǫ 1 imply that A I,ǫ t ≥ A I t P z -a.s. for all t ≥ 0. As in the proof of Lemma 4.2, the independence of B and Z (by Lemma 2.3) implies that the processes (R In,ǫ , Z n ) and (r+B A I,ǫ , Z n ) are equal in law, where B A I,ǫ denotes the Brownian motion B time-changed by the precess A I,ǫ . Similarly, we have that the laws of (R In , Z n ) and (r + B A I , Z n ) coincide, where R In is given in (4.6). These observations imply the almost sure inequality, inf{t ≥ 0 : and the fact that the random variable on the left-hand side of this inequality has the same law as τ + 0 (R n,ǫ (V )) while the one on the right-hand side is distributed as τ + 0 (R n (V )). This therefore implies the inequality P r,z τ + 0 (R In,ǫ ) > T ≤ P r,z τ + 0 (R In ) > T which, together with (4.33) and the definition of ζ (I) in (4.12), yields (4.31) and hence concludes step one of the proof of Theorem 4.1.
In the second step of the proof we assume that the volatility process Z is a general (F t )-Markov n in (4.11)-(4.12) imply the following inequalities for any Brownian motion V ∈ V, P r,z τ + 0 (R IIn ) > T ≤ P r,z τ + 0 (R n (V )) > T ≤ P r,z τ + 0 (R In ) > T , (4.34) where R n (V ) is given in (4.4) and R In , R IIn are defined in (4.6). Furthermore, for any t in the stochastic interval [0, τ n ] the following equalities hold: where the process R(V ) is defined in (2.2) and the Brownian motions V I and V II are given in (1.5).
Therefore, we have that, on the event {τ n > T }, the random variables I {τ + 0 (R n (V ))>T } and I {τ + 0 (R(V ))>T } coincide. The same holds true for the pairs I {τ + 0 (R IIn )>T } and I {τ + 0 (R(V II ))>T } , and I {τ + 0 (R In )>T } and I {τ + 0 (R(V I ))>T } . Since (τ n ) n∈N is a non-decreasing sequence of stopping times, such that τ n ր ∞ P z -a.s. as n → ∞, we obtain the following almost sure limits: t ) t≥0 be two progressively measurable processes on (Ω, (F t ) t≥0 , F, P), such that t 0 E Σ (i) s 2 ds < ∞ for i = 1, 2 and any t ≥ 0. As usual, for any V ∈ V, define the difference process R(V ) = (R t (V )) t≥0 by s dV s , r ≤ 0, t ≥ 0. Let the candidate extremal Brownian motions V II = (V II t ) t≥0 and V I = (V I t ) t≥0 be given by Under these assumptions the process R(V ) is a martingale for each V ∈ V. Hence, by [ for all t ≥ 0.
It is clear from these definitions that the following inequalities hold almost surely for all times t ≥ 0: Let τ + 0 (R(V )), τ + 0 (r + W V A II ) and τ + 0 (r + W V A I ) denote the first-passage times over zero of the processes R(V ), r + W V A II and r + W V A I , respectively, and note that the inequalities in (4.36) imply on the entire probability space Ω for every Brownian motion V ∈ V.
It is tempting to conclude from this that the processes r + W V A II and R(V II ), where the Brownian motion V II is defined in (4.35) have the same law (ditto for the pair r + W V A I and R(V I )), which would together with (4.37), yield a generalisation or an alternative proof of Theorem 4.1. However, the counterexample in Section 1.2.1 demonstrates that the generalised mirror coupling in (4.35) can be suboptimal in this setting. The counterexamples to Theorem 4.1, based on the continuous-time Markov chains in Section 5.2, which are adapted non-Markovian processes with respect to the filtration (F t ) t≥0 , clearly show that this approach cannot be used as an alternative proof of Theorem 4.1, because it only requires the volatility processes to be (F t )-adapted. We should stress here however, that in the case of deterministic integrands Σ (1) and Σ (2) , Proposition 4.5 can be established. 3 Proposition 4.5. Let Σ (1) , Σ (2) be deterministic processes (i.e. measurable functions of time) that satisfy the integrability condition above, |Σ (1) s |, |Σ (2) s | > 0 for all s ≥ 0 and A II t , A I t ր ∞ as t ր ∞. Then for any time horizon T > 0 and Brownian motion V ∈ V, the following inequalities hold: Proof. The integrability assumption t 0 (Σ (i) s ) 2 ds < ∞, i = 1, 2, from the beginning of Section 4.3 implies that A II is a well-defined, finite, strictly increasing differentiable function. Its inverse E II , which is defined on [0, ∞) since the limit A II tends to infinity with increasing time, is also strictly increasing and differentiable and satisfies the following ODE: ds.
Since the left-hand side of (4.38) is finite for all u ≥ 0, for any V ∈ V the process W IIV = (W IIV t ) t≥0 , is well-defiend for all t ≥ 0, where W V denotes the (DDS)-Brownian motion introduced above.  Let R(V ) be the difference of X and Y (V ) and assume that it takes the form where B is the fixed ( and recall that P r τ + 0 (R(V I )) > T = F (1/|1 −σ|), P r τ + 0 (R(V II )) > T = F (1/(1 +σ)) (see e.g. T v and clearly satisfies F ′ (v) < 0 for all v > 0, the lemma follows. 4 5.2. (F t )-adapted non-(F t )-Markov processes on a discrete state space. In this section we construct two continuous-time (F t )-adapted processes with a countable discrete state space, neither of which are (F t )-Markov, and show that in both cases the strategies in Theorems 1.1 and 1.2 are suboptimal. In the first (resp. second) example, Section 5.2.1 (resp. Section 5.2.2), the constructed process is semi-Markov (resp. Markov) with respect to its natural filtration. This demonstrates that the assumption that the chain Z is an (F t )-Markov process, not just a Markov process with respect to its "natural" filtration, is indeed necessary in Theorems 1.1 and 1.2.
Define the processes N = (N t ) t≥0 and W = (W t ) t≥0 by For every t > 0 we have {T n ≤ t} ∈ F t for all n ∈ N and hence the process W is (F t )-adapted.
where the second equality follows from the facts that T Nt ≤ t, and that there are no jumps of W Consider the stochastic integral · 0 Z s dB s and note that the equality W Tn − W Tn− = B Tn − B T n−1 holds for all n ∈ N. Hence the stochastic integral can be expresses as follows: Therefore, since by definition we have |B t − B T N t | < ǫ and Z t > 0, the following inequalities hold: As in Section 1.2.1, define σ i : E → R by σ i (z) := −iz for any z ∈ E and i = 1, 2, and note that by (1.5) we have V I = B and V II = −B. Hence, for any starting points x, y ∈ R, definition (1.1) and inequality (5.1) yield the following almost sure inequalities: For any time horizon T > 0, counterexamples to the Conjecture in Section 1.2 (for both Problems (T) and (C)) can now be constructed in the same way as in Section 1.2.1. 5 An additional bijection is needed to define a chain with a state space that is a discrete subspace of a Euclidean space.

5.2.2.
Non-(F t )-Markov Markov chain. In order to define a process Z, which is a time-homogeneous Markov chain in its own filtration and has properties analogous to the ones in the previous section, we sample the path of the Brownian motion B at a sequence of holding times of a Poisson process.
The function g ǫ is supported in [−1, 1] and satisfies The idea is to use g ǫ in order to define a continuous-time random walk W ǫ with increments given by approximating those of the Brownian motion B. However, since these increments can be zero if during a holding-time interval the Brownian motion B has either moved in the positive direction by less than h(ǫ) or in either direction by more than 1, we first need to "prune" the Poisson process N ′ in the following way. 6 We mark N ′ at each time T ′ n , n ∈ N, with 1 (resp. 0) if the event T n − T n−1 , n ∈ N, are exponential IID random variables with mean larger than ǫ, (5.3) N t = max{n ∈ N ∪ {0} : T n ≤ t}, T Nt ≤ t < T Nt+1 and lim ǫ↓0 T Nt = t P-a.s., (5.4) where as before we have T −1 := T 0 := 0. Note that in the construction of the marked Poisson process it is key that the marks at distinct Poisson points of the original process N ′ are independent of each other, which is the case in our setting since the Brownian increments over disjoint holding-time intervals of N ′ are independent.
We can now define the process W ǫ = (W ǫ t ) t≥0 by Note that W ǫ has a countable discrete state space ǫZ and cádlág trajectories. It jumps only finitely many times on any compact time interval, has the same holding times as the Poisson process N , the jumps W ǫ t − W ǫ t− = g ǫ B T N t − B T N t− are distributed as g ǫ (B T 1 ) for all t > 0 and do not depend on the position of W ǫ when the jump occurs. Hence W ǫ is a continuous-time time-homogeneous random walk, making its stochastic exponential Z ǫ := z 0 E(W ǫ ) (see [10,Sec II.7,Thm. 37] for definition) a time-homogeneous Markov chain with a countable state space and cádlág paths (footnote 5 also applies here). If for some T > 0 it holds where Z is defined in (1.6), then, since the stochastic exponentials Z and Z ǫ are square integrable on compact intervals (see Lemma 5.2), the Burkholder-Davis-Gundy inequality [10,Sec IV.4,Thm. 48] implies the following almost sure convergence as ǫ → 0, for any Brownian motion V ∈ V, cost function φ and volatility functions σ 1 , σ 2 given in Section 1.2.1 (the processes X ǫ , Y ǫ (V ) are defined in (1.1) with Z replaced by Z ǫ and the stopping The counterexamples from Section 1.2.1 (with a bounded φ ∈ C 2 (R)) show that the conjecture in Section 1.2 fails (for both Problems (T) and (C)) in the case of the process Z ǫ if ǫ > 0 is small enough.
In order to complete our counterexample, we need to prove that the limit in (5.5) holds. To this end we establish the following lemma.
Lemma 5.2. Let Z ǫ be as defined above and let Z be given by (1.6). Then Z ǫ and Z are square integrable on compact intervals and there exists a constant C 0 > 0 such that the following holds: Proof. Recall that the definition of the stochastic exponential (see e.g. [10, Sec II.7, Thm. 37]) implies the following representations for Z ǫ : where the sum (resp. product) is taken to be zero (resp. one) if N t = 0. The second equality and the definition of the function g ǫ imply that 0 < Z ǫ t ≤ z 0 2 Nt (recall that z 0 > 0), which yields the stated square integrability of Z ǫ . The first equality and the fact Z t = z 0 + t 0 Z s dB s imply the following: The definition of the Poisson process N , the definition of g ǫ with the estimate in (5.2), the elementary fact (a + b + c) 2 ≤ 9(a 2 + b 2 + c 2 ) for any positive a, b, c, and the triangle inequality yield for some constants A 0 , B 0 , B 1 > 0. Since, by (5.3), the random variable N t is Poisson distributed with the rate less or equal to t/ǫ and h(ǫ) = exp(−1/ǫ 2 ), we have lim ǫ↓0 E exp(B 0 N t )h(ǫ) 2 = 0 for any t ≥ 0.
The inequality in the lemma now follows from (5.6), (5.7), (5.8) and (5.10). The last statement in the lemma follows from the fact that N t ≤ N T for any t ∈ [0, T ], the bound in (5.9), the right-hand side of which is independent of ǫ, and Doob's L 2 -martingale inequality.
Hence, for any T ∈ (0, ∞), we have Since T is fixed and α(·, ǫ) is bounded uniformly in ǫ on [0, T ], the Dominated Convergence Theorem and Lemma 5.2 imply that the right-hand side of this inequality tends to zero and (5.5) follows.

(F t )-
Feller process Z independent of B. The final counterexample shows that the "tracking" part of the conjecture in Section 1.2 fails for general Feller processes even if Z and B are independent.
Assume that there exist an (F t )-Brownian motion B ⊥ ∈ V, independent of B, and define the (F t )-Feller process Z := z 0 + B ⊥ with state space E := R for any starting point z 0 ∈ R. Let σ 1 (z) := 2z and σ 2 (z) := z, for any z ∈ R, and note that by (1.5) we have V I = B. We will now show that, for the cost function φ(x) := x 4 , the first inequality in Problem (T) fails, i.e. there exists a Brownian motion V ∈ V such that for any T > 0 it holds where R(V ) = X − Y (V ) (and X, Y (V ) given in (1.1) for any V ∈ V) and R 0 (V ) = r, Z 0 = z 0 .
To construct such a process V , define the family V c = (V c t ) t≥0 , c ∈ [−1, 1], of (F t )-Brownian motions by and note that V 0 = B = V I . Therefore the difference process R(V c ) takes the form Proof. The representation in the lemma for the expectation ψ c (r, z, t) follows from martingale arguments and stochastic calculus. Alternatively to verify the lemma, one can easily check that the function ϕ, given by the formula above, satisfies the PDE with boundary condition ϕ(r, z, 0) = r 4 and polynomial growth in r and z. An application of the Feynman-Kac formula then yields ψ c = ϕ.  The properties of the random variables o(∆t) listed in the paragraph above, the Dominated Convergence Theorem applied to the right-hand side of (A.4) as ∆t ↓ 0, the definition of the Lebesgue integral and the fact that Z jumps only finitely many times during the time interval [t, t ′ ] together imply the equality in (A.2). This concludes the proof of the lemma. ✷