Parameter-dependent optimal stopping problems for one-dimensional diffusions

We consider a class of optimal stopping problems for a regular one-dimensional diffusion whose payoff depends on a linear parameter. As shown in [ Bank and Föllmer(2003) ] problems of this type may allow for a universal stopping signal that characterizes optimal stopping times for any given parameter via a level-crossing principle of some auxiliary process. For regular one-dimensional diffusions, we provide an explicit construction of this signal in terms of the Laplace transform of level passage times. Explicit solutions are available under certain concavity conditions on the reward function. In general, the construction of the signal at a given point boils down to ﬁnding the inﬁmum of an auxiliary function of one real variable. Moreover, we show that monotonicity of the stopping signal corresponds to monotone and concave (in a suitably generalized sense) reward functions. As an application, we show how to extend the construction of Gittins indices of [ Karatzas(1984) ] from monotone reward functions to arbitrary functions.


Introduction
In this paper, we study optimal stopping problems of one-dimensional regular diffusions that depend on a parameter.A well-known example is the perpetual American put option written on an underlying process X with strike k whose discounted payoff at time t is given by e −r t (k − X t ) + .More generally, with denoting the set of stopping times, one can consider the optimal stopping problem with value function where r > 0 is a discount rate and the reward function is h k (•) u(•)− k.A general characterization of the value function in terms of excessive functions was proved by [Dynkin(1963)].Moreover, efficient methods for the calculation of the value function are available if X is a one-dimensional regular diffusion, see e.g.[Dayanik and Karatzas(2003)].This calculation is reduced to finding the smallest majorant of the reward function h k which exhibits a suitably generalized form of concavity.Optimal stopping times can then be identified as the first entrance time of the underlying diffusion into the set where the value function and the reward function coincide.A purely probabilistic approach to find optimal stopping times is to compute the Snell envelope of the payoff process; see e.g.[Shiryayev(1978)], [El Karoui(1981)] or [Karatzas(1988)].In any case, if the reward function depends on a parameter k as in the case of an American put option, both approaches outlined above have to be repeated anew for each choice of the parameter.By contrast, [Bank and Föllmer(2003)] show how to construct a universal stopping signal from the optional solution K to the representation problem Optimal stopping times can be obtained from the solution K to the representation problem via a level-crossing principle: it is optimal to exercise a put with strike k as soon as K passes the threshold k.While explicit formulas for the signal process K have been found in special cases, this paper provides an explicit construction of universal stopping signals for general one-dimensional diffusions and reward functions.Specifically, the computation of the optimal stopping signal at a given point is reduced to finding the infimum of an auxiliary function of one variable that can be computed explicitly from the Laplace transforms of level passage times for the underlying diffusion.Moreover, we show that the infimum can be determined in closed form if the reward function satisfies certain concavity conditions.We also prove that monotone stopping signals correspond to monotone and concave (again in a suitably generalized sense) reward functions and provide explicit solutions in that case.Finally, we illustrate our method by applying them to optimal stopping problems for American options and to the computation of Gittins indices, extending results of [Karatzas(1984)].
This paper is organized as follows: in Section 2 we outline those results theory of optimal stopping for one-dimensional regular diffusions which are needed for our approach.In Section 3 we introduce a class of optimal stopping problems depending on a parameter k in a linear way and discuss how to compute universal stopping signals in that case.Our results are illustrated by some examples.We close by pointing out the connection of our results to stochastic representation problems.acknowledge financial support of Quantitative Products Laboratory, Berlin.Christoph Baumgarten also thanks Berlin Mathematical School that supported him while writing his diploma thesis at Technische Universität Berlin.
Our assumptions are sufficient for the SDE (1) to have a weak solution (Ω, , = ( t ) t≥0 , {P x : x ∈ }, W, X ) up to an explosion time that is unique in the sense of probability law; see [Karatzas and Shreve(1991), Section 5.5 C].Define T y inf{t ≥ 0 : X t = y} the first hitting time of the point y ∈ .Assumption 1 implies that X is regular, i.e.P x T y < ∞ > 0 for all x ∈ , y ∈ (see [Dayanik and Karatzas(2003)]).Throughout this paper, we consider only natural boundaries: Assumption 2. The boundaries of are natural for X , i.e.
Next, we introduce our discounting process (R t ) t≥0 given by where we impose the following conditions on the function r: Assumption 3. The function r : → (0, ∞) is Borel-measurable and locally bounded.Moreover, there is r 0 > 0 such that r(x) ≥ r 0 for all x ∈ .
In particular, R is a continuous additive functional of X , i.e.R is a.s.nonnegative, continuous, vanishes at zero and has the additivity property where θ s is the usual shift operator: X t • θ s = X t+s .For an account on additive functionals, see, e.g., Chapter X in [Revuz and Yor(1991)].Moreover, we have R t → ∞ as t → ∞.Sometimes we write R(t) instead of R t for notational convenience.
Let us now recall how to compute the Laplace transforms of R(T y ) for a < y < b.To this end, consider the ODE where is the infinitesimal generator of X .This ODE has two linearly independent, positive solutions.These are uniquely determined up to multiplication by constants, if we require one of them to be increasing and the other decreasing.We denote the increasing solution by ψ and the decreasing one by ϕ.They are continuously differentiable and satisfy We refer to [Johnson and Zervos(2007)] for these results; see also [Dayanik and Karatzas(2003)] and [Itô and McKean, Jr.(1974)] for the case r(x) ≡ r > 0.
It will be convenient to introduce the increasing functions F and G defined by Lemma 3. [Dayanik(2008), Lemma 2.3] For every x ∈ [ y, z] ⊂ , we have Following [Dayanik and Karatzas(2003)], we use the following generalized notion of concavity.
Definition 4. Let f : → be a strictly monotone function.Then We speak of a strictly f -concave function if the inequality above holds in the strict sense whenever y < x < z.
Clearly, if f (x) = x, then a f -concave function is concave in the usual sense.Some facts about f -concave functions can be found in [Dynkin(1965)].
Remark 5. Note that by Lemma 3, the function 1/ϕ is strictly F -concave whereas 1/ψ is strictly G-concave on .
With these concepts at hand, we may now recall some very useful results of [Dayanik(2008)] that show how to solve optimal stopping problems in our setting; see [Dayanik and Karatzas(2003)] for these results in case of a constant discount rate.Let h be a Borel function such that E x e −R τ h(X τ ) is well defined for every -stopping time τ and x ∈ .Moreover, we assume that h is bounded on every compact subset of .By convention, we set h(X τ ) = 0 on {τ = ∞}.Denote by the value function of the optimal stopping problem with reward function h and discount rate r(•), where the supremum is taken over the set of -stopping times.
Proposition 6. [Dayanik(2008), Prop. 3.1 -3.4, 3.10] 1.The value function V is either finite or infinite everywhere on .Moreover, V ≡ ∞ on if and only if at least one of the limits a lim sup and b lim sup 2. If h is continuous on and a = b = 0, the stopping time 3. If a and b are both finite, the value function V is continuous on .Moreover, V is the smallest nonnegative majorant of h on such that V /ϕ is F -concave (equivalently, V /ψ is G-concave) on .
4. With W : F ( ) → denoting the smallest nonnegative concave majorant of the function the value function V is given by

Parameter-dependent optimal stopping
In many applications, the reward function h of an optimal stopping problem depends on some parameter k, i.e. sup τ∈ A very prominent example is the American put option with strike k, i.e. h k (x) = (k − x) + .In the previous section, we have seen how to compute the corresponding value function for a fixed value of k by finding the smallest majorant of the reward function h k which is concave in some suitably generalized sense.An optimal stopping region Γ k is then given by the set where the value function and the reward function coincide (cf.Proposition 6).If one has to determine optimal stopping rules corresponding to many different values of k, this approach might be quite tiresome, particularly if the structure of the stopping region is complex.In such a case, it would be desirable to have a universal stopping signal γ that characterizes the optimal stopping regions for any given parameter k, for instance as the level set Γ k = {x : γ(x) ≥ k}.In the sequel, we describe a method to compute such a universal stopping signal for reward functions of the form h k (x) = u(x) − k, i.e. for the optimal stopping problem where u satisfies the following assumption: Assumption 4. The function u: → is continuously differentiable and Remark 7. In view of Proposition 6, Assumption 4 ensures that the value function V k of (4) is finite for any k.

Computation of optimal stopping signals
Let us discuss how to compute a universal stopping signal yielding an optimal stopping time for any k.To this end, let denote the optimal stopping region (cf.Proposition 6).
Assumption 5.For any k, the stopping time A sufficient condition for optimality of the stopping times τ k was given in Proposition 6.Moreover, granted there is an optimal stopping time, τ k is optimal as well (see e.g.[El Karoui(1981)]).
Remark 8. Recall that we use the convention h(X τ ) = 0 on the set {τ = ∞} for any stopping time τ.
The existence of an optimal stopping time τ k requires that the supremum in ( 4) is attained.
We aim to derive a function γ on such that the optimal stopping regions Γ k can be written as level sets Γ k = {x : γ(x) ≥ k}.In other words, knowledge of the function γ suffices to derive the optimal stopping regions for any parameter k.Our main result, Theorem 13 below, shows that γ(x) can be found as the infimum over an auxiliary function η x (•) of one real variable which is expressed in terms of the Laplace transforms ϕ and ψ of level passage times.The infimum can be computed explicitly under specific concavity conditions on the reward function u.In the sequel, we show how to obtain this result and give an explicit expression for η x in (12).
Let T U inf{t ≥ 0 : X t ∈ U} denote the first exit time from a measurable subset U of (a, b) and the class of all exit times from open neighborhoods of x.The following lemma shows that the stopping signal γ is the solution to a non-standard optimal stopping problem (cf.equation ( 25) in [Bank and Föllmer(2003)], also [El Karoui and Föllmer(2005)]).
Proof.For any k, we have x ∈ Γ k if and only if To see this, note that if x ∈ Γ k , then V k (x) = u(x)−k and the inequality above is true by definition of the value function.On the other hand, note that the sets Γ k are closed in since V k is continuous for all k by Proposition 6.Thus, if x ∈ Γ k , τ k = inf{t : X t ∈ Γ k } is an exit time from an open neighborhood of x, i.e. τ k ∈ ˜ (x).Since τ k is optimal by Assumption 5, we get Thus, for any k, we have Let us now discuss how to compute the function γ of (5).The following lemma reduces the optimal stopping problem of the preceding lemma to finding the infimum of a function of two variables.
Lemma 10.For any x ∈ , where The claim now follows from Lemma 1 and Lemma 3.
Remark 11.According to Lemma 3, the functions ϕ and F may be replaced by ψ and G in the definition of f x , i.e. for a < y < x < z < b, .
For simplicity of notation, let For a < y < x < z < b, simple manipulations yield We remark that Let us discuss next how to compute the infimum of f x ( y, z) over y and z for fixed x in order to find γ(x) in (6).In view of the definition of f x , it seems natural to ask what happens if one lets y ↑ x or z ↓ x in (7).One notices that a difference quotient similar to the definition of the usual derivative appears.Let us therefore recall the following definition.
Definition 12. Let f : → be a strictly monotone function.We say that U : exists.
For f (s) = s, the definition above is just the usual derivative.Also, if U and f are differentiable at x, then d f U(x) = U (x)/ f (x).In particular, under Assumption 4, α ϕ and α ψ are F -and Gdifferentiable.β ϕ and β ψ are always F -and G-differentiable.
From ( 7), we find that Using l'Hôspital's rule, one computes We now define η x (•) can also be written in terms of G, α ψ and β ψ instead of F, α ϕ and β ϕ in view of ( 8) and ( 9).In some proofs, it will be more convenient to use this alternative representation.Note that η x may fail to be continuous at the boundaries a or b.In fact, we have lim We now show that the computation of the optimal stopping signal γ at a point x ∈ amounts to finding the infimum of the function η x of one real variable instead of the function f x of two variables.
In order to prove Theorem 13, we need the following lemma.Lemma 14.For any a ≤ y < x < z ≤ b, it holds that f x ( y, z) ≥ min {η x ( y), η x (z)}.
Proof.Suppose first that a < y < x < z < b.If f x ( y, z) = 0, then either η x ( y) ≤ 0 or η x (z) ≤ 0. Indeed, if both expressions were positive, then since f x ( y, z) = 0, we would get (considering only the numerators of f x in (7) and η x in ( 12)) that If f x ( y, z) > 0, assume for the purpose of contradiction that the claim of the lemma is false, i.e.
Strict F -concavity of β ϕ implies that all denominators are positive and the numerator of the righthand side is also positive since f x ( y, z) > 0. Thus, we can rewrite the inequalities above as .
If we add both equations, we obtain 1 > 1, which is absurd.
The case f x ( y, z) < 0 can be handled analogously.
The proof in the case a < y < x, z = b is also along similar lines.Assume again that If f x ( y, b) = 0, then α ϕ (x) = α ϕ ( y), so we are led to the contradictory statement If f x ( y, b) = 0, rearranging terms and adding the resulting equations as before, we also obtain a contradiction.
If y = a and z < b, assume again that the claim of the lemma is false, i.e.
Here we have used the representation of η x involving G and ψ stated in (8).Since β ψ = 1/ψ is G-concave and G is decreasing, this leads to a contradiction as before.Finally, if y = a, z = b, since ψ > 0 and ϕ < 0, clearly Proof.(Theorem 13).By Lemma 10 and Lemma 14, it is immediate that for all x ∈ (a, b) To see the reverse inequality, note that for z > x, Similarly, for y < x, Thus, inf

Concave envelopes
According to Theorem 13, the computation of the universal stopping signal γ amounts to minimizing the function η x given by (12) for every x ∈ .In the sequel, we show that under certain concavity conditions on u, the infimum of η x is attained at the boundaries a or b of which yields a closed form solution for γ(x) since η x (a) = κ(x) (see ( 10)) and η x (b) = ρ(x) (see ( 11)).To this end, if ς: → is a strictly monotone function, denote by u ς the ς-concave envelope of u, i.e.
Provided that u ς is finite, u ς is itself ς-concave.We remark that u ς (x) = u • ς −1 (ς(x)) where u = u id refers to the usual concave envelope.With this concept at hand, we may now state the main result of this section.
1. Then u is ς-concave iff for all x, y ∈ Proof. 1.Note that u is ς-concave on if and only if u • ς −1 is concave on ς( ).This is equivalent to The claim follows upon noting that for all x ∈ ς( ) 2. Assume that x ∈ {u = u ς }.By definition of the ς-concave envelope, the differentiability of u and ς and the first part of the lemma, it must hold for all y that On the other hand, the ς-concave function f ( y) u(x) and f ≥ u if (13) holds.This implies u(x) = u ς (x).
Proof.(Theorem 15) Let x, y ∈ .For y < x, η x ( y) ≥ κ(x) is equivalent to We have used the fact that β ϕ = 1/ϕ is F -concave (Remark 5), F is increasing and part 1 of Lemma 16.Since κ(x) = d F α ϕ (x)/d F β ϕ (x), we may write Some algebra shows that this is further equivalent to If y > x, a similar calculation leads to the same result showing that η x ( y) ≥ κ(x) = η x (b) for all y ∈ if and only if In view of part 2 of Lemma 16, we deduce from ( 14) that κ(x) ≤ η x ( y) for all y ∈ if and only if which is equivalent to u (x) ≤ 0. This proves the first assertion of the theorem.
The proof of the second claim is along the same lines.For a < y < x, η x ( y) ≥ ρ(x) is equivalent to One can show that η x ( y) ≥ ρ(x) for all y ∈ if and only if Since ρ(x) ≤ κ(x) = η x (a) iff u (x) ≥ 0, the proof is complete in view of (15) and Lemma 16.
We can now prove that a monotone stopping signal γ is equal either to κ or to ρ.Thus, by Theorem 15, a nondecreasing (resp.nonincreasing) signal corresponds to a nondecreasing (resp.nonincreasing) reward function u that is ψ-concave (resp.ϕ-concave).In order to prove this claim, we first show that γ is upper semi-continuous.
Lemma 17.The function γ is upper semi-continuous on .
Proof.If {γ > k} = , we have that γ( y) = k for all y ∈ (a, x] and Since k → V k ( y) is continuous for any y ∈ , we deduce that u( y) − k = 0 for all y ∈ (a, x].In particular, u (x) = 0 and therefore, Assume next that {γ > k} = .If {γ = k} = {x}, then for y > x, we have due to Lemma 1 that Thus, g attains its maximum at the interior point x and x must satisfy g (x) = 0 which is equivalent to κ(x) = k.By definition, also γ(x) = k.
Thus, for any y ∈ (x 1 , x 2 ], we have that showing that u is an affine transformation of ϕ on (x 1 , x 2 ], i.e. u( y) = c •ϕ( y)+ k for some constant c = c(x 1 ) > 0. Now The same reasoning applies in the remaining case {γ = k} = (x 1 , b).
On the other hand, if γ = κ on , u must be nonincreasing and ϕ-concave on due to Theorem 15.By Lemma 16, we have for all x, y ∈ that which we may rewrite as Proof.By Theorem 15, we have γ( From the definition of the ϕconcave envelope, we see that This implies that κ(l n ) = κ(r n ).
In particular, the function κ : → defined by x ∈ (l n , r n ) for some n, defines a nonincreasing majorant of γ and coincides with the optimal stopping signal on {u = u ϕ }.
Note that if the diffusion starts at x 0 ∈ {u = u ϕ }, optimal stopping times can be derived directly from κ.Thus, the computation of γ on the intervals (l n , r n ) as the infimum of η x as suggested by Theorem 13 is not required in that case.
Of course, an analogous result holds if u is nondecreasing.Again, { u ψ > u} is the countable union of disjoint open intervals denoted by (l n , r n ) and we have

Applications and illustrations
For simplicity, we illustrate our results for a constant discount rate r > 0 which implies that R t = r t.Examples with random discounting can be found in [Dayanik(2008)] and [Beibel and Lerche(2000)].

Optimal stopping
Example 21.Let us consider the well-known example of a perpetual put option written on an underlying whose dynamics are given by a geometric Brownian motion with volatility σ > 0 and drift µ ∈ , i.e. consider the diffusion The functions ϕ and ψ are the solutions to the ODE (2) which is One finds that ϕ(x) = x α and ψ(x) = x β , x ∈ where α < 0 and β > 0 are the roots of q(x) = 1 2 σ 2 x(x − 1) + µx − r.Let us show how to compute the optimal stopping signal for the family of American put options whose payoff profile is given by h k (x) = (k − x) + where k > 0. The corresponding optimal stopping problem is Clearly, the quantities 0 and ∞ defined in Proposition 6 are equal to zero for any k since h k is bounded.Thus, the stopping times τ k inf{t : timal by Proposition 6 for any strike k.Note that V k > 0 since X reaches the point k/2 with positive probability which in turn leads to a positive payoff.It follows that Example 22.This example considers a reward function u that is not monotone.Nonetheless, u is ϕand ψ-concave and the optimal stopping signal can be computed explicitly in view of Theorem 15.
Assume that X is a standard Brownian motion and r ≤ 1.The functions ϕ and ψ are given by Consider the reward function u(x) = − cosh(x).Then Note that γ(−x) = γ(x) for all x ∈ which is clear by symmetry.
Example 23.Finally, we provide an example of a diffusion and a decreasing reward function u such that u is not ϕ-concave.The optimal stopping signal γ is computed numerically on the interval where u and its concave envelope do not coincide.To this end, consider a diffusion X specified by the SDE for σ, µ, c > 0. We would like to derive the optimal stopping signal related to the perpetual American put option, i.e. u(x) = −x.The functions ψ and ϕ must solve the ODE (2), i.e.
One computes where α 1 < 0, α 2 > 0 are the roots of q 1 (x) = 1 2 σ 2 x(x − 1) − µx − r and β 1 < 0, β 2 > 0 are the roots of q 2 (x) = 1 2 σ 2 x(x − 1) + µx − r.In order to compute the universal stopping signal, we apply Theorem 15.Since u is decreasing, let us check whether u is ϕ-concave which is equivalent to convexity of ϕ.Clearly, the restriction of ϕ to [c, ∞) is convex.One can show that ϕ has a root on (0, c) iff Note that the right-hand side of the inequality above is increasing in µ.Thus, if the drift coefficient µ is sufficiently large compared to σ and r, ϕ changes its curvature and is not convex on (0, ∞).For simplicity, assume that c = 1 and r = µ.Then u • ϕ −1 is not concave (or equivalently, ϕ is not convex) if σ 2 /2 < r, see Figure 1.
It is now easy to determine the set {u = u ϕ }.In view of Lemma 16, we find that x ∈ {u = u ϕ } if and only if ϕ( y) ≥ ϕ(x) + ϕ (x)( y − x) for all y.
The equation above has an obvious graphical interpretation: If (and only if) the tangent that touches the graph of ϕ at x remains below the graph of ϕ everywhere, then x ∈ {u = u ϕ }; see Figure 1.We see that { u ϕ > u} = (â, b) for unique 0 < â < c < b.According to Theorem 15, we have γ(x) = κ(x) for x / ∈ (â, b) whereas the value of γ(x) can be computed numerically as the infimum over η x given by ( 12) for x ∈ (â, b).The resulting stopping signal is shown in Figure 1.Note that not all the optimal stopping regions Γ k are connected in this case.

Gittins indices
The approach described in this section can also be applied to the class of optimal stopping problems that appears in [Karatzas(1984)], namely These stopping problems arise in the context of multi-armed bandit problems where one seeks to dynamically allocate limited resources among several independent projects in order to maximize the discounted reward generated by these projects.The evolution of each project is modeled by a stochastic process such as a one-dimensional diffusion or a Lévy process (see [Kaspi and Mandelbaum(1995)]) and u(x) specifies the reward generated by allocating resources to a project in state x.Optimal allocation rules can be found by the projects' Gittins indices, see e.g.[Gittins(1979), Gittins and Jones(1979)].An optimal strategy is to engage in the project whose index is maximal.Specifically, the Gittins index of a project with diffusion dynamics X as in [Karatzas(1984)] and reward function u can be derived from the solution to (17).This optimal stopping problem can be reduced to a problem of type (4).To this end, set p(x) E x ∞ 0 e −r t u(X t ) d t .It follows from the strong Markov property that e −rτ p(X τ ) = E x ∞ τ e −r t u(X t ) d t| τ P x -a.s.Thus, This is now an optimal stopping problem of type (4).Note that the function p is a particular solution to the non-homogeneous linear ODE f − r f = u on since p = U r u where U denotes the resolvent corresponding to the diffusion X .
If the state of each project is given by a one-dimensional regular diffusion, one can show (Theorem 4.1 in [Karatzas(1984)], also [Gittins and Glazebrook(1977)]) that the Gittins index M is given by This show that the Gittins index is basically the optimal stopping signal γ associated with the optimal stopping problem (4) for the reward function −p.
In [Karatzas(1984)], the function u is assumed to be increasing, continuously differentiable and bounded.Then p inherits the same properties.Note that adding a positive constant c to the reward function u only shifts M upwards by the amount c/r.Thus, we may assume without loss of generality that u and p are positive.Using that ( − r)p = u and ( − r)ϕ = 0, one can check that −p • ϕ −1 is strictly concave in that case.Therefore, we apply Theorem 15 to deduce that the optimal stopping signal is given by γ Karatzas(1984)].We may also use Theorem 15 to compute the Gittins index if u is not increasing.

A stochastic representation problem
Let us now consider the stochastic representation problem i.e. for a given function u and a constant r > 0, find an optional process K such that the equation above holds.If the representation above holds for some K, then for any k ∈ and any stopping time T , we have Moreover, if we choose T k inf{t : K t ≥ k}, the inequality above becomes an equality provided that the process K has upper semicontinuous paths (ensuring that K T k ≥ k if T k is finite).Thus, the process K characterizes optimal stopping times for any parameter k just as the process (γ(X t )) t≥0 in the previous section.Hence, there is a close relation of optimal stopping and stochastic representation problems and we refer to [Bank and Föllmer(2003)] for an overview, see also [Bank and El Karoui(2004)].
Note that if (18) holds, then choosing T = 0 yields On the other hand, if ( 19) is satisfied, then by the strong Markov property, it holds for any T ∈ that Therefore, solving (18) is equivalent to solving (19).The latter problem was considered by [El Karoui and Föllmer(2005)] in a more general setting without discounting.One can proceed in an analogous fashion in order to prove that (K t ) t≥0 defined by K t γ(X t ) solves the representation problem (18) if the diffusion X satisfies some additional uniform integrability assumptions.Let us just briefly outline the main ideas.
Let (x) denote the class of all exit times from relatively compact open neighborhoods of x.Recall that any T ∈ (x) is P x -a.s.finite [Revuz and Yor(1991), Prop.VII.As in [El Karoui and Föllmer(2005)], we say that a measurable function u: → is of class (D) if the family {e −r T u(X T ) : T ∈ (x)} is uniformly integrable with respect to P x for all x ∈ .Assumption 6.The function u is continuous on , of class (D) and for all x ∈ , it holds that lim t→∞ e −r t u(X t ) = 0 P x -a.s.
Proof.The inequality γ(x) ≤ Du(x) is clear since (x) ⊂ ˜ (x).In order to prove the reverse inequality, denote by (U n ) n a sequence of relatively compact open sets increasing to , and denote by S n the exit time from the set U n .For T ∈ ˜ (x), the stopping times T n T ∧ S n ∈ (x) increase to T , and so we have We can now prove the following uniqueness result which corresponds to Theorem 3.1 of [El Karoui and Föllmer(2005)] in our modified setting.(20) This implies that Du(x) ≥ f (x).
To prove the reverse inequality, fix α > f (x) and denote by T α the exit time from the set { f < α}.Since lim sup y→x f ( y) ≤ f (x) for all x by upper-semicontinuity of f , the set { f ≥ α} is closed.Thus, T α is indeed an exit time from an open neighborhood of x.For t ≥ T α , it holds P x -a.s. that sup 0≤s≤t f (X s ) = sup The proof is complete due to Lemma 24.
In order to prove that the process (γ(X t )) t≥0 does indeed solve (18) under Assumption 6 or equivalently, u(x) = G Du(x) for x ∈ , one can proceed in the spirit of [El Karoui and Föllmer(2005)].
3.1].Using the notation of El-Karoui and Föllmer, we setDu(x) inf T ∈ (x) u(x) − E x e −r T u(X T ) E x 1 − e −r T , G f (x) E x ∞ 0 r e −r t sup 0≤s≤t f (X s ) d t .
E x 1 − e −r T n ↑ E x 1 − e −r T .Moreover, since u is of class (D) by Assumption 6, it follows thatlim n→∞ E x e −r T n u(X T n ) = E x e −r T u(X T ) .Therefore, for any T ∈ ˜ (x),E x u(x) − e −r T u(X T ) E x 1 − e −r T = lim n→∞ E x u(x) − e −r T n u(X T n ) E x 1 − e −r T n ≥ Du(x),proving thatγ(x) = inf T ∈ ˜ (x) E x u(x) − e −r T u(X T ) E x 1 − e −r T ≥ Du(x).

Proposition 25 .
Under Assumption 6, suppose that the function f : → is upper-semicontinuous andu(x) = G f (x) = E x ∞ 0 r e −r t sup 0≤s≤t f (X s ) d t .Then f (x) = Du(x).Proof.By the strong Markov property, it holds for any T∈ (x) that G f (X T ) = E x ) − E x e −r T u(X T ) = G f (x) − E x e −r T G f (X T ) −r t sup 0≤s≤t f (X s ) d t   ≥ f (x)E x   T 0 r e −r t d t   = f (x)E x 1 − e −r T .
T α ≤s≤t f (X s ) and the first inequality in (20) becomes an equality for T = T α .Thus,E x u(x) − e −r T u(X T α ) = E x x 1 − e −r T α .Now take a sequence α n ↓ f (x) to conclude that γ(x) = inf T ∈ ˜ (x) u(x) − E x e −r T u(X T ) E x 1 − e −r T ≤ f (x).
its derivative is nonincreasing showing that d ϕ u(•) is nondecreasing.Thus, if x ≤ y, we see from (16) that κ(x) ≥ κ( y).The proof of the second assertion is similar.Let us further investigate the structure of the stopping signal γ if the reward function u is monotone.For instance, if u is nonincreasing, we know from Theorem 15 that γ(x) = κ(x) iff u(x) = u ϕ (x).Since the set { u ϕ > u} is open, it is the countable union of disjoint open intervals which we denote by (l n , r n ), n ∈ .In this setting, we have the following result: Proposition 19.If u is nonincreasing on and [l n , r n ] ⊂ , then