Stopping with expectation constraints: 3 points suffice

We consider the problem of optimally stopping a one-dimensional continuous-time Markov process with a stopping time satisfying an expectation constraint. We show that it is suﬃcient to consider only stopping times such that the law of the process at the stopping time is a weighted sum of 3 Dirac measures. The proof uses recent results on Skorokhod embeddings in order to reduce the stopping problem to a linear optimization problem over a convex set of probability measures.


Introduction
Let (Y t ) t∈R ≥0 be a one-dimensional strong Markov process with respect to a right-continuous filtration (F t ). Let f : R → R be measurable and denote by T (T ) the set of (F t )-stopping times such that E[τ ] ≤ T ∈ R ≥0 . In the following we consider the optimal stopping problem maximize E[f (Y τ )] subject to τ ∈ T (T ). (0.1) The problem (0.1) arises whenever an average time constraint applies for any stopping rule. If a process has to be stopped repeatedly and independently of the previous stopping times, then it is reasonable to impose an average time constraint instead of a sharp constraint of the type τ ≤ T , a.s. For example think of the question of when to stop searching for a parking space. If you face this question every morning when driving to your work, it is more likely 1 that you impose an average constraint on your searching time than just a sharp upper bound. Notice that there is no deterministic dependence of the constraint on time. For solving the stopping problem (0.1) one needs to turn the expectation constraint into a scenario-dependent constraint.
In this article we show that for the stopping problem (0.1) it is sufficient to consider only stopping times τ such that the law of Y τ is a weighted sum of at most 3 Dirac measures. Any such stopping time can be interpreted as a composition of exit times from intervals.
We also show that in general a reduction to weighted sums of 2 Dirac measures is not possible. In particular, one can not split the state space into a deterministic stopping and continuation region. This is in contrast to stopping problems with a sharp bound on the stopping time and to stopping problems with infinite time horizon and discounting.
Our idea for proving a reduction to 3 Dirac measures is to rewrite the stopping problem (0.1) as a linear optimization problem over a set of probability measures. To this end we use recent results on the Skorokhod embedding problem characterizing the set A(T ) of probability distributions that can be embedded into Y (see [1] and [9]) with stopping times having expectation smaller than or equal to T . As for standard linear problems the maximal value of the optimization is attained by extreme points. The extreme points of A(T ) turn out to be contained in the set of probability measures that can be written as a weighted sums of at most 3 Dirac measures.
To the best of our knowledge, the idea of using Skorokhod embeddings to analyze optimal stopping problems first appeared in [19], where the authors solve an optimal stopping problem for the geometric Brownian motion, under the Choquet integral, and where the only condition imposed on the stopping times is that they are almost surely finite. When it comes to optimal stopping problems with constraints on the stopping time distribution, the literature is rather scarce: Kennedy [10] solves an optimal stopping problem with an expectation constraint for a discrete time process. The author uses Lagrangian techniques to write the problem as an unconstrained one. Within a continuous time setting, the article [2] formulates a dynamic programming principle for stopping problems with expectation constraints and derives a verification theorem. Different constraints have been recently studied: Bayraktar and Miller [3] consider the problem of optimally stopping the Brownian motion with a stopping time whose distribution is atomic with finitely many points of mass. Miller [11] analyzes stopping problems with time inconsistent constraints. In [4], the authors use optimal transport techniques to treat the problem of optimally stopping the Brownian motion with a stopping time having a fixed specified distribution. Further stopping problems with an expectation constraint on the stopping time have been solved by Urusov [17].
Let θ ∈ [0, 1] be the moment at which a standard Brownian motion attains its maximal value on [0, 1] and let α ≥ 0. Then Urusov [17] characterizes the stopping time that minimizes E[(τ − θ) + ] over all stopping times τ satisfying the expectation constraint The article is organized as follows. In Section 1 we describe the precise setting of stopping problems considered. In Section 2 we show how one can reduce the stopping problem to an optimization over the set of probability measures that are weighted sums of 3 Dirac measures. In Section 3 we provide sufficient conditions for the existence of an optimal stopping time. Throughout we assume that the process to stop is in natural scale -this is, as explained in Section 4, not a restriction.

Stopping after consecutive exit times
In this section we rigorously set the framework for the optimal stopping problem. The process to stop is assumed to be a one-dimensional continuous the family of shift operators on Ω defined by (θ t ω)(s) = ω(t + s), s ≥ 0. Let (P x ) x∈J be a family of probability measures on (Ω, F 0 ) that is a regular diffusion in the sense of [16,Chapter V.45]. In particular, we have P x [Y 0 = x] = 1 for all x ∈ J. Regularity means that for every y ∈ I and x ∈ J we have that P y [τ For a probability measure µ on (J, B(J)) let Let F µ be completion of F 0 with respect to µ and set F µ t = σ(F 0 t , N ), t ≥ 0, where N denotes the collection of P µ -null sets in F µ . One can show that (Ω, F µ , (F µ t ), P µ ) satisfies the usual conditions. We set F t = µ F µ t and F = µ F µ . Observe that (F t ) is right-continuous, but that in general (Ω, F, (F t ), P µ ) does not satisfy the usual conditions. The process (Y t ) t≥0 fulfills the strong Markov property (cf. Theorem 9.4, Chapter III, in [15]): For any bounded F-measurable mapping η and any finite (F t )-stopping time τ we have Let m be the speed measure of the diffusion (P x ) x∈J on J. Since Y is regular we have for all a < b ∈ I 0 < m([a, b]) < ∞.
For simplicity we assume that the diffusion Y is in natural scale. If Y is not in natural scale, then there exists a function s : J → R, the so-called scale function, such that s(Y t ), t ≥ 0, is in natural scale. In Section 4 below we show how to reduce the general stopping problem to the case where the process to stop is in natural scale.
In addition, we assume that if an endpoint is accessible, then it is absorbing. This implies Y is a local martingale (see Corollary 46.15 in [16]).
For y ∈ I and x ∈J we define q y : with the convention that m((y, u)) = −m((u, y)) whenever u < y. Moreover, , is a local martingale with respect to P y and (F t ) (this follows e.g. from Theorem 3.6 in Chapter VII, [14]). Moreover, the behavior of q y at l and r determines whether the process attains the boundary points with a positive probability or not.
Proof. Since we have not found a reference with a proof, we provide a sketch of the proof in the appendix.
Let f : J → R be a Borel-measurable function determining the payoff of the stopping problem. Throughout we make the following assumption on f : exists for all y ∈ J, T ≥ 0 and τ ∈ T (T, y). Indeed, for an appropriately chosen localizing sequence of stopping times (τ n ), it holds that We consider the problem of finding the stopping time in T (T, y) that maximizes the expected payoff for all T ≥ 0 and y ∈ J.
is replaced by the stronger assumption that for every y ∈ J there exists a C(y) ∈ R ≥0 such that then the value function is finite. Indeed, it follows by using similar arguments as in Remark 1.
The following example shows that in general one can not dispense with condition For stopping problems without an expectation constraint an optimal stopping time is given by the exit time of the continuation region (see Corollary 2.9, Chapter I in [13]). In particular, for solving unconstrained stopping problems it is enough to consider exit times from intervals. For constrained stopping problems a reduction to simple exit times is not possible. We show, however, that it is enough to consider at most three consecutive exit times.
To give a precise statement, we denote for a, b ∈ R with a ≤ b the first hitting time of a by τ a = inf{t ≥ 0 : Y t = a} and the first exit time from the interval (a, b) after time r ≥ 0 by τ a,b (r) = inf{t ≥ r : Y t / ∈ (a, b)}. Moreover, we write T 3 (T, y) for the collection of stopping times τ ∈ T (T, y) for which there exist p 1 , p 2 , p 3 ∈ [0, 1] with p 1 + p 2 + p 3 = 1 and a, c, d ∈ R with a ≤ c ≤ d such that . One of our main results is that the stopping problem (1.3) can be simplified to the set T 3 (T, y). (1.5) We prove Theorem 1.4 in the following section. We do so by reducing problem (1.3) to an optimization over a set of probability measures. Theorem 1.4 brings up the question whether the supremum is attained in T 3 (T, y). In Section 3 below we provide sufficient conditions guaranteeing the existence of an optimal stopping time in T 3 (T, y).

Optimal stopping as a measure optimization
In this section we first explain how one can reduce the stopping problem (1.3) to a linear optimization problem over a set of probability measures satisfying some integrability constraints. The linear nature of the measure optimization allows us then to conclude that the maximum values are attained by extreme points, which here are weighted sums of three Dirac measures.
We denote by M = M(J) the set of all probability measures on R with support in J and by M 1 the set of all measures in M with finite first moment µ = x µ(dx). Let A(T, y) be the set of measures µ ∈ M 1 satisfying the following properties: 2. µ integrates q y such that Results from [9] on the Skorokhod embedding problem for diffusions (and from [1] for processes described in terms of SDEs) imply that A(T, y) coincides with the set of probability measures that can be embedded into Y under P y with stopping times τ satisfying E y (τ ) ≤ T . More precisely, we have the following: There exists a stopping time τ ∈ T (T, y) with Y τ ∼ µ under P y if and only if µ ∈ A(T, y).
Proof. Let τ ∈ T (T, y) be an embedding of µ in Y under P y , i.e. let Y τ have the distribution µ under P y . Then [7] and [12] imply that Thus, µ ∈ M 1 whenever r or l is finite. Section 3.5 in [9] shows that if I = (−∞, ∞) and τ is an integrable embedding for µ, then µ ∈ M 1 . If µ = y, then it follows from Theorem 2.4. in [9] that H(y, µ).
For the reverse direction let µ ∈ A(T, y) and assume first that µ is centered around y. Then µ can be embedded in Y under P y for −∞ ≤ l < r ≤ ∞ by [7] and [12]. It follows from Theorem 3.4 in [9] that there exists a minimal stopping time τ with Y τ ∼ µ under P y and Hence, τ ∈ T (T, y). Now let µ ∈ A(T, y) withμ < y. Then we have r = ∞. Theorem 3.6 in [9] shows the existence of a minimal embedding τ of µ in Y under P y with where the last inequality follows from the second property of µ. Hence, τ ∈ T (T, y). Finally, for µ ∈ A(T, y) withμ > y, using similar arguments, one can show that there exists a stopping time τ with Y τ ∼ µ under P y and E y [τ ] ≤ T .
Remark 2.2. The function q y appearing in the definition of the set of measures A(T, y) plays for the Markov process Y the same role than the function x → x 2 plays for the Brownian motion. Indeed, we know that when Y is a Brownian motion starting in y = 0, we can find an embedding of µ with an integrable stopping time if and only if µ is centered and in L 2 . The papers [9] and [1] identify the function q y as the counterpart of the second-order moment when Y is a general diffusion. Remark 2.3. When µ is not centered around y, there is a non zero function H in the constraint 2. of A(T, y). To understand this condition, notice that H(y,μ) = E y [inf{t ≥ 0 : Y t =μ}]. This tells that to embed a measure µ such thatμ = y, we first wait until the first hitting time ofμ and then we embed µ in Y , started atμ. and for any optimal µ ∈ A(T, y) there exists an optimal stopping time τ ∈ T (T, y) in (1.3) with Y τ ∼ µ under P y .
Notice that the functional µ → f (x)µ(dx) is linear on A(T, y). We have thus obtained a linear problem over a set of probability measures µ with some integrability constraints. Recall that for standard linear problems the maximum value is attained by extreme points. We have a similar result for an optimization problem gdµ over measures µ ∈ M satisfying moment constraints of the form f i dµ ≤ c i , g and f i measurable, c i ∈ R, 1 ≤ i ≤ n. The maximum value of gdµ is also attained in the set of extreme points, see [18]. Furthermore, the extreme points are contained in the set of all weighted Dirac measures with at most n + 1 mass points satisfying the moment constraints.
In the following we denote the extreme points of a convex set A ⊆ M by where D(t, y) = {µ ∈ M 1 :μ = y and q y (x)µ(dx) = t}, 0 ≤ t ≤ T . Theorem 2.1(b) and Theorem 3.2 in [18] imply that because D 3 (t, y) coincides with E (D(t, y)). For t ∈ [0, T ] we have D 3 (t, y) ⊆ A 3 (T, y). Therefore, v(T, y) = sup In the second case the set A(T, y) also contains uncentered measures. We define ifμ ≤ y for all µ ∈ A(t, y), ifμ ≥ y for all µ ∈ A(t, y), Observe that at least one of the sets A + (T, y) or A − (T, y) is nonempty and that (2.1) can be reduced to the two optimization problems sup µ∈A + (T,y) f (x)µ(dx) and sup µ∈A − (T,y) f (x)µ(dx), where we follow the convention that the supremum over the empty set is equal to −∞. If A + (T, y) is nonempty, then f (x)µ(dx).
Proof of Theorem 1.4. Let µ ∈ A 3 (T, y) with exactly three mass points a < c < d. First observe that we can assume that µ is centered around y. Otherwise the first hitting time ofμ is integrable wrt. P y (Theorem 2.4 in [9]) and we wait until Y hitsμ and then continue as in the centered case. Extending the Balayage method developed by Chacon and Walsh in [6], we construct consecutive exit times as follows: The stopping time τ 2 is an embedding of µ into Y under P y . By using that If µ has two mass points a < c, then τ = inf{t ≥ 0 : Y t / ∈ (a, c)} ∈ T 3 (T, y). And similar, if µ = δ a , then τ = inf{t ≥ 0 : Y t / ∈ J\{a}} ∈ T 3 (T, y).
The following example shows that in general a reduction to A 2 (T, y), the set of probability measures in A(T, y) that are weighted sums of at most 2 Dirac measures, is not possible.
Example 2.6. Let (Y t ) t≥0 be a Brownian motion starting in 0 and let f (x) = 1 {|x|≥1} , x ∈ R, be the payoff function. We claim that v = sup Now we derive the value of the optimization problems over all measure µ ∈ A(T, 0) resp. µ ∈ A 2 (T, 0). For T ≥ 1 an optimal measure µ * is given by Now let T < 1 and observe that for every measure µ ∈ A 2 (T, 0) at least one atom is contained in (−1, 1). Due to the symmetry of the problem and the form of f , we can restrict ourselves to measures of the form where S ∈ (0, T ]. Then we obtain Next we turn to measures with three atoms. Observe that all measures in A 3 (T, y) are centered. Theorem 1.4 and Theorem 2.5 imply that it is sufficient to consider measures µ of the form where 0 < b < 1 and c < b < 1. The measure µ corresponds to the distribution of Y τ under P y , where µ is a centered probability measure and Then we obtain c ∈ (−1, b) and q 0 (x)µ(dx) = S. The payoff of the measure µ is given by which is maximized for b S = S 2−S ∈ (0, S) with corresponding optimal c S = 0. Hence, we obtain

Existence of an optimizer
The next example shows that the supremum in (2.2) is not always attained.
Example 3.1. Let f 1 (x) = x 2 |x| 1+|x| , x ∈ R, and Y be a Brownian motion starting in 0 under P y . In this case there does not exist an optimal stopping time. To prove this let v 1 : Moreover, consider the second payoff function f 2 (x) = x 2 . Note that for any integrable stopping On the other hand, for the stopping times τ n = τ −1/n,nT we have E y [τ n ] = T and E 0 [f 1 (Y τn )] = nT 1/n + nT 1 n 2 1/n 1 + 1/n + 1/n 1/n + nT n 2 T 2 nT 1 + nT −→ T, as n → ∞, and hence v 1 ≥ v 2 .
From v 1 = v 2 we can deduce that the supremum can not be attained in v 1 , because for any stopping time In the case where I is bounded we have the following existence result. Proof. Let (µ n ) be a sequence in A(T, y) such that lim n→∞ f dµ n = v. SinceĪ is compact, (µ n ) is tight. By Prokhorov's Theorem (µ n ) converges weakly along a subsequence to a probability measure µ on J. We show that µ ∈ A(T, y). Observe that y = µ n := xµ n (dx) → xµ(dx) =μ as n → ∞. Thus,μ = y. Next, lower-semicontinuity of q on J implies qdµ ≤ lim inf n→∞ qdµ n ≤ T . Since f is upper-semicontinuous we have v = lim n→∞ f dµ n ≤ f dµ ≤ v. Hence µ is optimal in (2.1) and by Corollary 2.4 there exists an optimal stopping in (1.3).
If the state space equals R, there exists an optimizer under conditions on the payoff function f and the speed measure m.
qy(x) ≤ 0 and that f is upper semi-continuous. Then there exists an optimal stopping time in T 3 (T, y) for (1.5).
If the sequence (x 1 n ) n∈N is unbounded, choose a subsequence, also denoted by (x 1 n ) n∈N , such that either lim n→∞ x 1 n = −∞ =: x 1 or lim n→∞ x 1 n = ∞ =: n ) n∈N is bounded, extract a subsequence such that lim n→∞ x 1 n = x 1 ∈ R. By extracting further subsequences, proceed in the same way with (x 2 n ) n∈N and (x 3 n ) n∈N . Then, refine once again the sequence to obtain that (p 1 n , p 2 n , p 3 n ) → (p 1 , p 2 , p 3 ) ∈ [0, 1] 3 as n → ∞. Overall we obtain for n → ∞ that From the fact that Observe that it follows from (3.2) that Consequently, we have that µ ∈ A 3 (T, y) and in particular v(T, y) ≥ f dµ.  f (x)µ(dx).

A. Proof of auxiliary results
We proof in this appendix Lemma 1.1 that characterizes in terms of q y whether a boundary point is attained with a positive probability or not.
Proof of Lemma 1.1. We prove the statement only for r. If r = ∞, then q y (r) = ∞. Next suppose that r ∈ J (in particular r < ∞). Fix a ∈ (l, y) and let τ a,r be the first exit time of the interval (a, r). Since q y (Y t )−t is a local martingale, we have E y [τ a,r ] = E y q y Y τa,r . This further entails E y [τ a,r ] = r − y r − a q y (a) + y − a r − a q y (r) ≥ y − a r − a q y (r).
Proposition 3.1 in Chapter VII, [14], implies that τ a,r is integrable and, hence, q y (r) < ∞. Now consider the case when r / ∈ J and r < ∞. Then the first hitting time τ r of the point r satisfies τ r = ∞, P y -almost surely. Assume that q y (r) < ∞ for some y ∈ I. Let l < a < y < b < r and denote by τ a,b the first exit time of the interval (a, b). Similar to the first case one can show E y [τ a,b ] = E y q y Y τ a,b = b − y b − a q y (a) + y − a b − a q y (b).
Hence, = r − y r − a q y (a) + r − a b − a q y (r).
In particular, τ a < ∞, P y -almost surely, and τ a embeds the Dirac measure δ a into Y y . But if r < ∞, a measure µ can be embedded into Y under P y if and only ifμ = x µ(dx) ≥ y by [12] and [8]. Therefore, q y (r) = ∞.