Two explicit Skorokhod embeddings for simple symmetric random walk

Motivated by problems in behavioural finance, we provide two explicit constructions of a randomized stopping time which embeds a given centered distribution $\mu$ on integers into a simple symmetric random walk in a uniformly integrable manner. Our first construction has a simple Markovian structure: at each step, we stop if an independent coin with a state-dependent bias returns tails. Our second construction is a discrete analogue of the celebrated Az\'ema-Yor solution and requires independent coin tosses only when excursions away from maximum breach predefined levels. Further, this construction maximizes the distribution of the stopped running maximum among all uniformly integrable embeddings of $\mu$.


Introduction
We contribute to the literature on Skorokhod embedding problem (SEP). The SEP, in general, refers to the problem of finding a stopping time τ , such that a given stochastic process X, when stopped, has the prescribed distribution µ: X τ ∼ µ. When such a τ exists we say that τ embeds µ into X. This problem was first formulated and solved by Skorokhod (1965) when X is a standard Brownian motion. It has remained an active field of study ever since, see Ob lój (2004) for a survey, and has recently seen a revived interest thanks to an intimate connection with the Martingale Optimal Transport, see Beiglböck et al. (2016) and the references therein.
In this paper, we consider the SEP for the simple symmetric random walk. Our interest arose from a casino gambling model of Barberis (2012) in which the gambler's cumulative gain and loss process is modeled by a random walk. The gambler has to decide when to stop gambling and her preferences are given by cumulative prospect theory (Tversky and Kahneman, 1992). Such preferences lead to dynamic inconsistency, so this optimal stopping problem cannot be solved by the classical Snell envelop and dynamic programming approaches. By applying the Skorokhod embedding result we obtain here, He et al. (2016) convert the optimal stopping problem into an infinite-dimensional optimization problem, find the (pre-committed) optimal stopping time, and study the gambler's behavior in the casino.
To discuss our results, let us first introduce some notation. We let X = S = (S t : t ≥ 0) be a simple symmetric random walk defined on a filtered probability space (Ω, F, F, P), where F = (F t ) t≥0 . We work in discrete time so here, and elsewhere, {t ≥ 0} denotes t ∈ {0, 1, 2, . . .}. We let T (F) be the set of F-stopping times and say that τ ∈ F(F) is uniformly integrable (UI) if the stopped process (S t∧τ : t ≥ 0) is UI. Here, and more generally when considering martingale, one typically restricts the attention to UI stopping times to avoid trivialities and to obtain solutions which are of interest and use. We let Z denote the set of integers and M 0 (Z) the set of probability measures on Z which admit finite first moment and are centered. Our prime interest is in stopping times which solve the SEP: SEP(F, µ) := {τ ∈ T (F) : S τ ∼ µ and τ is UI} . (1) Clearly if SEP(F, µ) = ∅ then µ ∈ M 0 (Z). For embeddings in a Brownian motion, the analogue of (1) has a solution if and only if µ ∈ M 0 (R). However, in the present setup, the reverse implication depends on the filtration F. If we consider the natural filtration F S = (F S t : t ≥ 0) where F S t = σ(S u : u ≤ t), then Cox and Ob lój (2008) showed that the set of probability measures µ on Z for which SEP(F S , µ) = ∅ is a fractal subset of M 0 (Z). In contrast, Rost (1971) and Dinges (1974) showed how to solve the SEP using randomized stopping times, so that if F is rich enough then SEP(F, µ) = ∅ for all µ ∈ M 0 (Z). We note also that the introduction of external randomness is natural from the point of view of applications. In the casino model of Barberis (2012) mentioned above, He et al. (2017) and Henderson et al. (2017) showed that the gambler is strictly better off when she uses extra randomness, such as a coin toss, in her stopping strategy instead of relying on τ ∈ T (F S ). Similarly, randomized stopping times are useful in solving optimal stopping problems, see e.g. Belomestny and Krätschmer (2016) and He et al. (2016).
Our contribution is to give two new constructions of τ ∈ SEP(F, µ) with certain desirable optimality properties. Our first construction, in Section 2 below, has minimal (Markovian) dependence property: a decision to stop only depends on the current state of S and an independent coin toss.
The coins are suitably biased, with state dependent probabilities, which can be readily computed using an explicit algorithm we provide. Such a strategy is easy to compute and easy to implement, which is important for applications, e.g. to justify its use by economic agents, see He et al. (2017).
We also link our construction to the embedding in Cox et al. (2011) and show that the former corresponds to a suitable projection of the latter. Our second construction, presented in Section 3, is a discrete-time analogue of the Azéma and Yor (1979) embedding. It is also explicit and its first appeal lies in the fact that it only stops when the loss, relative to the last maximum, gets too large. It also has an inherent probabilistic interest: it maximizes P(max t≤τ S t ≥ x), simultaneously for all x ∈ R, among all τ ∈ T (F), attaining the classical Blackwell et al. (1963) bound for x ∈ N.
We conclude the paper with explicit examples worked out in Section 4.

Randomized Markovian Solution to the SEP
To formalize our first construction, consider r = (r x : x ∈ Z) ∈ [0, 1] Z and a family of Bernoulli random variables ξ = {ξ x t } t≥0,x∈Z with P(ξ x t = 0) = r x = 1 − P(ξ x t = 1), which are independent of each other and of S. Each ξ x t stands for the outcome of a coin toss at time t when S t = x with 1 standing for heads and 0 standing for tails. To such ξ we associate which is a stopping time relative to F S,ξ = (F S,ξ . The decision to stop at time t only depends on the state S t and an independent coin toss. Accordingly, we refer to τ (r) as a randomized Markovian stopping time. It is clear however that the distribution of S τ (r) is a function of r and does not depend on the particular choice of the random variable ξ.
The following shows that such stopping times allow us to solve (1) for all µ ∈ M 0 (Z).

Proof of Theorem 1
We establish the theorem by embedding our setup into a Brownian setting and then using Theorem 5.1 in Cox et al. (2011). We reserve t for the discrete time parameter and use u ∈ [0, ∞) for the continuous time parameter. We assume our probability space (Ω, F, P) supports a standard Brownian motion B = (B u ) u≥0 . We let G = (G u ) u≥0 denote its natural filtration taken rightcontinuous and complete and L y u denote its local time at time u ≥ 0 and level y ∈ R. Recall that µ ∈ M 0 (Z) is fixed. Cox et al. (2011) show that there exists a measure m on R such that, for a Poisson random measure ∆ m with intensity du × m(dx), independent of B, is minimal and embeds µ, i.e., B T m ∼ µ and (B T m ∧u : u ≥ 0) is uniformly integrable, the latter being equivalent to minimality of T m , see Ob lój (2004, Sec. 8). Moreover, by the construction in Cox et al. (2011), m(I c ) = +∞ for any interval I that contains the support of µ. For µ ∈ M 0 (Z) it follows that m is a measure on Z: m(dy) = x∈Z m x δ x (dy) ; otherwise, ∆ m (R u ) can possibly hit 1 when the local time L y u accumulates at certain non-integer level y, in which case B T m takes value y.
In addition, if µ has support bounded from above, i.e., µ([x, ∞)) = µ({x}) > 0 for a certainx ∈ Z, then mx = ∞ and T m ≤ inf{u ≥ 0 : B u ≥x}, with analogous expressions when the support is bounded from below. We note that N x u = ∆ m ({(s, x) : s ≤ u}) is a Poisson process with parameter m x , x ∈ Z and its first arrival time ρ x = inf{u ≥ 0 : N x u ≥ 1} is exponentially distributed with parameter m x . We can now rewrite the embedding time as Consider now consecutive hitting times of integers and note that X t := B σt is a simple symmetric random walk. Recall that the measure dL x u is supported on {u : B u = x}. This implies a particularly simple structure of the stopping time T m .
First, note that T m = σ t unless m Xt = ∞. Then, let us describe T m conditionally on T m > σ t .
In particular, ρ Xt > L Xt σt and ρ Xt − L Xt σt is again exponential with parameter m Xt . T m happens in (σ t , σ t+1 ) if and only if the local time accumulated at level X t is greater than this exponential variable. Clearly this event depends on the past only through the value of X t . More formally, considering the new Brownian motion B t u = B u+σt − B σt , u ≥ 0, and denoting its local time in on X t = x, is independent of F σt and has probability which only depends on x. Further, in this case B T m = X t . If we let τ B,X = t on {σ t ≤ T m < σ t+1 } then it follows that τ B,X is a stopping time relative to the natural filtration of X enlarged with a suitable family of independent random variables, it has the precise structure of τ (r) in (2), embeds µ and is UI. More precisely, using the space homogeneity of Brownian motion, we can define the probabilities using just the local time in zero, and it follows that if we take then τ (r) ∈ SEP(F S,ξ , µ) as required.
Remark 1 We note that by following the methodology in Cox et al. (2011) one could write a direct proof of Theorem 1, albeit longer and more involved than the one above. In particular, it is insightful to point out that if µ has a finite support -µ([x,x]) = 1 with µ({x}) > 0, µ({x}) > 0 for some x < 0 <x -then r as constructed above can be shown to be the maximal element in the set

Algorithmic computation of the stopping probabilities r µ
In this section, we work under the assumptions of Theorem 1 and provide an algorithmic method for computing {r i } i∈Z obtained therein. We let τ = τ (r µ ) and g i denote the expected number of visits of S to state i strictly before τ , i.e.: Denote a + := max{a, 0}. It is a matter of straightforward verification to check that for any i ≤ 0 ≤ j the processes are martingales. To compute g i , we then apply the optional sampling theorem at τ ∧ t and let t → ∞. Using the fact that {S τ ∧t } is a UI family of random variables together with monotone convergence theorem, we deduce that Writing p i := µ({i}) = P(S τ = i), we now compute , which is the case if and only if i is on the boundaries of the support, we have r i = 1, i.e., we have to stop instantly. If p i + g i = 0, then p i = g i = 0 and this can only happen for states outside the boundaries of the support. In this case, we set r i = 1, which is consistent with the characterisation in Remark 1.
Thus r µ = (r i ) in Theorem 1 is given by where p i = µ({i}) and g i can be calculated from (4) and (5). This can be seen as the equation on the bottom of page S22 in Cox et al. (2011) specialised to our setup. While that equation is only argued heuristically therein, in our setup we can give it a rigorous meaning and proof.

Randomized Azéma-Yor solution to the SEP
Let us recall the celebrated Azéma-Yor solution to the SEP for a standard Brownian motion (B u : u ≥ 0). As above, we reserve t for the discrete time parameter and use u ∈ [0, ∞) for the continuous time parameter. To a centered probability measure µ on R we associate its barycenter function whereμ(x) := µ [x, +∞) and ψ µ (x) := x for x such that µ([x, +∞)) = 0. We let b µ denote the right-continuous inverse of ψ µ ; i.e., b µ (y) := sup{x : ψ µ (x) ≤ y}, y ≥ 0. Then for any other such solution T to the SEP, and any maximizes the distribution of the maximum in the stochastic order. Hereμ HL is the Hardy-Littlewood transform of µ and the boundμ HL (x) is due to Blackwell et al. (1963), and has been extensively studied since; see e.g. Carraro et al. (2012) for details.
A direct transcription of the Azéma-Yor embedding to the context of a simple symmetric random walk only works for measures µ ∈ M 0 (Z) for which ψ µ (x) ∈ N for all x ∈ R, which is a restrictive condition; see Cox and Ob lój (2008) for details. For a general µ ∈ M 0 (Z) we should seek instead to emulate the structure of the stopping time: the process stops when its drawdown hits a certain level, i.e., when Before we state the theorem we need to introduce some notation. Let µ ∈ M 0 (Z) and denotē x := sup{n ∈ Z : µ({n}) > 0} and x := inf{n ∈ Z : µ({n}) > 0} the bounds of the support of µ.
The barycentre function ψ µ is piece-wise constant on (−∞,x] with jumps in atoms of µ, is non- The inverse function b µ is right-continuous, non-decreasing with b µ (0) = x, and is integer valued consequently, µ({y}) = 0 for any y that is not in the range of b µ .
For each 1 ≤ n <x, {b µ (y) : y ∈ [n, n + 1]} ∩ (−∞, n] is a nonempty set of finitely many integers which we rank and denote as x n 1 > x n 2 > · · · > x n mn+1 . Similarly, we rank {b µ (y) : y ∈ [0, 1]}, which is a nonempty set of finitely many integers x 0 1 > x 0 2 > · · · > x 0 m 0 +1 if b µ (0) > −∞ and a set of countably many integers x 0 1 > x 0 2 > . . . otherwise. Then, for each 1 ≤ n <x, x n−1 Note that we may have m n = 0, in which case x n−1 1 = x n mn+1 = x n 1 . For each 1 ≤ n <x and for n = 0 when x > −∞, define When x = −∞, define Then, as we will see in the proof of Theorem 2, for each 1 ≤ n <x and for n = 0 when x > −∞, ρ n k := 1 − (f n k /f n k−1 ) is in [0, 1) for k = 1, . . . , m n and we let ρ n mn+1 := 1; when x = −∞, ρ 0 k := 1 − (f 0 k /f 0 k−1 ) is in (0, 1) for each k ≥ 1 and we set m 0 + 1 := +∞. Let η = (η n k : 0 ≤ n <x, k ∈ Z ∩ [1, m n + 1]) be a family of mutually independent Bernoulli random variables, independent of S, with P(η n k = 0) = ρ n k = 1 − P(η n k = 1). We let S * t := sup r≤t S r and define the enlarged filtration We are now ready to define our Azéma-Yor embedding for S. It is an F S,η -stopping time which, in analogy to (8), stops when an excursion away from the maximum breaches a given level. However, since the maximum only takes integer values, we emulate the behaviour of B u∧T AY µ between hitting times of two consecutive integers in an averaged manner, using independent randomization.
Specifically, if we let H n := inf{t ≥ 0 : S t = n} then after H n but before H n+1 we may stop at each of x n 1 > x n 2 > . . . > x n mn depending on the independent coin tosses (η n k ), while we stop a.s. if we hit x n mn+1 . If we first hit n + 1 then a new set of stopping levels is set. Finally, we stop upon hittingx.
The optimality property in (11) is analogous to the optimality of T AY µ in a Brownian setup, as described above and the bound in (11) coincides withμ HL (n).
Remark 2 Note that by considering our solution for (−S t ) t≥0 we obtain a reversed Azéma-Yor solution which stops when the maximum drawup since the time of the historical minimum hits certain levels. It follows from (11) that such embedding maximizes the distribution of the running minimum in the stochastic order.

Remark 3
We do not claim that τ AY µ is the only embedding which achieves the upper bound in (11). Our construction inherits the main structural property of the Azéma-Yor embedding for B: when a new maximum is hit, a lower threshold is set (which may depend on an independent randomisation) and we either stop when this threshold is hit or else a new maximum is set. This, in effect, averages out the behaviour of B u∧T AY µ between hitting times of two consecutive integers.
Instead, we could consider averaging out only the behaviour of B u∧T AY µ between the first hitting time of an integer n and the minimum between the hitting time of n + 1 and the return of the embedded walk to n. The resulting embedding would have the following structure: each time the random walk is at its maximum, S t = S * t = n a (randomized) threshold is set and the walk stops if it hits this threshold. If not, a new instance of the threshold level is drawn when the walk returns to n. This is iterated until the walk is stopped at the current threshold level or when n + 1 is hit which changes the distribution of the threshold level. This embedding appears less natural for us, having the casino gambling motivation in mind, however it should share the optimality property of τ AY µ in (11).

Proof of Theorem 2
It is straightforward to verify that all the conclusions of the theorem hold for the case in which µ({0}) = 1, so we assume in the following that µ({0}) < 1 and thus x < 0 <x. Throughout the proof we let τ = τ AY µ and recall that H j = inf{t ≥ 0 : S t = j}. We first prove constructively that ρ n i 's are well defined and the constructed stopping time τ embeds µ into the random walk. Specifically, we argue by induction that for any 0 ≤ j <x the stopping time τ satisfies For clarity of the presentation, somewhat lengthy and technical, proof of the above equalities is relegated to Section 3.2 below.
To conclude that τ ∈ SEP(F S,η , µ) it remains to argue that τ is UI, which is equivalent to lim K→+∞ KP(sup t≥0 |S τ ∧t | ≥ K) = 0, see e.g. Azéma et al. (1980). We first show that Because {S τ ∧t } never visits states outside any interval that contains the support of µ, we only need to prove this whenx = +∞, and hence lim y→+∞ b µ (y) = +∞, and taking K ∈ N.
By (15) and the construction of τ , we see that for n ∈ N, n <x, we have where the last equality is the case becausē consequently, For a < c < d the function y → y(d − y)/(y − a) attains its maximum on [c, d] in y = c. Taking , we can bound the first term bȳ which goes to zero with n → ∞ since µ ∈ M 0 (Z). Similarly, ψ µ (b µ (n)+) > n gives and we conclude that nP(S * τ ≥ n) → 0 as n → ∞. It remains to argue that This is trivial if x > −∞. Otherwise b µ (0) = x = −∞ and x 0 i 's are infinitely many. For n ∈ N, by the construction of τ , inf t≥0 S t∧τ ≤ −n implies that S visits −n before hitting 1 and S is not stopped at any x 0 i > −n. Denote the i n := sup{i ≥ 1|x 0 i > −n} and note that i n → ∞ as n → ∞. By construction, the probability that S does not stop at x 0 i given that S reaches x 0 On the other hand, the probability that S visits −n before hitting 1 is 1/(n + 1). Therefore, the probability that S visits −n before hitting 1 and S is not stopped at any x 0 i > −n is From (10), lim k→+∞ f 0 k = 0 since µ has a finite first moment. Therefore, The above concludes the proof of τ ∈ SEP(F S,η , µ).
Consider first the case when m n = 0, so that x n 1 = x n mn+1 = x n−1 1 and τ stops if x n 1 is hit between H n and H n+1 . Consequently, and we conclude that (12) holds for j = n.
Next, we consider the case in which m n ≥ 1. A direct calculation yields where the inequality follows from x n mn+1 = b µ (n) < n <x and ψ µ (b µ (y)+) > y for any y <x.

Examples
We end this paper with an explicit computation of our two embeddings for two examples.

Optimal Gambling Strategy
The first example is a measure µ arising naturally from the casino gambling model studied in He et al. (2016). Therein, a gambler whose preferences are represented by cumulative prospect theory (Tversky and Kahneman, 1992) is repeatedly offered a fair bet in a casino and decides when to stop gambling and exit the casino. The optimal distribution of the gambler's gain and loss at the exit time is a certain µ ∈ M 0 (Z), which may be characterised explicitly, see He et al. (2016, Theorem 2). With a set of reasonable model parameters 1 , we obtain Next we follow Theorem 2 to construct a randomized Azéma-Yor stopping time τ AY µ embedding µ. To this end, we compute x n k 's and ρ n k 's, which stand for the drawdown levels that are set after reaching maximum n and the probabilities that the coins tossed at these levels turn up tails, respectively: m 2 = 0, x 2 1 = 1, m 3 = 0, x 3 1 = 1, m 4 = 0, x 4 1 = 1, m 5 = 1, x 5 1 = 2, ρ 5 1 = 0.0049, x 5 2 = 1, m 6 = 0, x 6 1 = 1, . . .
The stopping time is then illustrated in the right pane of Figure 1: S is represented by a nonrecombining binomial tree. Again, black nodes stand for "stop", white nodes stand for "continue", and grey nodes stand for the cases in which a random coin is tossed and one stops if and only if the coin turns up tails. The probability that the random coin turns up tails is shown on the top of each grey node.
By definition, τ (r) is Markovian: at each time time t, the decision to stop depends only on the current state and an independent coin toss. However, to implement the strategy, one needs to toss a coin most of the times. In contrast τ AY µ requires less independent coin tossing: e.g. in the first five periods at most one such a coin toss, but it is path-dependent. For instance, consider t = 3 and S t = 1. If one reaches this node along the path from (0,0), through (1,1) and (2,2), and to (3,1), then she stops. If one reaches this node along the path from (0,0), through (1,1) and (2,0), and to (3,1), then she continues. Therefore, compared to the randomized Markovian strategy, the randomized Azéma-Yor strategy may involve less independent coin tosses but is typically path-dependent 2 as it considers relative loss when deciding if to stop or not. (1,1) (   (16) into the simple symmetric random walk S. Black nodes stand for "stop", white nodes stand for "continue", and grey nodes stand for the cases in which a random coin is tossed and one stops if and only if the coin turns up tails. Each node is marked on the right by a pair (t, x) representing time t and S t = x. Each grey node is marked on the top by a number showing the probability that the random coin tossed at that node turns up tails.
The randomized Markovian stopping time that embeds µ given by (17) can be derived analytically. Indeed, according to the algorithm given in Section 2.2, the probability of a coin tossed at S t = i turning up tails is The randomized Azéma-Yor stopping time that embeds µ given by (17) can also be derived analytically. Because the formulae for x n k 's and ρ n k 's are tedious, we chose not to present them here. Instead, we illustrate the two embeddings in Figure 2 by setting q + = γ + = 5/12 and q − = γ − = 13/24. As in the previous example, the randomized Azéma-Yor stopping time involves less randomization than the randomized Markovian stopping time at the cost of being path-dependent.