Optimal Strategies in a Production Inventory Control Model

Abstract

We consider a production-inventory control model with finite capacity and two different production rates, assuming that the cumulative process of customer demand is given by a compound Poisson process. It is possible at any time to switch over from the different production rates but it is mandatory to switch-off when the inventory process reaches the storage maximum capacity. We consider holding, production, shortage penalty and switching costs. This model was introduced by Doshi, Van Der Duyn Schouten and Talman in 1978. In their paper they found a formula for the long-run average expected cost per unit time as a function of two critical levels, in this paper we consider expected discounted cumulative costs instead. We seek to minimize this discounted cost over all admissible switching strategies. We show that the optimal cost functions for the different production rates satisfy the corresponding Hamilton-Jacobi-Bellman system of equations in a viscosity sense and prove a verification theorem. The way in which the optimal cost functions solve the different variational inequalities gives the switching regions of the optimal strategy, hence it is stationary in the sense that depends only on the current production rate and inventory level. We define the notion of finite band strategies and derive, using scale functions, the formulas for the different costs of the band strategies with one or two bands. We also show that there are examples where the switching strategy with two critical levels is not optimal.

Appendix

1.1 Proof of Proposition 3.2

We call $S_{i}$ the positive upper bounds of the functions $V_{i}$ for $i=1,2.$ Given initial inventory level $x\in [l,b)$ and initial phase $i=1,2,$ take $\delta \in (0,b-x]$ and consider an admissible strategy $\pi _{x+\delta }\in \Pi _{x+\delta ,i}$ such that $V_{i}^{\pi _{x+\delta }}(x+\delta )\le V_{i}(x+\delta )+\varepsilon ,$ where $0<\varepsilon <\delta$. Let us now define the admissible strategy $\pi _{x}\in \Pi _{x,i}$ as follows: stay in phase i until the controlled inventory level $X_{t}^{\pi _{x}}$ reaches $x+\delta$ and then follow $\pi _{x+\delta }\in \Pi _{x+\delta ,i}$. Then, from (3.1) and Proposition 3.1, we get

$$\begin{aligned} \begin{array}{lll} V_{i}(x) &{} \le &{} V_{i}^{\pi _{x}}(x)\\ &{} \le &{} \int _{0}^{\frac{\delta }{\sigma _{i}}}e^{-qt}h_{i}(x+\sigma _{i}t)dt+\mathbb {P}[\tau _{1}^{i}>\frac{\delta }{\sigma _{i}}]e^{-q\frac{\delta }{\sigma _{i}}}V_{i}^{\pi _{x+\delta }}(x+\delta )\\ &{} &{} +\mathbb {P}\left[ \tau _{1}^{i}\le \frac{\delta }{\sigma _{i}}\right] S_{i}\\ &{} \le &{} \overline{h}\frac{\delta }{\sigma _{i}}+e^{-\left( \lambda _{i}+q\right) \frac{\delta }{\sigma _{i}}}\left( V_{i}(x+\delta )+\varepsilon \right) +(1-e^{-\lambda _{i}\frac{\delta }{\sigma _{i}}})S_{i}. \end{array} \end{aligned}$$

Hence, we have

$$\begin{aligned} \begin{array}{lll} V_{i}(x)-V_{i}(x+\delta ) &{} \le &{} \overline{h}\frac{\delta }{\sigma _{i}}+e^{-\left( \lambda _{i}+q\right) \frac{\delta }{\sigma _{i}}}\left( V_{i}(x+\delta )+\varepsilon \right) -V_{i}(x+\delta )+(1-e^{-\lambda _{i}\frac{\delta }{\sigma _{i}}})S_{i}\\ &{} \le &{} \overline{h}\frac{\delta }{\sigma _{i}}+\varepsilon +\lambda _{i}\frac{\delta }{\sigma _{i}}S_{i}. \end{array} \end{aligned}$$

So, taking

$$\begin{aligned} m_{i}^{1}:=\frac{\overline{h}}{\sigma _{i}}+1+\frac{\lambda _{i}}{\sigma _{i} }S_{i}, \end{aligned}$$

we obtain

$$\begin{aligned} V_{i}(x)-V_{i}(x+\delta )\le m_{i}^{1}\delta . \end{aligned}$$

(8.1)

Let us prove now that there exists $m_{i}^{2}>0$ such that,

$$\begin{aligned} V_{i}(x+\delta )-V_{i}(x)\le m_{i}^{2}\delta . \end{aligned}$$

(8.2)

We start showing that there exists m such that,

$$\begin{aligned} V_{i}(y)-V_{i}(l)\le m\delta \end{aligned}$$

(8.3)

for all $y\in [l,l+\delta ]$. Given $\varepsilon >0$ and an initial inventory level l, consider the strategy $\pi _{l}\in \Pi _{l,i}$ for $i=1,2$ such that $V_{i}^{\pi _{l}}(l)\le V_{i}(l)+\varepsilon$ and call $X_{t}^{\pi _{l}}$ the associated process with initial inventory level l. Take also a strategy $\pi _{b}\in \Pi _{b,0}$ such that $V_{0}^{\pi _{b}}(b)\le V_{0}(b)+\varepsilon$.

Let us define the admissible strategy $\pi _{y}\in \Pi _{y,i}$ for initial inventory level $y\in [l,l+\delta ]$ as:

• For $0\le t\le T,$ follow $\pi _{l}$ (and so the associated controlled processes $X_{t}^{\pi _{y}}=X_{t}^{\pi _{l}}+(y-l)$ for $t<T$), where

  $$\begin{aligned} T:=\min \{t:X_{t}^{\pi _{y}}=b\text { or }\overset{\vee }{X}_{t}^{\pi _{y}}-(y-l)=\overset{\vee }{X}_{t}^{\pi _{l}}<l\}. \end{aligned}$$

• If $X_{T}^{\pi _{y}}=b$, follow $\pi _{b}$ for $t\ge T$.

• If $\overset{\vee }{X}_{T}^{\pi _{y}}<l$ (and so $X_{T}^{\pi _{y}}=X_{T}^{\pi _{l}}=l$), follow $\pi _{l}$ for $t\ge T$.

• If $l\le X_{T}^{\pi _{y}}<y$ (and so $X_{T}^{\pi _{l}}=l$ and $X_{T}^{\pi _{y}}=\overset{\vee }{X}_{T}^{\pi _{y}}$), also follow the strategy $\pi _{l}$ for $t\ge T$.

Given any stopping time $\tau$, let us define $\widehat{V}_{i}^{\pi _{y}}(y,\tau )$ as the expected discounted cost of the strategy before $\tau$ and $\widetilde{V}_{i}^{\pi _{y}}(y,\tau )$ as the expected discounted cost of the strategy after $\tau$. Thus,

$$\begin{aligned} \begin{array}{l} V_{i}(y)-V_{i}(l)-\varepsilon \\ \begin{array}{ll} \le &{} V_{i}^{\pi _{y}}(y)-V_{i}^{\pi _{l}}(l)\\ \le &{} \mathbb {E}\left[ \int _{0}^{T}e^{-qt}\left( h_{\mathcal {J}_{t}}(X_{t}^{\pi _{l}}+(y-l))-h_{\mathcal {J}_{t}}(X_{t}^{\pi _{l}})\right) dt\right] +\\ &{} +\mathbb {E}\left[ 1_{\left\{ X_{T}^{\pi _{y}}=b\right\} }\left( e^{-qT}\left( K_{\mathcal {J}_{T^{-}},0}+V_{0}(b)+\varepsilon \right) -\widetilde{V}_{i}^{\pi _{l}}(l,T)\right) \right] \\ &{} +\mathbb {E}\left[ 1_{\left\{ \overset{\vee }{X}_{T}^{\pi _{y}}<l\right\} }e^{-qT}\left( V_{\mathcal {J}_{T}}(l)+\varepsilon +p_{\mathcal {J}_{T^{-}}}(l-\overset{\vee }{X}_{T}^{\pi _{y}})-(V_{\mathcal {J}_{T}}(l)+p_{\mathcal {J}_{T^{-}}}(l-\overset{\vee }{X}_{T}^{\pi _{y}}+y-l))\right) \right] \\ &{} +\mathbb {E}\left[ 1_{\left\{ l\le X_{T}^{\pi _{y}}<y\right\} }e^{-qT}\left( V_{\mathcal {J}_{T}}^{\pi _{X_{T}^{\pi _{y}}}}(X_{T}^{\pi _{y}})-V_{\mathcal {J}_{T}}^{\pi _{l}}(l)+2\varepsilon \right) \right] . \end{array} \end{array} \end{aligned}$$

Let $n_{D}$ be the sum of the numbers of discontinuities of $h_{1}$ and $h_{2}$. Note that between two customer demands, the inventory level $X_{t}^{\pi _{y}}$ goes through at most $n_{D}$ points of discontinuities of $h_{\mathcal {J}_{t}}$. Hence, calling $\tau _{0}=0$ and $\overline{\lambda }=\max _{i=0,1,2}\lambda _{i}$, we have

$$\begin{aligned} \mathbb {E}\left[ \int _{0}^{T}e^{-qt}\left( h_{\mathcal {J}t}(X_{t}^{\pi _{l}}+(y-l))-h_{\mathcal {J}t}(X_{t}^{\pi _{l}})\right) dt\right] \le \left( \frac{m_{h}}{q}+n_{D}\frac{\overline{h}}{\sigma _{2}}\left( 1+\frac{\overline{\lambda }}{q}\right) \right) \delta . \end{aligned}$$

(8.4)

Let us call $\widetilde{T}:=\inf \left\{ t:X_{t}^{\pi _{l}}=b\right\}$ and $\widetilde{\tau }$ the time of the first customer demand after T;  we have that $\mathbb {P}\left[ \widetilde{T}>\widetilde{\tau }\right] \le 1-e^{-\frac{\delta }{\sigma _{2}}\overline{\lambda }}$ and so

$$\begin{aligned} \begin{array}{l} \mathbb {E}\left[ 1_{\left\{ X_{T}^{\pi _{y}}=b\right\} }\left( e^{-qT}\left( K_{\mathcal {J}_{T^{-}},0}+V_{0}(b)\right) -\widetilde{V}_{i}^{\pi _{l}}(l,T)\right) \right] \\ \begin{array}{ll} \le &{} \left( 1-e^{-\overline{\lambda }\frac{\delta }{\sigma _{2}}}\right) \left( V_{0}(b)+\left( K_{1,0}\vee K_{2,0}\right) \right) \\ &{} +\mathbb {E}\left[ 1_{\left\{ X_{T}^{\pi _{y}}=b,\widetilde{T}<\widetilde{\tau }\right\} }e^{-qT}\left( K_{\mathcal {J}_{T^{-}},0} +V_{0}(b)\right) -\widetilde{V}_{i}^{\pi _{l}}(l,T)\right] \\ \le &{} \overline{\lambda }\frac{\delta }{\sigma _{2}}\left( V_{0}(b)+\max \left\{ K_{1,0},K_{2,0}\right\} \right) \\ &{} +\mathbb {E}\left[ 1_{\left\{ X_{T}^{\pi _{y}}=b,\widetilde{T}<\widetilde{\tau }\right\} }e^{-qT}\left( K_{\mathcal {J}_{T^{-}},0}+V_{0}(b)\right) -\widetilde{V}_{i}^{\pi _{l}}(l,T)\right] . \end{array} \end{array} \end{aligned}$$

(8.5)

Let $\Delta$ be the length of time after T in which the process $X_{t}^{\pi _{l}}$ reaches b in the event of no arrivals of demands. In this case, we have

$$\begin{aligned} X_{T+\Delta }^{\pi _{l}}=b-(y-l)+\int _{T}^{T+\Delta }e^{-qs}\sigma _{\mathcal {J} _{T}}ds=b \end{aligned}$$

and so $\frac{\delta }{\sigma _{1}}\le \Delta \le \frac{\delta }{\sigma _{2}}$. Hence, from (2.3),

$$\begin{aligned} \begin{array}{l} \mathbb {E}\left[ 1_{\left\{ X_{T}^{\pi _{y}}=b\right\} }1_{\left\{ \widetilde{T}<\widetilde{\tau }\right\} }\widetilde{V}_{i}^{\pi _{l} }(l,T)\right] \\ \begin{array}{ll} \ge &{} \mathbb {P}\left[ \text {no demands in }t\in \left[ T,T+\frac{\delta }{\sigma _{2}}\right] \right] \text { }\mathbb {E}\left[ e^{-q\left( T+\Delta \right) }\left( K_{\mathcal {J}_{T^{-}},0}+V_{0}(b)\right) \right] \\ \ge &{} e^{-\left( q+\overline{\lambda }\right) \frac{\delta }{\sigma _{2}} }\mathbb {E}\left[ e^{-qT}\left( K_{\mathcal {J}_{T^{-}},0}+V_{0}(b)\right) \right] . \end{array} \end{array} \end{aligned}$$

Therefore,

$$\begin{aligned} \begin{array}{l}\mathbb {E}\left[ 1_{\left\{ X_{T}^{\pi _{y}}=b\right\} }1_{\left\{ \widetilde{T}<\tau _{1}\right\} }e^{-qT}\left( K_{\mathcal {J}_{T^{-}},0}+V_{0}(b)\right) -\widetilde{V}_{i}^{\pi _{l}}(l,T)\right] \\ \begin{array}{ll}\le &{} \left( 1-e^{-\left( q+\overline{\lambda }\right) \frac{\delta }{\sigma _{2}}}\right) \left( (K_{1,0}\vee K_{2,0})+V_{0}(b)\right) \\ \le &{} \frac{q+\overline{\lambda }}{\sigma _{2}}\left( (K_{1,0}\vee K_{2,0})+V_{0}(b)\right) \delta . \end{array} \end{array} \end{aligned}$$

(8.6)

Since the penalty functions $p_{i}$ are non-decreasing, we also have,

$$\begin{aligned} \mathbb {E}\left[ 1_{\left\{ \overset{\vee }{X}_{T}^{\pi _{y}}<l\right\} }e^{-qT}\left( p_{\mathcal {J}_{T^{-}}}(l-\overset{\vee }{X}_{T}^{\pi _{y} })-p_{\mathcal {J}_{T^{-}}}(l-\overset{\vee }{X}_{T}^{\pi _{y}}+y-l)\right) \right] \le 0. \end{aligned}$$

(8.7)

Finally, since the event $l\le X_{T}^{\pi _{y}}<y$ coincides with the arrival of a customer demand,

$$\begin{aligned} \begin{array}{l} \mathbb {E}\left[ 1_{\left\{ l\le X_{T}^{\pi _{y}}<y\right\} }e^{-qT}\left( V_{\mathcal {J}_{T}}^{\pi _{X_{T}^{\pi _{y}}}}(X_{T}^{\pi _{y}})-V_{\mathcal {J}_{T}}(l)\right) \right] \\ \begin{array}{ll} = &{} \mathbb {E}\left[ 1_{\left\{ l\le X_{T}^{\pi _{y}}<y\right\} }1_{\left\{ T=\tau _{n}^{\mathcal {J}_{T^{-}}}\text { for some }n\right\} }e^{-qT}\left( V_{\mathcal {J}_{T}}^{\pi _{X_{T}^{\pi _{y}}}}(X_{T}^{\pi _{y}})-V_{\mathcal {J}_{T}}(l)\right) \right] \\ \le &{} \mathbb {E}\left[ e^{-q\tau _{1}^{\mathcal {J}_{T}}}\max _{z\in [l,y]}\left( V_{\mathcal {J}_{T}}^{\pi _{z}}(z)-V_{\mathcal {J}_{T}}(l)\right) \right] \\ \le &{} \frac{\overline{\lambda }}{q+\overline{\lambda }}\max _{z\in [l,l+\delta ]}\left( V_{\mathcal {J}_{T}}^{\pi _{z}}(z)-V_{\mathcal {J}_{T} }(l)\right) . \end{array} \end{array} \end{aligned}$$

(8.8)

Hence, from (8.5), (8.6), (8.7) and (8.8), there exists $\overline{m}$ large enough such that

$$\begin{aligned} \frac{q}{q+\overline{\lambda }}\max _{z\in [l,l+\delta ]}\left( V_{i}^{\pi _{y}}(z)-V_{i}^{\pi _{l}}(l)\right) \le \overline{m}\delta . \end{aligned}$$

So, we obtain (8.3) with $m=\overline{m}\left( q+\lambda \right) /q$. The argument to show (8.2) is analogous.

1.2 Proof of Proposition 4.3

Consider $\pi \in \Pi _{x,j}$. Let us extend $\overline{u}_{1}$ and $\overline{u}_{2}$ as $\overline{u}_{i}(x)=\overline{u}_{i}(l)$ and $\overline{u}_{0}(x)=\overline{u}_{0}(l)$ for $x<l$. Consider the controlled risk process $X_{t}^{\pi }$ starting at x and the function $\mathcal {J}_{t}$ defined in (2.5). Since $\overline{u}_{i}$ is Lipschitz for $i=1,2$, we obtain that the function $t\rightarrow e^{-qt}~\overline{u}_{\mathcal {J}_{t}}(X_{t}^{\pi })$ is absolutely continuous in between the stopping times $\left\{ 0\right\} \cup \left\{ \tau _{n}:n\ge 1\right\} \cup \left\{ T_{k}:k\ge 1\right\}$. So, taking

$$\begin{aligned} m_{t}:=\max \{k:T_{k}\le t\}\text {,} \end{aligned}$$

we have

$$\begin{aligned} \begin{array}{l} \overline{u}_{\mathcal {J}_{t}}(X_{t}^{\pi })e^{-qt}-\overline{u}_{j}(x)\\ \begin{array}{ll} = &{} \sum _{k=0}^{m_{t}-1}\left( \overline{u}_{_{J_{k+1}}}(X_{T_{k+1}}^{\pi })e^{-qT_{k+1}}-\overline{u}_{_{J_{k}}}(X_{T_{k}}^{\pi })e^{-qT_{k}}\right) +(\overline{u}_{J_{m_{t}}}(X_{t}^{\pi })e^{-qt}-\overline{u}_{_{J_{m_{t}}}}(X_{T_{m_{t}}}^{\pi })e^{-qT_{m_{t}}}). \end{array} \end{array} \end{aligned}$$

(8.9)

Let us define

$$\begin{aligned} \begin{array}{lll} M^{i}(z_{0},t_{0},t) &{} = &{} \overline{u}_{i}(Z_{t}^{i})e^{-qt}-\overline{u}_{i}(z_{0})e^{-qt_{0}}+\sum _{n=N_{t_{0}}^{i}}^{N_{t}^{i}}e^{-q\tau _{n}^{i}}p_{i}(l-Z_{\tau _{n}^{i-}}^{i}+Y_{n}^{i})1_{\{Z_{\tau _{n}^{i-}}^{i}-Y_{n}^{i}-l<0\}}\\ &{} &{} -\int _{t_{0}}^{t}e^{-qs}\left( \sigma _{i}\overline{u}_{i}^{\prime } (Z_{s}^{i})-(q+\lambda _{i})\overline{u}_{i}(Z_{s}^{i})+\lambda _{i}\int _{0}^{Z_{s^{-}}^{i}-l}\overline{u}_{i}(Z_{s^{-}}^{i}-\alpha )dF_{i} (\alpha )\right) ds\\ &{} &{} -\int _{t_{0}}^{t}e^{-qs}\left( \lambda _{i}\int _{Z_{s^{-}}^{i}-l} ^{\infty }\left( p_{i}(\alpha -Z_{s^{-}}^{i}+l)+\overline{u}_{i}(l)\right) dF_{i}(\alpha )\right) ds\text {,} \end{array} \end{aligned}$$

(8.10)

with

$$\begin{aligned} \text { }Z_{t}^{i}=z_{0}+\sigma _{i}\left( t-t_{0}\right) -\sum \nolimits _{n=N_{t_{0}}^{i}}^{N_{t}^{i}}\min \{Y_{n}^{i},Z_{\tau _{n}^{i-.}} ^{i}-l\}\text { for }t\ge t_{0}\ge 0, \end{aligned}$$

it can be seen that $M^{i}(z_{0},t_{0},t)$ is a martingale with zero expectation for $t\ge t_{0}$.

Consider first the case $J_{k}=i$ and $J_{k+1}=j$ with $i=1,2$, $j=0,1,2$ and $i\ne j$. Since $\overline{u}_{i}$ is absolutely continuous, the function $t\rightarrow \overline{u}_{i}(X_{t}^{\pi })e^{-qt}$ is also absolutely continuous, between customer demands. Using an extension of the Dynkin’s Formula, we obtain

$$\begin{aligned} \begin{array}{l} \overline{u}_{j}(X_{T_{k+1}}^{\pi })e^{-qT_{k+1}}-\overline{u}_{i}(X_{T_{k}}^{\pi })e^{-qT_{k}}\\ \begin{array}{ll} = &{} \overline{u}_{j}(X_{T_{k+1}}^{\pi })e^{-qT_{k+1}}-\overline{u}_{i}(X_{T_{k+1}}^{\pi })e^{-qT_{k+1}}+\overline{u}_{i}(X_{T_{k+1}}^{\pi })e^{-qT_{k+1}}-\overline{u}_{i}(X_{T_{k}}^{\pi })e^{-qT_{k}}\\ \ge &{} -K_{ij}e^{-qT_{k+1}}+\overline{u}_{i}(X_{T_{k+1}}^{\pi })e^{-qT_{k+1}}-\overline{u}_{i}(X_{T_{k}}^{\pi })e^{-qT_{k}}\\ = &{} -K_{ij}e^{-qT_{k+1}}+\int _{T_{k}}^{T_{k+1}}e^{-qs}\mathcal {L}_{i}(\overline{u}_{i})(X_{s}^{\pi })ds\\ &{} -\left( \int _{T_{k}}^{T_{k+1}}e^{-qs}h_{i}(X_{s}^{\pi })ds+\sum _{n=N_{T_{k}}^{i}}^{N_{T_{k+1}}^{i}}e^{-q\tau _{n}^{i}}p_{i}(l-X_{\tau _{n}^{-}}^{\pi }+Y_{n}^{i})1_{\{X_{\tau _{n}^{i-}}^{\pi }-Y_{n}^{i}-l<0\}}\right) \\ &{} +M^{i}(X_{T_{k}}^{\pi },T_{k},T_{k+1}); \end{array} \end{array} \end{aligned}$$

and so, since $\overline{u}_{i}$ is a supersolution of (4.5), we get that

$$\begin{aligned} \begin{array}{l} \mathbb {E}\left[ \left. \overline{u}_{j}(X_{T_{k+1}}^{\pi })e^{-qT_{k+1}}-\overline{u}_{i}(X_{T_{k}}^{\pi })e^{-qT_{k}}\right| \mathcal {F}_{T_{k}}\right] \\ \begin{array}{ll} \ge &{} -\mathbb {E}\left[ K_{ij}e^{-qT_{k+1}}+\left. \int _{T_{k}}^{T_{k+1}}e^{-qs}h_{i}(X_{s}^{\pi })ds+\sum _{n=N_{T_{k}}}^{N_{T_{k+1}}}e^{-q\tau _{n}^{i}}p_{i}(l-X_{\tau _{n}^{-}}^{\pi }+Y_{n}^{i})1_{\{X_{\tau _{n}^{i-}}^{\pi }-Y_{n}^{i}-l<0\}}\right| \mathcal {F}_{T_{k}}\right] . \end{array} \end{array} \end{aligned}$$

In the case $J_{k}=0$ we have $\mathcal {J}_{T_{k+1}}\ne 0$, then $X_{s}^{\pi }=b$ in $[T_{k},T_{k+1})$ , $T_{k+1}=\tau _{n}^{0}$ for some n and so, analogously to the previous case,

$$\begin{aligned} \begin{array}{l} \overline{u}_{\mathcal {J}_{T_{k+1}}}(X_{T_{k+1}}^{\pi })e^{-qT_{k+1}}-\overline{u}_{0}(X_{T_{k}}^{\pi })e^{-qT_{k}}\\ \begin{array}{ll} = &{} e^{-qT_{k}}\left( (\overline{u}(b-Y_{n}^{0})1_{\{b-Y_{n}^{0}-l\ge 0\}}+\overline{u}_{\mathcal {J}_{T_{k+1}}}(l)1_{\{b-Y_{n}^{0}-l<0\}})e^{-q(T_{k+1}-T_{k})}-\overline{u}_{0}(b)\right) -K_{0\mathcal {J}_{T_{k+1}}}e^{-qT_{k+1}}\\ = &{} e^{-qT_{k}}\left( \overline{u}(b-Y_{n})1_{\{b-Y_{n}^{0}-l\ge 0\}}+\overline{u}_{\mathcal {J}_{T_{k+1}}}(l)1_{\{b-Y_{n}^{0}-l<0\}}+p_{0}(l-b+Y_{n}^{0})1_{\{b-Y_{n}^{0}-l<0\}}\right) e^{-q(T_{k+1}-T_{k})}\\ &{} -e^{-qT_{k}}\left( \frac{\lambda _{0}}{q+\lambda _{0}}\left( {\textstyle \int _{0}^{b-l}} \overline{u}(b-\alpha )dF_{0}(\alpha )+ {\textstyle \int _{b-l}^{\infty }} \left( p_{0}(\alpha -b+l)+\overline{u}(l)\right) dF_{0}(\alpha )\right) \right) \\ &{} -\left( K_{0\mathcal {J}_{T_{k+1}}}e^{-qT_{k+1}}+\int _{T_{k}}^{T_{k+1} }e^{-qs}h_{0}(b)ds+e^{-qT_{k+1}}p_{0}(l-b+Y_{n}^{0})1_{\{b-Y_{n}^{0} -l<0\}}\right) \\ &{} +\int _{T_{k}}^{T_{k+1}}e^{-qs}h_{0}(b)ds-\frac{\lambda _{0}}{q+\lambda _{0} }e^{-qT_{k}}h_{0}(b)\text {,} \end{array} \end{array} \end{aligned}$$

and, since $T_{k+1}-T_{k}$ is distributed as $exp(\lambda _{0})$, we obtain that

$$\begin{aligned} \begin{array}{ll} 0= &{} \mathbb {E}\left[ \left. e^{-qT_{k}}\left( \overline{u}(b-Y_{n}^{0})1_{\{b-Y_{n}^{0}-l\ge 0\}}+\overline{u}_{\mathcal {J}_{T_{k+1}}}(l)1_{\{b-Y_{n}^{0}-l<0\}}\right) \right| \mathcal {F}_{T_{k}}\right] \\ &{} +\mathbb {E}\left[ \left. e^{-qT_{k+1}}p_{0}(l-b+Y_{n}^{0})1_{\{b-Y_{n}^{0}-l<0\}}\right| \mathcal {F}_{T_{k}}\right] \\ &{} -\mathbb {E}\left[ \left. \frac{e^{-qT_{k}}}{q+\lambda _{0}}\lambda _{0}\left( {\textstyle \int _{0}^{b-l}} \overline{u}(b-\alpha )dF_{0}(\alpha )+ {\textstyle \int _{b-l}^{\infty }} \left( p_{0}(\alpha -b+l)+\overline{u}(l)\right) dF_{0}(\alpha )\right) \right| \mathcal {F}_{T_{k}}\right] . \end{array} \end{aligned}$$

and so

$$\begin{aligned} \begin{array}{l} \mathbb {E}\left[ \left. \overline{u}_{\mathcal {J}_{T_{k+1}}}(X_{T_{k+1}}^{\pi })e^{-qT_{k+1}}-\overline{u}_{0}(X_{T_{k}}^{\pi })e^{-qT_{k}}\right| \mathcal {F}_{T_{k}}\right] \\ \begin{array}{cc} = &{} -\mathbb {E}\left[ \left. K_{0\mathcal {J}_{T_{k+1}}}e^{-qT_{k+1}}+\int _{T_{k}}^{T_{k+1}}e^{-qs}h_{0}(b)ds+e^{-qT_{k+1}}p_{0}(l-b+Y_{n}^{0})1_{\{b-Y_{n}^{0}-l<0\}}\right| \mathcal {F}_{T_{k}}\right] . \end{array} \end{array} \end{aligned}$$

Analogously, we can prove that

$$\begin{aligned} \begin{array}{c} \begin{array}{l} \mathbb {E}\left[ \left. \overline{u}_{J_{m_{t}}}(X_{t}^{\pi })e^{-qt}-\overline{u}_{_{J_{m_{t}}}}(X_{T_{m_{t}}}^{\pi })e^{-qT_{m_{t}}}\right| \mathcal {F}_{T_{m_{t}}}\right] \\ \ge -\mathbb {E}\left[ \left. \int _{T_{m_{t}}}^{t}e^{-qs}h_{_{J_{m_{t}}}}(X_{s}^{\pi })ds+\sum _{n=N_{T_{m_{t}}}^{J_{m_{t}}}}^{N_{t}^{J_{m_{t}}}}e^{-q\tau _{n}^{J_{m_{t}}}}p_{J_{m_{t}}}(l-X_{\tau _{n}^{J_{m_{t}}-}}^{\pi }+Y_{n}^{J_{m_{t}}})1_{\{X_{\tau _{n}^{J_{m_{t}}-}}^{\pi }-Y_{n}^{J_{m_{t}}}-l<0\}}\right| \mathcal {F}_{T_{m_{t}}}\right] . \end{array} \end{array} \end{aligned}$$

Taking the expected value in (8.9), we obtain

$$\begin{aligned} \begin{array}{lll} \mathbb {E}\left[ \overline{u}_{\mathcal {J}_{t}}(X_{t}^{\pi })e^{-qt}\right] -\overline{u}_{j}(x) &{} = &{} \mathbb {E}\left[ \sum _{k=0}^{m_{t}-1} \mathbb {E}\left[ \left. \left( \overline{u}_{_{J_{k+1}}}(X_{T_{k+1}}^{\pi })e^{-qT_{k+1}}-\overline{u}_{_{J_{k}}}(X_{T_{k}}^{\pi })e^{-qT_{k}}\right) \right| \mathcal {F}_{T_{k}}\right] \right] \\ &{} &{} +\mathbb {E}\left[ \mathbb {E}\left[ \left. (\overline{u}_{J_{m_{t}} }(X_{t}^{\pi })e^{-qt}-\overline{u}_{_{J_{m_{t}}}}(X_{T_{m_{t}}}^{\pi })e^{-qT_{m_{t}}})\right| \mathcal {F}_{T_{m_{t}}}\right] \right] \\ &{} \ge &{} -V_{j}^{\pi }(x) \end{array} \end{aligned}$$

taking the limit with t going to infinity, and using that $X_{t}^{\pi }\in [l,b]$ we obtain that ${\overline{u}}_{j}(x)\le {V}_{j}^{\pi}(x)$ for $j=1,2$.

Considering instead the controlled risk process $X_{t}^{\pi }$ starting at b, we obtain with a similar proof that $\overline{u}_{0}(b)\le V_{0}^{\pi }(b)$.