Extensions of the sequential stochastic assignment problem

Abstract

The sequential stochastic assignment problem (SSAP) allocates N workers to N IID sequentially arriving tasks so as to maximize the expected total reward. This paper studies two extensions of the SSAP. The first one assumes that the values of any two consecutive tasks are dependent on each other while the exact number of tasks to arrive is unknown until after the final arrival. The second extension generalizes the first one by assuming that the number of workers is also random. Optimal assignment policies for both problems are derived and proven to have the same threshold structure as the optimal policy of the SSAP.

Appendix: Proofs

Proof of Lemma 6

Using an approach analogous to that applied in equation (3.6) of Kennedy (1986), one can obtain

$$\begin{aligned} {1 \over \alpha } \sum _{i=1}^{M}p_{\pi (i,n)}\big \vert \bar{Z}_{i,n}^{(1)} \big \vert \le {1 \over \alpha } \left( \sum _{i=1}^{+\infty } p_{i} \right) E\left[ \sup _{r \ge n} X_{r} \big \vert X_{n} \right] , \end{aligned}$$

(57)

for any $M \ge 1$ and $n \ge 1$, implying that

$$\begin{aligned} \big \vert F^{\pi }_{n} \big \vert \le \sum _{r=1}^{n-1} p_{\pi _{r}}X_{r} + {1 \over \alpha } \sum _{i=1}^{+\infty }p_{\pi (i,n)}\big \vert \bar{Z}_{i,n}^{(1)} \big \vert \le \left( \sum _{i=1}^{+\infty } p_{i} \right) \left( \sup _{r} X_{r} + {1 \over \alpha } E\left[ \sup _{r \ge n} X_{r} \big \vert X_{n} \right] \right) , \end{aligned}$$

(58)

and hence,

$$\begin{aligned} E\left[ \big \vert F^{\pi }_{n} \big \vert \right]\le & {} \left( \sum _{i=1}^{+\infty } p_{i} \right) \left( E\left[ \sup _{r} X_{r} \right] + {1 \over \alpha } E\left[ E\left[ \sup _{r \ge n} X_{r} \big \vert X_{n}\right] \right] \right) \nonumber \\\le & {} \left( \sum _{i=1}^{+\infty } p_{i} \right) \left( E\left[ \sup _{r} X_{r} \right] + {1 \over \alpha } E\left[ E\left[ \sup _{r} X_{r} \big \vert X_{n} \right] \right] \right) \nonumber \\= & {} \left( \sum _{i=1}^{+\infty } p_{i} \right) \left( E\left[ \sup _{r} X_{r} \right] + {1 \over \alpha }E\left[ \sup _{r} X_{r} \right] \right) \nonumber \\= & {} \left( 1+{1 \over \alpha }\right) \left( \sum _{i=1}^{+\infty } p_{i} \right) E\left[ \sup _{r} X_{r} \right] , \end{aligned}$$

(59)

for any $n \ge 1$ which results in

$$\begin{aligned} \sup _{n}E\left[ \big \vert F^{\pi }_{n} \big \vert \right] < +\infty , \end{aligned}$$

(60)

by (33) and the assumption that $\sum _{i=1}^{+\infty }p_{i}<+\infty$. Now, observe that

$$\begin{aligned} E\left[ F^{\pi }_{n+1} \big \vert X_{n},\ldots ,X_{1} \right] =&\sum _{r=1}^{n}p_{\pi _{r}}X_{r}+E\left[ {1 \over \alpha }\sum _{i=1}^{+\infty }p_{\pi (i,n+1)}\bar{Z}^{(1)}_{i,n+1} \big \vert X_{n},\ldots ,X_{1} \right] \nonumber \\ =&\sum _{r=1}^{n-1}p_{\pi _{r}}X_{r}+p_{\pi _{n}}X_{n}+\sum _{i=1}^{+\infty }p_{\pi (i,n+1)}\nonumber \\&E\left[ {1 \over \alpha }\bar{Z}^{(1)}_{i,n+1} \big \vert X_{n},X_{n-1},\ldots ,X_{1} \right] \nonumber \\ =&\sum _{r=1}^{n-1}p_{\pi _{r}}X_{r}+p_{\pi _{n}}X_{n}+\sum _{i=1}^{+\infty }p_{\pi (i,n+1)} E\left[ {1 \over \alpha }\bar{Z}^{(1)}_{i,n+1} \big \vert X_{n} \right] , \end{aligned}$$

(61)

where the second equality follows from (57) and the last equality is obtained since

$$\begin{aligned} E\left[ {1 \over \alpha }\bar{Z}^{(1)}_{i,n+1} \big \vert X_{n}, X_{n-1},\ldots ,X_{1} \right]= & {} \lim _{N \rightarrow +\infty }E\left[ {1 \over \alpha }Z^{(1),N}_{i,n+1} \big \vert X_{n},X_{n-1},\ldots ,X_{1} \right] \nonumber \\= & {} \lim _{N \rightarrow +\infty }E\left[ {1 \over \alpha }Z^{(1),N}_{i,n+1} \big \vert X_{n} \right] \nonumber \\= & {} E\left[ {1 \over \alpha }\bar{Z}^{(1)}_{i,n+1} \big \vert X_{n} \right] , \end{aligned}$$

(62)

where the first and the last equalities follow from the dominated convergence theorem and the second one is a direct result of the way $\left\{ Z^{(1),N}_{m,n}\right\}$ and $\left\{ Z^{(2),N}_{m,n}\right\}$ are defined in (9) and (10). Note that by Lemma 4 and the dominated convergence theorem,

$$\begin{aligned} E\left[ {1 \over \alpha } \bar{Z}_{i,n+1}^{(1)} \big \vert X_{n}\right] =E\left[ \bar{Z}_{i,n+1}^{(1)} \big \vert X_{n+1}>0,X_{n}\right] + \bar{Z}_{i,n+1}^{(2)}, \end{aligned}$$

(63)

for $i=1,2,\ldots$, and observe that $\left\{ {1 \over \alpha }\bar{Z}^{(1)}_{i,n}, i=1,2,\ldots \right\}$ is the set of ordered values of

$$\begin{aligned} \left\{ X_{n}, \left( E\left[ \bar{Z}^{(1)}_{i,n+1} \big \vert X_{n+1}>0,\,X_{n}\right] +\bar{Z}^{(2)}_{i,n+1}\right) , i=1,2,\ldots \right\} , \end{aligned}$$

(64)

which, by the extension of Lemma 3 (Hardy’s Theorem) to the infinite-support case (see Kennedy (1986)), implies that

$$\begin{aligned} E\left[ F^{\pi }_{n+1} \big \vert X_{n},X_{n-1},\ldots ,X_{1} \right] \le \sum _{r=1}^{n-1}p_{\pi _{r}}X_{r}+{1 \over \alpha }\sum _{i=1}^{+\infty }p_{\pi (i,n)} \bar{Z}^{(1)}_{i,n} = F^{\pi }_{n}. \end{aligned}$$

(65)

By Doob’s forward convergence theorem, (60) and (65) indicate that the super Martingale $\left\{ F^{\pi }_{n}: n\ge 1 \right\}$ converges almost surely to a finite limit $F^{\pi }_{\infty }$ as $n \rightarrow +\infty$, where

$$\begin{aligned} F^{\pi }_{\infty }=\sum _{r=1}^{+\infty }p_{\pi _{r}}X_{r} \end{aligned}$$

(66)

since

$$\begin{aligned} \lim _{n \rightarrow +\infty }{1 \over \alpha } \sum _{i=1}^{+\infty }p_{\pi (i,n)}\big \vert \bar{Z}_{i,n}^{(1)} \big \vert =0 \end{aligned}$$

(67)

by (57), (33), and the assumption that $\sum _{i=1}^{+\infty }p_{i} <+\infty$. Recall from (58) that

$$\begin{aligned} \big \vert F^{\pi }_{n} \big \vert \le \left( \sum _{i=1}^{+\infty } p_{i} \right) \left( \sup _{r} X_{r} + {1 \over \alpha } E\left[ \sup _{r \ge n} X_{r} \big \vert X_{n}, \right] \right) , \end{aligned}$$

(68)

for any $n \ge 1$, where the expected value of the right-hand side of (68) is finite, and hence, $\left\{ F^{\pi }_{n}: n\ge 1 \right\}$ is uniformly integrable, which along with (65) yields

$$\begin{aligned} E\left[ F^{\pi }_{\infty } \big \vert X_{n},X_{n-1},\ldots ,X_{1} \right] \le F^{\pi }_{n}. \end{aligned}$$

(69)

$\square$

Proof of Theorem 2

Using Lemma 6 and an approach similar to that applied in the final part of Section 3 in Kennedy (1986), the optimal assignment policy is derived as follows. For any fixed $\pi \in \pi \mid _{1}$ and an arbitrary policy $\phi \in \pi \mid _{n}$, by Lemma 6,

$$\begin{aligned} F^{\pi }_{n}=F^{\phi }_{n} \ge E\left[ \sum _{r=1}^{+\infty }p_{\phi _{r}}X_{r} \big \vert X_{n},X_{n-1},\ldots ,X_{1} \right] \end{aligned}$$

Therefore,

$$\begin{aligned} F^{\pi }_{n} \ge R^{\pi }_{n}, \end{aligned}$$

(70)

for any $\pi \in \pi \mid _{1}$ and $n \ge 1$. In addition, note that under policy $\bar{\pi }$, (65) becomes

$$\begin{aligned} E\left[ F^{\bar{\pi }}_{n+1} \big \vert X_{n},X_{n-1},\ldots ,X_{1} \right] = F^{\bar{\pi }}_{n}, \end{aligned}$$

(71)

which leads to

$$\begin{aligned} F^{\bar{\pi }}_{1}=E\left[ F^{\bar{\pi }}_{\infty } \big \vert X_{1} \right] =E\left[ \sum _{r=1}^{+\infty }p_{\bar{\pi }_{r}}X_{r} \big \vert X_{1} \right] \ge R^{\bar{\pi }}_{1}, \end{aligned}$$

(72)

where the inequality follows from (70). Recall that $R^{\bar{\pi }}_{1}$ is the optimal expected total reward by definition; therefore, (72) implies that $\bar{\pi }$ is the optimal policy. Moreover, the optimal expected total reward under $\bar{\pi }$ is equal to

$$\begin{aligned} F^{\bar{\pi }}_{1}={1 \over \alpha }\sum _{i=1}^{+\infty }p_{\bar{\pi }(i,1)}\bar{Z}^{(1)}_{i,1}, \end{aligned}$$

(73)

from (72) and the definition of $F^{\pi }_{n}$. $\square$