Abstract
Westudy the optimal stopping problem for a class of continuoustime random evolutions described by stochastic differential equationswith alternating renewal processes as noise sources. The exactsolution of this stopping problem provides, in explicit form,an expression for the Gittins' indices needed to derive the optimalscheduling of a class of multi-armed bandit problems in continuoustime. The underlying random processes to which the bandits' armsobey are random velocity models. Such processes are commonlyused to describe, in the fluid limit, the random production flowsdelivered by failure prone machines.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bielicki, T., and Kumar, T. R. 1988. Optimality of zero inventory policies for unreliable manufacturing systems. Oper. Res 36: 532–541.
Bertsimas, D., and Nino-Mora, J. 2000. Restless bandits, linear programming relaxations and primal-dual index heuristic. Oper. Res 48: 80–90.
Dalang, R. C., and Cairoli, R. 1996. Sequential Stochastic Optimization. J. Wiley.
Davis, M. H. A. 1984. Piecewise-deterministic Markov processes: A general class of non-diffusion stochastic models. J. Roy. Statist. Soc.B 46: 353–388.
Dusonchet, F., and Hongler, M. O. Continuous time restless bandits and dynamic scheduling for make-to-stock production. Working Paper, EPFL-DMT/IPM.
El Karoui, N., and Karatzas, I. 1994. Dynamic allocation problems in continuous time. The Annals of Prob. 2: 255–286.
El Karoui, N., and Karatzas, I. 1997. Synchronization and optimality for multi-armed bandit problems in continuous time. Special invited paper in Computational and Applied Math. 16-2: 117–152.
Eplett, W. J. R. 1986. Continuous-time allocation indices and their discrete time approximation. Adv. Appl. Probab. 18: 724–746.
Gershwin, S. B. 1994. Manufacturing Systems Engineering. Prentice Hall.
Gittins, J. C., and Jones, D. M. 1974. A dynamic allocation index for the sequential design of experiments. Progress in Statistics. J. Gani, ed., North Holland, pp. 241–266.
Gittins, J. C. 1989. Multi-Armed Bandits Allocation Indices. J. Wiley.
Karatzas, I. 1984. Gittins indices in the dynamic allocation problem for diffusion processes. Ann. of Probab. 12: 173–192.
Kaspi, H., and Mandelbaum, A. 1995. Levy bandits; multi-armed bandits driven by Levy processes. Ann. Appl. Probab. 5: 541–565.
Mandelbaum, A. 1987. Continuous multi-armed bandits and multi-parameter processes. Annals of Probab. 15: 1527–1556.
Menaldi, J. L., and Robin, M. 1990. On the optimal reward function of the continuous time multiarmed bandit problem. SIAM J. Optim. and Control 28: 97–112.
Miller, R. G. 1963. Continuous time stochastic storage processes with random linear inputs and outputs. J. of Math. and Mech. 12: 275–291.
Nino-Mora, J. On certain greedoid polyhedra, partially indexable scheduling problems, and extended restless bandit allocation indices. Submitted to Mathematical Programming.
Pinsky, M. A. 1991. Lectures on Random Evolution. World Scientific.
Whittle, P. 1982. Optimization over Time. Dynamic Programming and Stochastic Control. New York: J. Wiley.
Whittle, P. 1988. Restless bandits: Activity in a changing world. A celebration of applied probability. J. Gani, ed., J. of Appl. Probab. 25A: 287–298.
Veatch, M. H., and Wein, L. M. 1996. Scheduling a make-to-stock queue: Indexpolicies and hedging points. Oper. Res. 44: 634–647.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Hongler, MO., Dusonchet, F. Optimal Stopping and Gittins' Indices for Piecewise Deterministic Evolution Processes. Discrete Event Dynamic Systems 11, 235–248 (2001). https://doi.org/10.1023/A:1011205206089
Issue Date:
DOI: https://doi.org/10.1023/A:1011205206089