Abstract
The paper is concerned with a variant of the continuous-time finite state Markov game of control and stopping where both players can affect transition rates, while only one player can choose a stopping time. The dynamic programming principle reduces this problem to a system of ODEs with unilateral constraints. This system plays the role of the Bellman equation. We show that its solution provides the optimal strategies of the players. Additionally, the existence and uniqueness theorem for the deduced system of ODEs with unilateral constraints is derived.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Guo, X.P., Hernández-Lerma, O.: Continuous-Time Markov Decision Processes: Theory and Applications. Springer, New York (2009)
Guo, X., Hernández-Lerma, O., Prieto-Rumeau, T.: A survey of recent results on continuous-time Markov decision processes (with comments and rejoinder). TOP 14(2), 177–261 (2006). https://doi.org/10.1007/BF02837562
Mordecki, E.: Optimal stopping and perpetual options for Lévy processes. Finance Stoch. 6, 473–493 (2002)
Shiryaev, A.N.: Optimal Stopping Rules. Springer, New York (2008)
Horiguchi, M.: Markov decision processes with a stopping time constraint. Math. Methods Oper. Res. 53(2), 279–295 (2001). https://doi.org/10.1007/PL00003996
Zachrisson, L.: Markov games. Ann. Math. Stud. 52, 211–253 (1964)
Dynkin, E.B.: Game variant of a problem on optimal stopping. Soviet Math. Dokl. 10, 270–274 (1969)
Frid, E.B.: The optimal stopping rule for a two-person Markov chain with opposing interests. Theory Probab. Appl. 14(4), 713–716 (1969). https://doi.org/10.1137/1114091
Ghosh, M.K., Rao, K.S.M.: Zero-sum stochastic games with stopping and control. Oper. Res. Lett. 35(6), 799–804 (2007). https://doi.org/10.1016/j.orl.2007.02.002
Pal, C., Saha, S.: Continuous-time zero-sum stochastic game with stopping and control. Oper. Res. Lett. 48(6), 715–719 (2020). https://doi.org/10.1016/j.orl.2020.08.012
Averboukh, Y.V.: Approximation of value function of differential game with minimal cost. Vestn. Udmurt. Univ. Mat. Mekhanika Komp’yuternye Nauki 31(4), 536–561 (2021). https://doi.org/10.35634/vm210402
Averboukh, Y.: Approximate solutions of continuous-time stochastic games. SIAM J. Control. Optim. 54, 2629–2649 (2016). https://doi.org/10.1137/16M1062247
Barron, E.N.: Differential games with maximum cost. Nonlinear Anal. Theory Methods Appl. 14(11), 971–989 (1990)
Subbotin, A.I.: Generalized Solutions of First-Order PDEs. The Dynamical Perspective. Birkhauser, Boston (1995)
Krasovskii, N.N., Subbotin, A.I.: Game-Theoretical Control Problems. Springer, New York (1988)
Kallenberg, O.: Foundations of Modern Probability. Springer, New York (2001)
Author information
Authors and Affiliations
Contributions
The author confirms sole responsibility for the whole study and manuscript preparation.
Corresponding author
Ethics declarations
Conflict of Interest
The author confirms that this article content has no conflict of interest.
Additional information
The article was prepared within the framework of the HSE University Basic Research Program in 2023.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Averboukh, Y. Zero-Sum Continuous-Time Markov Games with One-Side Stopping. J. Oper. Res. Soc. China 12, 169–187 (2024). https://doi.org/10.1007/s40305-023-00502-3
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40305-023-00502-3