Abstract
Discrete-time Markovian processes with a multidimensional compact state space are considered, where coordinate processes are locally interacting and change their states synchronously. Conditions are given which guarantee that in the class of local randomized strategies there exist deterministic stationary Markovian strategies which maximize asymptotic average expected rewards. If a reward structure is separable, these strategies are even globally optimal.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
REFERENCES
T. Liggett, Interacting Particle Systems, Springer-Verlag, New York (1989).
R. K. Chornei, H. Daduna, and P. S. Knopov, “Controlled Markov fields with finite state space on graphs,” Prepr. Inst. Math. Stochastics, Univ. of Hamburg, Hamburg (2000).
F. P. Kelly, Reversibility and Stochastic Networks, Wiley, New York (1979).
P. Whittle, Systems in Stochastic Equilibrium, Wiley, New York (1986).
E. J. Balder, “On compactness of the space of policies in stochastic dynamic programming,” Stoch. Processes and Their Appl., 32, 141–150 (1989).
M. Schäl, “On dynamic programming: compactness of the space of policies,” Stoch. Processes and Their Appl., 3, 345–364 (1975).
P. A. David and D. Foray, “Percolation structures, Markov random fields,” in: The Economics of EDI Standards Diffusion, Center for Economical Policy Research, Stanford University (1992).
P. A. David, D. Foray, and J.-M. Dalle, “Marshallian externalities and the emergence and spatial stability of technological enclaves,” Prepr. Center for Economical Policy Research, Stanford Univ., Stanford (1996).
R. L. Dobrushin, V. I. Kryukov, and A. L. Toom (ed.), Locally Interacting Systems and Their Application in Biology, Lect. Notes in Math., Springer, Berlin (1978).
N. B. Vasilyev, “Bernoulli and Markov stationary measures in discrete local interactions,” in: R. L. Dobrushin, V. I. Kryukov, and A. L. Toom (ed.), Locally Interacting Systems and Their Application in Biology, Lect Notes in Math., Berlin, Springer (1978), pp. 99–112.
L. G. Gubenko and E. S. Shtatland, “On controlled Markovian processes with discrete time,” Teor. Ver. Mat. Stat., 7, 51–64 (1972).
P. S. Knopov, “Markovian fields and their application in economy,” Obchisl. Prykl. Mat., 80, 33–46 (1996).
K. Kuratowski, Topology [Russian translation], Vol. 2, Mir, Moscow (1969).
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Daduna, G., Knopov, P.S. & Chornei, R.K. Local Control of Markovian Processes of Interaction on a Graph with a Compact Set of States. Cybernetics and Systems Analysis 37, 348–360 (2001). https://doi.org/10.1023/A:1011985609994
Issue Date:
DOI: https://doi.org/10.1023/A:1011985609994