This paper shows the convergence of the value iteration (or successive approximations) algorithm for average cost (AC) Markov control processes on Borel spaces, with possibly unbounded cost, under appropriate hypotheses on weighted norms for the cost function and the transition law. It is also shown that the aforementioned convergence implies strong forms of AC-optimality and the existence of forecast horizons.
@article{bwmeta1.element.bwnjournal-article-zmv23i2p219bwm,
author = {Evgueni Gordienko and On\'esimo Hern\'andez-Lerma},
title = {Average cost Markov control processes with weighted norms: value iteration},
journal = {Applicationes Mathematicae},
volume = {23},
year = {1995},
pages = {219-237},
zbl = {0829.93068},
language = {en},
url = {http://dml.mathdoc.fr/item/bwmeta1.element.bwnjournal-article-zmv23i2p219bwm}
}
Gordienko, Evgueni; Hernández-Lerma, Onésimo. Average cost Markov control processes with weighted norms: value iteration. Applicationes Mathematicae, Tome 23 (1995) pp. 219-237. http://gdmltest.u-ga.fr/item/bwmeta1.element.bwnjournal-article-zmv23i2p219bwm/
[000] [1] D. P. Bertsekas, Dynamic Programming : Deterministic and Stochastic Models, Prentice-Hall, Englewood Cliffs, N.J., 1987. | Zbl 0649.93001
[001] [2] E. B. Dynkin and A. A. Yushkevich, Controlled Markov Processes, Springer, New York, 1979. | Zbl 0073.34801
[002] [3] J. Flynn, On optimality criteria for dynamic programs with long finite horizons, J. Math. Anal. Appl. 76 (1980), 202-208. | Zbl 0438.90100
[003] [4] E. Gordienko and O. Hernández-Lerma, Average cost Markov control processes with weighted norms: existence of canonical policies, this volume, 199-218. | Zbl 0829.93067
[004] [5] O. Hernández-Lerma, Adaptive Markov Control Processes, Springer, New York, 1989.
[005] [6] O. Hernández-Lerma and J. B. Lasserre, A forecast horizon and a stopping rule for general Markov decision processes, J. Math. Anal. Appl. 132 (1988), 388-400. | Zbl 0646.90090
[006] [7] O. Hernández-Lerma and J. B. Lasserre, Average cost optimal policies for Markov control processes with Borel state space and unbounded costs, Systems Control Lett. 15 (1990), 349-356. | Zbl 0723.93080
[007] [8] O. Hernández-Lerma and J. B. Lasserre, Linear programming and average optimality of Markov control processes on Borel spaces-unbounded costs, SIAM J. Control Optim. 32 (1994), 480-500. | Zbl 0799.90120
[008] [9] O. Hernández-Lerma and J. B. Lasserre, Discrete-Time Markov Control Processes, book in preparation. | Zbl 0724.93087
[009] [10] G. P. Klimov, Existence of a final distribution for an irreducible Feller process with invariant measure, Math. Notes 37 (1985), 161-163. | Zbl 0659.60101
[010] [11] R. Montes-de-Oca and O. Hernández-Lerma, Value iteration in average cost Markov control processes on Borel spaces, Acta Appl. Math., to appear. | Zbl 0843.93093
[011] [12] E. Nummelin, General Irreducible Markov Chains and Non-Negative Operators, Cambridge University Press, Cambridge, 1984. | Zbl 0551.60066
[012] [13] R. Rempała, Forecast horizon in a dynamic family of one-dimensional control problems, Dissertationes Math. 315 (1991). | Zbl 0754.90063
[013] [14] H. L. Royden, Real Analysis, 2nd ed., Macmillan, New York, 1971. | Zbl 0197.03501
[014] [15] M. Schäl, Conditions for optimality and for the limit of n-stage optimal policies to be optimal, Z. Wahrsch. Verw. Gebiete 32 (1975), 179-196. | Zbl 0316.90080
[015] [16] L. I. Sennott, Value iteration in countable state average cost Markov decision processes with unbounded costs, Ann. Oper. Res. 28 (1991), 261-272. | Zbl 0729.90088
[016] [17] D. J. White, Dynamic programming, Markov chains, and the method of successive approximations, J. Math. Anal. Appl. 6 (1963), 373-376. | Zbl 0124.36404