Infinite-horizon Markov control processes with undiscounted cost criteria: from average to overtaking optimality
Hernández-Lerma, Onésimo ; Vega-Amaya, Oscar
Applicationes Mathematicae, Tome 25 (1998), p. 153-178 / Harvested from The Polish Digital Mathematics Library
Access to full text
Full (PDF)

We consider discrete-time Markov control processes on Borel spaces and infinite-horizon undiscounted cost criteria which are sensitive to the growth rate of finite-horizon costs. These criteria include, at one extreme, the grossly underselective average cost

Publié le : 1998-01-01
EUDML-ID : urn:eudml:doc:219198 
@article{bwmeta1.element.bwnjournal-article-zmv25i2p153bwm,
     author = {On\'esimo Hern\'andez-Lerma and Oscar Vega-Amaya},
     title = {Infinite-horizon Markov control processes with undiscounted cost criteria: from average to overtaking optimality},
     journal = {Applicationes Mathematicae},
     volume = {25},
     year = {1998},
     pages = {153-178},
     zbl = {0906.93062},
     language = {en},
     url = {http://dml.mathdoc.fr/item/bwmeta1.element.bwnjournal-article-zmv25i2p153bwm}
}

Hernández-Lerma, Onésimo; Vega-Amaya, Oscar. Infinite-horizon Markov control processes with undiscounted cost criteria: from average to overtaking optimality. Applicationes Mathematicae, Tome 25 (1998) pp. 153-178. http://gdmltest.u-ga.fr/item/bwmeta1.element.bwnjournal-article-zmv25i2p153bwm/

[000] A. Arapostathis, V. S. Borkar, E. Fernández-Gaucherand, M. K. Ghosh and S. I. Marcus (1993), Discrete-time controlled Markov processes with average cost criterion: a survey, SIAM J. Control Optim. 31, 282-344. | Zbl 0770.93064

[001] R. Bellman (1957), A markovian decision process,z J. Math. Mech. 6, 679-684. | Zbl 0078.34101

[002] D. P. Bertsekas and S. E. Shreve (1978), Stochastic Optimal Control: The Discrete Time Case, Academic Press, New York. | Zbl 0471.93002

[003] B. W. Brown (1965), On the iterative method of dynamic programming on a finite space discrete time Markov process, Ann. Math. Statist. 33, 719-726.

[004] D. A. Carlson, A. Haurie and A. Leizarowitz (1991), Infinite Horizon Optimal Control: Deterministic and Stochastic Systems, Springer, New York. | Zbl 0758.49001

[005] E. V. Denardo and U. G. Rothblum (1979), Overtaking optimality for Markov decision chains, Math. Oper. Res. 4, 144-152. | Zbl 0409.90084

[006] P. K. Dutta (1991), What do discounted optima converge to? A theory of discount rate asymptotics in economic models, J. Econom. Theory 55, 64-94. | Zbl 0743.90024

[007] E. B. Dynkin and A. A. Yushkevich (1979), Controlled Markov Processes, Springer, New York. | Zbl 0073.34801

[008] A. Ephremides and S. Verdú (1989), Control and optimization methods in communication network problems, IEEE Trans. Automat. Control 34, 930-942. | Zbl 0709.94667

[009] E. Fernández-Gaucherand, M. K. Ghosh and S. I. Marcus (1994), Controlled Markov processes on the infinite planning horizon: weighted and overtaking criteria, Z. Oper. Res. 39, 131-155. | Zbl 0809.90128

[010] J. Flynn (1980), On optimality criteria for dynamic programs with long finite horizons, J. Math. Anal. Appl. 76, 202-208. | Zbl 0438.90100

[011] D. Gale (1967), On optimal development in a multi-sector economy, Rev. Econom. Stud. 34, 1-19. P. W. Glynn and S. P. Meyn (1996), A Lyapunov bound for solutions of Poisson's equation, Ann. Probab. 24, 916-931.

[012] E. Gordienko and O. Hernández-Lerma (1995a), Average cost Markov control processes with weighted norms: existence of canonical policies, Appl. Math. (Warsaw) 23, 199-218. | Zbl 0829.93067

[013] E. Gordienko and O. Hernández-Lerma (1995b), Average cost Markov control processes with weighted norms: value iteration, ibid., 219-237. | Zbl 0829.93068

[014] O. Hernández-Lerma (1989), Adaptive Markov Control Processes, Springer, New York. | Zbl 0698.90053

[015] O. Hernández-Lerma, J. C. Hennet and J. B. Lasserre (1991), Average cost Markov decision processes: optimality conditions, J. Math. Anal. Appl. 158, 396-406. | Zbl 0739.90072

[016] O. Hernández-Lerma and J. B. Lasserre (1996), Discrete-Time Markov Control Processes: Basic Optimality Criteria, Springer, New York. | Zbl 0840.93001

[017] O. Hernández-Lerma and J. B. Lasserre (1997), Policy iteration for average cost Markov control processes on Borel spaces, Acta Appl. Math. 47, 125-154. | Zbl 0872.93080

[018] O. Hernández-Lerma, R. Montes-de-Oca and R. Cavazos-Cadena (1991), Recurrence conditions for Markov decision processes with Borel state space: a survey, Ann. Oper. Res. 28, 29-46. | Zbl 0717.90087

[019] O. Hernández-Lerma and M. Muñoz de Ozak (1992), Discrete-time Markov control processes with discounted unbounded cost: optimality criteria, Kybernetika (Prague) 28, 191-212. | Zbl 0771.93054

[020] C. J. Himmelberg, T. Parthasarathy and F. S. Van Vleck (1976), Optimal plans for dynamic programming problems, Math. Oper. Res. 1, 390-394. | Zbl 0368.90134

[021] A. Leizarowitz (1988), Controlled diffusion processes on infinite horizon with the overtaking criterion, Appl. Math. Optim. 17, 61-78. | Zbl 0637.60055

[022] P. Mandl and M. Lausmanová (1991), Two extensions of asymptotic methods in controlled Markov chains, Ann. Oper. Res. 28, 67-79. | Zbl 0754.60081

[023] S. P. Meyn (1995), The policy improvement algorithm for Markov decision processes with general state space, preprint, Coordinated Science Laboratory, Univ. of Illinois, Urbana, Ill.

[024] S. P. Meyn and R. L. Tweedie (1993), Markov Chains and Stochastic Stability, Springer, London. | Zbl 0925.60001

[025] R. Montes-de-Oca and O. Hernández-Lerma (1996), Value iteration in average cost Markov control processes on Borel spaces, Acta Appl. Math. 42, 203-222. | Zbl 0843.93093

[026] A. S. Nowak (1992), Stationary overtaking optimal strategies in Markov decision processes with general state space, preprint, Institute of Mathematics, Technical Univ. of Wrocław.

[027] E. Nummelin (1984), General Irreducible Markov Chains and Non-Negative Operators, Cambridge Univ. Press, Cambridge. | Zbl 0551.60066

[028] S. Orey (1971), Lecture Notes on Limit Theorems for Markov Chain Transition Probabilities, Van Nostrand Reinhold, London. | Zbl 0295.60054

[029] M. L. Puterman (1994), Markov Decision Processes, Wiley, New York. | Zbl 0829.90134

[030] F. P. Ramsey (1928), A mathematical theory of savings, Econom. J. 38, 543-559.

[031] U. Rieder (1978), Measurable selection theorems for optimization problems, Manuscripta Math. 24, 115-131. | Zbl 0385.28005

[032] P. J. Schweitzer (1985), On undiscounted Markovian decision processes with compact action spaces, RAIRO Rech. Opér. 19, 71-86. | Zbl 0571.90095

[033] S. Stidham and R. Weber (1993), A survey of Markov decision models for control of networks of queues, Queueing Systems Theory Appl. 13, 291-314. | Zbl 0772.90082

[034] O. Vega-Amaya (1996), Overtaking optimality for a class of production-inventory systems, preprint, Departamento de Matemáticas, Universidad de Sonora.

[035] A. F. Veinott, Jr. (1966), On finding optimal policies in discrete dynamic programming with no discounting, Ann. Math. Statist. 37, 1284-1294. | Zbl 0149.16301

[036] C. C. von Weizsäcker (1965), Existence of optimal programs of accumulation for an infinite horizon, Rev. Econom. Stud. 32, 85-104.

[037] A. A. Yushkevich (1973), On a class of strategies in general Markov decision models, Theory Probab. Appl. 18, 777-779. | Zbl 0311.90081