We deal with semi-Markov control processes (SMCPs) on Borel spaces with unbounded cost and mean holding time. Under suitable growth conditions on the cost function and the mean holding time, together with stability properties of the embedded Markov chains, we show the equivalence of several average cost criteria as well as the existence of stationary optimal policies with respect to each of these criteria.
@article{bwmeta1.element.bwnjournal-article-zmv27i3p343bwm,
author = {Oscar Vega-Amaya and Fernando Luque-V\'asquez},
title = {Sample-path average cost optimality for semi-Markov control processes on Borel spaces: unbounded costs and mean holding times},
journal = {Applicationes Mathematicae},
volume = {27},
year = {2000},
pages = {343-367},
zbl = {1050.90030},
language = {en},
url = {http://dml.mathdoc.fr/item/bwmeta1.element.bwnjournal-article-zmv27i3p343bwm}
}
Vega-Amaya, Oscar; Luque-Vásquez, Fernando. Sample-path average cost optimality for semi-Markov control processes on Borel spaces: unbounded costs and mean holding times. Applicationes Mathematicae, Tome 27 (2000) pp. 343-367. http://gdmltest.u-ga.fr/item/bwmeta1.element.bwnjournal-article-zmv27i3p343bwm/
[000] [1] R. B. Ash, Real Analysis and Probability, Academic Press, New York, 1972.
[001] [2] S. Bhatnagar and V. S. Borkar, A convex analytic framework for ergodic control of semi-Markov processes, Math. Oper. Res. 20 (1995), 923-936. | Zbl 1035.93511
[002] [3] R. N. Bhattacharya and M. Majumdar, Controlled semi-Markov models under long-run average rewards, J. Statist. Plann. Inference 22 (1989), 223-242. | Zbl 0683.49003
[003] [4] B. S. Borkar, Topics in Controlled Markov Chains, Pitman Res. Notes Math. Ser. 240, Longman Sci. Tech., 1991. | Zbl 0725.93082
[004] [5] R. Cavazos-Cadena and E. Fernández-Gaucherand, Denumerable controlled Markov chains with average reward criterion: Sample path optimality, Z. Oper. Res. Math. Methods Oper. Res. 41 (1995), 89-108. | Zbl 0835.90116
[005] [6] A. Federgruen, A. Hordijk and H. C. Tijms, Denumerable state semi-Markov decision processes with unbounded costs, average cost criterion, Stochastic Process. Appl. 9 (1979), 223-235. | Zbl 0422.90084
[006] [7] A. Federgruen, P. J. Schweitzer and H. C. Tijms, Denumerable undiscounted semi-Markov decision processes with unbounded rewards, Math. Oper. Res. 8 (1983), 298-213. | Zbl 0513.90085
[007] [8] A. Federgruen and H. C. Tijms, The optimality equation in average cost denumerable state semi-Markov decision problems. Recurrence conditions and algorithms, J. Appl. Probab. 15 (1978), 356-373. | Zbl 0386.90060
[008] [9] E. A. Feinberg, Constrained semi-Markov decision processes with average rewards, Z. Oper. Res. (Math. Methods Oper. Res.) 39 (1994), 257-288. | Zbl 0824.90136
[009] [10] P. W. Glynn and S. P. Meyn, A Liapunov bound for solutions of Poisson's equation, Ann. Probab. 24 (1996), 916-931. | Zbl 0863.60063
[010] [11] E. Gordienko and O. Hernández-Lerma, Average cost Markov control processes with weighted norms: existence of canonical policies, Appl. Math. (Warsaw) 23 (1995), 199-218. | Zbl 0829.93067
[011] [12] P. Hall and C. C. Heyde, Martingale Limit Theory and Its Application, Academic Press, 1980. | Zbl 0462.60045
[012] [13] U. G. Haussman, On the optimal long-run control of Markov renewal processes, J. Math. Anal. Appl. 36 (1971), 123-140.
[013] [14] O. Hernández-Lerma and J. B. Lasserre, Discrete-Time Markov Control Processes: Basic Optimality Criteria, Springer, New York, 1996. | Zbl 0840.93001
[014] [15] O. Hernández-Lerma and J. B. Lasserre, Further criteria for positive Harris recurrence of Markov chains, Proc. Amer. Math. Soc., to appear. | Zbl 0970.60078
[015] [16] O. Hernández-Lerma and O. Vega-Amaya, Infinite-horizon Markov control processes with undiscounted cost criteria: from average to overtaking optimality, Appl. Math. (Warsaw) 25 (1998), 153-178. | Zbl 0906.93062
[016] [17] O. Hernández-Lerma, O. Vega-Amaya and G. Carrasco, Sample-path optimality and variance-minimization of average cost Markov control processes, SIAM J. Control Optim., to appear. | Zbl 0951.93074
[017] [18] M. Kurano, Semi-Markov decision processes and their applications in replacement models, J. Oper. Res. Soc. Japan 28 (1985), 18-29. | Zbl 0564.90090
[018] [19] M. Kurano, Average optimal adaptive policies in semi-Markov decision processes including an unknown parameter, ibid., 252-266. | Zbl 0579.90098
[019] [20] J. B. Lasserre, Sample-path average optimality for Markov control processes, IEEE Trans. Automat. Control, to appear. | Zbl 0956.93066
[020] [21] S. A. Lippman, Semi-Markov decision processes with unbounded rewards, Management Sci. 19 (1973), 717-731. | Zbl 0259.60044
[021] [22] S. A. Lippman, On dynamic programming with unbounded rewards, ibid. 21 (1975), 1225-1233. | Zbl 0309.90017
[022] [23] F. Luque-Vásquez and O. Hernández-Lerma, Semi-Markov control models with average costs, Appl. Math. (Warsaw) 26 (1999), 315-331. | Zbl 1050.90566
[023] [24] S. P. Meyn and R. L. Tweedie, Markov Chains and Stochastic Stability, Springer, London, 1993. | Zbl 0925.60001
[024] [25] M. L. Puterman, Markov Decision Processes, Wiley, New York, 1994.
[025] [26] S. M. Ross, Average cost semi-Markov decision processes, J. Appl. Probab. 7 (1979), 649-656. | Zbl 0204.51704
[026] [27] M. Schäl, On the second optimality equation for semi-Markov decision models, Math. Oper. Res. 17 (1992), 470-486. | Zbl 0773.90091
[027] [28] P. J. Schweitzer, Iterative solutions of the functional equations of undiscounted Markov renewal programming, J. Math. Anal. Appl. 34 (1971), 495-501. | Zbl 0218.90070
[028] [29] L. I. Sennott, Average cost semi-Markov decision processes and the control of queueing systems, Probab. Engrg. Inform. Sci. 3 (1989), 247-272. | Zbl 1134.60408
[029] [30] O. Vega-Amaya, Sample path average optimality of Markov control processes with strictly unbounded cost, Appl. Math. (Warsaw) 26 (1999), 363-381. | Zbl 1050.93523
[030] [31] O. Vega-Amaya, Markov control processes in Borel spaces: undiscounted cost criteria, doctoral thesis, UAM-Iztapalapa, México, 1998 (in Spanish).