In this paper we consider an absorbing Markov chain with finite number of states. We focus especially on random walk on transient states. We present a graph reduction method and prove its validity. Using this method we build algorithms which allow us to determine the distribution of time to absorption, in particular we compute its moments and the probability of absorption. The main idea used in the proofs consists in observing a nondecreasing sequence of stopping times. Random walk on the initial Markov chain observed exclusively in the stopping times τ1, τ2, … is equivalent to some new Markov chain.
@article{bwmeta1.element.doi-10_2478_v10062-009-0009-7,
author = {Mariusz G\'orajski},
title = {Reduction of absorbing Markov chain},
journal = {Annales UMCS, Mathematica},
volume = {63},
year = {2009},
pages = {91-107},
zbl = {1239.60072},
language = {en},
url = {http://dml.mathdoc.fr/item/bwmeta1.element.doi-10_2478_v10062-009-0009-7}
}
Mariusz Górajski. Reduction of absorbing Markov chain. Annales UMCS, Mathematica, Tome 63 (2009) pp. 91-107. http://gdmltest.u-ga.fr/item/bwmeta1.element.doi-10_2478_v10062-009-0009-7/
Billingsley, P., Probability and Measure, John Wiley & Sons, New York, 1979. | Zbl 0411.60001
Chandrasekharan, M. P., Madhusudanan Pillai, V., An absorbing Markov chain model for production systems with rework and scrapping, Computers and Industrial Engineering 55, Issue 3 (2008), 695-706.
Engel, A., The probabilistic abacus, Educational Studies in Mathematics 6 (1975), 1-22. | Zbl 0303.60056
Engel, A., Why does the probabilistic abacus work?, Educational Studies in Mathematics 7 (1976), 59-56. | Zbl 0338.60041
Feller, W., An Introduction to Probability Theory and Its Applications, John Wiley & Sons, New York, 1968. | Zbl 0155.23101
Gosselin, F., Asymptotic behavior of absorbing Markov chains conditional on nonabsorption for application in conservation biology, Ann. Appl. Probab. 11, no. 1 (2001), 261-284. | Zbl 1019.60082
Iosifescu, M., Finite Markov Processes and Their Applications, John Wiley & Sons, Chichester, 1980. | Zbl 0436.60001
Kemeny, J. G., Snell, J. L., Finite Markov Chain, D. Van Nostrand, Princeton, 1960.
Keming Gu, Sadiku, M. N. O., Absorbing Markov chain solution for Possion's equation, Southeastcon 2000. Proceedings of the IEEE (2000), 297-300.
Norris, J. R., Markov Chain, Cambridge University Press, Cambridge, 1997.
Płocki, A., Introduction to probability calculus and mathematical statistics for teachers, Propedeutyka rachunku prawdopodobieństwa i statystyki matematycznej dla nauczycieli, PWN, Warszawa 1992 (Polish).
Swan, Y. C., Bruss, F. T., A matrix-analytic approach to the N-player ruin problem, J. Appl. Probab. 43, no. 3 (2006), 755-766. | Zbl 1137.60035