On Transient Markov Processes with a Countable Number of States and Stationary Transition Probabilities
Blackwell, David
Ann. Math. Statist., Tome 26 (1955) no. 4, p. 654-658 / Harvested from Project Euclid
We consider a Markov process $x_0, x_1, \cdots$ with a countable set $S$ of states and stationary transition probabilities $p(t \mid s) = P\{x_{n+1} = t \mid x_n = s\}$. Call a set $C$ of states almost closed if (a) $P\{x_n \varepsilon C$ for an infinite number of $n\} > 0$ and (b) $x_n \varepsilon C$ infinitely often implies $x_n \varepsilon C$ for all sufficiently large $n$, with probability one. It is shown that there is a set $(C_1, C_2, \cdots)$ essentially unique, of disjoint almost closed sets such that (a) all except at most one of the $C_i$ are atomic, that is, $C_i$ does not contain two disjoint almost closed subsets, (b) the non-atomic $C_i$, if present, contains no atomic subsets, (c) the process is certain to enter and remain in some set $C_i$. A relation between the sets $C_i$ and the bounded solutions of the system of equations \begin{equation*}\tag{1} \alpha(s) = \sum_t \alpha(t)p(t \mid s)\end{equation*} is obtained; in particular there is only one atomic $C_i$ and no non-atomic $C_i$ if and only if the only bounded solution of (1) is $\alpha(t)$ = constant. This condition is shown to hold if the process is the sum of independent identical (numerical or vector) variables; whence, for such a process, the probability of entering a set $J$ infinitely often is zero or one. The results are new only if the process has transient components. The main tool is the martingale convergence theorem.
Publié le : 1955-12-14
Classification: 
@article{1177728425,
     author = {Blackwell, David},
     title = {On Transient Markov Processes with a Countable Number of States and Stationary Transition Probabilities},
     journal = {Ann. Math. Statist.},
     volume = {26},
     number = {4},
     year = {1955},
     pages = { 654-658},
     language = {en},
     url = {http://dml.mathdoc.fr/item/1177728425}
}
Blackwell, David. On Transient Markov Processes with a Countable Number of States and Stationary Transition Probabilities. Ann. Math. Statist., Tome 26 (1955) no. 4, pp.  654-658. http://gdmltest.u-ga.fr/item/1177728425/