Let $\{X_t\}$ be a stationary ergodic process with distribution $P$ admitting densities $p(x_0,\ldots, x_{n-1})$ relative to a reference measure $M$ that is finite order Markov with stationary transition kernel. Let $I_M(P)$ denote the relative entropy rate. Then $n^{-1}\log p(X_0,\ldots, X_{n-1}) \rightarrow I_M(P) \mathrm{a.s.} (P).$ We present an elementary proof of the Shannon-McMillan-Breiman theorem and the preceding generalization, obviating the need to verify integrability conditions and also covering the case $I_M(P) = \infty$. A sandwich argument reduces the proof to direct applications of the ergodic theorem.
Publié le : 1988-04-14
Classification:
Shannon-McMillan-Breiman theorem,
asymptotic equipartition property (AEP),
ergodic theorem of information theory,
relative entropy rate,
likelihood ratio,
sandwich argument,
Markov approximation,
asymptotically mean stationary,
28D05,
94A17,
28A65,
28D20,
60F15
@article{1176991794,
author = {Algoet, Paul H. and Cover, Thomas M.},
title = {A Sandwich Proof of the Shannon-McMillan-Breiman Theorem},
journal = {Ann. Probab.},
volume = {16},
number = {4},
year = {1988},
pages = { 899-909},
language = {en},
url = {http://dml.mathdoc.fr/item/1176991794}
}
Algoet, Paul H.; Cover, Thomas M. A Sandwich Proof of the Shannon-McMillan-Breiman Theorem. Ann. Probab., Tome 16 (1988) no. 4, pp. 899-909. http://gdmltest.u-ga.fr/item/1176991794/