In a recent paper R. M. Dudley [2] obtained very interesting results on the speed of mean Glivenko-Cantelli convergence where the underlying random variables are supposed to be independent identically distributed (i.i.d.) taking their values in a separable metric space. In the present paper we shall show that his method of proof also applies for the case of Markov processes with stationary transition probabilities fulfilling Doeblin's condition $(D_0)$. The main additional tool in the treatment of this more general case is the determination of an appropriate upper estimate for the variance of the empirical $p$-measure for the transitions performed in a given set (Lemma 3.2 and Corollary 3.3 (i)) using a mixing property (Lemma 3.1). As indicated by Dudley, the results obtained in this way are applicable to problems in testing statistical hypotheses. An application concerning the speed of convergence of asymptotically normal estimates for Markov processes will be given in a separate paper [3].