Reversible nearest particle systems are certain one-dimensional interacting particle systems whose transition rates are determined by a probability density $\beta(n)$ with finite mean on the positive integers. The reversible measure for such a system is the distribution $\nu$ of the stationary renewal process corresponding to this density. In an earlier paper, we proved under certain regularity conditions that the system converges exponentially rapidly in $L_2(\nu)$ if and only if the system is supercritical. This in turn is equivalent to $\beta(n)$ having exponential tails. In this paper, we consider the critical case, and give moment conditions on $\beta(n)$ which are separately necessary and sufficient for the convergence of the process in $L_2(\nu)$ at a specified algebraic rate. In order to do so, we develop conditions on the generator of a general Markov process which correspond to algebraic $L_2$ convergence of the process. The use of these conditions is also illustrated in the context of birth and death chains on the positive integers.