Consider a system of identical particles moving on the integer lattice with mutual annihilation of any pair of particles which collide. Apart from this interference, all particles move independently according to the same random walk $p$. A system will be called site recurrent if a.s. each site is occupied at arbitrarily large times. The following generalization of a conjecture by Erdos and Ney was open: the system of annihilating simple random walks on $Z_2$, starting with all sites except the origin occupied, is site recurrent. We prove, for general $p$ and a reasonably broad class of initial distributions, that the annihilating system is site recurrent. Loosely speaking, this condition is that the initial configuration does not have any fixed sequence of holes with diameters tending to infinity.