Building on work of Davenport and Schmidt, we mainly prove two results. The first one is a version of Gel'fond's transcendence criterion which provides a sufficient condition for a complex or $p$-adic number $\xi$ to be algebraic in terms of the existence of polynomials of bounded degree taking small values at $\xi$ together with most of their derivatives. The second one, which follows from this criterion by an argument of duality, is a result of simultaneous approximation by conjugate algebraic integers for a fixed number $\xi$ that is either transcendental or algebraic of sufficiently large degree. We also present several constructions showing that these results are essentially optimal.