A result from Gromov ensures the existence of a contact structure on any connected non-compact odd dimensional Lie group. But in general such structures are not invariant under left translations of the Lie group. The problem of finding which Lie groups admit a left invariant contact structure (contact Lie groups), is then still open. While Lie groups with left invariant symplectic structures are widely studied by a number of authors (amongst which A. Lichnerowicz; E.B. Vinberg; I.I. Pjateckii-Sapiro; S. G. Gindikin; A. Medina; Ph. Revoy; M. Goze, J. Dorfmeister; K. Nakajima; etc.), contact Lie groups still remain quite unexplored. We perform a 'contactization' method to construct, in every odd dimension, many contact Lie groups with a discrete centre and discuss some applications and consequences of such a construction. We give classification results in low dimensions. In any dimension greater than or equal to 7, there are infinitely many locally non-isomorphic solvable contact Lie groups. We also classify and characterize contact Lie groups having some prescribed Riemannian or semi-Riemannian structure and derive some obstructions results.