We determine the action of the Torelli group on the equivariant cohomology of the
space of flat $SL(2, C)$ connections on a closed Riemann surface. We show that the trivial part of
the action contains the equivariant cohomology of the even component of the space of flat $PSL(2, C)$
connections. The non-trivial part consists of the even alternating products of degree two Prym
representations, so that the kernel of the action is precisely the Prym-Torelli group. We compute the
Betti numbers of the ordinary cohomology of the moduli space of flat $SL(2, C)$ connections. Using
results of Cappell-Lee-Miller we show that the Prym-Torelli group, which acts trivially on equivariant
cohomology, acts non-trivially on ordinary cohomology.