It is well known that the ring of polynomial invariants of a finite matrix group {\small $G$} becomes more and more messy the further one moves away from groups generated by pseudoreflections. Also, the number of generators has a tendency to become large. In this experimental study, we try to create evidence for the observation that these unpleasant properties might improve if one enlarges the ring of invariants in one way or another. For instance, {\small $G$} might fix a symplectic form. It can be used to turn the ring of invariants into a Lie algebra by introducing a {\small $G$}-invariant Poisson bracket. In the absence of an invariant symplectic form, one might still consider the Lie algebra of invariant polynomial vector fields simultaneously with the ring of polynomial invariants. If one is prepared to leave the realm of commutative rings and Lie algebras, one can also take {\small $G$}-invariant differential operators into account. Under these additional operations the number of generators necessary to create all invariants is often drastically decreased. A particularly nice situation arises if the group fixes a quadratic form and a symplectic form at the same time, because together they give rise to an endomorphism on the space of homogeneous invariants of any given degree that, for instance, can be used to single out more effective generators in the new sense. In the classical situation of fields of characteristic zero, to which this paper is restricted, the averaging operator is both theoretically and algorithmically an important tool, whose computational feasibility, however, decreases with increasing degrees. The methods presented here might also help to avoid this difficulty.