This paper shows that various relevant dynamical systems can be described as vector fields associated to smooth functions via a bracket that defines what we call a Leibniz structure. We show that gradient flows, some dissipative systems, and nonholonomically constrained simple mechanical systems, among other dynamical behaviors, can be described using this mathematical construction that generalizes the standard Poisson bracket currently used in Hamiltonian mechanics. The symmetries of these systems and the associated reduction procedures are described in detail. A number of examples illustrate the theoretical developments in the paper.