This paper shows that various well-known dynamical systems can be described as vector fields associated to smooth functions via a bracket that defines what we call a Leibniz structure. We show that gradient flows, some control and dissipative systems, and non-holonomically constrained simple mechanical systems, among other dynamical behaviors, can be described using this mathematical construction that generalizes the standard Poisson bracket currently used in Hamiltonian mechanics. The symmetries of these systems and the associated reduction procedures are described in detail. A number of examples illustrate the theoretical developments in the paper.