Let $\mu$ be a probability measure on a Riemannian manifold. It is known that if the semigroup $e^{-t\nabla *\nabla}$ is hypercontractive, then any function $g$ for which $\|\nabla g\|_{\infty}\leq 1$ will satisfy a Herbst inequality, $\int \exp (\alpha g^{2})d\mu < \infty$, for small $\alpha > 0$. If the semigroup is supercontractive, then the above inequality will hold for all $\alpha > 0$. For any $\alpha > 0$ for which $Z = \int \exp(\alpha g^{2})d\mu < \infty$, we define a measure $\mu _{g}$ by $d\mu _{g}=Z^{-1}\exp (\alpha g^{2})d\mu$. We show that if $\mu$ is hyper- or supercontractive, then so is $\mu _{g}$. Moreover, under standard conditions on logarithmic Sobolev inequalities which yield ultracontractivity of the semigroup, Gross and Rothaus have shown that $Z = \int \exp (\alpha g^{2}|\log |g||^{c})d\mu < \infty$ for some constants $\alpha ,c$. We in addition show that the perturbed measure $d\mu _{g} = Z^{-1} \exp (\alpha g^{2}|\log |g||^{c})d\mu$ is ultracontractive.