We propose a new analytic formula for evaluating the derivatives of a matrix exponential. In contrast to the diagonalization method, eigenvalues and eigenvectors do not appear explicitly in the derivation, although we show that a necessary and sufficient condition for the validity of the formula is that the matrix has distinct eigenvalues. The new formula expresses the derivatives of a matrix exponential in terms of minors, polynomials, the exponential of the matrix as well as matrix inversion, and hence is algebraically more manageable. For sparse matrices, the formula can be further simplified. Two examples are discussed in some detail. For the companion matrix of a continuous-time autoregress\-ive moving average process, the derivatives of the exponential of the companion matrix can be computed recursively. The second example concerns the exponential of the tri\-diagonal transition intensity matrix of a finite-state-space continuous-time Markov chain whose instantaneous transitions must be between adjacent states. We present a numerical study to show that the new method may yield numerically more accurate results than the diagonalization method, at the expense of a slight increase in computation.