PA is Peano arithmetic. The formula $\operatorname{Interp}_\mathrm{PA}(\alpha, \beta)$ is a formalization of the assertion that the theory $\mathrm{PA} + \alpha$ interprets the theory $\mathrm{PA} + \beta$ (the variables $\alpha$ and $\beta$ are intended to range over codes of sentences of PA). We extend Solovay's modal analysis of the formalized provability predicate of PA, $\mathrm{Pr}_\mathrm{PA}(x)$, to the case of the formalized interpretability relation $\operatorname{Interp}_\mathrm{PA}(x, y)$. The relevant modal logic, in addition to the usual provability operator `$\square$', has a binary operator `$\triangleright$' to be interpreted as the formalized interpretability relation. We give an axiomatization and a decision procedure for the class of those modal formulas that express valid interpretability principles (for every assignment of the atomic modal formulas to sentences of PA). Our results continue to hold if we replace the base theory PA with Zermelo-Fraenkel set theory, but not with Godel-Bernays set theory. This sensitivity to the base theory shows that the language is quite expressive. Our proof uses in an essential way earlier work done by A. Visser, D. de Jongh, and F. Veltman on this problem.