We are deriving optimal rank-based tests for the adequacy of a vector autoregressive-moving average (VARMA) model with elliptically contoured innovation density. These tests are based on the ranks of pseudo-Mahalanobis distances and on normed residuals computed from Tyler’s [Ann. Statist. 15 (1987) 234–251] scatter matrix; they generalize the univariate signed rank procedures proposed by Hallin and Puri [J. Multivariate Anal. 39 (1991) 1–29]. Two types of optimality properties are considered, both in the local and asymptotic sense, a la Le Cam: (a) (fixed-score procedures) local asymptotic minimaxity at selected radial densities, and (b) (estimated-score procedures) local asymptotic minimaxity uniform over a class ℱ of radial densities. Contrary to their classical counterparts, based on cross-covariance matrices, these tests remain valid under arbitrary elliptically symmetric innovation densities, including those with infinite variance and heavy-tails. We show that the AREs of our fixed-score procedures, with respect to traditional (Gaussian) methods, are the same as for the tests of randomness proposed in Hallin and Paindaveine [Bernoulli 8 (2002b) 787–815]. The multivariate serial extensions of the classical Chernoff–Savage and Hodges–Lehmann results obtained there thus also hold here; in particular, the van der Waerden versions of our tests are uniformly more powerful than those based on cross-covariances. As for our estimated-score procedures, they are fully adaptive, hence, uniformly optimal over the class of innovation densities satisfying the required technical assumptions.