Asymptotic properties of multivariate time series with characteristic roots on the unit circle are considered. For a vector autoregressive moving average (ARMA) process, we derive the limiting distributions of certain statistics which are useful in understanding nonstationary processes. These distributions are derived in a unified manner for all types of characteristic roots and are expressed in terms of stochastic integrals of Brownian motions. For applications, we use the limiting distributions to establish the consistency properties of the ordinary least squares (LS) estimates of various autoregressions of a vector process, e.g., the ordinary, forward and shifted autoregressions. For a purely nonstationary vector ARMA($p, q$) process, the LS estimates are consistent if the order of the fitted autoregression is $p$; for a general ARMA model, the limits of the LS estimates exist, but these estimates can only provide consistent estimates of the nonstationary characteristic roots.