13.2 Linear time-series models: ARMA

Motivation

ARMA models are the baseline forecasters for any stationary series. For index returns they forecast almost nothing, and knowing why is the point. For spreads, basis and volatility proxies they forecast a great deal, and the same algebra gives half-lives of mean reversion.

Setup#

A process $\{X_t\}$ is weakly stationary if $\mathbb E X_t = \mu$ and $\operatorname{Cov}(X_t, X_{t-k}) = \gamma_k$ do not depend on $t$ . Its autocorrelation is $\rho_k = \gamma_k/\gamma_0$ . White noise $\{e_t\}$ is uncorrelated with mean 0 and variance $\sigma^2$ . The lag operator is $L X_t = X_{t-1}$ .

An AR(1) is

X_t = c + \phi X_{t-1} + e_t. \tag{13.3}

Proposition 13.3AR(1)

If $|\phi| < 1$ , (13.3) has a unique weakly stationary solution

X_t = \mu + \sum_{j=0}^{\infty} \phi^j e_{t-j}, \qquad \mu = \frac{c}{1-\phi}, \tag{13.4}

with $\gamma_0 = \sigma^2/(1-\phi^2)$ and $\rho_k = \phi^{|k|}$ . If $|\phi| \ge 1$ there is no stationary causal solution.

Proof

Iterate (13.3) $k$ times: $X_t = c\sum_{j=0}^{k-1}\phi^j + \phi^k X_{t-k} + \sum_{j=0}^{k-1}\phi^j e_{t-j}$ . If $|\phi|<1$ and $X$ is stationary, $\mathbb E[(\phi^k X_{t-k})^2] = \phi^{2k}(\gamma_0 + \mu^2) \to 0$ , and the partial sums of $\sum\phi^j e_{t-j}$ are Cauchy in $L^2$ because $\sum \phi^{2j} < \infty$ . Letting $k\to\infty$ gives (13.4). Then $\gamma_0 = \sigma^2\sum_j \phi^{2j} = \sigma^2/(1-\phi^2)$ and $\gamma_k = \sigma^2\sum_j \phi^j\phi^{j+k} = \phi^k\gamma_0$ . If $|\phi| \ge 1$ , a stationary solution would need $\operatorname{Var}(X_t) = \phi^2 \operatorname{Var}(X_{t-1}) + \sigma^2$ , i.e. $\gamma_0(1-\phi^2) = \sigma^2 > 0$ , impossible for $\phi^2 \ge 1$ .∎

An AR( $p$ ) is $\Phi(L) X_t = c + e_t$ with $\Phi(z) = 1 - \phi_1 z - \dots - \phi_p z^p$ .

Proposition 13.4AR( $p$ ) stationarity

The AR( $p$ ) has a stationary causal solution $X_t = \mu + \sum_{j\ge0}\psi_j e_{t-j}$ with $\sum|\psi_j|<\infty$ iff all roots of $\Phi(z) = 0$ lie outside the unit circle; equivalently, all eigenvalues of the companion matrix

F = \begin{pmatrix} \phi_1 & \phi_2 & \cdots & \phi_p \\ 1 & 0 & \cdots & 0 \\ \vdots & \ddots & & \vdots \\ 0 & \cdots & 1 & 0 \end{pmatrix}

lie strictly inside it.

Sketch

Factor $\Phi(z) = \prod_{i=1}^p (1 - \lambda_i z)$ , where the $\lambda_i$ are the reciprocals of the roots and are also the eigenvalues of $F$ (expand $\det(\lambda I - F) = \lambda^p - \phi_1\lambda^{p-1} - \dots - \phi_p = \lambda^p \Phi(1/\lambda)$ ). Each factor $(1-\lambda_i L)$ is invertible with absolutely summable inverse $\sum_j \lambda_i^j L^j$ iff $|\lambda_i|<1$ , by Proposition 13.3. The product of absolutely summable filters is absolutely summable. The converse follows as in Proposition 13.3 applied to the state vector $(X_t, \dots, X_{t-p+1})'$ , which evolves as $\xi_t = F\xi_{t-1} + (e_t, 0, \dots, 0)'$ . Hamilton (1994, ch. 1 and 3.4) has the full argument.∎

For an MA( $q$ ), $X_t = \mu + \Theta(L)e_t$ is always stationary. It is invertible (expressible as an AR( $\infty$ ) in past $X$ ) iff the roots of $\Theta(z)$ lie outside the unit circle; this makes the forecast a function of observed data. An ARMA( $p,q$ ) combines both, and the conditions apply to $\Phi$ and $\Theta$ separately.

Proposition 13.5Yule–Walker

For a stationary AR( $p$ ) with $\mu = 0$ ,

\rho_k = \sum_{j=1}^p \phi_j \rho_{k-j}, \quad k \ge 1, \qquad \sigma^2 = \gamma_0\Big(1 - \sum_{j=1}^p \phi_j\rho_j\Big). \tag{13.5}

In matrix form, $R\boldsymbol\phi = \boldsymbol\rho$ with $R_{ij} = \rho_{|i-j|}$ , $i,j = 1..p$ , and $\boldsymbol\rho = (\rho_1, \dots, \rho_p)'$ .

Proof

Multiply $X_t = \sum_j \phi_j X_{t-j} + e_t$ by $X_{t-k}$ , $k\ge1$ , and take expectations. Causality means $X_{t-k}$ is a function of $e_{t-k}, e_{t-k-1},\dots$ , so $\mathbb E[e_t X_{t-k}] = 0$ . This gives $\gamma_k = \sum_j\phi_j\gamma_{k-j}$ ; divide by $\gamma_0$ . For $k=0$ : $\gamma_0 = \sum_j \phi_j\gamma_j + \mathbb E[e_t X_t] = \sum_j\phi_j\gamma_j + \sigma^2$ , since $\mathbb E[e_tX_t] = \mathbb E[e_t^2]$ . Rearrange.∎

The Yule–Walker estimator replaces $\rho_k$ by sample autocorrelations and solves the linear system. The second equation shows that the population one-step $R^2$ of an AR( $p$ ) is $1 - \sigma^2/\gamma_0 = \sum_j \phi_j \rho_j$ . For an AR(1), $R^2 = \phi^2$ .

Proposition 13.6AR(1) forecasts

For (13.3) with $|\phi|<1$ , the minimum-MSE $h$ -step forecast and its error variance are

\mathbb E_t[X_{t+h}] = \mu + \phi^h (X_t - \mu), \qquad \operatorname{Var}_t(X_{t+h}) = \sigma^2\,\frac{1-\phi^{2h}}{1-\phi^2}. \tag{13.6}

Proof

Iterate: $X_{t+h} - \mu = \phi^h(X_t - \mu) + \sum_{j=0}^{h-1}\phi^j e_{t+h-j}$ . The first term is $\mathcal F_t$ -measurable; the sum has mean zero given $\mathcal F_t$ and variance $\sigma^2\sum_{j=0}^{h-1}\phi^{2j}$ . The conditional mean minimises MSE among $\mathcal F_t$ -measurable forecasts.∎

The deviation from the mean halves after $h_{1/2} = \ln 0.5/\ln\phi$ periods. This is the half-life used for mean-reverting spreads (M10 (10.19); for an OU spread, M13A · 13A.7 gives the continuous-time version $\ln 2/\kappa$ and the precision of its estimate; Chan, Algorithmic Trading, ch. 2) and, with $\phi$ replaced by $\alpha+\beta$ , for GARCH variance (13.4).

Worked example: Yule–Walker on daily index returns#

Suppose the sample autocorrelations of a large-cap Indian index's daily returns are $\hat\rho_1 = 0.06$ and $\hat\rho_2 = -0.03$ (illustrative values of the size M02 reports for index data; measure your own). Solve

\begin{pmatrix}1 & 0.06\\ 0.06 & 1\end{pmatrix}\begin{pmatrix}\phi_1\\ \phi_2\end{pmatrix} = \begin{pmatrix}0.06\\ -0.03\end{pmatrix}.

The determinant is $1 - 0.0036 = 0.9964$ . So $\phi_1 = (0.06 - 0.06\cdot(-0.03))/0.9964 = 0.0618/0.9964 = 0.06202$ and $\phi_2 = (-0.03 - 0.06\cdot 0.06)/0.9964 = -0.0336/0.9964 = -0.03372$ . The one-step $R^2$ is $\phi_1\rho_1 + \phi_2\rho_2 = 0.003721 + 0.001012 = 0.47\%$ . With 2,500 days of data, the standard error of each $\hat\rho_k$ under the null of no autocorrelation is about $1/\sqrt{2500} = 0.02$ , so $\hat\rho_1 = 0.06$ is "significant" at 3 standard errors. It is also economically almost useless: a forecast with $R^2 = 0.5\%$ of a daily return with 1% volatility has a standard deviation of $\sqrt{0.0047}\times 1\% \approx 0.07\%$ , or 7 basis points. M01 shows that a round trip in Indian intraday equity costs about 8 basis points. The forecast does not cover costs.

Practitioner notes

Positive lag-1 autocorrelation in index returns is partly an artefact of stale prices of less liquid constituents (nonsynchronous trading). An index future, which trades continuously, shows less of it. Fit models to what you can trade.
Where ARMA earns its keep: the futures basis (M07), calendar spreads, pairs spreads, and volatility proxies such as $\ln|r_t|$ or realised variance. These have large $\phi$ and therefore large $R^2$ .
Order selection by AIC/BIC on the full sample, followed by an "out-of-sample" test on the same sample, is a leak. Select within each training window (13.6).