13.3 Return predictability and out-of-sample $R^2$

Setup#

The predictive regression with a persistent predictor $x_t$ (dividend yield, earnings yield, term spread) is

with $(u_t, v_t)$ iid, mean zero, variances $\sigma_u^2, \sigma_v^2$ and covariance $\sigma_{uv}$. A recursive out-of-sample forecast at $t$ fits (13.7) on data $1..t$ and predicts $\hat r_{t+1} = \hat\alpha_t + \hat\beta_t x_t$. The benchmark is the historical mean $\bar r_t = t^{-1}\sum_{s\le t} r_s$.

$R^2_{OS} > 0$ means the model beat the historical mean in squared error. Unlike in-sample $R^2$, it can be (and usually is) negative, because estimation error in $\hat\beta_t$ adds variance to the forecast.

Evidence

• Welch and Goyal (2008, RFS 21(4):1455–1508) examined the predictors proposed in the literature and concluded that none would have helped a real-world investor outpredict the then-prevailing historical mean out of sample; the models were also unstable.
• Campbell and Thompson (2008, RFS 21(4):1509–1531) replied that many predictors do beat the historical mean once weak restrictions are imposed (the slope must have the theoretically expected sign, and the forecast equity premium must be non-negative). They stated that the out-of-sample explanatory power is small but economically meaningful for mean-variance investors. Proposition 13.7 makes that claim precise.
• Goyal, Welch and Zafirov (2024, RFS 37(11):3490–3557) revisited 29 later variables plus the original 17. They report that more than a third of the new variables no longer show in-sample significance, half of those that do perform poorly out of sample, and a small number perform reasonably in both.

How much is a small $R^2$ worth?#

A mean-variance investor with risk aversion $\gamma$ who uses the forecast (weight $w_t = (\mu+x_t)/(\gamma\sigma_\varepsilon^2)$) earns expected excess return $S^{*2}/\gamma$. One who ignores it (constant weight $\mu/(\gamma(\sigma_x^2+\sigma_\varepsilon^2))$) earns $S^2/\gamma$. The proportional increase is

For a pure timer ($\mu = 0$, so $S=0$), $S^{*2} = R^2/(1-R^2) \approx R^2$.

Proof

$\mathbb E[(\mu+x)^2] = \mu^2 + \sigma_x^2$. Write $\sigma^2 = \sigma_x^2+\sigma_\varepsilon^2$, so $\sigma_\varepsilon^2 = \sigma^2(1-R^2)$ and $\sigma_x^2 = \sigma^2 R^2$. Then $\mu^2/\sigma_\varepsilon^2 = S^2/(1-R^2)$ and $\sigma_x^2/\sigma_\varepsilon^2 = R^2/(1-R^2)$; add. For the investor: the conditional mean and variance of $r_{t+1}$ are $\mu + x_t$ and $\sigma_\varepsilon^2$, so the conditional mean-variance optimum is $w_t$ above, and $\mathbb E_t[w_t r_{t+1}] = (\mu+x_t)^2/(\gamma\sigma_\varepsilon^2)$. Take expectations. The constant-weight investor faces mean $\mu$ and variance $\sigma^2$, giving $\mu^2/(\gamma\sigma^2) = S^2/\gamma$. Then $(S^{*2}-S^2)/S^2 = [S^2 + R^2 - S^2(1-R^2)]/[S^2(1-R^2)] = R^2(1+S^2)/[S^2(1-R^2)]$.∎

$S^{*2}$ is the right measure of what a mean-variance investor gains, but it is not the unconditional Sharpe ratio of the timing strategy's P&L, which also has to pay for the variance of the position size. The next result makes the difference concrete.

Proof

$\mathbb E\pi = \mathbb E[x^2] + \mathbb E[x]\mathbb E[\varepsilon] = \sigma_x^2$. $\mathbb E\pi^2 = \mathbb E[x^2(x+\varepsilon)^2] = \mathbb E x^4 + \mathbb E x^2\,\mathbb E\varepsilon^2 = 3\sigma_x^4 + \sigma_x^2\sigma_\varepsilon^2$, using the Gaussian fourth moment and independence. So $\operatorname{Var}\pi = 2\sigma_x^4 + \sigma_x^2\sigma_\varepsilon^2$ and the squared Sharpe ratio is $\sigma_x^2/(2\sigma_x^2+\sigma_\varepsilon^2)$. Divide numerator and denominator by $\sigma_x^2+\sigma_\varepsilon^2$.∎

A simulation with $R^2 = 0.05$ and $4\times10^6$ draws gives a squared Sharpe of 0.0477 against $R^2/(1+R^2) = 0.0476$ and $R^2/(1-R^2) = 0.0526$. Both expressions are $R^2 + O(R^4)$, so for realistic $R^2$ the rule of thumb is simply per-period Sharpe ratio ≈ √R².

Worked example: what 0.5% buys#

Take a broad equity index with a monthly Sharpe ratio of $S = 0.15$ (about $0.15\sqrt{12} = 0.52$ a year; an assumption for illustration, not a Nifty estimate). A predictor with monthly $R^2_{OS} = 0.5\%$ gives, by (13.9),

By (13.10) the mean-variance investor's expected excess return rises by $0.005\cdot1.0225/(0.0225\cdot0.995) = 22.8\%$ (the approximation $R^2/S^2$ gives 22.2%). With $R^2 = 1\%$ the rise is 45.9%. A pure timer with $R^2 = 0.5\%$ has $S^* = \sqrt{0.005/0.995} = 0.0709$ a month, or 0.246 a year, before costs.

Turn this around. A daily strategy with an annualised Sharpe ratio of 1.0 has a daily Sharpe of $1/\sqrt{252} = 0.063$, so it corresponds to a daily $R^2$ of about $0.063^2 = 0.40\%$. An $R^2$ indistinguishable from zero is what a good trading signal looks like. This is why OOS $R^2$ is the correct yardstick and why in-sample $R^2$ of 5% on daily data should make you suspicious, not excited.

Stambaugh bias#

Predictors like the dividend yield are highly persistent ($\rho$ near 1), and their innovations are strongly negatively correlated with return innovations: when prices jump, the yield falls. This creates a finite-sample bias in $\hat\beta$.

Proof

Project $u$ on $v$: $u_{t+1} = \gamma v_{t+1} + e_{t+1}$, where $e$ is uncorrelated with $v$ at all leads and lags and therefore (under joint normality, independent of) the whole path of $x$. Let $\tilde x_t = x_t - \bar x$ be demeaned regressors over the estimation sample. Both regressions in (13.7) use the same regressor, so

$$\hat\beta - \beta = \frac{\sum \tilde x_t u_{t+1}}{\sum\tilde x_t^2} = \gamma\frac{\sum\tilde x_t v_{t+1}}{\sum\tilde x_t^2} + \frac{\sum\tilde x_t e_{t+1}}{\sum\tilde x_t^2} = \gamma(\hat\rho - \rho) + \frac{\sum\tilde x_t e_{t+1}}{\sum\tilde x_t^2}.$$

Conditional on the path of $x$, the last term has mean zero. Take expectations. The approximation uses Kendall's (1954) result for the AR(1) with estimated intercept, $\mathbb E[\hat\rho - \rho] \approx -(1+3\rho)/T$.∎

Since $\sigma_{uv} < 0$ for valuation ratios, $\gamma < 0$ and the bias in $\hat\beta$ is positive: OLS overstates predictability exactly in the direction researchers hope for.