2.4 Sampling error: what you can and cannot estimate

Motivation

A backtest reports a mean return, a volatility, a skewness and a Sharpe ratio. These estimates have very different precision. Volatility is estimated well from a year of data. The mean is barely estimated from twenty. A trader who does not know this will size positions on noise.

Estimating the mean#

Setup

Let $r_1, \dots, r_n$ be iid observations sampled at interval $\Delta$ (in years; $\Delta = 1/252$ for daily) over a span $T = n\Delta$ years, with per-year mean $\mu$ and per-year volatility $\sigma$ , so each observation has mean $\mu\Delta$ and variance $\sigma^2\Delta$ . The annual Sharpe ratio is $S = (\mu - r_f)/\sigma$ ; in this subsection set $r_f = 0$ or read $\mu$ as an excess return.

Proposition 2.9the mean depends only on the span

The estimator $\hat\mu = \frac{1}{n\Delta}\sum_{t=1}^{n} r_t$ satisfies

\hat\mu = \frac{\ln P_T - \ln P_0}{T}, \qquad \operatorname{sd}(\hat\mu) = \frac{\sigma}{\sqrt{T}}, \qquad t\text{-stat} = \frac{\hat\mu}{\sigma/\sqrt{T}} \approx S\sqrt{T}. \tag{2.13}

Consequently the span needed for an expected $t$ -statistic of $t_0$ is $T = (t_0/S)^2$ ; for $t_0 = 2$ ,

T \approx \frac{4}{S^2} \text{ years.} \tag{2.14}

Proof

Log returns telescope: $\sum_t r_t = \ln P_T - \ln P_0$ . $\operatorname{Var}(\hat\mu) = n\sigma^2\Delta/(n\Delta)^2 = \sigma^2/T$ . The $t$ -statistic with true parameters is $\mu\sqrt{T}/\sigma = S\sqrt{T}$ ; set it equal to $t_0$ and solve.∎

Sampling more often does not help: the daily, weekly and monthly estimates of $\mu$ are the same number (Merton 1980). By contrast, the variance estimator $\hat\sigma^2 = \frac{1}{(n-1)\Delta}\sum (r_t - \bar r)^2$ has, under normality, $\operatorname{Var}(\hat\sigma^2) \approx 2\sigma^4/n$ , so its relative precision improves with $n$ for a fixed span. Ten years of daily data give $\operatorname{sd}(\hat\sigma)/\sigma \approx 1/\sqrt{2n} = 1/\sqrt{5040} \approx 1.4\%$ (worse with fat tails; see Problem 7), while $\operatorname{sd}(\hat\mu) = \sigma/\sqrt{10} = 6.3\%$ a year at $\sigma = 20\%$ .

Annual Sharpe $S$	0.2	0.3	0.43	0.5	1.0	2.0
Years for $t = 2$	100	44	21.6	16	4	1

Worked example 2.4.1Nifty sample

Over 19.13 years the Nifty price index had $\hat\mu = 8.75\%$ and $\hat\sigma = 20.2\%$ , so $t = 0.0875/(0.202/\sqrt{19.13}) = 1.89$ . Nineteen years of data on India's benchmark index do not establish, at the 5% level, that its price-only log return is positive, and that is before subtracting a risk-free rate of several percent. A strategy backtest of 3 years with a reported Sharpe of 1 has $t = \sqrt3 = 1.7$ ; the same arithmetic applies. M04 adds the multiple-testing correction, which makes this worse.

Skewness, kurtosis and normality#

Proposition 2.10skewness and kurtosis under normality

Let $r_t$ be iid $N(\mu, \sigma^2)$ and define sample central moments $m_k = \frac1n\sum (r_t - \bar r)^k$ , skewness $\hat S = m_3/m_2^{3/2}$ , excess kurtosis $\hat K = m_4/m_2^2 - 3$ . Then

\sqrt{n}\,\hat S \Rightarrow N(0, 6), \qquad \sqrt{n}\,\hat K \Rightarrow N(0, 24), \qquad \text{asymptotically independent.} \tag{2.15}

Proof

Standardise so $\mu = 0$ , $\sigma = 1$ ( $\hat S$ , $\hat K$ are location-scale invariant). Write $\bar x = \bar r$ and $\mu_k' = \frac1n \sum r_t^k$ . Expanding, $m_2 = \mu_2' - \bar x^2$ , $m_3 = \mu_3' - 3\bar x\mu_2' + 2\bar x^3$ , $m_4 = \mu_4' - 4\bar x\mu_3' + 6\bar x^2\mu_2' - 3\bar x^4$ . Each $\mu_k' - E r^k$ and $\bar x$ is $O_p(n^{-1/2})$ , and $E r^3 = 0$ , so to first order $m_2 = 1 + (\mu_2' - 1)$ , $m_3 = \mu_3' - 3\bar x$ , $m_4 = 3 + (\mu_4' - 3)$ , each with $o_p(n^{-1/2})$ error. Linearise $g(m_2, m_3) = m_3 m_2^{-3/2}$ at $(1, 0)$ : $\partial g/\partial m_3 = 1$ , $\partial g/\partial m_2 = -\tfrac32 m_3 = 0$ . So $\hat S = m_3 + o_p(n^{-1/2}) = \frac1n\sum (r_t^3 - 3r_t) + o_p(n^{-1/2})$ . Linearise $h(m_2, m_4) = m_4 m_2^{-2} - 3$ at $(1, 3)$ : $\partial h/\partial m_4 = 1$ , $\partial h/\partial m_2 = -2m_4 = -6$ . So $\hat K = (\mu_4' - 3) - 6(\mu_2' - 1) + o_p(n^{-1/2}) = \frac1n\sum (r_t^4 - 6r_t^2 + 3) + o_p(n^{-1/2})$ . The summands are the Hermite polynomials $He_3(x) = x^3 - 3x$ and $He_4(x) = x^4 - 6x^2 + 3$ , which are orthogonal under $N(0,1)$ with $E[He_k^2] = k!$ . Hence by the CLT $\sqrt n \hat S \Rightarrow N(0, 3!) = N(0, 6)$ , $\sqrt n\hat K \Rightarrow N(0, 4!) = N(0, 24)$ , and $\operatorname{Cov}(He_3, He_4) = 0$ gives joint normality with zero covariance, hence independence. Direct check: $E[(x^3 - 3x)^2] = 15 - 18 + 9 = 6$ ; $E[(x^4 - 6x^2 + 3)^2] = E[x^8] - 12E[x^6] + 42E[x^4] - 36E[x^2] + 9 = 105 - 180 + 126 - 36 + 9 = 24$ .∎

Corollary 2.10aJarque–Bera

Under normality $JB = n\left(\frac{\hat S^2}{6} + \frac{\hat K^2}{24}\right) \Rightarrow \chi^2_2$ , whose survival function is $e^{-x/2}$ . (Jarque & Bera 1980.)

Worked example 2.4.2Nifty sample

With $n = 4{,}693$ : $\sqrt{6/n} = 0.036$ and $\sqrt{24/n} = 0.072$ . Observed $\hat S = -0.28$ ( $-7.8$ standard errors) and $\hat K = 15.2$ (212 standard errors). $JB = 4693(0.0778/6 + 230.1/24) = 45{,}050$ , $p \approx e^{-22{,}525} = 0$ to machine precision. Normality is rejected beyond any doubt. But Proposition 2.10 assumes normality, so the "212 standard errors" is itself unreliable. When the tail index is below 4 (2.6), the population kurtosis is infinite and $\hat K$ does not converge. The annual sample kurtosis of Nifty returns ranged from 0.04 (2011) to 12.4 (2020) in this sample: one or two days per year decide it.

Practitioner notes

Report a confidence interval for every Sharpe ratio you see: $\hat S \pm 2/\sqrt{T}$ (annual units, iid approximation; Lo 2002 gives the autocorrelation and non-normality corrections, and M04 · 4.5 derives the probabilistic Sharpe ratio).
Volatility can be forecast and estimated; means essentially cannot. That asymmetry is why professional risk management sizes positions by volatility (M03, M14) rather than by forecast return.
Skewness estimates are dominated by single days. A strategy that sells options shows positive skewness and a high Sharpe ratio until the one day that reverses both (M09).