Few aspects of risk management come with higher stakes than estimating tail risk.[1] Just as a chain is no stronger than its weakest link, an investment strategy will be judged (at least in part) by the depth of its biggest losses. Unfortunately, tail risk is one of the toughest challenges for investment analysts for a simple reason: the supply of data for studying extreme events, by definition, is scarce.

In some cases, there may be just enough historical data to offer a hint of how returns behave at the outer edges of worst-case scenarios. The deepest 1% losses, for example, are a rare breed in terms of data points in a given track record. That’s well short of what’s necessary for developing a robust estimate of tail risk.

The challenge is made even more daunting for testing new portfolio management strategies. A backtest can help, but it’s only one path of many that can be taken. In fact, it’s fair to say that drawing conclusions from the past (either through historical numbers or a backtest) is a blunt tool at best for developing context on how much damage the next black swan event[2] will unleash on an otherwise well-behaved strategy. Overlooking this issue can get you into all kinds of trouble.

## Searching for Extremes

A recent study demonstrates that a poorly thought-out backtest can be deeply misleading. “Many investment strategies such as risk-premia strategies are (back-)tested using historical data,” note Enoch Cheng (University of Colorado, Denver) and Clemens Struck (University College Dublin) in “Time-Series Momentum: A Monte Carlo Approach.” That’s an obvious first step, but it shouldn’t be the last. Why? Stuff happens, as they say.

The research literature (along with common sense) “emphasize the importance of tail events for understanding risk-premia,” the authors remind. But there are several traps lying in wait when navigating this analytical path. Historical data, they advise, “contain few tail events, making it difficult for a backtest to assess i) the true risks involved in a strategy and ii) the chances of a strategy to outperform a benchmark in the long-term.”

What’s a prudent risk manager to do? Analysts have devised numerous techniques for evaluating worst-case scenarios with econometric tools in recent decades. But quantity

doesn’t automatically confer quality and so the degree of reliability can – and usually does -vary widely, depending on the methodology.

Generating simulated data with a bootstrapping procedure is a common econometric modeling technique.[3] There are many cases when conventional bootstrapping works fine, but estimating tail risk isn’t one, Cheng and Struck explain.

Existing bootstrapping procedures are not suitable for our analysis as they i) re-employ the same extreme residuals again and again [and] ii) do not simultaneously preserve both the time-series and cross-sectional dependencies of returns.

The issue boils down to the reality that “while bootstrapping can replicate past historic tail events, it cannot generate a sufficiently large variety of possible *future* tail events to robustly examine a strategy.”

## A Better Way to Estimate Tail Risk

In search of a solution, Cheng and Struck introduce an innovative Monte Carlo procedure to address the drawbacks that afflict bootstrapping consisting of two components. The first is simulating residuals (differences between observed and predicted values) via bootstrapping rather than focusing on returns. The second employs “a combination of time-series models and copulas” to preserve the return dependencies.

What’s the practical application? The paper reviews an example in a set of charts that highlight the analysis of a Long-Short Time-Series Momentum strategy (LS-TSM) vs. a Buy-and-Hold strategy (BH) during 1985-2009 for the S&P 500 Index.[4] The first chart (A. S&P 500 1985.1-2009.12 Cumulative Returns) shows a standard backtest based on historical data. The momentum strategy outperforms, although Cheng and Struck note that “tail events are crucial to understand this outperformance.” In particular, “without the severe market decline around month 200 (during a recession in the early 2000s), the strategy underperforms BH in the overall sample.”

The second chart presented below (C. Generic Cumulative Returns (Model 8), Path #1335) reflects one of 10,000 simulated paths using the authors’ methodology. In this case (path number 1335), the LS-TSM trails the buy-and-hold results by a substantial degree. In fact, the paper advises that the momentum strategy “underperforms BH in more than 84% of the 10,000 simulated paths,” which contrasts with a bias for LS-TSM outperformance via basic backtesting procedures and standard Monte Carlo simulations.

The third chart (E. Generic Cumulative Returns (Model 8), Path #1335) zooms in on a single simulation and reveals a “hidden risk” that’s exposed “during an innocently looking sideways movement of the market”—a period when “LS-TSM underperforms by almost 50% in 25 months.”

The key takeaway: tail-risk analysis via simulations is essential for stress testing a strategy, but analysts should be aware of a crucial caveat. The results are highly dependent on the reliability of the simulation model. Standard bootstrapping methods usually fall short, which means that alternative solutions are necessary. Cheng and Struck suggest one way forward for minimizing the uncertainty that can impair, if not invalidate, some tail-risk estimates.

[1] Tail risk is the risk of portfolio losses that would occur in a worst-case scenario. More specifically, it refers to a portfolio underperforming by more than three standard deviations from its mean.

[2] A black swan refers to very unlikely and severe event capable of having very negative repercussions for the financial markets and/or the economy.

[3] Bootstrapping is a statistical technique that falls under the broader heading of resampling. More specifically, it is a type of resampling where large numbers of smaller samples of the same size are repeatedly drawn, with replacement, from a single original sample.

[4] The authors define LS-TSM: “Long-Short Time-Series Momentum strategy [uses] a 9 months lookback window on the S&P 500. Such a strategy shorts the S&P 500 if the cumulative return of the past 9 months is negative and buys the S&P 500 if the cumulative return of the past 9 months is positive. A rebalancing is conducted at the last day of each month.”

*By James Picerno, Director of Analytics*

__IMPORTANT DISCLOSURES__: PLEASE REMEMBER THAT PAST PERFORMANCE MAY NOT BE INDICATIVE OF FUTURE RESULTS. DIFFERENT TYPES OF INVESTMENTS INVOLVE VARYING DEGREES OF RISK, AND THERE CAN BE NO ASSURANCE THAT THE FUTURE PERFORMANCE OF ANY SPECIFIC INVESTMENT, INVESTMENT STRATEGY, OR PRODUCT MADE REFERENCE TO DIRECTLY OR INDIRECTLY FROM THE MILWAUKEE COMPANY™, WILL BE PROFITABLE, EQUAL ANY CORRESPONDING INDICATED HISTORICAL PERFORMANCE LEVEL(S), OR BE SUITABLE FOR YOUR PORTFOLIO. DUE TO VARIOUS FACTORS, INCLUDING CHANGING MARKET CONDITIONS, THE CONTENT MAY NO LONGER BE REFLECTIVE OF CURRENT OPINIONS OR POSITIONS. MOREOVER, YOU SHOULD NOT ASSUME THAT ANY DISCUSSION OR INFORMATION CONTAINED IN THE MILWAUKEE COMPANY™ SERVES AS THE RECEIPT OF, OR AS A SUBSTITUTE FOR, PERSONALIZED INVESTMENT ADVICE FROM THE MILWAUKEE COMPANY™