Perspectives

When Uncertainty Rises, Your Model Faces Higher Failure Risk

Prediction is difficult, Nils Bohr famously warned — especially about the future.

The caveat needs no explanation in economics and finance, although a bit of clarification is required, starting with the obvious.

If you spend more than 20 minutes studying the history of forecasting that routinely populates the world of macro and markets, it’s blazingly apparent that true sages are as common as hail storms in the Sahara.

That’s a potent reason to steer clear of forecasts, right? Well, yes… sort of.  In reality, most investors don’t have the luxury to shun the prediction game entirely.  Investing and portfolio management, after all, inherently involve predictions.  Even a buy-and-hold approach carries an inherent forecast that future returns will resemble the historical record — if you wait long enough.

The goal, then, is not to mindlessly ignore forecasting.  Instead, an enlightened investor will selectively favor models that appear to dispense some degree of value.

That’s still a high bar, yet some models offer a small edge over the crowd’s collective wisdom, when results are compared over several business cycles.  The dividend discount model, for instance, has proven to be useful for estimating future equity returns, particularly when valuations are stretched to extreme.

But every model – even the relatively reliable ones – are prone to failure at times. Knowing when models are vulnerable to higher-than-average failure rates would be enormously valuable, which is why a recent study on the effects of uncertainty in money management is so intriguing.

Risk vs. Uncertainty

The first challenge in this research niche is quantifying uncertainty, which is no mean feat. Critics will say, and not without merit, that’s it’s a fool’s errand to even try. That may be too harsh, but it’s a point worth considering, if only to keep us humble.

Meantime, should you agree to proceed down this thorny path, let’s not be fooled into confusing uncertainty with risk. Frank Knight famously distinguished between the two in his 1921 book, “Risk, Uncertainty, and Profit.” As he persuasively outlined, risk applies to those conditions where the outcome is unknown but the probabilities can be measured and estimated with some degree of certainty. In sharp distinction, uncertainty describes those situations where relevant data is limited (if not missing altogether) and so calculating probabilities is virtually impossible. 

We can reasonably project the distribution of returns for the stock market over the next 20 years, which is to say that we can model the risk of performance variation with a high degree of confidence. Deciding what next year’s distribution will be, on the other hand, is uncertain.

What’s The Connection?

Rumpa Biswas, a researcher at the University of New Orleans, has crunched the numbers on the link between economic policy uncertainty and forecast errors.  She finds that the higher the ambiguity, the greater the potential for inaccuracy. Surprising? Not really, but documenting the relationship with data serves as a reminder that model failure, like financial markets and the business cycle, is dynamic.

Although Ms. Biswas focuses on corporate earnings estimates to illustrate her thesis, there are broader implications for studying the linkage between macro and markets. The thousand-mile journey of mapping this landscape, however, can start with earnings forecasts.

The “average forecast error becomes higher during periods of high uncertainty,” Biswas observes in “Does Economic Policy Uncertainty Affect Analyst Forecast Accuracy?”, a January 2019 paper.

This isn’t exactly news, either to the investment practitioner or the academic. Indeed, Biswas cites several predecessor studies that have informed her research. Dzielinski (2012), for instance, finds that a spike in economic uncertainty, based on the frequency of internet searches, is related to low equity market returns. Pástor and Veronesi (2012) demonstrate that economic policy changes – a proxy for elevated uncertainty – are related to lower stock prices.

Quantifying Policy Uncertainty: A First Approximation

Knightian uncertainty is a considerable challenge in the quest for measuring the black hole of the future, but a partial solution can arguably be found in a project run by a group of economists via the Economic Policy Uncertainty Index (EPU). The methodology focuses on U.S. “policy-related economic uncertainty,” based on three sources: news coverage of policy-related uncertainty; tax code expiration data; and the dispersion of economic projections via the Federal Reserve Bank of Philadelphia’s Survey of Professional Forecasters.

Biswas taps into EPU to model the connection between forecasting accuracy and the level of future ambiguity. She posits, “When a high level of EPU affects a firm’s performance and earnings adversely, analysts (as outsiders) find difficulty in anticipating the firm’s earnings before the firm releases its earnings reports”.

Additionally, during the periods of high uncertainty, fluctuations in overall investment levels and stock markets’ performances create another level of difficulty for the analysts. When analysts do not get enough or reliable information that are required to predict firms’ performances, prospects for making less accurate earnings forecasts increases. Thus, average forecast error becomes higher during periods of high uncertainty.

The study’s results, she concludes, offer “evidence that the economic policy uncertainty has a negative effect on the analyst forecast accuracy, and this effect persists for 13 months in the future”.

The question is whether uncertainty by way of EPU data can be extended to other applications for modeling economic conditions and financial markets?

A Simple Test

As a preliminary experiment, consider how EPU stacks up against a measure of financial stress in the US (Kansas City Financial Stress Index). When the two are in alignment, there may higher-than-usual information in the data sets for estimating the near-term economic future.

The first step is transforming the two time series to Z-scores to create a level playing field for easy visual comparison (a necessary task because each data set is on a vastly different scale). The resulting chart below shows the standard deviations for the full sample.

Not surprisingly, both indicators show an upside bias (readings above 0) ahead of and during the last three US economic recessions (gray bars). By contrast, a spike in one index without a commensurate and convincing rise in the other appears to be a head fake for confidently calling the start of a new contraction.

For instance, financial stress (blue line) spiked several years ahead of the 2001 downturn (middle gray bar). Initially, EPU followed suit (red line), but quickly retreated. Soon after, EPU spiked again and this time it remained elevated, confirming the ongoing rise in financial stress. Just around that time, a US recession kicked in.

For the other two recessionary periods in the chart above, financial stress and policy uncertainty surged together and generally remained elevated.

The implication: a sharp jump in financial stress reflects a relatively convincing case of danger ahead. But until and unless it’s accompanied by heightened policy uncertainty, the odds appear to be low that economic growth will hit a wall.

By that reasoning, conditions in May 2019 were mixed. Policy uncertainty was slightly elevated, but financial stress was low. As such, the case remained weak for expecting a new recession, as of that date.

The modeling above is intentionally simple, intended to illustrate a concept rather than deliver robust analysis. But even a toy example suggests that policy uncertainty via EPU opens an analytical door with many possibilities for modeling markets and macro. In fact, the Economic Policy Uncertainty project’s data sets span the world and target several flavors of risk as well as policy uncertainty. This is far from the final word on quantifying the unquantifiable, but it’s a reasonable first approximation and is certainly worthy of deeper research.

At the center of the subject is a simple question? How does causality flow? From markets and macro to uncertainty, or vice versa? Unclear. But even a little bit of insight could yield huge benefits.

To the extent that there’s an answer (or at least informed guidance), the EPU data sets offer a path for exploration. It may be a dead end, of course, which wouldn’t be surprising, given the nature of uncertainty. But perhaps that’s overly pessimistic, as Biswas’ paper and related research imply. Uncertainty is a slippery beast, but it may not be entirely resistant to some level of quantification.              

By James Picerno, Director of Analytics

IMPORTANT DISCLOSURES:  PLEASE REMEMBER THAT PAST PERFORMANCE MAY NOT BE INDICATIVE OF FUTURE RESULTS.  DIFFERENT TYPES OF INVESTMENTS INVOLVE VARYING DEGREES OF RISK, AND THERE CAN BE NO ASSURANCE THAT THE FUTURE PERFORMANCE OF ANY SPECIFIC INVESTMENT, INVESTMENT STRATEGY, OR PRODUCT MADE REFERENCE TO DIRECTLY OR INDIRECTLY FROM THE MILWAUKEE COMPANY™, WILL BE PROFITABLE, EQUAL ANY CORRESPONDING INDICATED HISTORICAL PERFORMANCE LEVEL(S), OR BE SUITABLE FOR YOUR PORTFOLIO.  DUE TO VARIOUS FACTORS, INCLUDING CHANGING MARKET CONDITIONS, THE CONTENT MAY NO LONGER BE REFLECTIVE OF CURRENT OPINIONS OR POSITIONS.  MOREOVER, YOU SHOULD NOT ASSUME THAT ANY DISCUSSION OR INFORMATION CONTAINED IN THE MILWAUKEE COMPANY™ SERVES AS THE RECEIPT OF, OR AS A SUBSTITUTE FOR, PERSONALIZED INVESTMENT ADVICE FROM THE MILWAUKEE COMPANY™

Leave a Reply

Theme by Anders Norén