Prediction: The long and the short of it

By Global Priorities Institute @ 2019-11-30T14:32 (+3)

This is a linkpost to https://globalprioritiesinstitute.org/antony-millner-and-daniel-heyen-prediction-the-long-and-the-short-of-it/

Note: We are grateful to audiences at EEA, EAERE, Imperial, Heidelberg, LSE, Montpellier, UCSB, to Leo Simon, Larry Karp, Geoff Heal, and Derek Lemoine for valuable discussions, and to the CCCEP, Grantham Foundation, and the German Research Foundation (DFG) grant HE 7551/1 for support.

Abstract

Commentators often lament forecasters’ inability to provide precise predictions of the long-run behaviour of complex economic and physical systems. Yet their concerns often conflate the presence of substantial long-run uncertainty with the need for long-run predictability; short-run predictions can partially substitute for long-run predictions if decision-makers can adjust their activities over time. So what is the relative importance of short- and long-run predictability? We study this question in a model of rational dynamic adjustment to a changing environment. Even if adjustment costs, discount factors, and long-run uncertainty are large, short-run predictability can be much more important than long-run predictability.

Introduction

Scientific progress over the past four hundred years has rendered a staggering range of phenomena more predictable. Atmospheric scientists forecast the weather, epidemiologists predict the spread of infectious diseases, macroeconomists forecast economic growth, and demographers predict population change. Yet despite many successes, reliable predictions of the long run behaviour of complex social or natural systems often remain elusive (Granger & Jeon, 2007; Palmer & Hagedorn, 2006). Inability to predict the long run is frequently seen as a barrier to effective decision-making, and can be a source of emotional distress and planning inertia (Grupe & Nitschke, 2013). Concomitantly, improving long-run predictability is often a major goal of the scientific communities that produce forecasts. But just how important is it to be able to predict the distant future? Does substantial long-run uncertainty necessarily imply that accurate long-run predictions would be highly valuable? Or can long-run predictions be effectively substituted by short-run forecasts when decisions can be adjusted dynamically as new information arrives? This paper attempts to shed light on these questions.

It is not uncommon to find the presence of long-run uncertainty identified with the need for improved long-run predictions.^[1] For example, a recent report by The National Academy of Sciences (2016) on planned improvements in long-range weather forecasting suggests that ‘Enhancing the capability to forecast environmental conditions outside the well-developed weather timescale – for example, extending predictions out to several weeks and months in advance – could dramatically increase the societal value of environmental predictions, saving lives, protecting property, increasing economic vitality, protecting the environment, and informing policy choices.’ Similarly, many commentators have suggested that the lack of reliable projections of the local impacts of climate change, most of which will occur many decades hence, is a significant barrier to effective adaptation planning. F¨ussel (2007), for example, contends that ‘the effectiveness of pro-active adaptation to climate change often depends on the accuracy of [long run] regional climate and impact projections’. One can find a similar identification of the presence of long-run uncertainty with the importance of long-run predictions in economics. Lindh (2011), for example, states that ‘Very long-run...forecasts of economic growth are required for many purposes in long-term planning. For example, estimates of the sustainability of pension systems need to be based on forecasts reaching several decades into the future.’

While one-shot decisions with fixed lead times between actions and outcomes (e.g. agricultural planting decisions) doubtless benefit from predictability at decision-relevant time-scales, most long-run decision processes are at least partially flexible, and can thus be adjusted over time. Firms or individuals who anticipate long-run changes in market conditions, regulation, or their physical environments will adjust their actions dynamically as new information becomes available. Similarly, governments concerned with policies that depend on conditions in the distant future (e.g. social security or adaptation to climate change) can alter the level of policy instruments dynamically as the future unfolds. The sequential nature of many long-run adjustment processes implies that there is no generic association between the presence of long-run uncertainty and the importance of long-run predictions. Since the long-run today will become the short-run tomorrow, short-run predictions can play an important role in informing decision-making, even when long-run uncertainty is large. Indeed, it is intuitive that short-run predictability is a perfect substitute for long-run predictability if adjustment is costless. In general however adjustment is costly, and large abrupt changes in response to short-run warnings are often significantly more costly than managed gradual transitions that may be informed by accurate long-run predictions. This suggests that long run predictions could play an important role in informing anticipatory planning, and avoiding excessive adjustment costs. It is however unclear a priori how the importance of predictability at different lead times depends on the magnitude of adjustment costs. We develop a simple analytical model in which this question is answerable.

Our model considers a decision-maker whose period payoffs depend on how well adapted her choices are to the current state of the world. The state of the world is uncertain, and may change over time in a non-stationary manner. The decision-maker may adjust her choices in every period to account for expected changes in her environment, but faces convex adjustment costs. This cost structure makes rapid adjustments in response to short-run warnings more costly than gradual incremental shifts of equal magnitude (which may be informed by longrun predictions).^[2] Optimal decisions thus balance the benefits of exploiting current conditions with the need to anticipate future conditions in order to avoid costly rapid adjustments in the future. The decision-maker has access to a prediction system that generates forecasts of all future states in every period. These forecasts have a fixed profile of accuracy as a function of lead time. Thus, if τm is a measure of the accuracy of forecasts of lead time m, the decision-maker receives a forecast of accuracy τ1, τ2, . . . of states of the world 1, 2, . . . time steps from the present in every period. For example, the decision-maker receives a forecast of accuracy τ2 about a state two time steps from now in the current period, but knows that in the next period she will receive a new forecast of the same state, this time with accuracy τ1. She may change her decisions in order to react to new predictions once they become available, but doing so entails a cost. Although the model reduces to a stochastic-dynamic control problem with an infinite number of state variables, we find an analytic expression for the decision-maker’s discounted expected payoffs V as a function of the profile of predictive accuracy that the prediction system exhibits:

V = V (τ1, τ2, τ3, . . .).

By exploring the dependence of V on its arguments, and the parameters of the decision problem, we quantify the value of predictability at different lead times. Our central finding is that if we account for sequential forecast updating and agents’ ability to adjust their activities over time, short-run predictability can be more important than long-run predictability, even if adjustment costs, discount factors, and long-run uncertainty are large.

Although there is a sizeable literature on the value of information and its role in dynamic decision-making, as far as we know there are few direct antecedents to the questions we seek to address in this paper. The literature on the value of information began with the pathbreaking work of Blackwell (1953) and Marschak & Miyasawa (1968), who defined an incomplete ordering of the ‘informativeness’ of arbitrary information structures. We share this work’s micro-oriented focus on the value of exogenous information sources for individual decision-makers, but also differ from it in important respects. In order to ensure tractability, our model makes strong assumptions about the nature of agents’ payoff functions and the forecasts they receive. The return for this specificity is that we are able to study a much richer set of dynamic decisions than is typically used in this literature. Our focus on the dynamic characteristics of predictions, i.e., their accuracy as a function of lead time, is absent from this literature, and necessitates a pared down approach.

Work on the role of information in optimal dynamic decision-making falls into two categories: two period models that examine the effect of second period learning on optimal first period decisions (e.g. Arrow & Fisher, 1974; Epstein, 1980; Gollier et al., 2000), or infinite horizon models that involve learning about the realizations of a stochastic state variable (e.g. Merton, 1971), or a parameter of a structural dynamic-stochastic model (e.g. Ljungqvist & Sargent, 2004). Neither of these standard approaches can capture the effects we study here. Two period models cannot capture the repeatedly updated nature of prediction and the dependence of predictability on lead time, both essential features of our model. Finite horizon models also suffer from an inherent bias towards short-run forecasts, as in a model with horizon H there will be H lead time 1 forecasts, but only one lead time H forecast. On the other hand, models based on familiar stochastic processes, or learning about parameters of structural models, do not allow the accuracy of predictions at different lead times to be controlled independently, meaning that it is impossible to ask questions about the relative importance of short- and long-run predictability (see the discussion on p. 8 for an elaboration of this point). We thus need a different approach if we are to define a model that is tractable, unbiased, disentangles lead times, and nevertheless retains coarse features of dynamic prediction.

A small applied literature studies the effect of forecasts at different lead times on dynamic decision-making. Costello et al. (2001) study a finite horizon stochastic renewable resource model and show that forecasts of shocks more than one step ahead carry no value for a resource manager. This result follows directly from the fact that their model is linear in the control variable; this removes the interactions between decisions in different periods, rendering long-run forecasts irrelevant. Costello et al. (1998) use numerical methods to study the effect of one and two period ahead forecasts in a calibrated nonlinear resource management model, showing that for some parameter values perfect information at these lead times provides substantial value. Our work considerably generalizes these findings. We analyze a nonlinear model that exhibits non-trivial interactions between time periods, use an infinite time horizon that removes bias against long-run forecasts, obtain analytic solutions that enable clean comparative statics without the need for a calibrated numerical model, calculate the contribution of forecasts at all lead times to the overall value of a prediction system, and allow forecasts of arbitrary accuracy.

Finally, a substantial literature delineates the difficulties of long-run forecasting in contexts as diverse as climate science, macroeconomics, demography, epidemiology, and national security (see e.g. Palmer & Hagedorn, 2006; Granger & Jeon, 2007; Lindh, 2011; Lee, 2011; Myers et al., 2000; Yusuf, 2009). A common refrain in much of this work is that accurate long-run forecasting is difficult, but would be of considerable value for decision-makers if achievable. Yet to our knowledge there is no existing analytical framework that provides intuition for if, and when, this is likely to be true. Our work provides a first step towards such a framework, illustrating in a simple model how a decision-maker’s ability to adapt to changes in her environment dynamically, and the costs she sustains in doing so, co-determine the relative importance of short-run and long-run predictability.

Read the rest of the paper