Do Analyst Forecasts Vary Too Much?
by
Russell Lundholm
and
Rafael Rogo
First draft: October 16, 2013
Prepared for the 2013 Columbia Business School Burton Workshop
1
I. Introduction
Different analysts make different forecasts, and individual analysts change their forecasts, for a variety of reasons. The rational forecasting explanation for this variability in forecasts is that it is caused by variation in information across analysts and over time. This explanation is difficult to refute without access to analysts’ underlying information. Alternatively, analysts may make different forecasts, or change their forecasts, for strategic reasons unrelated to their information, or they may respond non-optimally to the information they receive. In this paper we introduce a new test of the optimality of analyst forecasts based on their time series and cross sectional variability. For a given firm at one point in time, do the forecasts of different analysts differ from each other too much? And for a given firm and analyst, do forecasts fluctuate too much over time?
For both questions we offer a novel approach to identifying when forecasts vary “too much.” We derive a bound for the variance of forecasts that is completely general – it only assumes that a variance of the underlying company earnings exists – and then we examine forecast data to see how frequently the bound is violated. Importantly, our test does not require any knowledge of the underlying information available to analysts. While we derive our bound mathematically, a loose translation would be that the variance of rational forecasts about a random variable must be lower than the actual variance of the random variable. Collections of forecasts (either in time series or in cross section) that violate this bound are unquestionably too volatile to be rational; that is, there is no amount of variation in information that can justify this much volatility. Consequently, they are clear cases where forces beyond rational forecasting have come into play.
Cases where analyst forecasts are excessively volatile are of interest beyond providing evidence of non-optimal forecasts. Changes in analyst forecasts of earnings are often accompanied by changes in the firm’s stock price. If forecasts are excessively volatile then this could contribute to excessively volatile stock prices. Whether or not stock prices change too much has been studied extensively in finance. The short answer to this question is ‘Yes” although the results are not without controversy (see Shiller, 1981 for the original study and Gilles and LeRoy 1991 for a comprehensive review of the evidence). In the stock price setting, the researcher must assume that the observed price fluctuations are driven by fluctuations in the forecasts about the underlying “true value” of the company. The researcher observes neither the actual forecast nor the “true value” of the company, and must make restrictive assumptions and imprecise estimates of each. In our setting, we observe the forecasts directly because analysts regularly report their forecasts to IBES. We also observe the underlying series being forecasted – company earnings. Consequently, we can precisely establish a variance bound on forecasts of earnings.
There are at least two reasons why an analyst might produce forecasts that are too volatile over time, or why a group of analysts’ forecasts might be collectively too disperse at a point in time. First, regardless of their information, analysts may have incentives to make non-optimal forecasts. For instance, they may make a “bold” forecast in order to attract the attention of the investment community (see Clarke and Subramanian 2006). Second, they may simply respond too aggressively to new information, as would be the case if they suffered from a saliency bias (Kahneman and Tversky 1973). Working against these forces are an analyst’s incentive to herd toward the consensus, either rationally because the consensus is a good aggregator of private information, or strategically because the analyst doesn’t want to appear unusual (see Hirshliefer and Teoh 2003 for a review of the herding literature).
Based on both cross-sectional tests and time-series tests, and using both annual and quarterly data, we find a non-trivial number of cases where analyst forecasts are excessively volatile. Individual analysts forecasts are too volatile in time series approximately 17 percent of the time, with roughly 20 percent of the analysts producing at least one excessively volatile forecast series one quarter of the time. The cross-section of analyst forecasts are better behaved, although we still find roughly five to seven percent of the cases are excessively volatile.
We supplement our analysis with an exploration of the analyst characteristics, firm characteristics, and time period characteristics that are associated with excessively volatile forecasts. We find that periods of excessively volatility tend to precede large market corrections. The frequency of excessively volatile forecasts peak and then plummet around Black Monday (1987-1988), the Dot Com bubble and bust (2001-2002), and the Global Financial Crisis (2008-2010). We also find that analysts are very different in terms of their propensity to produce an excessively volatile series of forecasts, with some analysts contributing to this phenomenon regularly and others who have never produced such a series of forecasts. Similarly, some firms are considerably more prone to violations of the variance bound than others.
In the next section we discuss related literature, in section three we derive our variance bound, and in section four we talk about how to estimate the variances that go into the variance bound test. We present our results are in section five and conclude in section six.
II. Related Literature
Barron et al. (1998) present a rational model of analyst forecasts where each analyst forms a posterior belief about the firm’s upcoming earnings based on common public information and a noisy private signal. Dispersion in forecasts is caused by dispersion in the private signal errors. The authors then map the statistics found on the IBES consensus database onto the parameters of their model. The result is a powerful tool that has be used to estimate the average amount of private signal precision that analysts have at a point in time. For instance, Barron et al. (2002a) find that consensus, measured as the cross-sectional correlation in forecast errors, decreases around earnings announcements, and Barron et al. (2002b) find that consensus is lower for firms with relatively more intangible assets. Based on the Barron et al. (1998) model, the interpretation is that analysts collect more private information after earnings announcements, and for firms with more intangible assets, and that this is the source of the cross-sectional dispersion in their forecasts. However, this interpretation places considerable faith in the structure of the model, including the assumption that all random variables are normally distributed, that analysts are Bayesian processors of information, and that their only motivation is to make an accurate forecast. As we show later, in this setting theoretical forecasts never violate the variance bounds that we propose. Our test takes a very different tack. We assume very little about the structure of the information used by analysts, including no restrictions on the realizations or distributions of the random variables, but can only identify one type of irrational forecasts – those that are too variable either in the cross-section across analysts or in time-series for each analyst.
Although there is a wealth of analyst forecast literature (see Ramnath et al. 2008 for an excellent review), very little has focused on the variance of the forecasts as a collection. In terms of cross-sectional variation in analyst forecasts, there is evidence that public information lowers dispersion. Lang and Lundholm (1996) find that firms who provide better information to analysts, as measured by their AIMR score, have lower dispersion. And Bowen et al. (2002) find that dispersion decreases following earnings announcement conference calls.
There is also some indirect evidence that analysts are influenced by the forecasts of other analysts, which will affect the cross-sectional variance. For instance, if analysts herd toward the consensus estimate then this will lower the cross-sectional variance in their forecasts. Graham (1999) finds that analysts with high reputation, or low ability, herd toward the consensus. And Welch (2000) finds that analysts herd toward the consensus when there is little information available. In contrast, Clement and Tse (2005) find that bold forecasts – those that move away from the consensus – are more accurate. Clarke and Subramanian (2006) find that both very accurate, and very inaccurate, forecasters produce bold forecasts. And Bernhardt et al. (2006) report evidence of “anti-herding,” meaning that analyst forecasts are repelled away from the consensus. Finally, Hong et al. (2000) find that inexperienced analysts are more likely to be fired for issuing a “bold” forecast, giving them an incentive to herd toward the consensus. The evidence of herding, or boldness, is based on comparing analyst forecast revisions to the consensus, with movements toward the consensus labeled as herding and movements away from the consensus labeled as boldness. But without access to the information used by the analysts, these studies cannot rule out that the forecastswere simply the result of rational use of information. By considering a collection of forecasts together, we can unambiguously say when the forecasts are collectively too variable to be consistent with rational forecasting. Note that this will identify “boldness” generally, and any countervailing forces that create herding will work against our measure.
In terms of the time-series variation in forecasts, Gleason and Lee (2003) find that bold forecast revisions generate larger stock return responses. In addition, there is evidence that forecast revisions are positively serially correlated (Zhang 2006, Chen et al. 2013). Zhang (2006) also provides some results that link the time-series forecast variance to the cross-sectional variance. In particular, he finds that the drift in analyst forecast revisions is greater for firms with greater cross-sectional dispersion, and that the effect is more pronounced following bad news. This type of incomplete adjustment to new information will lower the estimated time-series variance, and work against violations of our bound.
There are behavioural reasons that analysts might make forecasts that are too volatile as well. Kahneman and Lovallo (1993) describe a judgement bias wherein forecasters take an “inside view” of the problem, causing them to overweight the specific details of the forecast at hand and underweight baseline priors derived from previous forecasting exercises. This is a specific version of a general judgement bias wherein agents overweight salient information (Kahneman and Tversky 1973). Such a bias will therefore overweight recently-received private information, causing the forecasts to excessively respond to the private signals and increase the variance of the collection to a possibly irrational level. DeBondt and Thaler (1990) provide some related evidence on this judgement bias based on IBES consensus analyst earnings forecasts from 1976-1984. They find that differences in forecasts across firms and years appears to be too extreme, insofar as the level of the forecast is negatively related to the forecast error. In other words, analysts forecast as if the differences in firms and years are greater than they actually are, and they would be more accurate if they tempered their extreme forecasts. In contrast, we examine the excess volatility in forecasts within firm-years. For a given firm-year, are the time-series of forecasts too volatile, or the cross-section of forecasts, too volatile?
III. A Variance Bound for Earnings Forecasts
Let X be the underlying random variable being forecasted, which has density g(x). Let Y be a summary statistic for all public and private information used by a rational forecaster in constructing a posterior distribution of X, denoted as g(x|y). If Y is a set of information, rather than a single summary statistic, the derivation of the variance bound is almost the same, but uses conditioning sets of random variables. The variance bound is based on the following condition (see DeGroot1975, p.183):
V(X) = V[E(X|Y)] + E[V(X|Y)].[1] (1)
Rearranging (1) gives
V[E(X|Y)] = V(X) - E[V(X|Y)]. (2)
Since E[V(X|Y)] is strictly positive for all non-degenerate posteriors g(x|y),
V[E(X|Y)] < V(X).(3)
Equation (3) is the basis for our tests. It says that the variance of expectations of X, seen as a random variable in Y, must be less than the ex ante variance of X.
To illustrate the bound in a specific context, consider the Barron et al. (1998) model. Let X denote unknown future earnings and Y = X + denote the analyst’s private signal, where X and are independently normally distributed, is mean zero, X has mean , V(X) = 1/r, and V() = 1/s. In this case
and
which is strictly less thanV(X) = 1/r.
In the context of the Barron et al. (1998) model, a rough intuition for why V[E(X|Y)] is less than V(X) goes as follows. There are two reasons why the signal Y might be highly variable; either X is highly variable or is highly variable. If X is highly variable then the RHS of the bound,V(X), will be large as well. Alternatively, if Y is highly variable becauseis highly variable, then the weight on Y in the expectation will be low and it won’t flow through to variation in the posterior expectation.
The Barron et al. (1998) model illustrates the bound, but we emphasize the extremely general nature of boundas given in (3) – as long as the densities g(x) and g(x|y) have variances, equation (3) holds. The distributions do not need to be normal and the signal does not need to havean additive error. In fact, the signal can be a multidimensional set of signals.
IV. Estimation
The next challenge is to estimate V[E(X|Y)] and V(X) using analyst forecasts and earnings realizations.
A. Estimating the Variance of Earnings Changes
A crucial estimate in our analysis is V(X). This establishes the benchmark variance that bounds the forecast variance. Because earnings for a given firm evolve in a time series, we need to consider its time series properties. We consider two units of observation: forecasts and outcomes for firm-years, and forecasts and outcomes for firm-quarters. Early literature has found that quarterly earnings evolve approximately as a seasonal random walk and annual earnings evolve approximately as a simple random walk (Brown et al 1987). More recently, Gerakos and Gramacy (2012) and Li and Mohanram (2013) find that at a one-year forecasting horizon, a random walk performs about as well as many other more complicated models. Therefore, for firm-quarters, we define the object of the analyst forecast as the seasonal change in the firm’s earnings. That is, we specify the unknown object of interest to be
Xjt = Ejt – Ejt-4,
whereEjt is the realized quarterly earnings from IBES for firm j announced at time t, and Ejt-4 is the realized earnings from the same quarter a year earlier. When the unit of observation is the firm-year, we define the object of the analyst forecast as the annual change in earnings. That is, we specify the unknown object of interest to be
Xjt = Ejt – Ejt-1,
whereEjt is in this case is the realized annual earnings from IBES for firm j announced at time t, and Ejt-1 is the realized annual earnings from a year earlier.
By focusing on forecasts and realizations of changes in earningsweincrease the likelihood that the time-series variances are computed from a stationary series. In particular, for each realized earnings change (annual or seasonal quarter)and for each firm in the sample, we estimate the variance based on previous realizations of Xjt. Denote this estimate as.
Because the estimate ofis such an important value for our tests, we consider three different estimation periods; the previous eight realizations, all previous realizations, and the previous seven realizations plus the current period’s realization of theXjt that is being forecast.Which estimation period is the best depends on the appropriate horizon that a firm’s Xjt series is stationary, and what an analyst could possibly know about the distribution of Xjt at the time she makes her forecast. If a firm’s change-in-earnings process is stationary over its entire history, then all prior realizations would be the best choice as it maximizes the number of observations in the estimate, and all this information would be available to analysts. Denote this as the [-, -1] window, where time zero is when the outcomeXjtis announced. However, if the nature of a firm’s earnings process changes over time, then a shorter window might yield a more accurate estimate of the variance that an analyst could reasonably expect, and so we also consider an estimation period based on periods [-8, -1]. Finally, we consider the window [-7, 0] to rule out the possibility that the earnings process has fundamentally changed in period zero, and the analyst knows this, but our estimates based on periods before time zero do not take this into account. That is, suppose that in period zero there is extreme news, which the analyst discovered and accurately forecast near the end of her time-series of forecasts. In addition, this extreme news is indicative of a new earnings regime with significantly higher variance. The time-series variance of analyst forecasts would be increased by this late extreme news, but a estimate based on prior information would not, potentially leading to false violations of our variance bound. However, by including the realized Xjt in the estimate, we will wrongly inflate the estimate whenever there is an extreme realizationof Xjt, even when the underlying process variance has not changed. This will cause us to wrongly eliminate true violations of the variance bound. Finally, the estimates based on the [-7, 0] window gives the analyst clairvoyance for one of the variance estimate inputs, so it isn’t surprising that it will result in fewer variance bound violations. We consider this window as a specification check, but not as a legitimate estimate of .