Introducing Probability Into Energy Forecasting

Introducing Probability into Energy Forecasting

Lynn Kaack, Carnegie Mellon University, Phone: (919) 904-1258, E-mail:

Jay Apt, Carnegie Mellon University, Phone: (412) 268-3003, E-mail:

M. Granger Morgan, Carnegie Mellon University, Phone: (412) 268-2672, E-mail:

Inês Azevedo, Carnegie Mellon University, Phone: (412) 268-3754, E-mail:

Evan Sherwin, Carnegie Mellon University, Phone: (805) 705-8761, E-mail:

Overview

Agencies and research institutions make an effort to frequently forecast energy-related quantities such as electricity and fuel demands, commodity prices, and specific energy consumption and production rates. Energy forecasts, like the Annual Energy Outlook by the EIA and the World Energy Outlook by the IEA, are important because they are often used as the basis for investment and policy decisions. As retrospective analysis of past energy projections have shown, such forecasts can mispredict with errors of up to 200%. A thorough treatment of uncertainty is essential for good decision-making. This work aims to establish methods for probabilistic projections that are suitable for the type of prediction models used by the EIA and IEA, while preserving their core forecasting process and being as quantitative as possible.

Methods

We extend the method of empirical prediction intervals to partial-equilibrium models that are not stationary in time and therefore violate essential requirements of the known framework.The paper analyzes the scope and limitations of the method of empirical prediction intervals for the given data set by systematically investigating the mathematical assumptions for obtaining a well-calibrated prediction interval. We focus onassessing the stationarity of the forecast error distribution, finding the quantiles in a non-parametric fashion, and discussing bias.In the light of the inherent non-stationarity of forecasting errors, we propose a method to improve calibration of the predictive distribution.By using change-point analysis, we identify stationary intervals. Calibration can be improved by giving more weight to forecast errors in more recent stationary intervals.

Results

We find that error distributions do not follow a known probability density function, but require non-parametric quantile regression. Calibration and sharpness of prediction intervals are improved by identifying non-stationarity with change point analysis. The prediction intervals for the EIA and IEA forecasts often span a different range than captured by the institution’s scenario analysis. Moreover, we challenge the assumption that forecasts are systematically biased.

Conclusions

Empirical prediction intervals by retrospective errors appear to be the most feasible method for finding probabilistic uncertainty for these types of forecasting processes. Energy forecast data introduce a number of limitations to the method that have to be considered with care in the process of creating the prediction interval. In this work we developed an approach that systematically reduces these limitations and makes the method suitable for energy forecasts. The paper entails specific guidance for institutions on how to construct and incorporate sound prediction intervals in future energy outlooks, but findings are also applicable to forecasts in other fields.

References

[Auffhammer, 2007] Auffhammer, M. (2007). The rationality of eia forecasts under symmetric and asymmetric loss. Resource and Energy Economics, 29(2):102–121.

[EIA, 2014] EIA (2014). Annual Energy Outlook 2014. U.S. Energy Information Administration.

[Fischer et al., 2009] Fischer, C., Herrnstadt, E., and Morgenstern, R. (2009). Understanding errors in eia projections of energy demand. Resource and Energy Economics, 31(3):198–209.

[Gneiting et al., 2007] Gneiting, T., Balabdaoui, F., and Raftery, A. E. (2007). Probabilistic forecasts, calibration and sharpness. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 69(2):243–268.

[Gneiting and Katzfuss, 2014] Gneiting, T. and Katzfuss, M. (2014). Probabilistic forecasting. Annual Review of Statistics and Its Application, 1:125– 151. 3.5, 4.2

[Knüppel, 2014] Knüppel, M. (2014). Efficient estimation of forecast uncertainty based on recent forecast errors. International Journal of Forecasting, 30(2):257–267.

[Lee and Scholtes, 2014] Lee, Y. S. and Scholtes, S. (2014). Empirical prediction intervals revisited. International Journal of Forecasting, 30(2):217–234.

[Linderoth, 2002] Linderoth, H. (2002). Forecast errors in iea-countries’ energy consumption. Energy Policy, 30(1):53–61.

[Morgan and Keith, 2008] Morgan, M. G. and Keith, D. W. (2008). Improving the way we think about projecting future energy use and emissions of carbon dioxide. Climatic Change, 90(3):189–215. 2.3

[Shlyakhter et al., 1994] Shlyakhter, A. I., Kammen, D. M., Broido, C. L., and Wilson, R. (1994). Quantifying the credibility of energy projections from trends in past data: The us energy sector. Energy Policy, 22(2):119–130.

[Smil, 2000] Smil, V. (2000). Perils of long-range energy forecasting: reflections on looking far ahead. Technological Forecasting and Social Change, 65(3):251– 264.

[Wallis, 1999] Wallis, K. F. (1999). Asymmetric density forecasts of inflation and the bank of england’s fan chart. National Institute Economic Review, 167(1):106–112.

[Wara et al., 2015] Wara, M., Cullenward, D., and Teitelbaum, R. (2015). Peak electricity and the clean power plan. The Electricity Journal.

[Williams and Goodman, 1971] Williams, W. H. and Goodman, M. L. (1971). A simple method for the construction of empirical confidence limits for eco- nomic forecasts. Journal of the American Statistical Association, 66(336):752– 754.

[Winebrake and Sakva, 2006] Winebrake, J. J. and Sakva, D. (2006). An evaluation of errors in us energy forecasts: 1982–2003. Energy Policy, 34(18):3475– 3483.