Overview of New Method for Determining LOR Levels - Feb 2018

Overview of new method for determining Lack of Reserve (LOR) levels – Feb 2018

A change to the National Electricity Rules[1] in December 2017 revised the principles for determining Lack of Reserve (LOR) in the National Electricity Market (NEM). In conjunction with the rule change, AEMO developed the Reserve Level Declaration Guidelines[2] to set out how the new determinations will be made. This overview summarises the new method.

This is a simplified explainer of AEMO’s process for determining lack of reserve levels, as at February 2018. We’ve taken care in preparing it, but it’s not comprehensive and we can’t guarantee its accuracy. Things change over time, and this document might not be up to date when you read it. Please read the current version of the Reserve Level Declaration Guidelines on AEMO’s website if you require a full understanding of the process.

Until February 2018, LOR1 and LOR2 levels were determined solely on the basis of the largest credible contingencies in a region. These values, while not static, were generally relatively constant. AEMO reviewed how these thresholds took into account risks of unexpected reductions in reserves due to factors that exist now in the changing power system, but were not covered in the existing Rule.

The new process introduces a probabilistic element into the determination, which allows for the impact of estimated reserve forecasting uncertainty in the prevailing conditions. These estimates are made on the basis of past reserve forecasting performance for:

• Demand.
• Output of intermittent generation.
• Availability of scheduled generation.

This means LOR1 and LOR2 levels will tend to vary with operating conditions and over different forecasting horizon, and thus the levels will be more changeable than in the past.

LOR levels

Under the new arrangements, LOR threshold levels in forecasts are determined as follows. TheForecasting Uncertainty Measure (FUM)in each formula is explained on the next page.

The LOR1 threshold is determined by the formula MAX (LCR2, FUM) whereLCR2 is the sum of the two largest credible risks in the region (effectively the former LOR1 threshold).

The LOR2 threshold is determined by the formula MAX (LCR, FUM) where LCR is the largest credible risk in the region(effectively the former LOR2 threshold).

The LOR3 threshold is when the forecast reserve in a region is at or below zero. This remains unchanged.

Figure 1 shows how each LOR level would be determined based on extreme FUM values.

Figure 1Schematic representation of LOR formulation in circumstances of extreme FUM values

Forecasting uncertainty measure

AEMO applies the historical data and the conditions expected to determine a distribution of reserve forecasting error across all forecasts for the first 72 hours of the one-week LOR forecasting horizon. The input states that will be taken into account in developing the distribution will be:

• Forecast lead time.
• Forecast regional reference node temperatures.
• Current demand forecast error for forecast lead times below eight hours.
• Unconstrained intermittent generation forecast.

The FUM for a region, point in time, and set of expected conditions, is the number of MWs representing a level that is not expected to be exceeded for the specified confidence level. For instance, if the specified confidence level is 95% and the FUM value is 900MW, there is a 95% chance that the actual error will not exceed 900 MW.

These confidence levels are intended to be set at a level that AEMO reasonably expects to reduce the chances of load shedding occurring because potential reserve shortfalls were not flagged due to forecasting errors whilst not unduly increasing the likelihood of unnecessary declarations.

To achieve this balance, confidence levels will change with the forecasting horizon. The initial confidence levels are in Table 1 for all NEM regions.

Table 1Confidence levels for determination of FUM values

Region(s) / Forecasting horizon (hrs) / Confidence level
All / 0.5 to 15 / 98%
All / 15.5 to 18 / 97%
All / 18.5 to 21 / 96%
All / 21.5 to 72 / 95%

Example:If, for a region, the FUM value for the forecast 12 hours ahead is 800 MW, this means there is a 98 % chance that the actual reserve 12-hour ahead forecast error (due to a combination of the errors in demand, intermittent generation, and scheduled generation availability forecasts) will not exceed 800 MW.

FUM will only be determined for the first 72 hours of the LOR forecasting horizon. This means that LOR forecasts beyond 72 hours ahead will remain determined as previously (that is, by the reserves needed to cover the one or two largest credible contingencies).

Figure 2 shows a typical example of how FUM values change over the 72 hours for each region.It shows that FUM values will tend to reduce as the forecasting horizon reduces, particularly for forecasting horizons of six hours ahead or less. In this particular example, conditions on the Saturday afternoon are such that forecast uncertainty is estimated as higher in Victoria and South Australia.

Figure 2Figure 2 FUM Values for run at 1600hrs on 8 February 2018

How AEMO estimates the FUM

For every 30 minute trading interval since July 2011, AEMO calculated the reserve forecast based on the forecasting errors for demand, intermittent generation, and schedule generator availability for the next 384 trading intervals (eight days ahead).

Each 30 minute forecast was assessed against the actual conditions for each of the next 144 trading intervals. The known prevailing conditions present just prior to the forecast run were included, to develop an understanding of how these conditions affect the forecasting error. Those prevailing conditions were:

• Current temperature (temperature by region).
• Forecast temperature (temperature by region).
• Prior trading interval forecasting error (Semi-Scheduled and Non Scheduled generation and Operational Demand).
• Regional reference price ($/MWh).
• Time of day (daytime / night-time forecast).

Not all the prevailing conditions were found to be significant to the error distribution. These were discarded, to simplify the calculation and enable the distributions to be built using a greater input sample size.

This data was then used to train a Bayesian Belief Network (BBN) to produce an error distribution for each of the next 144 trading intervals. This is dynamic, in that the error distributions will update based on the current prevailing conditions when the forecast is produced.

BBNs are probabilistic models used in artificial intelligence to deal with problems that are associated with uncertainty. They can be used to investigate and present causal relationships between essential elements and output values of a system in a simple and understandable manner. They are useful for calculating the impact of interventions such as examining alternative policies or decisions for optimizing a desired outcome. In addition, at the same time the uncertainties integrated with these causal relationships can be investigated.

The data used for initial training of the BBN was from the period 1 July 2011 to 1 August 2017. The BBN will be retrained on quarterly basis. At the time of retraining, additional data available since the last retraining will be added to the training data set. Any changes to forecasting systems which result in a change in any of the error distributions (for example, an upgrade to the forecasting system resulting in an improvement to forecasting accuracy) will be reflected in the BBN (and subsequent FUM values) following the next scheduled retraining.

Once a BBN network has been trained, it is possible to statistically assess the impact that each of the prevailing conditions (input nodes) has on the forecasting errors. From this analysis, AEMO determined that the most significant prevailing conditions (the conditions that cause the largest change in forecasting uncertainty) are:

• Temperature forecast.
• Wind forecast.
• Last Operational Demand Error (the error of the demand forecast from the previous 30 minute trading interval)for forecast lead times below eight hours.

These were included in the model so that changes in these inputs may result in changes to the calculated FUM values.

Before FUM values are used to calculate LOR levels, the calculated FUM values are subject to a reasonability check[3].

Next steps

At the end of the 2017-18 summer, the BBN model will be retrained and AEMO will undertake a consultation to review the Reserve Level Declaration Guidelines. This may result in changes to the settings of the confidence levels and some details of the design of the BBN models.

© AEMO 2018|OVERVIEW OF NEW METHODS FOR DETERMINING LOR LEVELS / 1

[1] Refer

[2] Refer

[3] For information on the initial reasonability limits refer