29 November 2015
Project-Based Assessments in the Victorian Energy Efficiency Target (VEET) Scheme
Energy Policy and Programs
Department of Economic Development, Jobs, Transport and Resources
GPO Box 4509
Melbourne VIC 3001
Re: Opower’s comments on “Project-based assessments in the VEET scheme
consultation Paper 2015”
To Whom It May Concern:
Opower appreciates this opportunity to submit comments in response to the “Project based
assessments in the VEET scheme October 2015 stakeholder consultation paper.”
We commend the Victorian Government (the “Government”) for continuing to lead on
energy efficiency by working to refine and strengthen the Victorian Energy Efficiency
Target (VEET) scheme, and we welcome the proposed changes.
Opower supports inclusion of project-based assessment (PBA) methods in the VEET
scheme. Further, we strongly support the Government’s proposal to include a method
similar to the Commonwealth ERF Aggregated Small Energy Users (ASEU) methodi as
one of the first PBA methods approved for VEEC crediting in mid-2016. Our responses
to the consultation questions below emphasize the following key points:
● Including proven PBA methods in the VEET scheme can dramatically
broaden the scope of energy savings activities taken up in Victoria. PBA
methods for Project Impact Assessment and Aggregated Small Energy Users will
unlock two important sources of cost-effective energy savings at scale:
commercial building efficiency and residential behavioural energy efficiency (BEE).
● Enabling forward creation of certificates for rigorous PBA methods will
help ensure a robust market for proven, large-scale EE activities. Forward
creation, subject to ex-post verification of energy-saving performance, would
remove financial barriers to investment in large-scale metered-baseline programs
and ensure a level playing field for all VEET activities.
● Adopting an Aggregated Small Energy Users PBA method would unlock
reliable, cost-effective energy savings at scale. Behavioral energy efficiency
programs alone have annual potential to generate 300,000 VEECs and AUD 60
million in customer bill savings on electricity and natural gas.
● Experimental program design that requires randomised-controlled trials is
the global “gold standard” for ASEU PBA methods. Such methods are
rigorous and widely accepted in Australia and the United States to measure energy savings resulting from behavioural energy efficiency programmes. The
Government’s proposed ASEU method can draw from well-qualified precedents,
and should not require new investment in infrastructure or capacity development.
About Opower
Opower is an enterprise software company that is transforming the way energy retailers
and distribution companies engage with their customers. Opower’s customer
engagement platform enables these energy providers to reach their customers at
moments that matter through proactive communications that drive energy savings and
demand reduction, increase customer engagement and satisfaction, and lower customer
operation costs. Opower’s software has been deployed to more than 95 utility partners
around the world and reaches more than 55 million households and businesses. Having
run hundreds of large scale field tests, and been subject to over 45 independent
programme evaluations, Opower has amassed the world’s largest body of experience in
delivering behavioural demand-side management programmes.
Opower Response to Consultation
Opower is pleased to provide the following responses to the questions raised by the PBA
consultation. Note that we have taken the opportunity to address consultation questions
3-8 with answers more specifically applicable to an ASEU method than to the M&V
method for project impact assessment (PIA) of commercial building upgrades that is the
primary focus of this consultation. While our direct experience with PIA-type activities is
limited, we engage daily with regulators and energy providers on projects measured
using methods similar to the proposed ASEU method. Though our input may not match
the scope that the consultation has in mind, it is our sincere hope that the information
that follows is helpful to the Government as it proceeds with the larger effort to
incorporate PBA into the VEET scheme.
1. Proposed M&V Method
~ Do you envisage a market for the proposed M&V Method? Why/why not?
Yes. Adoption of the proposed PBA methods can dramatically broaden the scope of
proven, large-scale energy-saving activities taken up in Victoria. However, the extent to
which the Government allows forward creation of certificates from PBA-verified activities
will strongly influence the market for each of the proposed methods.
PBAs unlock substantial savings
In our experience, cost-effective end-use EE at scale results primarily from three types of
activity:
1) Lighting efficiency upgrades;
2) Commercial building efficiency upgrades;
3) Residential behavioural energy efficiency.
The existing VEET scheme has successfully incentivized broad uptake of efficient
lighting. Indeed, more than 90% of VEECs issued in 2015 stem from such activities.ii
Inclusion of PBA methods could unlock substantial energy savings from activity types
that have not enjoyed VEET incentives to date. The proposed ”M&V Method” for project
impact assessment will incentivize commercial building efficiency upgrades, and the
proposed ASEU method will incentivize residential BEE.
Timing of certificate creation matters
Opower strongly agrees with the Government’s observation that “forward creation is a
key feature” for PBA methods, because it removes financing barriers for proven, largescale
energy-saving activities by offsetting upfront costs.iii Commercial building upgrades
and BEE programs alike generate substantially greater energy savings than a typical
VEET activity today, but they also incur higher initial costs, which may limit market
interest if there is an interval between investment and certificate creation.
Moreover, if there were PBA-eligible activities that could benefit from forward creation,
they would be at a disadvantage versus activities that have deemed values (e.g.
replacing incandescent lights with fluorescent ones). This would be despite the fact that
PBA ensures greater environmental integrity than deeming, given that PBA awards
certificates on the basis of verifiable performance and deeming awards certificates on
the basis of strong assumptions about likely performance. Forward creation is already a
key de facto feature for deeming savings from VEET-eligible installed-device activities:
an AP gets credit for 100% of “deemed” lifetime savings for a given installed device
when its installation is certified by the Government.
The Government can ensure a level playing field for all activities by treating the timing of
certificate creation as consistently as possible, despite necessary practical differences in
the means by which savings are deemed or measured.
We therefore support the Government’s proposal to allow forward creation of certificates
subject to the proposed M&V method. We also strongly recommend that the Government
allow forward creation of certificates subject to its proposed ASEU method, to the extent
that it requires rigorous ex-post verification and periodic true-ups.
2. Other proposed project-based assessment methods
~ Do you envisage a market for these proposed methods? Why/why not?
Yes. The Government’s proposal to adopt an “aggregated method for quantifying energy
savings from large groups of small energy users using statistical analysis” corresponds
with global best practice for M&V of behavioral energy efficiency programs. We expect
such BEE programs to deliver substantial energy and bill savings for hundreds of
thousands of Victorians. Market uptake of such programs will depend on the timing of certificate creation for activities subject to the proposed ASEU method. Victoria can build
on precedent from other regulatory jurisdictions to ensure both robust experimental
design in advance of program implementation and rigourous ex-post verification of
measured savings.
Randomized Controlled Trials are Best Practice
Behavioural energy efficiency programs that aggregate energy saving activities
undertaken by many small energy users, such as those administered by Opower, use
RCTs to measure energy savings with over 90% statistical confidence. Under this
approach, the program implementer uses parameters to create an eligible group of
recipients, randomly assigns households to control and treatment groups, tests these
groups to ensure statistical equivalence, implements the intervention (such as sending
reports) only to the treatment group, and measures the difference in energy usage
between the two groups using statistical billing analysis. This approach measures
savings with precision and without bias. Key components of this approach include:
● Statistically equivalent and randomly allocated control and treatment groups;
● Opt-out design;
● Ex-post measurement;
● Panel data methodology billing analysis for comparison of control and treatment
groups.
The following is the basic calculation of impact:
kWh saved (treatment group) = kWh used (control group) – kWh used (test
group) – kWH saved by rebated equipment (product participation) for the
same time period and same customers.
This energy savings calculation can be easily converted into the corresponding amount
of emissions reductions achieved by applying the appropriate grid emission factor to
account for carbon intensity of the local fuel mix.
This methodology matches the prescribed protocols for measuring residential behavioral
energy efficiency programs as detailed by the Australian Ministry of the Environment,iv
the New South Wales Office of Environment and Heritage,v the Essential Services
Commission of South Australia (ESCOSA),vi the US Department of Energy’s National
Renewable Energy Laboratoryvii and numerous state-level jurisdictions in the US.viii
Further, an ASEU approach is consistent with the recommendations of the US
government-led State & Local Energy Efficiency (SEE) Action Network’s definitive report:
“EM&V of Residential Behavior-Based Energy Efficiency Programmes: Issues and
Recommendations.” SEE Action is a consensus group comprised of utilities, consumer
advocates, commission staff, and government officials. This SEE Action report
concludes:
“We recommend using a randomized controlled trial for behaviour-based
efficiency programmes, which will result in robust, unbiased programme
savings impact estimates, and a panel data analysis method, which will
result in precise estimates.”ix
This is a low-risk approach because the results are proven and predictable, but also
because they are measured ex post, so the credit is given for results actually achieved.
This is different from many other efficiency programmes, which have expected values
but no means by which to measure after savings have occurred. It is an approach that
has been recognized as the gold standard across the industry.
RCT enables straightforward avoidance of double-counting
RCT program design allows for robust measurement and exclusion from double-counting
of energy savings that are jointly-attributable to the BEE program, and to another energy
saving activity. This ability does depend on being able to having an accessible and
accurate registry of credited energy savings activities that have been taken by customers
within the BEE program population. Opower’s understanding is that Victoria’s registry
is very good, and will thus facilitate in ensuring that double-counting of savings
does not occur.
For a detailed explanation of how an RCT approach should be used by an ASEUtype
method to identify jointly-attributable savings, please refer to the technical
appendix that covers this very topic in the US Department of Energy’s
“Residential Behavior Protocol.”x
BEE programs deliver energy savings equitably at scale
More than 55 independent evaluations by academics, consultants, and economists have
verified that residential behavioural energy efficiency programs empower customers to
save an average of 1% to 3% on their energy bills. These savings are consistent across
fuel type, geography, income, and age.xi
Opower BEE programs have been proven reliably effective, delivering over 8.2 TWh of
energy savings and AUD 1.3 billion in bill savings for customers at over 95 utilities, while
yielding emissions reductions of over 5.5 million tonnes of CO2e. Opower estimates that
with the inclusion of an ASEU PBA method in the VEET, the Government will unlock the
potential to deliver more than 300,000 VEECs and AUD 60 million in customer bill
savings per year via residential BEE programs.
Rigourous M&V enables prudent forward-creation of certificates
As described in detail above, proven ASEU or aggregated “metered baseline” methods
require ex-post verification of savings. Compared with traditional deemed-savings approaches, this step provides a unique level of certainty that savings have occurred.
Yet in practice, requiring this step before issuing credits (or otherwise recognizing
savings for compliance) penalizes that additional rigor, given that months typically pass
between initial program investment and the first robust measurement of program
savings. This timing issue creates a financial barrier that often limits initial program size
to a fraction of cost-effective potential, and in competitive retail markets such as
Victoria,xii it often discourages investment in BEE altogether.
As touched on in our response to question #1, forward creation of certificates already
takes place as a matter of course in EE markets around the world – whenever an
administrator “deems” energy savings up front for an installed measure such as efficient
lighting. This “standardized baseline” approach to forward-crediting efficiency
improvements implies a tradeoff in favor of financial certainty for the AP or EE program
implementer, at the cost of environmental and regulatory certainty (i.e. that savings will in
fact occur according to assumptions for typical device usage that are embedded in the
deeming formula). This tradeoff is prudent and perfectly reasonable, because in practice,
a requirement to monitor usage from each energy-saving device would create an
insurmountable barrier to market participation.
Opower submits that it is equally prudent to permit forward creation of certificates from
programs designed as randomized-controlled trials and subject to ex post verification
and regular periodic true-ups.
The US state of Michiganxiii applies such a hybrid “deem and verify” approach to the
crediting of energy savings from BEE programs, and to our knowledge, represents its
longest-standing precedent. RCT-based programs in the state that address residential
behavioural energy efficiency are assumed to achieve a 1% - 3% energy efficiency
improvement. This initial savings estimate is deemed based on documented results from
similar programs that have been recently implemented in the same or a similar market,
for the same or a similar duration. At a fixed time each year, actual savings for such a
program are measured ex-post using RCT-based methods to verify that the ex-ante
deemed efficiency improvements were achieved. If verified savings exceed the initial
deemed value, the state credits the program with the additional savings. If verified
savings fall short of the deemed value, the state holds the BEE program administrator
liable for the amount of the shortfall during the following year.