From: Kasman, Robert
Sent: Friday, June 20, 2008 12:16 PM
To: Wan, Mike
Subject: FW: MF NTFR
Mike, this is the justification for the RNC MF NTFR=1.0. There is additional supporting documentation (examples of poor Title 24 modeling) if you want it.
Robert Kasman
Tel: 415-973-4094
Email:
From: Douglas Mahone [mailto:
Sent: Thursday, June 12, 2008 11:22 AM
To: Kasman, Robert
Cc: Cathy Chappell; Wan, Mike; Friedmann, Rafael
Subject: RE: MF NTFR
Robert, et al -
How does this meet your request?
Under the California MultiFamily New Homes program (CMFNH), which HMG administers on a third party basis for PG&E, we have been accepting developers' Title 24 submittals as part of their application process for the program. These are the same submittals that the applicants have submitted to their local building departments, and on the basis of which they have been granted building permits (i.e., they were acceptable to the building departments). In providing these submittals, most of them are also claiming to exceed Title 24 requirements by at least 15% (a few are asking for advice on what they would need to change in order to reach that threshhold). Our in-house Title 24 experts then do a detailed review of the submittals, comparing them to the plans and also reviewing for modeling errors. Here's what we have found:
Of10 submittals we reviewed, we found substantial errors (incorrect take-offs from plans, modeling mistakes, etc.), fixed those errors, and re-ran the Title 24 compliance runs. All of the 10 projects then FAILED to meet basic Title 24 performance, let alone exceed it by 15%.
Of an additional 17 submittals that we reviewed, we noted substantial errors and sent them back to their original Title 24 analysts for correction. These were sent to us with the understanding that they exceeded Title 24 by at least 15%, but clearly none of them did. If they had been ordinary submittals that aimed to just-meet Title 24, they would have failed.
On the basis of our experience with 27 of these project submittals, including detailed reviews of plans and compliance models, we conclude that curent standard practice is worse than Title 24. As you know, the Title 24 standards for multifamily buildings were dramatically tightened with the 2005 T24 revision, a process that closed many loopholes and made compliance much more difficult. While we would expect some practitioners to have difficulty in meeting these new requirements, we have been shocked by the nearly universal failure of projects to produce accurate Title 24 compliance runs, and by the degree to which real-world projects are performing below the minimum requirements.
Once projects are accepted into the CMFNH program, we are confident that they have been accurately modeled, and that they actually exceed current Title 24 requirements by 15% or more. We further verify this after the projects are constructed, through site inspections and HERS verification of installed measures.
Consequently, we have good reason to believe that the CMFNH program participants are not only performing better than Title 24, but that the standard practice baseline is worse than Title 24. The NTG for this program, therefore, should be greater than 1.0. We recommend, however, that the default NTFR value be set at 1.0 as a conservative estimate. Subsequent ex poste measurement should, of course, verify the actual values. That study should be sufficiently technical that is can uncover the actual energy performance of program non-participants, not just what their Title 24 submittals claim, in order to accurately determine the true energy baseline for MF new construction.
As a side note, it is apparent, from CPUC ED replies to the NTFR comments regarding the new construction programs, that the people who are determining the default NTFR values for these programs are confused about the distinction between Title 24 baselines and actual practice baselines, and that they are mixing code compliance issues into the NTFR determination. It is a long-established principle of impact evaluations for new construction programs, both in California and nationwide,that the true baseline for establishing energy savings is actual practice, not the nominal baseline defined by the energy code. This is simply a corollary to the principle that programs are credited only for savings that are caused by the program and that would not have happened absent the program. Savings claimed by the program that would have happened anyway are considered free-ridership and are deducted from the claimed savings.
However, there is nothing in these principles that says the actual practice baseline is always better than Title 24;it could be worse. In practice, commercial new construction tends to perform somewhat better than Title 24, and residential new construction tends to perform worse. Multifamily new construction, prior to the 2005 Title 24 update, tended to perform better; since then it looks to be performing worse.
There also seems to be confusion about the method for determining the actual practice baseline. During the 90's, over the course of several impact evaluation studies, various methods were tried to determine that baseline. Ultimately, it was agreed that the most repeatable and reliable approach was the so-called difference-of-differences method (we can cite papers reporting on this agreement). In this method, a sample of nonparticipant projects are evaluated to determine their energy performance difference relative to the nominal Title 24 baseline. The same procedure is applied to the sample of participant projects. The difference for the nonparticipant projects represents the free-ridership portion of the participant projects' energy savings. For example, if the nonparticipant projects are 5% better than Title 24, and the participant projects are 20% better, then the program was credited with savings of 15% better than baseline, or NTFR. (Other methods, such as using regression techniques, billing analysisorSAE, were also tried but were deemed to be less reliable means of determining free-ridership.)
What seems to be the source of the confusion, in the CPUC ED comments, is the failure to recognize that the nonparticipant projects could just as readily be worse than Title 24. So if the nonparticipant projects are 5% worse than Title 24, and the participant projects are 20% better, then the participants are 25% better than baseline, or NTFR.
It is a mistake to introduce codes and standards (C&S) compliance considerations into this calculation. Those are matters for determining how much savings should be attributed to the C&S programs. The C&S programs are credited with savings attributable to new code requirements, and they are debited for non-compliance with those requirements. So if, as in the second example above, the standard practice baseline is below the Title 24 baseline, that is the amount of the non-compliance penalty charged against the C&S program savings. If the new construction programs are not allowed to reclaim those lost savings when their participants perform above code, then the utility portfolio is double debited for the non-compliance.
DOUGLAS MAHONE
HESCHONG MAHONE GROUP, INC.
(916) 962-7001, ext 30
From: Kasman, Robert [mailto:
Sent: Tuesday, June 10, 2008 5:14 PM
To: Douglas Mahone
Cc: Cathy Chappell; Wan, Mike; Friedmann, Rafael
Subject: RE: MF NTFR
Hi Doug,
We had another statewide IOU NTFR call today. I presented PG&E’s position on NTFR values, and we are probably going to move forward with a NTFR recommendation of 1.0 for RNC MF projects statewide. However, in order to do so we need to provide some evidence/documentation supporting that value. I also understand that HMG implements both PG&E’s and SCE’s MF RNC program.
Up to now I’ve been primarily basing the 1.0 recommendation on conversations with HMG that “…most MF projects, once properly modeled, fall out of compliance.” Could HMG pull together a paragraph backing up that statement? In particular I think it needs to say x% of projects fall out of compliance, or average compliance goes from +x% to –y% based on looking at N projects, or something like that. Is that reasonable or do you have another idea?
I’m available most of the day tomorrow to discuss. Thanks,
Robert Kasman
Tel: 415-973-4094
Email:
From: Douglas Mahone [mailto:
Sent: Friday, June 06, 2008 3:06 PM
To: Kasman, Robert
Subject: RE: Materials for call today @2
Here's the history of thinking on net-to-gross in California's new construction impact evaluations (using SBD as the focus). The theory is completely applicable to RNC as well.
Doug.
From: Kasman, Robert [mailto:
Sent: Friday, June 06, 2008 2:14 PM
To: Douglas Mahone
Subject: RE: Materials for call today @2
Some comments I’ve developed in response to supporting RNC NTG values.
PG&E SF RNC Comments
1) The DEER/ED Team is correct that self reports were conducted for single family and multi-family projects, but the single family self-report NTG findings were not used by RLW in reporting final results. (This is readily observable since reported Net savings = Gross savings * NTG, only when the correct NTG value is used.) PG&E supports the adoption of NTG values that RLW actually used in reporting their final net savings values. Alternatively, PG&E would consider the use of a lower NTG value, and an appropriately higher gross savings value, as the DEER/ED team suggests in its comments, but the net results should be the same.
2) We believe the difference-of-differences methodology is appropriate for single family net savings estimation and NTG, and this method was approved for use by the CPUC, and has been in use since 2002. The baseline gross assumption, that non-participants construct homes just to code, was used to determine gross savings, but this is an arbitrary baseline.
3) The difference-of-differences evaluation method is consistent with the definition of free ridership (CA Framework, CPUC) since it compares what was actually built, to what would have been built absent the program (i.e., non participant behavior). “Behavior absent the program” is the intent and essence of the definition of free ridership.
4) The majority of non-participant homes were legally built to Title24 through the performance method of compliance, but gas related measures tended to over-comply, while electric related measures tended to under-comply. Therefore, the main driver of electric NTG values greater than one has little to do with overall code “non-compliance” but rather legitimate tradeoffs of building measures that tend to favor gas savings at the expense of electric savings to reach compliance. Participant homes tended to achieve increased savings through electric (cooling) measures, since the program intentionally required positive electric savings for at least two IOUs, including PG&E.
5) To our knowledge there is no double counting of savings with the Codes and Standards efforts.
6) As in the past, PG&E continues to support collaboration between implementers, evaluators and the Commission, to address these types of issues to promote more meaningful evaluation results and interpretation.
MF Comments:
TBD.
Robert Kasman
Tel: 415-973-4094
Email:
From: Douglas Mahone [mailto:
Sent: Friday, June 06, 2008 12:18 PM
To: Kasman, Robert
Subject: RE: call today 1:30-2?
Actually, I'm running about a half hour late so far today. How about 2:00?
DOUGLAS MAHONE
HESCHONG MAHONE GROUP, INC.
(916) 962-7001, ext 30
From: Kasman, Robert [mailto:
Sent: Friday, June 06, 2008 11:41 AM
To: Douglas Mahone
Subject: call today 1:30-2?
Hi Doug, are you free from 1:30-2 to discuss RNC and C&S evaluation?
In case you haven’t seen them, the CPUC posted the staff responses to the comments posted on the DEER Update at the www.deeresources.com website, including RNC. The login information for the website is below and the comments are the second link on the page. It would be helpful if you have a chance to review comments 33-36 before we talk.
Robert Kasman
Customer Energy Efficiency
Pacific Gas & Electric Company
Tel: 415-973-4094
Email:
From: Lai, Peter [mailto:
Sent: Friday, May 30, 2008 8:05 PM
To: ; ; ; ; ; ; ; ; ; Lawrie, Sandy; ; Wan, Mike; Ramaiya, Shilpa R; Kasman, Robert; ; ; McLennan, Robert (Law); ; ; ; ; Tisdale, Matthew; ; Barnes, Jennifer; ; ; ; Reyhner, Tsosie; Pham, Trinh; ; ; ; Mah, Ed; Hather, David T; Roffey, Robert; Ansar, Jasmin; ; ; ; ; Homsher, Ila; ; Cooper, Lee; ; ; ; ; ; Cho, Theresa; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; Watts-Zagha, Karen; ; ; ; ; ; ; ; Campbell, Michael (MPM); ; ; ; ; Ardell, Megan; Seager, Jonathan
Subject: DEER May 2008 Update
Dear DEER 2008 Update Discussion Topics Subscribers:
The Final DEER May 2008 Update for utilities energy efficiency portfolios planning purposes will be available at the URL: http://www.deeresources.com. To access this website, please use:
User ID: deer
Passcode: 2008
In the website, you will find:
1. Measure costs data
2. Net-to-Gross data
3. Effective Useful Life and Remaining Useful Life data
4. Energy Unit Savings and Load Shapes data
Responses to comments will also be posted at this website by Monday, June 2, 2008.
The DEER Team is in the process of posting the data onto the web server. So you will be able to access the data from the website in a few hours from now.
Please note that that there was some confusion about which EUL values the DEER team is recommending for the measures shown in the spreadsheet table attached. This spreadsheet table shows the proposed DEER EUL values. Energy Division will receive comments on the EUL values for these measures at http://www.energydataweb.com/cpuc, under the DEER EUL/RUL Discussion Topic, from now through June 6, 2008. Based on review and consideration of comments received, Energy Division will finalize the EUL values for these measures by June 11, 2008.