HVAC Road Map Summary

1 Summary of EM&V Activity for HVAC

1.1 HVAC Team Organization

The HVAC Research Roadmap lead is Jarred Metoyer. He is currently DNV KEMA’s lead on all HVAC-related Work Orders under the CPUC 2010-2012 EM&V prime contract. Mr. Metoyer is highly knowledgeable regarding Energy Division and IOU policies, programs and EM&V activities, given his recent experience planning 2010-2012 HVAC impact and market research and managing the HVAC PCG. He also has previous experiences managing the 2006-2008 HVAC HIM studies, advising on DEER 2009 HVAC updates.

Each project has a DNV KEMA manager assigned to avoid administrative issues, even if the study is likely to include extensive use of subcontractors. The project managers and key subcontractors are identified in the organization chart below. There are also cross roadmap advisors which are listed in particular for HVAC. All of the ED-led Research Roadmaps will include sampling, analysis, and data management. Additional firms on contract will also be added to the studies as needed including Cadmus Group and UC Davis.

1.2 Overview of Original Studies from EM&V Road Map Inventory

Impact Evaluation of Commercial and Residential Upstream HVAC Programs

Commercial Upstream HVAC is a major component of the HVAC portfolio and the IOUs were directed to include upstream incentive pilots for residential sector in their 2013-2014 portfolio, per Decision D.12.11.014.

Impact evaluation of the Commercial Upstream HVAC incentive was included in the 2010-12 HVAC evaluations, but critical questions still need to be addressed. New commercial HVAC equipment efficiency standards will make it more difficult to claim savings from Upstream HVAC programs. The installed efficiency, distribution system efficiency, and loads imposed on program incentivized HVAC equipment is not well known and is an important area of uncertainty. Installations after July 2014 will be subject to the new Title 24 code, including changes for fault diagnostic detection requirements.

Impact Evaluation of Commercial and Residential Quality Installation Programs and “to Code” HVAC installations Pilot

Both residential and commercial QI programs were to be part of the 2010-12 evaluation, but the programs were slow to start. Commercial QI pilots have been mentioned but there have been no savings claims. The CPUC has also directed the IOUs to create a pilot program for “to code” installations. These programshave the following known uncertainties and research risks:

Savings are highly uncertain, which causes low projected budgets and savings goals for 13-14 program cycle.
Benefits of requirements beyond Title 24 are not well known
There is a need to understand how well HVAC contractors utilize “whole building” design aspects when sizing and installing units (such as duct sealing, insulation, and building envelope integrity). The building envelope is not changed by QI but duct sealing and system sizing are primary aspects; so there is a need to understand coordination between QI and Energy Upgrade CA.
Need to determine the gross and net first year and lifecycle impacts for kWh, kW and Therms.

Impact Evaluation of Commercial and Residential Quality Maintenance Programs

The program has the following known uncertainties and research needs:

Uncertainties
Program participation has been slow to ramp up. The IOUs, however, areprojecting increased participation for both of these programs as theymove forward.
The program design for the 10-12 cycle introduced multiple service actions for which data is sparse.
Program design may continue to evolve as 10-12 impact data becomes available. Impact evaluations for 13-14 will need to take these changes into account.
Uncertainties in impacts of service actions on equipment efficiency, in-situ v. manufacturers’ name plate efficiency, distribution system efficiency and building loads are important cross-cutting issues.
Research Needs
There is a need to determine the gross and net first year and lifecycle impacts for kWh, kW and Therms.
Need to determine the baseline market conditions for small and medium commercial HVAC units.
What are the current maintenance procedures for non-participating contractors
How do customers view ACCA 180 maintenance?
What motivates customers and contractors to participate in the CQM program?
What are the costs to perform each maintenance measure to the customer and the contractor? This research should include new measures now incorporated in the programs, e.g., demand controlled ventilation, notched belts, etc.

Impact Evaluation of Deemed HVAC Measures

A gap exists for measures that are not addressed in QI, QM, and upstream programs. Examples include ductless mini splits, HVAC controls, VFDs for HVAC, and boilers and furnaces. Key research questions include:

Within this group of measures, which measures and engineering parameters have the highest priority?
What field data can inform DEER updates and workpaper revisions?
What field data are required to improve ex-post impact estimates?

Potential EM&V methods should include:

Scope will depend on the magnitude of the savings as well as the uncertainty in the DEER or workpaper estimates.
Field data collection should focus on obtaining parameter level data to reduce savings uncertainty.

Laboratory Testing

Lab testing is an ongoing activity spanning multiple program cycles. Lab testing is an integral part of the overall HVAC Program impact evaluation as it provides insights into the impact of installation techniques and service actions that are not possible in a field only study. Potential EM&V methods should include:

Any testing identified in the 10-12 plans or during the 10-12 cycle that was not completed will be addressed during the 13-14 cycle.
Additional tests will be identified based on the needs of the DEER updates, non-DEER workpaper reviews and ex-post impact evaluation.
Tests will be conducted in qualified laboratories using AHRI, ASHRAE and/or ANSI standards.

Market Assessment: Identifying Baselines and Barriers for Existing HVAC Conditions, Building Permit and Title 24 Compliance

Objectives for these assessments are as follows:

Investigatemagnitude of the failure of contractors to obtain required building permits and to comply with building energy efficiency standards (Title 24, Part 6).anddetermine why contractors are not pulling permits
Identify baselines for CA industry standard practice in obtaining building permits and achieving 2013 Title 24 compliance for residential and commercial HVAC equipment installations. Anecdotal information suggests that both compliance baselines are low. Investigate permitting and compliance separately including whether non-permitted installs are compliant and also if permitted installs are compliant.
Estimate a baseline for “existing conditions” found in the population of HVAC units across CA using site assessments performed during the 2010-12 cycle and additional non-participants in 2013-14.
Coordinate with and leverage the results of similar Codes & Standards EM&V (past cycles and 2013-14) research effort to identify market barriers to obtaining HVAC installation permits and Title 24 compliance. Determine the variance of permit cost and the scope across jobs.
Examine the awareness of contractors on the code requirements and inspection processes.
Examine the awareness of building code officials and HERS rater on the code requirements and inspection process.Identify knowledge gaps where training could help building code officials and HERS raters to better enforce HVAC requirements.
Improvethe market intervention strategies for HVAC market actors identified in the “Road to ZNE” study to pursue the state’s Big Bold Initiatives and ZNE goals for HVAC.

1.3 Proposed Changes to the Roadmap not redlined above

The current studies are not being changed. Changes to update the roadmap that do not entail major reorganization include creating multiple projects within study areas and providing some updates to the ED studies since the August 2013 HVAC roadmap.

Multiple projects within study areas

Multiple projects have been created in the large study areas to accommodate the high priority ESPI studies and better manage the large study areas in the HVAC roadmap.

Mini-split systems, which include variable refrigerant flow systems, are on the ESPI list and will have a high priority evaluability assessment prior to conducting the planned upstream program impact evaluation.
Quality maintenance is on the ESPI list and will have a high priority field and laboratory component. The 2010-12 evaluation has shown synergies in conducting field observations and laboratory testing.

Specific clarifications to August 2013 HVAC roadmap

Work Order 32: 2010-12 Residential and Small Commercial HVAC Impact Evaluation
Completion Date Q1 2014 changed from Q4 2013
Both residential and commercial QI were to be components of the 2010-2012 evaluation
Only residential QI included in WO32 due to slow start for commercial and no commercial QI workpaper in 2010-12. Not clear that there is an approved WP for CQI in 2013-14
Both Residential and Commercial QM programs are a component of the 2010-12 evaluation
Only commercial QM included due to slow start for residential
Note uncertainty in in-situ efficiency of equipment and systems and uncertainty in loads is common across HVAC measures. HVAC end-use calibration data lacking for commercial buildings.

2 Need and Application of Results

According to D.13-09-023 which established the Energy Savings Performance Index (ESPI); the ED Staff is instructed to create a list of measures that would require an ex post evaluation of energy savings to support an earnings claim should the IOUs include the measure in their programs/portfolio. These measures are selected for ex post measurement due to several factors, including the lack of recent program evaluation to inform/update ex ante estimates and for measures included in relatively new programs where performance is uncertain. In addition to the ESPI list, the ED Staff also conducted an analysis (P4) to identify measures that carry probability factors that may pose a risk to programs and/or the portfolio reaching their goals; thereby placing higher priority towards validating any outdated input variables for these measures that are needed for calculating energy savings.

Commission staff issued a Final List of ESPI measures memo on December 6, 2013. Mini-splits and quality maintenance are on the ESPI list while other upstream HVAC measures are on the P4 list.

We do not have sufficient data about HVAC market behavior to make well informed decisions regarding long term market transformation efforts. The PCG is interested to know how current and future HVAC research can best guide programs to target and fix the worst performing HVAC units. The savings estimates for the HVAC sector remain an area of considerable uncertainty. Questions exist about the true baseline of HVAC efficiency in CA:

What is the true as-found condition of HVAC systems regarding their design and installation, as well as changes since installation (charge level, non-condensables, etc.)
How do HVAC units perform in-situ relative to the manufacturers’ specifications?
What are the building heating and cooling requirements; and distribution system efficiencies that determine the loads imposed on the HVAC systems?
What are the energy improvements associated with bringing HVAC systems to baseline?
What are the baseline maintenance, installation, sales and distribution practices?
What is the baseline level of technician skill and diligence?
What is the role of building codes since not all HVAC work triggers code compliance?

3 General Approach, Issues, and List of Studies and Activities

Across studies there are a few primary types of data collection that can be used to conduct impact analysis. Field observations, field and laboratory measurements of HVAC unit performance and surveys are the primary methods. The cost of collecting field and laboratory measurements is a key constraint on the studies in terms of sample design. For the market assessment and net-to-gross components, market actor interviews and market data are also primary data that can be collected.

For each study the sources of uncertainty contributing to the ESPI and P4 assessments was reviewed and the priority types of data collection were defined. A large table was created to show the timing of work orders alongside the uncertain parameters. A second table was also created to show the specific issues or challenges with the approach in the original roadmap and a proposed approach to overcome the issue or challenge. These tables also form the basis for the work order proposals for the high priority and first year activities.

In all cases the Year 2 activities are roughly planned and the research activities may be completely changed based on the WO32 report and Year 1 progress.

Uncertainty
Studies / Measure / ESPI - P4 / UES / IRate / NTG / Proposed Work Orders and Timing
Upstream HVAC Program Impact Evaluation / Mini-splits / ESPI / Baseline and Measure performance / Medium / High / High Priority: Evaluability Assessment
Year 1: Surveys to determine if Mini-splits replace traditional systems
Year 2: Measure performance of installed mini-splits
Upstream HVAC Program Impact Evaluation / Package and split systems / P4 / Uncertainty in applications / High / High / Year 1: Phone and onsite surveys to determine install rate and applications of installed units;
Year 1: Separate WO: Pilot remote metering and correlation to AMICoordinate with WB interval data study conducted by IOUs, where meters are installed on individual HVAC units to validate WB disaggregation and energy savings algorithms.
Year 2: Either more verifications or begin monitoring starting with highest savings climate zone (1) and building type (1) per IOU
Impact Evaluation of QI and To Code Program / Quality Installation, To Code Installation / N/A / Baseline and Measure performance / Low / Medium / Year 1: Participant and Non participant sample design with project review followed by monitoring and testing,
Year 2: Small WO or add-on for direct observations if needed
Impact Evaluation of QM Program / Quality Maintenance / ESPI / Major Measure performance issues / High / Medium / High Priority: Continue Lab work and site visits with interim reports on findings
Year 1: WO for ex-post sample followed by case studies of achieved and remaining energy savings potential.
Year 2: WO for pre-post monitoring with direct observation… coordination challenges
Impact Evaluation of Deemed HVAC Measures / Multiple deemed HVAC measures / N/A / Varies / Varies / Varies / Year 1: Uncertainty analysis and focused study. Perhaps on gas measures not studied in 2010-12 and not mentioned in 2013-14 (e.g. residential furnaces)
HVAC Laboratory Testing / Laboratory testing / N/A / Informs QM and QI / N/A / N/A / Year 1: WO to test 2 large and 2 small units already at Intertek
Year 2: Set aside contingency to plan and conduct additional lab tests based on new field findings
Permit/Compliance Market Assessment / Obtain permit data for participants and non-participants installations / N/A / Informs QI Baseline, To Code Baseline / Permit Rate / NA
Year 1: Permit rate study, market actor interviews; Await IOU customer surveys before fielding non-participants site visits
Year 2: Compliance study, same onsite procedure as QI and To Code
Studies / Measure / Original Roadmap Approach / Issues and Considerations / Proposed Approach
Upstream HVAC Program Impact Evaluation / Mini-splits / Same as upstream, one of 3-4 types / Unknown if these systems are added as new cooling capacity in previously un-cooled space, installed in lieu of standard efficiency mini-splits or are replacing traditional systems; Many cooling performance issues documented by Taylor Engineering, most data from field is for these replacing resistance heaters in the Northwest / Evaluability assessment: Include data collection to first determine baseline and applications through surveys. Several metering challenges can be outlined through literature review, with metering protocols established through a pilot/case study.
Upstream HVAC Program Impact Evaluation / Package and split systems / Field monitoring will be done for both participant and non-participant units in the residential and commercial sectors across a variety of climate zones and building types / Consider participant non-participant, 3 HVAC types, 6 key climate zones and at least 5 building types means 180 strata for sampling. Therefore even minimal samples will create costs far exceeding the budget for even simple metering of unit power. Sampling detailed performance would further limit the sample by increasing cost per point. / Conduct a metering pilot to determine whether AMI data can be leveraged to fill all the strata needed based on comparison to direct end use meters with remote communications.
A phone survey and followup onsite effort to verify installations and characterize installed application may be able to achieve large samples.
Impact Evaluation of QI and To Code Program / Quality Installation, To Code Installation / Either conduct a follow-up to 10-12 study or a new study depending on participation levels and program design changes. / 2010-12 Study sample doesn’t offer for much follow-up beyond the PG&E Compliance follow-up; Field measurements without monitoring may be required for the allocated budget or combine this with the Upstream effort / Focus on participants in the two 2013-14 programs (QI and To Code) since the budget is $500k. The larger market assessment should conduct the non-participant assessments since the budget is $1.25M. Non-participants under 2008 Title 24 would include installs from 2009 to June 2014.
Conduct field monitoring of participants (both QI and “To Code”) as well as sites where a permit was not pulled for a variety of climate zones and building types.
Impact Evaluation of QM Program / Quality Maintenance / Evaluation should focus on measures/treatments that have the highest uncertainty in projected savings for both the residential and commercial sectors. Pre/post monitoring and/or billing analysis of participating units. / Several issues with the pre-post design as documented by WO32, but pre-post is still the preferred ex post method / Ex-post observations followed by debrief to program and technician. Sub-sample may include case study which may be presented to the technicians as a training opportunity and to estimate achieved and remaining energy savings potential.
Without pre-measurement, installation rate is primary parameter measured, some UES assumptions in workpapers can be modified with measurements after measures are installed, but does not inform measure baseline without pre-measurement.
Work on pre-post coordination, but not on the annual reporting timeline. This means “Pre” in 2014 and “Post” in 2015. Otherwise the program measures may need to be all installed at the same time which is not part of program design, but is needed to streamline ex post M&V.
Calibrated simulation modeling based on monitored data. Estimate savings by individual service action. / Billing analysis will capture “noise” based on numerous observed issues and program allows technician discretion increasing variability
Impact Evaluation of Deemed HVAC Measures / Multiple deemed HVAC measures / Scope will depend on the magnitude of the savings as well as the uncertainty in the DEER or workpaper estimates. Field data collection should focus on obtaining parameter level data to reduce savings uncertainty. / Budget will not allow much field work unless focused to a single measure / Perhaps focus on gas measures not studied in 2010-12 and not mentioned in 2013-14 (e.g. residential furnaces)given all other efforts focusing on cooling
HVAC Laboratory Testing / Laboratory testing / Any testing identified in the 10-12 plans or during the 10-12 cycle that is not included in the current scope will be addressed during the 13-14 cycle. / No issues with scope to do tests not completed in 10-12. / Two Phases: Conduct lab work not completed in 10-12 and reserve funds to test based on new field findings
Tests will be conducting in qualified laboratories using AHRI, ASHRAE and/or ANSI standards / Do need to conduct tests at off AHRI conditions which includes economizers and faults. Follow AHRAE/ANSI instrumentation protocols and applicable. Test conditions per test plan.
Permit/Compliance Market Assessment / Obtain permit data for participants and non-participants installations / Obtain total permit counts and permit/compliance costs. / Using QM sites for permitting has issues since they are existing systems.
Permitting rate only requires counts of non-participant units with permits, while compliance requires more detailed file review and onsite inspections. The performance of systems that are close to compliance is also different than those with duct leakage, sizing, and airflow far from the standard / Site surveys for non-participants only as they will serve as a baseline for both QI and To Code. QM participants should be excluded.
Market Actor interviews should include: HERS Raters, Code Officials (part of permit data collection), CSLB, Contractors, Distributors, CEC Code setting staff.
The IOU Market Assessment may be a good gateway survey of customers to identify non-participants and then offer a follow-up onsite.
• Conduct site surveys at a sample of QI/QM program participants and non-participants (permitted/unpermitted, if available) to obtain T24 compliance baseline estimate and collect data related to HVAC Market Transformation Indicators.
• Study how effectively HVAC replacements, including emergency replacements, are complying with permitting obligations under SB 454 and directed in the Guidance Decision 09-11-014.
• Conduct phone interviews with market actors.
• Coordinate sample design and data collection with “Market Assessment of Residential and Commercial Customer Decision-making on High Efficiency HVAC Systems, Quality Maintenance and Quality Installation Purchases.”