National Academies Press: OpenBook

Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards (2021)

Chapter: 3 Assessing Current Models: Engineering and Technology Cost and Technology Performance

« Previous: 2 Framework for Developing Regulations
Suggested Citation:"3 Assessing Current Models: Engineering and Technology Cost and Technology Performance." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards. Washington, DC: The National Academies Press. doi: 10.17226/25992.
×

3

Assessing Current Models: Engineering and Technology Cost and Technology Performance

INTRODUCTION

The committee conducted a review of the Engineering and Screening Analysis based primarily on the Technical Support Documents (TSDs) that support the Notice of Proposed Rulemaking for three rulemakings supplied by the U.S. Department of Energy (DOE): standards for Residential Furnaces (DOE, 2016), Residential Dishwashers (DOE, 2014b), and Commercial Refrigeration Equipment (DOE, 2014a). DOE tasked the committee with reviewing the regulatory analysis of these three products as exhibits of its overall process. The committee also considered: the Federal Register notices promulgating standards regulations and those describing the process for setting such regulations; briefings from DOE and its contractors; and replies from DOE to the committee’s written queries. Overall the committee found that the approach DOE used for the Engineering and Technology Cost and Technology Performance, including the Pre-ANOPR (Pre-Advanced Notice of Proposed Rulemaking) Screening and Analysis of Design Options was sound but could be improved.

SCREENING AND ANALYSIS OF DESIGN OPTIONS

There are at least two analyses DOE conducts prior to the publication of the Advanced Notice of Proposed Rulemaking. The DOE Process Rule of 1996,1 which prevailed at the time the rulemakings the committee considered were conducted, keeps these together under the heading, Pre-ANOPR Screening and Analysis of Design Options.2,3,4 These include, first, the Screening Analysis, discussed further below. This is followed by the Engineering Analysis, which includes a Performance Analysis and a Cost Analysis.5 With respect to the Performance Analysis (also referred to as the Efficiency Analysis), DOE

___________________

1 U.S. Department of Energy (DOE), “Energy Conservation Program for Consumer Products: Procedures for Consideration of New or Revised Energy Conservation Standards for Consumer Products: Notice of Final Rulemaking,” Federal Register 61: 36974-36987.

2 This pre-ANOPR stage is to determine whether “candidate standard level is likely to produce the maximum improvement in energy efficiency that is both technologically feasible and economically justified or constitute significant energy savings.” Federal Register 85: 8705c.

3 See p. 36976b of DOE, “Energy Conservation Program for Consumer Products: Procedures for Consideration of New or Revised Energy Conservation Standards for Consumer Products: Notice of Final Rulemaking.” Federal Register 61: 36974-36987.

4 The “Process Rule” was updated in 2020. See DOE, “Energy Conservation Program for Appliance Standards: Procedures for Evaluating Statutory Factors for Use in New or Revised Energy Conservation Standards,” Federal Register 85(161): 50937-50944.

5 See, for example, slides 16 through 25 of DOE, “Commercial Refrigeration Equipment,” presentation to the committee, November 19, 2019.

Suggested Citation:"3 Assessing Current Models: Engineering and Technology Cost and Technology Performance." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards. Washington, DC: The National Academies Press. doi: 10.17226/25992.
×

TABLE 3.1 Sequencing of Analyses Under the Process Rule

Preliminary Analyses NOPR Final Rule
Market and technology assessment Revised preliminary analyses Revised NOPR analyses
Screening analysis Life-cycle cost sub-group analysis
Engineering analysis Manufacturer impact analysis
Energy use determination Emissions impacts analysis
Markups for equipment price determination Monetization of emissions analysis
Life-cycle cost and payback period analysis Utility impact analysis
Shipments analysis Employment impact analysis
National impact analysis Regulatory impact analysis

NOTES: In the Residential Dishwashers rulemaking, DOE conducted the analyses listed in the column, “Preliminary Analyses,” as part of the NOPR analysis. NOPR = Notice of Proposed Rulemaking.

SOURCE: DOE (2014b).

uses two different methods separately or in combination: an efficiency-level approach and a design-option approach. Table 3.1 shows the broader context and sequencing of these analyses as the process of standards progresses.

SCREENING ANALYSIS

Overview

The Screening Analysis precedes the Engineering Analysis.6 This first step screens all known design options and sets aside those that (1) DOE deems technologically infeasible, (2) DOE views as impractical, (3) will present an adverse impact on product utility or product availability, or (4) will cause an adverse impact on health or safety. However, DOE revises screening decisions based on new information.7 The analysis applies a strict filter that removes from further consideration technologies that do not meet screening criteria. These criteria for exclusion will vary by the end-use appliance, but include the following criteria:

  • The technology option could cause impacts on product safety. Can another, safer option replicate the increase in efficiency of this technology adoption?
  • The option is technologically feasible but unproven. Because the definition for unproven is vague, it would prove useful to label technologies according to their Technological Readiness Level.
  • The option causes a significant decrease in the functionality (product utility) of the appliance. While there is no convenient method to measure functionality across the range of appliances covered by DOE’s equipment standards, reviewing the appliance’s original purpose can determine if compromises have been made.

___________________

6 See p. 36982c of DOE, “Energy Conservation Program for Consumer Products: Procedures for Consideration of New or Revised Energy Conservation Standards for Consumer Products: Notice of Final Rulemaking,” Federal Register 61: 36974-36987.

7 Ibid.

Suggested Citation:"3 Assessing Current Models: Engineering and Technology Cost and Technology Performance." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards. Washington, DC: The National Academies Press. doi: 10.17226/25992.
×
  • The option is not practical to manufacture, install, and service. For example, DOE screens out dishwashers that save energy and water by relying on a cold-water supply connection because it assumes that all dishwashers in the United States are connected only to hot-water lines and is impractical to change this. Consumers can take advantage of appliances that manage their own water heating by turning down the thermostat on the household water heater thereby reducing water heating energy.
  • Done properly, the preliminary technical assessment phase will not exclude any reasonable technology option and enhanced functionality, including, potentially, those not yet included in appliances and equipment currently on the market. These may include attributes such as home safety and improved indoor air quality. DOE should continue to consider how a standard can reduce household energy beyond just the appliance’s energy, enhance broader household living quality, and deliver other attributes.

In a preliminary assessment, the evaluated technologies need not be commercially available. This is not explicitly stated in DOE’s documentation, and, for example, for Commercial Refrigeration Equipment (CRE)—one of the three rulemakings shared in detail with the committee—DOE’s briefing material states that: “DOE identifies technologies through general research, discussion with manufacturers and other experts, review of equipment literature, and reverse engineering of equipment.”8 Similarly, by eliminating alternatives that seem impractical to install -such as dishwashers certified for a cold-water connection, DOE may be depriving customers of appliance features that can be advantageous—in this case, reducing water-heating energy and enhancing home safety.

FINDING: Screening analyses are designed to be conservative and can therefore have the unintended consequence of excluding specific technology options that, while not currently in use, may have significant potential to enter the market.

Proprietary Technology

A further consideration is the treatment of propriety technology in the screening analysis. In a briefing to the committee, DOE also suggested that they may screen out proprietary technology options. Proprietary technology innovations can have a substantial efficiency impact on appliances. Many state-of-the-art appliances are initially proprietary, but, later on, several firms manufacture them. Also, the time period between rulemakings on a product is such that missing a rulemaking cycle could create a barrier to deployment of more efficient technology lasting several years unless market forces result in deployment of a cheaper, more efficient product or equipment. However, mandating a particular technology option as a standard can offer a given developer an unfair advantage, which DOE cannot endorse. The DOE program is further subject to review by the U.S. Department of Justice to ensure a candidate standard does not lessen competition. This is important for the standard setting process since considering these nascent technologies could provide DOE with justification for setting a higher energy efficiency standard for a whole class of technologies, since technology developers have demonstrated (at least in the mid TRL range) that advanced designs are possible and affordable.

Emergence of New Technology

There are several examples of DOE revising initial screening results based on new information. One example is regarding variable-speed compressor technology implemented in portable air conditioners. DOE’s initial evaluation for this technology determined that this technology option would offer no

___________________

8 DOE, “Commercial Refrigeration Equipment,” presentation to the committee, November 19, 2019.

Suggested Citation:"3 Assessing Current Models: Engineering and Technology Cost and Technology Performance." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards. Washington, DC: The National Academies Press. doi: 10.17226/25992.
×

measurable improvement (DOE, 2015). However, before DOE completed the analysis that accompanied the Notice of Proposed Rulemaking (NOPR), variable speed compressors became available in a capacity range suitable for portable air conditioners. DOE subsequently changed the screening results to allow variable speed compressors to be considered.9 A second example involves using isobutane refrigerant for residential refrigerators, refrigerator-freezers, and freezers as a replacement for HFC-134a refrigerant. DOE had screened out isobutane as a technology option since it is somewhat flammable.10 Subsequent studies of isobutane’s performance and safety in a controlled environment caused DOE to reconsider its screening and allow isobutane as a refrigerant.11

In future assessments, DOE may find it useful to make a broader solicitation for comments on screening and technical assessments while they remain in draft form. Transparency and public engagement would likely improve the quality of the data and improve many aspects of the later analysis. Similarly, for Residential Furnaces, DOE evaluated technology enhancements that they do not typically include in baseline models. These enhancements have the potential to improve the Annual Fuel Utilization Efficiency (AFUE) rating of furnaces. A key example is the development of a furnace that fully condenses its flue gases. The process of fully condensing flue gases extracts 95% or greater of the thermal energy of the furnace for beneficial use.

Technological Readiness

A well-developed and widely used method for identifying the readiness of a technology for dissemination into the marketplace is the technology readiness level (TRL) taxonomy (see Box 3.1). Developed by the NASA to manage its space-related research and development, TRLs are a convenient means of describing the development stage of increasingly efficient technologies, devices, and their components (NASEM, 2016).

___________________

9 DOE, “Examples of Screening Decisions Based on New Information,” personal communication from John Cymbalsky, Office of Buildings Technology, to Martin Offutt, National Academies, June 2, 2020.

10 DOE, 2009, “Energy Conservation Standards for Residential Refrigerators, Refrigerator-Freezers, and Freezers: Public Meeting and Availability of the Preliminary Technical Support Document,” Federal Register 74: 58915-58918. November 16.

11 DOE, 2011, “Energy Conservation Program: Energy Conservation Standards for Residential Refrigerators, Refrigerator-Freezers, and Freezers: Final Rule.” Federal Register 76: 57516-57612. September 15.

Suggested Citation:"3 Assessing Current Models: Engineering and Technology Cost and Technology Performance." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards. Washington, DC: The National Academies Press. doi: 10.17226/25992.
×

The current DOE Appliance and Equipment Standards Program screens out technologies below TRLs 7, 8, or 9 depending on how the Pre-ANOPR criteria are applied. There are nonetheless technologies that are at early, pre-competitive TRLs and have promise for use in consumer products and commercial/industrial equipment as part of the product population analyzed. Some earlier, pre-competitive technologies may ultimately have promise for use in appliances and equipment and bring these into volume production so that they have a broad, positive impact on the energy efficiency. Energy Star and utility demand side management programs, discussed in Chapter 5, may facilitate this.

Screening Analysis: Findings and Recommendations

FINDING: DOE has the ability to insert a new product for analysis after the screening analysis may have initially ruled it out. Such can be justified especially if new information becomes available about a screened-out technology.

RECOMMENDATION 3-1: DOE should consider technologies that are at early, pre-competitive technological readiness levels and have promise for use in consumer products and commercial/industrial equipment as part of product population analyzed, even if it seems plausible that they will be screened out in later stages of the analysis such as in the Screening Analysis made during the Notice of Proposed Rulemaking. DOE should continue to use the tools at its disposal, such as reconsidering a previously excluded technology, to avoid prematurely screening-out innovative technologies.

FINDING: The screening analysis may in some instances be removing new technologies that are otherwise good candidates from further consideration.

FINDING: Ranking technologies by their proximity to market-readiness may help rationalize the screening process.

RECOMMENDATION 3-2: DOE should adopt a taxonomy for labeling the readiness of technologies, such as technology readiness level or “TRL” as it is widely known.

ENGINEERING ANALYSIS

Objective of DOE Engineering and Technology Analysis

The objective of the Engineering Analysis is to develop cost-efficiency relationships. DOE does this by estimating the manufacturer’s costs of achieving increased efficiency levels (ELs) and determining the maximum technologically feasible ELs. This relationship facilitates subsequent or downstream analyses of consumer life-cycle cost, manufacturer impact, and employment impact. In the case of residential dishwashers, DOE considers both electricity and water consumption. They establish these cost-efficiency relationships for products with and without efficiency improvements.

It is unclear which future drivers of future costs, technology innovation, and consumer behavior DOE considers in these analyses and how it formulates a framework using models to assess how these market drivers will impact the creation and use of standards and other methods to improve end-use efficiency for building and equipment performance. The committee offers observations on these future changes in Chapter 6.

New technologies that DOE may consider within a product category can form two groups: (1) Those that improve the energy efficiency of buildings and equipment and (2) those that more broadly increase

Suggested Citation:"3 Assessing Current Models: Engineering and Technology Cost and Technology Performance." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards. Washington, DC: The National Academies Press. doi: 10.17226/25992.
×

the functionality of devices. The second category may add features to the appliance or device or may yield companion or secondary benefits such as reduced use of water consumption (e.g., in a dishwasher) or reduced emissions (e.g., soapy water discharge in a clothes washer). The second category will likely impact the market acceptance of products and therefore affect an evaluation of standards. DOE needs to consider these exogenous factors as well.

Performance Analysis (Efficiency Analysis)

Within the Engineering Analysis, DOE conducts the Performance Analysis (Efficiency Analysis) using either the efficiency-level approach (e.g., as with the residential dishwasher rulemaking), the design option approach (e.g., the CRE rulemaking), or a combination of both (e.g., the Residential Furnaces rulemaking).

Efficiency-Level Approach

This approach uses data from consumer purchases to estimate ELs for every product category. (These ELs are ordered and numbered as EL1, EL2 and so forth.) As products reach the market, consumers “vote” on the most desirable product that is available to them through their purchase actions. For any product class, there is a distribution of products available of various efficiencies. Efficiency levels are “discrete energy efficiency tiers that serve as potential standard levels.”12 DOE interviews manufacturers to get their feedback to validate the selections. These efficiency levels correspond to “Tiers” suggested by the Consortium for Energy Efficiency for utility energy efficiency appliance rebate programs.13

Design Option Approach

The Design Option approach uses detailed engineering calculations often based on computer simulations of various design options. These simulations use engineering calculations that deterministically calculate performance of end-use energy-consuming appliances and equipment. In some cases, DOE identifies design levels through a complete physical tear down of actual products. The design option approach is key to filling in gaps between efficiency levels and to determine the maximum level of efficiency a given product can obtain. (Life-cycle costs including disposal are not considered in this step.) The committee believes that understanding the impact of new and innovative technologies necessitates evaluating the energy efficiency of innovative technologies, the consumer’s behavior in accepting the new or updated products, and the consumer’s behavior in operating and maintaining the device where it impacts market acceptance and energy consumption. DOE relies on expert opinions for the impacts of consumer behavior.

FINDING: A structured program of market research that includes a combination of techniques would provide more reliable information than techniques relying solely on expert opinion to analyze consumer behavior. This is necessary to understand consumer behavior and appliance purchase decisions.

___________________

12 See slide 15 of DOE, “Commercial Refrigeration Equipment,” presentation to the committee, November 19, 2019.

13 Understanding New Energy Star and CEE Tiers, J. Linsenmeyer, AC & Heating Connect, Emerson Electric Company, 2020, http://www.ac-heatingconnect.com.

Suggested Citation:"3 Assessing Current Models: Engineering and Technology Cost and Technology Performance." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards. Washington, DC: The National Academies Press. doi: 10.17226/25992.
×

Consideration of Advanced Technology

The search for advanced technology involves understanding costs and performance both of devices currently in the marketplace and of new technologies, whether proprietary, pre-competitive, or emerging. Some of these technologies may offer still greater improvements in cost and performance. These new technologies are often not yet incorporated in products on the market. (See Box 3.2.) It is possible that in the course of considering which options are or are not technologically feasible,14 DOE may screen them out at the beginning of the standards-setting process. Use of TRLs may provide useful indicator during the process of considering this technological feasibility.

FINDING: DOE has thoroughly addressed the cost and performance of technologies on the market. Analysis that considers new and improved technologies including the most innovative technologies can complement the cost and performance analysis.

The pathway to develop new technologies notionally includes the following:

  1. Emergence of candidate technologies that have the potential to meet or exceed the requirements of existing products in the marketplace.
  2. Development of a prototype that demonstrates the functional requirements of the appliance.
  3. Funding the development, including an incubation period to achieve market acceptance.
  4. Determining how to reduce the cost of new components, often caused by low volume production.

___________________

14 See slide 11 of John Cymbalsky, DOE, “Appliance and Equipment Standards Program Buildings Technology Office,” presentation to the committee, November 19, 2019.

Suggested Citation:"3 Assessing Current Models: Engineering and Technology Cost and Technology Performance." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards. Washington, DC: The National Academies Press. doi: 10.17226/25992.
×
  1. Exploring options to optimize the efficiency of the appliance beyond that which the new technology contributes. This may be because of synergy between the disruptive technology and the existing components in the product.
  2. Creating a demand for the new technology and design production for high volume.

DOE participation in the development of new technology can, in certain instances, sharply accelerate development and ultimate deployment (NRC, 2001). DOE could in principal balance its Appliance and Equipment Standards Program with its other programs in the Building Technologies Office by crafting standards that are more liberal in incentivizing innovation. Assessing this complementarity was beyond the committee’s charge.

Alternative Means to Determine and Characterize Product Efficiency

Laboratory testing, field demonstrations, and modeling can augment both the efficiency-level approach and the design option approach. DOE can also use modeling to combine computer simulations with actual field data. These approaches are described below.

Laboratory Testing

Laboratory testing of small samples will not be sufficient for DOE’s purposes. The data reliability and demand characteristics of an appliance population can differ substantially from data derived by a small sample under laboratory test. However, laboratory testing can be useful to validate other results.

Empirical Data

The committee believes that engineering analysis is a useful activity but can be complemented by monitoring performance in deployment of new or improved appliances. The optimal evaluation would involve field tests of a population of similarly built homes or buildings to test two alternative technologies and obtain reliable data. In residential tests, there is a dramatic difference between the energy consumption measured in homes in differing climates and the same equipment measured in the laboratory or simulated by a computer model. For example, without careful evaluation, it may appear that appliances are poorly performing when in fact they were improperly installed. There can also be substantial differences between samples due to consumer behavior. In tests of building space conditioning, some of these could be obvious (i.e., thermostat settings) and others less so (opening windows and doors). For example, thermostat settings can change the performance of an appliance, effectively de-rating the BTU/kWh (British thermal units per kilowatt-hour) required.

Regarding heating, cooling and refrigeration appliances and devices, the Association of Heating, Refrigeration and Air-conditioning Engineers (ASHRAE) has published a large portfolio of “Method-OfTest” procedures for various appliance types, including room air conditioners (ACs), refrigerant expansion valves, refrigerant condensers, refrigerant compressors, liquid chillers, vending machines, unitary ACs, commercial refrigerant cases, and air coils.

Variability and Uncertainty

As discussed in Chapter 2, in the section “Criteria and Principles Guiding This Review,” uncertainty on factors affecting the costs and benefits of the standards must be propagated throughout the analysis to obtain an estimate of the probability distribution of all of the relevant quantities calculated. Following the regulatory analysis framework described in OMB Circular A-4, taking note of that circular’s guidance for understanding and presenting uncertainty, would facilitate greater clarity that could be reflected in DOE’s

Suggested Citation:"3 Assessing Current Models: Engineering and Technology Cost and Technology Performance." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards. Washington, DC: The National Academies Press. doi: 10.17226/25992.
×

analyses. The correct efficiency rating of a consumer appliance or commercial equipment would reflect performance from a “typical” installation. For example, in the case of residential air conditioning equipment, the in situ performance could be characterized at various levels such as ideal, mid-range, and non-ideal consumer behavior. These levels would reflect consumer behaviors involving fluctuating thermostat settings or the opening of windows and doors.

A sufficiently large number of units of the same product/appliance needs to be installed and metered to characterize the range of performance encountered in deployment. This may be costly and offer unworkable delays when an accurate performance estimate for an appliance population is needed. With a well-defined population and functioning metering equipment, the resultant load shape data collected will represent the appliance population. In any sizeable population of end-use devices, some will perform poorly or not at all, while other devices will perform as expected.

FINDING: The relationship of appliance performance to consumer behavior could be increasingly relevant as appliances become smarter and are able to dynamically change their in-situ efficiency by adapting to exogenous conditions.

RECOMMENDATION 3-3: DOE should characterize the uncertainty on the efficiency rating in its analyses. Rather than providing a “point” estimate of efficiency, DOE should provide a range that reflects the variability in energy consumption under different uses and consumer behavior. This should reflect the need to account for the uncertainty arising from the range of conditions under which a consumer may use a product and then using those use-case ranges to establish sensitivity studies so that efficiency can be communicated as a range rather than a single estimate.

New Data Sources

The ongoing explosion in the use of sensors, computational ability, and communications is evolving an ever-increasing data file of consumption data at the appliance, building, and system level. For instance, in web-connected appliances, some manufacturers are now incorporating. For example, once some smart appliances are connected, the consumer is prompted to enter connectivity data and then receives communications from the manufacturer offering the ability to monitor and manage consumptions. This has the advantage of potentially reducing manufacturing cost and enabling new revenue streams. These data can become the means to monitor end-use efficiency. Further discussion may be found in Chapter 6.

Data Aquistion and Cost Analysis

The Data Acquisition and Cost Analysis’ objective is to develop sound engineering cost analyses of appliances with and without certain features. DOE prepares a cost-benefit analysis for various levels of product efficiency to do so. The Engineering and Technology Cost Analysis is the first step in determining if efficiency improvements are economically justified and is a key driver for all subsequent analyses.

There are several elements to completing sound engineering analyses:

  • Establishing representative material and labor prices.
  • Choosing baseline models representative of the technology in the marketplace before implementing the proposed rule making.
  • Validating the models used to evaluate the cost-effectiveness of energy efficiency improvements.
  • The Cost Analysis is necessary in providing the financial data to support a rigorous Regulatory Impact Analysis (RIA) (see Chapter 2). DOE conducts the Cost Analysis portion of the
Suggested Citation:"3 Assessing Current Models: Engineering and Technology Cost and Technology Performance." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards. Washington, DC: The National Academies Press. doi: 10.17226/25992.
×
  • Engineering Analysis using one or more of a combination of cost approaches. (See Box 3.3.) The choice of techniques depends on the level of public and manufacturer information available. There are certain elements a cost analysis typically includes, which are listed and described below.
    • Ranges of cost data with accompanying bands of uncertainty. At present, DOE does not include a representation of uncertainty in this step. These estimates are very dependent on mark-ups by materials and costs incurred by manufacturers and internal pricing burdens. Extra features are often market-based (what the market will bear over a base price), which further obscures the real price, making it difficult to compare one manufacturer to another. Also, the country of manufacture and the distribution channel adds further uncertainty.
    • Energy consumption patterns. The energy consumption patterns that are projected for an appliance will typically include results by seasonal time-of-use (day, week, and month). DOE’s existing practice focuses on aggregate estimates of market penetration, which are not sufficient for the thorough Regulatory Impact Analysis (see Table 3.1 and Figure 2.1) that it will increasingly need.
    • Diversity factors. Diversity factors estimates to account for varying levels of performance for appliance populations at the same Trial Standard Level (TSL) in these studies. In addition to simulation or test results for one device, estimates should account for a statistically significant population of devices.
    • Energy demand estimates. Typically such estimates are derived for the product on an hourly basis. This in addition to the availability of data on performance in different climate zones—data that would beneficially reduce uncertainty and allow understanding the user’s influence on the uncertainty.
    • Ambient conditions. Variance of demand for Heating, Ventilation and Air Conditioning (HVAC) devices are subjected to variance in ambient temperature and humidity. Sampling studies done on a range of climate areas would also benefit understanding the real energy impact of the appliances’ use and time-of-use.

RECOMMENDATION 3-4: DOE should launch a study to investigate improved methods for data acquisition and analysis for use in setting and revising standards. This study should include an overview of where DOE helps enable collection of these data now and how they can improve that process at minimum impact on manufacturers.

Connecting Efficiency Levels (ELs) with Cost

DOE generates and uses cost curves to estimate life-cycle cost break-evens. These curves connect the TSL point for energy efficiency and cost. This allows DOE to choose efficiency levels that reflect costs of welfare-beneficial investments by consumers. Choosing ELs that yield the highest energy efficiency level may not be economically justified if the energy savings and reduced externalities do not outweigh the increased cost of purchase and installation. The choice of ELs considered is critical to the overall success of an efficiency standard.

Suggested Citation:"3 Assessing Current Models: Engineering and Technology Cost and Technology Performance." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards. Washington, DC: The National Academies Press. doi: 10.17226/25992.
×

Findings and Conclusions

FINDING: The DOE innovation programs that comprise the Buildings Technologies Office are supportive of new, innovative or disruptive technology used for or in consumer products and buildings/industrial equipment. The Appliance and Equipment Standards Program is not explicitly designed to do this. While disruptive products and equipment are a welcome find during the Engineering Analysis, DOE’s Appliance and Equipment Standards Program has the effect of raising the energy efficiency of existing devices and appliances incrementally.

FINDING: Disruptive or innovative technologies may emerge as a result alongside the standard setting process of DOE’s appliance efficiency standards, and these may have a large effect on increasing the efficiency with which energy services are delivered.

Suggested Citation:"3 Assessing Current Models: Engineering and Technology Cost and Technology Performance." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards. Washington, DC: The National Academies Press. doi: 10.17226/25992.
×

FINDING: A cost analysis typically includes a number of elements such as: ranges of cost data with accompanying bands of uncertainty; energy consumption patterns; diversity factors; energy demand estimates; and ambient conditions—all of which could be included in the Engineering Analysis to improve its fidelity.

RECOMMENDATION 3-5: DOE should expand the Cost Analysis segment of the Engineering Analysis to include ranges of costs, patterns of consumption, diversity factors, energy peak demand, and variance regarding environmental factors.

ELECTRIC POWER SYSTEM

Many of the devices subject to the Appliance and Equipment Standards Program are electricity-consuming. The standards consider the performance of such appliances and equipment to be the kilowatt hours (kWh) of electricity consumed at the point of end-use. Nonetheless, any change in consumption implied by the more efficient appliance is by itself incomplete without consideration of the timing and source of the electricity that is to be consumed. This section discusses a few of the notions that take this wider perspective on electricity consumption and how such notions dictate the value of the reduction or change in electricity consumption beyond the mere kWh that might be saved.

Load Shape Modeling

In addition to relying on Engineering Technical performance for standards-making, the efficiency data used to develop standards are critical for other energy analyses, sometimes referred to as “load shape modeling.” In planning and ultimately operating an electric utility, the collective electricity demand of the customer is critical. Load patterns determine the degree to which power plants are operated. Planners also use the patterns to determine what type and what generation capacity of power plants a balancing authority needs, how to operate them, what other resources to deploy, and the need for spinning reserves. Making estimates of the future patterns in the timing and amounts of electricity demand is called load shape modeling. These analyses rely on data that help characterize the end-use of energy-efficient appliances. In that regard, estimating the impact of DOE standards is critical. Examples of load shape modeling include Public Service Electric & Gas Company’s ELCS (Electric Load Curve Synthesis) model (Gellings and Taylor, 1981) and the Electric Power Research Institute’s HELM, the Hourly Electric Load Model.15 In the development of these models, it was important to replicate the behavior of major electric end-use appliances, both concerning their demand over various time horizons and their performance relative to variations in outside temperature and humidity. In addition, variations in consumer behavior were simulated by assuming variations in the demand relative to outside air temperature in winter conditions and weighted temperature-humidity in summer conditions.

Demand Response

The cost-efficiency relationship in the Engineering Analysis assumes that the only two determinants for consideration in appliance standards are cost and efficiency. However, demand response (DR) readiness is increasingly an important dimension in evaluating end-use appliances. There are three types of demand response: emergency demand response; economic demand response; and ancillary services demand response. DR readiness assures that a product can respond to each. This functionality is a

___________________

15 The Hourly Electric Load Model (HELM) was developed by ICF Incorporated for the Electric Power Research Institute under EPRI Project RP-1955-1.

Suggested Citation:"3 Assessing Current Models: Engineering and Technology Cost and Technology Performance." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards. Washington, DC: The National Academies Press. doi: 10.17226/25992.
×

prerequisite for the product to become part of a portfolio of demand response capability for responding to electricity market demands and system operator needs.

Electric power systems operate optimally when the demand imposed on them at each point in time is equal to the supply available. Demand for electricity varies over the course of a day and year, while the cost of generating electricity changes as well, depending on available resources. Economic demand response can judiciously decrease demand so as to offset expensive generation, for example on hot afternoons or when renewable power is unavailable.

Demand response programs can contribute to system operator needs as well. Ancillary services are electrical services necessary to support the transmission of electric power from producers to energy service companies and consumers. These services are determined in real time by grid operators within those control areas to maintain reliable operations of the interconnected transmission system. Typical services are those which provide various types of reserve capacity and assure system stability.

Emergency demand response is used in times of emergency such as a critical weather event, for example, Superstorm Sandy or a large area blackout such as the Northeast blackout of 2003.

Demand response programs may include utility activities such as: the use of innovative rate designs (e.g., Time-of-Use Pricing [TOU]; Critical Peak Pricing [CPP]; Extreme Day Pricing; or Real Time Pricing [RTP]); use of direct load control of air conditioners or electric water heaters; the use of utility-controlled customer-owned electric energy storage; or load control programs where a partial or full customer outage is pre-negotiated with individual large consumers. As the amount of renewable resources are added to the power system, the role of DR will become increasingly important in order to manage the variability of these resources and keep the system in balance.

Consumers who adopt appliances that are DR-ready have the potential to participate in demand response programs that lower consumer electric cost, increase the availability of electric service to everyone else during demand peaks and enhance system reliability. For example, a study by the National Renewable Energy Laboratory (NREL) revealed a potential saving ranging from 0.5% to 2.2% (Stoll et al., 2017). DR is already an element of virtually every electricity market in the United States and is part of the system economics, which affect nearly every consumer.

FINDING: The value of DR programs is substantial. While mandating Demand Response-ready is beyond the charter of appliance energy efficiency standards, since it may not reduce on-site energy use, DR often reduces system energy and reduces cost of supply capacity to meet demand.

FINDING: DOE Appliance Efficiency standards do not impede the inclusion of Demand Response functionality in appliances and, where possible, may encourage the ability to take advantage of it.

RECOMMENDATION 3-6: DOE should consider Demand Response readiness as a factor in cost-efficiency calculations. This necessitates the inclusion of power system benefits not currently considered. (See also Recommendation 4-10.)

FINAL OBSERVATIONS

The committee believes that the most effective appliance efficiency standards are those which incrementally improve the nation’s energy efficiency, and do not discourage or interfere with the development and eventual adoption of disruptive technology. The Market and Technology Assessment and Screening Analysis offer the best window through which to focus on that outcome. This is likewise the analytical stage in which to include technology options that are near-commercial and offer promising reductions in energy consumption and peak demand.

Some of the data and projections used in the suite of engineering and technology analyses discussed in this chapter will have important consequences both for appliance and commercial equipment standards and in other regulatory development mechanisms. The energy consumption patterns and amounts that are

Suggested Citation:"3 Assessing Current Models: Engineering and Technology Cost and Technology Performance." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards. Washington, DC: The National Academies Press. doi: 10.17226/25992.
×

projected for an appliance will need to include results by seasonal time-of-use (day, week, and month); these projections will be important for the RIA as well. The use of Actual Test Data, including experience from previously implemented standards, in place of computer simulation or to validate computer simulations will improve the analytical outcomes. The analysis of the TSLs will hinge on properly accounting for the variability and uncertainty in the cost and efficiency of the respective TSLs. Lastly, there is an opportunity to modify models and approaches used in future analyses by performing a retrospective review of some past predictions.

REFERENCES

DOE (U.S. Department of Energy). 2014a. Technical Support Document: Energy Efficiency Program for Consumer Products and Commercial and Industrial Equipment: Residential Dishwashers. Washington, DC. December. https://www.regulations.gov/document/EERE-2014-BT-STD-0021-0005.

DOE. 2014b. Technical Support Document: Energy Efficiency Program for Consumer Products and Commercial and Industrial Equipment: Commercial Refrigeration Equipment. Washington, DC. February. https://www.regulations.gov/document/EERE-2010-BT-STD-0003-0102.

DOE. 2015. Technical Support Document: Energy Efficiency Program for Consumer Products and Commercial and Industrial Equipment: Portable Air Conditioners. Washington, DC. February.

DOE. 2016. Technical Support Document: Energy Efficiency Program for Consumer Products and Commercial and Industrial Equipment: Residential Furnaces. Washington, DC. August 30. https://www.regulations.gov/document/EERE-2014-BT-STD-0031-0217.

Gellings, C.W., and R.W. Taylor. 1981. “Electric Load Curve Synthesis—A Computer Simulation of an Electric Utility Load Shape.” IEEE Transactions on Power Apparatus and Systems 100(1): 60-65. January.

NASEM (National Academies of Sciences, Engineering, and Medicine). 2016. The Power of Change: Innovation for Development and Deployment of Increasingly Clean Electric Power Technologies. Washington, DC: The National Academies Press.

NRC (National Research Council). 2001. Energy Research at DOE: Was It Worth It? Energy Efficiency and Fossil Energy Research 1978 to 2000. Washington, DC: National Academy Press.

Stoll, B., E. Buechler, and E. Hale. 2017. “The Value of Demand Response in Florida.” The Electricity Journal 30(9): 57-64. November.

Suggested Citation:"3 Assessing Current Models: Engineering and Technology Cost and Technology Performance." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards. Washington, DC: The National Academies Press. doi: 10.17226/25992.
×
Page 38
Suggested Citation:"3 Assessing Current Models: Engineering and Technology Cost and Technology Performance." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards. Washington, DC: The National Academies Press. doi: 10.17226/25992.
×
Page 39
Suggested Citation:"3 Assessing Current Models: Engineering and Technology Cost and Technology Performance." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards. Washington, DC: The National Academies Press. doi: 10.17226/25992.
×
Page 40
Suggested Citation:"3 Assessing Current Models: Engineering and Technology Cost and Technology Performance." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards. Washington, DC: The National Academies Press. doi: 10.17226/25992.
×
Page 41
Suggested Citation:"3 Assessing Current Models: Engineering and Technology Cost and Technology Performance." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards. Washington, DC: The National Academies Press. doi: 10.17226/25992.
×
Page 42
Suggested Citation:"3 Assessing Current Models: Engineering and Technology Cost and Technology Performance." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards. Washington, DC: The National Academies Press. doi: 10.17226/25992.
×
Page 43
Suggested Citation:"3 Assessing Current Models: Engineering and Technology Cost and Technology Performance." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards. Washington, DC: The National Academies Press. doi: 10.17226/25992.
×
Page 44
Suggested Citation:"3 Assessing Current Models: Engineering and Technology Cost and Technology Performance." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards. Washington, DC: The National Academies Press. doi: 10.17226/25992.
×
Page 45
Suggested Citation:"3 Assessing Current Models: Engineering and Technology Cost and Technology Performance." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards. Washington, DC: The National Academies Press. doi: 10.17226/25992.
×
Page 46
Suggested Citation:"3 Assessing Current Models: Engineering and Technology Cost and Technology Performance." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards. Washington, DC: The National Academies Press. doi: 10.17226/25992.
×
Page 47
Suggested Citation:"3 Assessing Current Models: Engineering and Technology Cost and Technology Performance." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards. Washington, DC: The National Academies Press. doi: 10.17226/25992.
×
Page 48
Suggested Citation:"3 Assessing Current Models: Engineering and Technology Cost and Technology Performance." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards. Washington, DC: The National Academies Press. doi: 10.17226/25992.
×
Page 49
Suggested Citation:"3 Assessing Current Models: Engineering and Technology Cost and Technology Performance." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards. Washington, DC: The National Academies Press. doi: 10.17226/25992.
×
Page 50
Suggested Citation:"3 Assessing Current Models: Engineering and Technology Cost and Technology Performance." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards. Washington, DC: The National Academies Press. doi: 10.17226/25992.
×
Page 51
Next: 4 The Economic Analysis of Standards »
Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards Get This Book
×
 Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards
Buy Paperback | $42.00 Buy Ebook | $33.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The U.S. Department of Energy (DOE) issues standards regulations for energy conservation pursuant to the Energy Policy and Conservation Act of 1975, as amended, and other authorities. These standards regulations apply to certain consumer products and commercial and industrial equipment. These can include air conditioning and heating systems, washing machines, and commercial refrigeration, among numerous other examples. DOE issues standards regulations by rulemaking and includes quantitative maximum water and energy use or minimum energy conservation standards. There are currently standards regulations for more than 70 product classes (i.e., a specific type of consumer product or commercial or industrial equipment). This report reviews the assumptions, models, and methodologies that DOE uses in setting the quantitative portion of the standards regulations following the Office of Management and Budget's guidance on the use of scientific information. Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards makes findings and recommendations on how DOE can improve its analyses and align its regulatory analyses with best practices for cost-benefit analysis.

READ FREE ONLINE

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!