The United States is responsible for nearly one-fifth of the world’s energy consumption (U.S. Energy Information Administration, 2011a), or approximately 100 quadrillion Btu of energy, which is the equivalent of over $1 trillion in energy expenditures every year (U.S. Energy Information Administration, 2011b). Population growth, and the associated growth in housing, commercial floor space, transportation, goods, and services, is expected to cause a 0.7 percent annual increase in energy demand for the foreseeable future (U.S. Energy Information Administration, 2011c). The scale of these numbers, combined with increasing concerns about U.S. economic competitiveness, energy security, national security, and climate change, underscores the importance of making sound energy policy and investment decisions for the future. In the words of U.S. Secretary of Energy Steven Chu, “Any path close to ‘business as usual’ will imperil future generations with dangerous and unacceptable economic, social, and environmental risks” (U.S. Department of Energy, 2011a).
The primary federal government authority charged with collecting, analyzing, and disseminating energy data to inform these decisions is the U.S. Department of Energy’s (DOE’s) statistical agency, the U.S. Energy Information Administration (EIA). EIA was established by the Department of Energy Organization Act of 1977, and its activities are funded through an annual appropriation from Congress. Although the data produced by EIA play an extremely important role in informing energy policy decisions, over the years the agency has struggled with budget limitations that have made
meeting its mandate difficult—and sometimes impossible. Figure 1-1 shows EIA’s funding levels between 1978 and 2011.
EIA’s portfolio of data collections includes three surveys of energy-consuming end use sectors: the Commercial Buildings Energy Consumption Survey (CBECS), the Residential Energy Consumption Survey (RECS), and the Manufacturing Energy Consumption Survey. Prior to 1994, EIA also conducted a transportation energy use survey, the Residential Transportation Energy Consumption Survey, but budget cuts forced this data collection to be discontinued after 1994.
EIA asked the National Research Council’s (NRC’s) Committee on National Statistics, with input from its Board on Energy and Environmental Systems, to convene a panel to study two of these surveys, the CBECS and RECS, which cover the commercial and residential end use sectors. The panel’s charge was to evaluate the designs of the two surveys and recommend updates based on current and expected future data needs. (See Box 1-1 for the exact wording of the panel’s charge.)
The CBECS and RECS were established in response to the energy shocks of the 1970s, which highlighted the need for coordinating the nation’s energy functions and for a comprehensive federal energy plan. At the time policy makers had accurate data regarding the sources of our energy (coal, petroleum, etc.) but had only crude guesses about how that energy was used to carry out basic functions such as space heating, lighting, and refrigeration. While the 1970s energy crisis is a distant memory to many, threats to energy security persist and are likely to persist for the foreseeable future. Mitigating these threats requires reliable information on how energy is used.
Today the energy used by the commercial and residential sectors represents approximately 40 percent of the nation’s total energy consumption, and the share of these two sectors is expected to increase in the future, in large part because of the 5.8 quadrillion Btu growth expected over the next 25 years in the commercial sector, shown in Figure 1-2 (U.S. Energy Information Administration, 2011c). On the other hand, various federal and local policies have been put into place with the goal of reversing this trend and reducing overall energy use to a more sustainable level. The importance of a thorough understanding of how commercial and residential buildings use energy has therefore never been greater. Only with good quality data can policy makers, industry, and consumers respond effectively to a variety of challenges, ranging from the mundane to potentially catastrophic scenarios.
The energy consumption surveys of EIA are the most relevant sources
The Committee on National Statistics will convene a panel to conduct a comprehensive 30-month study of the U.S. Energy Information Administration’s Commercial Buildings Energy Consumption Survey (CBECS) and the Residential Energy Consumption Survey (RECS). The panel will consider for these two surveys how to improve data quality, geographic coverage, timeliness of data releases, and relevance of the data for meeting user needs for energy end use information in the next decade and beyond. The panel will review survey design, frequency, and scope options, survey practice and operations, and the role that auxiliary data could play in improving survey coverage and editing and imputation methods. The panel will issue a letter report by spring 2010 that comments on design and data collection options for the 2010 CBECS to enable it to support U.S. Department of Energy program information needs, reduce respondent burden, and increase the quality and timeliness of the data.* The panel will issue a final report at the conclusion of a 24-month study that makes recommendations for the design and conduct of CBECS and RECS and the dissemination of CBECS and RECS data for the next decade and beyond, including consideration of the level of resources likely to be required in comparison with the current survey program. The last 6 months of the study will be used for dissemination.
of data available to policy makers seeking to make informed decisions and plan for the future by, for example, ensuring that energy production is in line with needs and that savings opportunities are identified wherever possible. Congress and the executive branch relied on EIA data to inform analyses evaluating the impact of the American Power Act of 2010, which was proposed to regulate emissions of greenhouse gases (U.S. Department of Energy, 2011b). EIA data also informed an analysis of several provisions of the discussion draft of the Domestic Manufacturing and Energy Jobs Act of 2010 (U.S. Department of Energy, 2011b).
Government investments in energy programs have long shaped in-
FIGURE 1-2 Primary energy use by end use sector, 2009–2035 (quadrillion Btu).
SOURCE: Annual Energy Outlook 2011 (U.S. Energy Information Administration, 2011c).
novation and growth in America, but the appropriate size and scope of these subsidies has always been a topic of debate among policy makers and the public, and subsidies for emerging fuel types have been particularly controversial (Pfund and Healey, 2011). Historical data on federal energy subsidies show that there has been strong support for oil and gas over the years and that subsidies for nuclear energy during the early days of support for this technology dwarfed all other energy subsidies (Pfund and Healey, 2011). At present there is great interest in renewable forms of energy, although subsidies for renewable energy have so far been significantly lower than the subsidies for other forms of energy during the early days of each subsidy’s life (Pfund and Healey, 2011).
As priorities shift, it is important to develop an increasingly clear understanding of energy demand and consumption patterns in order to evaluate the economic and environmental implications of subsidies, credits, and programs across the energy sector and to avoid investments in the wrong programs, standards, or technologies, which could be irresponsible and wasteful. Making wise choices regarding government support for the energy sector is most likely if those choices are supported by valid and reliable data.
The rapid pace of changes in technology and lifestyles, combined with ongoing building renovations and stock turnover makes it vital to perform periodic data collections that can identify trends and emerging issues. Data from the CBECS and RECS guide the development of building energy-rating tools, such as the HERS index, a scoring system established by the Residential Energy Services Network to quantify the energy performance of residential buildings, as well as simulation programs, such as EnergyPlus developed by DOE to model energy use and optimize building design in both the commercial and residential sectors. “Deep retrofit” projects that involve major renovations to an existing structure undertaken to increase energy efficiency are of increasing interest, and their effectiveness is being researched. “Net-zero energy” buildings that produce as much energy as they use are another emerging area of interest for which energy consumption data are necessary.
Furthermore, with the increase in the number of electrical devices in homes and workplaces, the role of appliance and commercial equipment standards that manufacturers must adhere to is increasingly important. Data from the CBECS and RECS are used to gather information about the quantity of Energy Star products in the market and about their energy consumption. The surveys are also used to develop and revise guidelines for product specifications to increase efficiency.
Over the past few decades miscellaneous energy use has grown rapidly, and researchers have been warning about the lack of adequate data concerning this area of energy consumption (Meier et al., 1992). Miscellaneous, or residual, energy use is generally defined as energy use other than the traditional end use categories, such as space heating and cooling, water heating, clothes drying, refrigeration, cooking, and lighting. In other words, this category includes a wide range of end uses, such as televisions, computers, electronic picture frames, ceiling fans, and so on. Residential miscellaneous energy consumption today accounts for more energy use than any other single end use in homes, and some devices consume particularly large amounts of energy (Roth et al., 2008). The Electric Power Research Institute estimates that if every household in the United States had a digital picture frame, five new natural gas power plants would be needed to operate these (Mansoor, 2008). In the commercial sector, miscellaneous end uses (such as medical devices and video displays) are expected to account for approximately 40 percent of all electricity consumption (U.S. Energy Information Administration, 2011c).
Inadequate measures of miscellaneous energy use can result in under-
estimates of the energy consumed by the rapidly growing use of consumer electronics and devices and, consequently, an overestimate of the energy use associated with more traditional activities such as heating, cooling, and lighting. This makes it difficult to evaluate the success—or lack thereof—of energy efficiency programs and could result in costly overinvestments in electricity generation and infrastructure, similar to what happened in the 1980s in the United States (Gold and Elliott, 2010).
An NRC committee argued that several recent trends in the energy industry, such as industry consolidation, a greater reliance on automation, and increasingly congested transmission corridors, have increased the vulnerability of the energy supply to terrorist attacks (National Research Council, 2002). The energy infrastructure is also vulnerable to natural disasters. When Hurricane Katrina struck in the summer of 2005, for instance, it hit a region critical to U.S. oil production and distribution, and sent prices surging. In addition to these risks at home, the U.S. dependence on foreign energy sources also contributes to instability. For example, about half of the petroleum consumed by the country in 2010 was imported (U.S. Energy Information Administration, 2011d). The growing vulnerability of the energy supply system to terrorist attacks and natural disasters is another factor that makes it important to develop a thorough understanding of the country’s energy needs.
The Energy Independence and Security Act of 2007 (EISA)1 directed EIA to “establish a 5-year plan to enhance the quality and scope of the data collection necessary to ensure the scope, accuracy and timeliness of the information needed for efficient functioning of energy markets and related financial operations.” (Appendix A includes the relevant section of the legislation.) The law highlighted the need to assess the gaps in the data obtained and compiled by EIA. Many data users have reported that such gaps represent the biggest shortcoming of EIA energy consumption surveys because in most cases the small sample sizes resulting from the low budgets make the data inadequate, particularly for state-level analyses. The CBECS and RECS sample sizes are approximately 6,000 cases nationally, although with funds from the American Recovery and Reinvestment Act of 2009, the most recent RECS was expanded to include 12,083 households, which made it possible to release data individually for 16 states for the first time since the survey was launched.
1Public Law 110-140, 110th Congress (December 19, 2007). The quotation is from § 805(a)(1).
While the increase in the RECS sample size was a promising development, the fiscal year 2011 budget reduced EIA’s funding by 14 percent compared to fiscal year 2010, which caused EIA to suspend work on a large number of products and services, including the 2011 CBECS. This is a concern because the CBECS is the only national source of data on energy consumption in commercial buildings (Michaels, 2010) and because for many purposes CBECS is the only source of data on other building characteristics as well. As such, policy decisions that will shape the future depend on it.
The panel believes that high-quality, valid, and reliable data are essential to informed policy making and that they enable rapid response to changing needs, support efficient markets for end use devices, and spur innovation by guiding cutting edge research. Extended lags or gaps in the collection of energy consumption information are of a particular concern because most missing data cannot be filled in retrospectively. Compromises in the design of the surveys, such as sample sizes that are too small to be adequate and shortcuts that affect the data collection approach, are more subtle problems that also affect the quality of the data available for research and decision making. Estimates based on surveys that are conducted with insufficient resources are subject to large uncertainties, which in turn affect the confidence in conclusions based on analyses that depend on the data.
The need is especially high for information about energy demand, which is as important for energy security as information about energy supply is. While information about current energy supplies and energy reserves is available from a variety of sources, including other EIA surveys, our understanding of how energy is used, where it goes, and the services it provides remains woefully inadequate. In the case of energy supply, there are programs and mechanisms in place to collect data about energy resources, and there are a limited number of energy types that must be considered (for example, petroleum, coal, nuclear, and renewables). On the other hand, the energy consumption community is much more diffuse and does not have significant data collection assets, while at the same time it has to consider a large and growing number of end uses.
ISSUES FOR THE PANEL
In 2009, EIA asked NRC to conduct a comprehensive review to determine how the CBECS and RECS can best take advantage of recent developments in survey methods and to ensure the relevance of the data for meeting increased user needs in a world characterized by a rapidly changing
energy landscape. Many of the design and operational procedures for the CBECS and RECS were developed in the 1970s and 1980s, and resource limitations during much of the time since then have prevented EIA from making significant changes to its survey methodology and operations. This study is an opportunity to focus on the future of the two surveys, and the study charge directs the panel to make recommendations “for the next decade and beyond.”
In response to this charge, the panel appointed by NRC undertook a range of activities over the course of approximately two years. The panel met with EIA on several occasions to learn as much as possible about the designs of the surveys and the challenges encountered. The panel also met with EIA’s data collection contractors to understand the specifics of the activities they performed as part of their contractual agreements with EIA and to obtain their perspectives on the surveys. The panel also discussed data collection strategies and methodological considerations with researchers conducting their own surveys in the energy sector, as well as with researchers conducting surveys on other topics with similar challenges.
An important aspect of the panel’s work was to gain a thorough understanding of data user needs. A data user workshop brought together researchers and other data users from the government, commercial, and academic sectors and provided the panel with an opportunity to hear about the most pressing data needs.2 The panel also received both formal and informal input from other researchers and organizations who did not participate in the workshop. For a list of data users who provided input see Appendix B (the panel reached out to several additional data users who failed to provide feedback). While publishing a written report of the data user input was not within the panel’s charge, data user considerations were at the heart of the panel’s deliberations. The common themes that emerged are woven throughout the discussion in this report and played an important role in shaping the recommendations.
The panel’s research and deliberations made clear the need for an energy consumption data collection program that can accurately characterize changes in technology and identify trends. Doing so will require maintaining data consistency over time, but in order to meet user needs the data collection program will also have to be nimble, flexible, responsive, and timely.
2Presentation slides from this workshop are available on the Committee on National Statistics website at http://www7.nationalacademies.org/cnstat/Presentations%20Main%20Page.html [December 2011].
In addition to its role in informing policy making, the data collected by the surveys should also serve as catalysts for innovation. Specifically, in an ideal world the United States would have energy consumption data collection programs characterized by the following:
• Annual data releases with minimal lags between the time of the data collection and the time of publication.
• Sample sizes that are large enough to enable valid state-level analysis for all states.
• An integrated survey design with all energy-consuming units (buildings, housing units, and so on) represented in the sample.
• Questionnaires and data collection forms that capture all important uses of energy.
• A longitudinal component of the study design to inform a better understanding of trends.
The panel, of course, understands that bringing the surveys closer to the ideal would come at a significant cost, and it is realistic about the need to find a balance between data needs and budget constraints. Indeed, the panel’s charge calls for “consideration of the level of resources likely to be required in comparison with the current survey program,” which the panel understood as the need for recommendations that are the result of careful consideration of these trade-offs.
When possible, the recommendations in this report are accompanied by a brief discussion of their cost implications, although the actual costs can vary greatly depending on the method of implementation selected. For example, many of the recommendations described in the individual sections are related and would likely be most efficient if implemented together, which would also lead to cost savings compared to the costs associated with any particular recommendation carried out independently of the others. Some of the panel’s suggestions are recommendations to evaluate the relative costs and benefits of a change to the design of the surveys or operational procedures. The reader should also note that although some of the recommendations involve an increase in survey costs in the short term, they were proposed by the panel based on the assumption that these increases will translate into more efficient surveys in the longer term.
On some issues the panel recommends alternatives for consideration by EIA, and the relative advantages of one option compared to another may depend on the amount of funding available and expected for subsequent
years. Survey design is a careful balance between survey errors and costs. The precision of the survey statistics is highly dependent on the resources that are invested into the collection and production of the data. For a given budget, survey design options can be evaluated to identify the option that leads to the smallest total survey error (Groves and Lyberg, 2010). Although the panel was not asked to base its recommendations on specific assumptions about the budget, its deliberations were centered around the need to be cost effective and achieve an optimal balance between data quality and costs. The last chapter of the report takes a slightly different perspective, revisiting some of the ideal program characteristics highlighted above and discussing some ideas that may be less feasible in the short term but which may help EIA envision the energy consumption data collections of the future.
The report discusses some overarching, transformational recommendations that, if implemented, would represent major changes in the designs of the surveys. Other recommendations are more technical and incremental in nature, and are focused on smaller-scale revisions that could improve and update the surveys. The two surveys will be discussed separately, which results in some duplication in the text and overlap in the recommendations.