Summary

At the request of the U.S. Energy Information Administration (EIA), a panel of the National Research Council conducted a study to review the designs of two of EIA’s energy consumption surveys: the Commercial Buildings Energy Consumption Survey (CBECS) and the Residential Energy Consumption Survey (RECS). The panel’s charge was to recommend updates to these surveys based on current and expected future data needs. The CBECS and RECS are used by Congress, the executive branch, state governments, local governments, energy producers, energy providers, investors, researchers, the media, and the public. As the energy demands associated with commercial and residential buildings grow and change, the need for reliable, valid, and increasingly more sophisticated data is likely to grow, making state-of-the-art energy consumption surveys even more necessary.

The designs of the CBECS and RECS are fundamentally similar, but there are certain differences that result from the differing characteristics of their two populations of interest. The CBECS is a survey of commercial buildings, and, as such, it faces some unique challenges. The RECS is a household survey with a more conventional—albeit also challenging—data collection approach. Both surveys collect data about the complex, highly technical topic of energy consumption and its determinants, and both are the surveys of record on this topic for researchers and policy makers in the United States. The CBECS, in particular, is the only national data source of detailed characteristics and energy use of commercial buildings (Michaels, 2010).



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1
Summary At the request of the U.S. Energy Information Administration (EIA), a panel of the National Research Council conducted a study to review the designs of two of EIA’s energy consumption surveys: the Commercial Buildings Energy Consumption Survey (CBECS) and the Residential En- ergy Consumption Survey (RECS). The panel’s charge was to recommend updates to these surveys based on current and expected future data needs. The CBECS and RECS are used by Congress, the executive branch, state governments, local governments, energy producers, energy providers, inves- tors, researchers, the media, and the public. As the energy demands associ- ated with commercial and residential buildings grow and change, the need for reliable, valid, and increasingly more sophisticated data is likely to grow, making state-of-the-art energy consumption surveys even more necessary. The designs of the CBECS and RECS are fundamentally similar, but there are certain differences that result from the differing characteristics of their two populations of interest. The CBECS is a survey of commercial buildings, and, as such, it faces some unique challenges. The RECS is a household survey with a more conventional—albeit also challenging—data collection approach. Both surveys collect data about the complex, highly technical topic of energy consumption and its determinants, and both are the surveys of record on this topic for researchers and policy makers in the United States. The CBECS, in particular, is the only national data source of detailed characteristics and energy use of commercial buildings (Michaels, 2010). 1

OCR for page 1
2 EFFECTIVE TRACKING OF BUILDING ENERGY USE Based on the panel’s research—and, in particular, on discussions with data users—two topics emerged as priority areas for both surveys: (1) the timeliness and frequency of the data and (2) data gaps. The panel concluded that the frequency of the surveys does not meet data user needs well and that the length of time it takes to process and prepare the data for release is a particular concern for many data users. The panel also identified a variety of data gaps, some of which are related to EIA’s inability to release all of the data collected because the sample sizes are too small to produce data that meet quality and confidentiality thresholds. In some cases, data users would like more detailed information about building characteristics and energy consumption than what the surveys currently collect or would like new measures added to the questionnaires to capture new topics of interest, such as new end uses. The panel’s main recommendations, which are briefly discussed below, focus on addressing the two priority areas mentioned, in the context of ad- ditional important considerations for the surveys, such as data quality and costs. The main recommendations, some of which involve major changes to the survey designs, are described first, followed by additional recommenda- tions that address more specific, technical aspects of the data collections. COMMERCIAL BUILDINGS ENERGY CONSUMPTION SURVEY Timeliness and Frequency of the CBECS Data The panel’s research, which included discussions with data users, found that one of the biggest concerns related to the surveys is the amount of time it takes to collect, process, and release the data. One option for addressing data users’ need for quicker access to data is to implement a rotating sample design by, for example, dividing the full sample into four subsamples and collecting a quarter of the data each year over a four-year period instead of collecting data from the entire sample once every four years. This change would enable annual data releases for a subset of the sample. Recommendation CBECS-1: EIA should evaluate the usefulness of implementing a rotating sample design for the CBECS to improve the timeliness of the data. Another way to shorten the time required for the survey would be to move some of the CBECS data collection from face-to-face interviews to

OCR for page 1
3 SUMMARY the web. For at least a subset of the sample, this will reduce the amount of time needed to complete the data collection. When transitioning the survey to the web, it will also be important to eliminate and simplify some of the editing procedures in order to reduce the associated delays. Recommendation CBECS-3: Informed by a methodological research program, EIA should begin developing procedures for a multimode approach and should begin moving some of the CBECS data collec- tion to the web. Recommendation CBECS-4: EIA should investigate strategies for re- leasing the CBECS data faster, for example, by revising the editing pro- cedures or by completing the editing for a subset of the variables and releasing these data prior to completing the editing for the full data set. CBECS Data Gaps The current CBECS sample design is best suited for producing descrip- tive statistics for larger geographical divisions, such as the entire country or census division levels. The relatively small sample sizes, in combination with strict quality control and confidentiality protections, severely limit the amount of data that can be released from the survey. This in turn limits, in terms of both geography and complexity, the analyses that can be conducted based on the data. Recommendation CBECS-5: As part of its efforts to address the needs of data users, EIA should make it a priority to identify opportunities for increasing the sample size in order to enable the release of more of the CBECS data that are currently being collected. Recommendation CBECS-6: EIA should consider establishing a research data center (RDC) or evaluate the option of using an existing RDC maintained by another organization to provide data users with secure access to CBECS data that are currently not publicly released. Other CBECS Recommendations In addition to the core recommendations highlighted above, the report includes additional recommendations that are more technical and specific

OCR for page 1
4 EFFECTIVE TRACKING OF BUILDING ENERGY USE in nature and that are focused on supporting the large-scale revisions and on further improving and updating the surveys. While the CBECS cannot meet all data user needs, the panel makes several recommendations aimed at ensuring that the questionnaire remains responsive to changes in the energy landscape. Recommendation CBECS-2: EIA should consider integrating a longitudinal element into the CBECS sample design to obtain better estimates of change. Recommendation CBECS-7: EIA should prepare for the more widespread availability of smart meter data in the future by evaluating potential uses of such data, strategies for collecting them, and ways of addressing new confidentiality challenges. Recommendation CBECS-8: EIA should develop a process for the periodic review of new energy end uses or end uses that are becom- ing more widespread in the commercial sector and which may need to be included on the CBECS questionnaire. The process should also identify end uses that are becoming obsolete and that can be removed from the questionnaire. Recommendation CBECS-9: To accommodate data user needs for more detailed information, EIA should evaluate the possibility of administering a short-form and a long-form CBECS questionnaire. Recommendation CBECS-15: EIA should invest in periodic reviews of the CBECS questionnaire content and wording. This should be understood as a routine updating of the instruments, separate from the concept of a major redesign of the survey. Given the challenges associated with developing and maintaining a sampling frame based on buildings, EIA should revisit the possibility of us- ing establishment-based lists as the primary source of the CBECS sampling frame. Recommendation CBECS-10: EIA should conduct research to evalu- ate the advantages and costs associated with using an establishment- based list for the CBECS sampling frame.

OCR for page 1
5 SUMMARY The panel also recommends several changes to the CBECS data collec- tion procedures with the goals of increasing efficiency and overcoming some of the challenges associated with collecting highly technical information from respondents who may find some of the questions challenging. This includes researching alternative ways of collecting some of the data that may be particularly difficult to collect from building respondents or that could be more efficiently collected from other sources. For example, EIA could ex- plore collecting some of the data from the headquarters of major centralized accounts (such as Walmart or McDonald’s) and relying more on energy sup- pliers. Alternatives sources of data, such as existing administrative records databases (for example, property tax databases), should also be evaluated. Recommendation CBECS-11: To increase CBECS data collection efficiency, EIA should explore the possibility of contacting the head- quarters of major centralized accounts and collecting data about all of the buildings in the sample from these centralized sources. Recommendation CBECS-12: EIA should evaluate whether working more closely with the energy suppliers of commercial buildings could lead to efficiencies in the data collection process. Recommendation CBECS-13: EIA should conduct an ongoing evalu- ation of administrative records as potential sources for substantive data for the CBECS. To address challenges associated with collecting highly technical infor- mation from respondents who are not necessarily experts on topics such as building characteristic and energy consumption, EIA should consider evaluating how involving professional energy auditors in the data collec- tion process would affect costs and data quality. Debriefings with field interviewers could also be useful in identifying questionnaire items that respondents find particularly difficult to answer and in developing possible improvements to these items. Recommendation CBECS-14: EIA should test the use of profes- sional energy auditors on a small scale to better understand the costs and benefits related to having experts collect data for a subset of the CBECS sample.

OCR for page 1
6 EFFECTIVE TRACKING OF BUILDING ENERGY USE Recommendation CBECS-16: Interviewer debriefings should become an integral part of the CBECS data collection process in order to iden- tify problems with the questionnaires and procedures and to serve as a source of ideas for increased efficiencies. RESIDENTIAL ENERGY CONSUMPTION SURVEY Timeliness and Frequency of the RECS Data As with the CBECS, one of the biggest concerns related to the RECS is the amount of time it takes to collect, process, and release the data. Recommendation RECS-1: EIA should evaluate the usefulness of implementing a rotating sample design for the RECS to improve the timeliness of the data. Another way of speeding up the survey operations would be to move some of the RECS data collection from face-to-face interviews to web in- terviews and to eliminate and simplify some of the editing procedures to reduce the associated delays. Recommendation RECS-3: Informed by a methodological research program, EIA should begin developing procedures for a multimode approach and should begin moving some of the RECS data collection to the web. Recommendation RECS-4: EIA should investigate strategies for re- leasing the RECS data faster, for example, by revising the editing pro- cedures or by completing the editing for a subset of the variables and releasing these data prior to completing the editing for the full data set. RECS Data Gaps The current RECS sample design is best suited for producing descrip- tive statistics for larger geographic areas, such as the entire country and census divisions. State-level data are available for only 16 states. The small sample sizes, in combination with strict quality control and confidentiality protections, severely limit the amount of data that can be released from the

OCR for page 1
7 SUMMARY survey. This in turn limits the complexity and geographic resolution of the analyses that can be conducted based on the data. Recommendation RECS-5: As part of its efforts to address the needs of data users, EIA should make it a priority to identify opportunities for increasing the sample size in order to enable the release of more of the RECS data that are currently being collected. Recommendation RECS-6: EIA should consider establishing a re- search data center or evaluate the option of using an existing RDC maintained by another organization to provide data users with secure access to RECS data that are currently not publicly released. Other RECS Recommendations In addition to the core recommendations described above, the report contains additional recommendations that are more technical and specific in nature and that are focused on supporting the large-scale revisions dis- cussed as well as on further improving and updating the survey. While it is important to be mindful of the fact that the RECS can- not meet all data user needs, the panel makes several recommendations to assure that the questionnaire remains responsive to changes in the energy landscape. Recommendation RECS-2: EIA should consider integrating a longi- tudinal element into the RECS sample design to obtain better estimates of change. Recommendation RECS-7: EIA should prepare for the more wide- spread availability of smart meter data in the future by evaluating potential uses of such data, strategies for collecting them, and ways of addressing new confidentiality challenges. Recommendation RECS-8: In anticipation of the spread of electric vehicles, EIA should prepare for RECS to collect more information about these, especially more detail about the capacity to charge electric vehicles.

OCR for page 1
8 EFFECTIVE TRACKING OF BUILDING ENERGY USE Recommendation RECS-9: EIA should develop a process for the periodic review of new energy end uses or end uses that are becoming more widespread in the residential sector and which may need to be included on the RECS questionnaire. The process should also identify end uses that are becoming obsolete and that can be removed from the questionnaire. Recommendation RECS-10: EIA should consider implementing a “whole building” supplement to the RECS to address the data gap related to multiunit residential buildings. Recommendation RECS-11: To accommodate data user needs for more detailed information, EIA should evaluate the possibility of ad- ministering a short-form and a long-form RECS questionnaire. Recommendation RECS-16: EIA should invest in periodic reviews of the RECS questionnaire content and wording. This should be un- derstood as a routine updating of the instruments, separate from the concept of a major redesign of the survey. Given the costs associated with area probability samples, combined with the recent advances made in the field of address-based sampling, the panel recommends evaluating address-based sampling for the RECS. Recommendation RECS-12: Research should be conducted to evalu- ate the possibility of replacing the RECS area probability sample with address-based sampling, using an address list developed by commercial vendors based on the U.S. Postal Service delivery sequence file. The panel also recommends several changes to the RECS data collec- tion procedures that are aimed at increasing efficiency and overcoming some of the challenges associated with collecting highly technical information from respondents who may find some of the questions challenging. Recommendation RECS-13: EIA should conduct an ongoing evalu- ation of administrative records as potential sources for substantive data for the RECS, especially for square footage data.

OCR for page 1
9 SUMMARY Recommendation RECS-14: EIA should evaluate whether working more closely with the energy suppliers could lead to efficiencies in the data collection process. Recommendation RECS-15: EIA should test the use of professional energy auditors on a small scale to better understand the costs and benefits related to having experts collect data for a subset of the RECS sample. Recommendation RECS-17: Interviewer debriefings should become an integral part of the RECS data collection process in order to iden- tify problems with the questionnaires and procedures and to serve as a source of ideas for increased efficiencies. A STATE-OF-THE-ART ENERGY CONSUMPTION DATA COLLECTION PROGRAM The panel’s recommendations in this report are based on a careful balance between what the data needs are and what is most realistic to implement in the short term. The last chapter of the report outlines some additional ideas that could be considered as part of envisioning a state-of- the-art data collection program for the energy surveys of the future. A comprehensive approach to collecting building energy consumption information could be based on a sampling frame that includes all buildings, regardless of type. Advances in computer technology and satellite mapping are greatly increasing the feasibility of implementing such a design, and are changing how much of the work can be performed by a computer program prior to, or instead of, field work. Other advances in technology are increas- ing opportunities for enhancing features of the data collection. For example, it is becoming cheaper and easier to integrate digital images that can help reduce ambiguities surrounding building characteristics or equipment. In- creasingly more sophisticated interactive online tools are becoming available and can be used to engage sample members and encourage participation in the surveys. In terms of possibilities for growth and meeting data needs, EIA could explore opportunities to partner with other government statistical agen- cies, organizations, or energy suppliers to collect data on specialized topics, without increasing the burden on the current CBECS or RECS. EIA could

OCR for page 1
10 EFFECTIVE TRACKING OF BUILDING ENERGY USE consider serving as the coordinator or data center for research on a number of special topics or special populations or performing additional functions that involve more active dissemination and closer collaboration with the research community.