Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1
Committee on National Statistics The Keck Center Division of Behavioral and Social Sciences and Education 500 Fifth Street, NW Washington, DC 20001 Phone: 202 334 3096 Fax: 202 334 3751 www.national-academies.org/cnstat May 18, 2010 Dr. Richard Newell Administrator U.S. Energy Information Administration 1000 Independence Avenue, SW Washington, DC 20585 Dear Dr. Newell: At the request of the Energy Information Administration (EIA), the Committee on National Statistics (CNSTAT) of the National Research Council convened a panel to conduct a comprehensive 30-month study of the Commercial Buildings Energy Consumption Survey (CBECS) and Residential Energy Consumption Survey (RECS). Many of the design and operational procedures for the CBECS and RECS were developed in the 1970s and 1980s, and resource limitations during much of the time since then have prevented EIA from making significant changes to the survey methodology or operations. With the possibility of additional funding available in the next few years, EIA asked the National Research Council to conduct a comprehensive review to assess how the CBECS and RECS can take advantage of recent developments in survey methods and to ensure the relevance of the data for meeting increased user needs in the next decade and beyond. The panel’s charge is to consider possible improvements to data quality, geographic coverage, relevance, and the timeliness of data releases. Because plans for the upcoming 2011 round of CBECS must be finalized in the near future, the panel was charged to comment as soon as possible on design and data collection options that would enable the upcoming round of this survey to better support U.S. Department of Energy program information needs, reduce respondent burden, and increase the quality and timeliness of the data. This letter responds to that request, and is limited in scope to discussing issues that the panel believes are realistic to consider in the timeframe leading up to the 2011 data collection. At the conclusion of the study, the panel will deliver its comprehensive report on the overall design and conduct of both CBECS and RECS. At the first meeting of the panel on February 1-2, 2010, EIA staff discussed preparations for the 2011 CBECS and indicated that work will begin on the 2011 CBECS sample design in the summer of 2010. Thus, any changes to this round of the data collection would have to be evaluated before then. EIA staff also informed the panel that the 2011 CBECS is anticipated to have more funding than it has had in the past. The panel also learned in those discussions that EIA has relatively little empirical data on how well the current design and procedures are 1

OCR for page 1
working in comparison with approaches that have been tried in the past and that EIA has not conducted an analysis of options considered but not pursued. Based on the factors described above, the panel’s overarching recommendation is to invest some of the currently available additional funding in research that will enable EIA to make future decisions based on empirical evidence about what is most likely to improve geographic coverage, data quality and relevance, while controlling costs. The panel’s specific recommendations for research as part of the 2011 CBECS are described below. BACKGROUND ON THE CBECS The CBECS is a survey of commercial buildings in the United States, mandated by Congress to provide comprehensive information about energy use in commercial buildings. In addition to energy consumption and expenditure data, the survey collects information about building characteristics, such as energy source, physical structure, equipment used, and activities performed, which provides researchers with detailed information about commercial sector energy use and how it relates to building characteristics. The CBECS is the only national source of these data, and is used for energy forecasting, program development, and policy development. The survey collects information from a sample of commercial buildings in the United States, and it is administered in two data-collection stages: a Building Characteristics Survey and an Energy Suppliers Survey. During the first stage of the data collection, interviewers visit the buildings selected into the sample and ask a representative of the building, such as the building’s owner, manager, or other knowledgeable person to complete the survey. During the second stage of the data collection, the energy suppliers of buildings that were not able to provide adequate information in the first stage are contacted to obtain actual usage and expenditure data from the supplier’s records. SAMPLING FRAME There is little comprehensive information about the stock of commercial buildings in the United States, and EIA indicated that the lack of a comprehensive national list of commercial buildings or another suitable source from which to select a sample of buildings to interview is one of the major challenges for the CBECS data collection. Because no complete list of buildings is available to use as a sampling frame, EIA builds a new area probability sampling frame for the CBECS on a decennial basis. The frame is based on field listings of commercial buildings within specified geographic areas. This sampling frame is updated between each data collection. However, field listings are resource intensive and relying on sources that are not comprehensive for updating the sampling frame leads to coverage problems. The CBECS sample design has undergone numerous revisions over the years, as EIA has attempted to address the cost and coverage issues, but most rounds of the CBECS have relied on a combination of an area frame and a list frame, based on existing lists of commercial buildings from a variety of sources and added at the second stage of the area frame sample. The primary sampling units have been counties or groups of counties, within which smaller geographic areas were randomly selected. All commercial buildings were listed and stratified within these smaller areas, and then a sample of buildings was randomly selected from each stratum. This approach 2

OCR for page 1
was supplemented with information from existing building lists from other sources to ensure adequate representation of buildings that were of special interest because of their size or type of primary activity. For the 2007 administration of the CBECS, the 2003 sampling frame had to be updated. At the recommendation of the data collection contractor, the National Opinion Research Center (NORC), the update was based on a U.S. Postal Service (USPS) Delivery Sequence File (DSF) purchased from a vendor licensed by USPS. The DSF is USPS’s list of all delivery points in the United States. Using the DSF for updating meant that this list had to be matched to the addresses in the second-stage area frame and the duplicates removed. NORC reported that the unduplication turned out to be a major challenge, in part because of imprecise address records. As EIA is aware, another major redesign of the CBECS sampling frame could be very productive, but due to the limited time and resources available, this is neither feasible nor recommended for the 2011 data collection. However, leading up to and during the 2011 CBECS, alternative approaches to building a good second-stage sampling frame should be the focus of EIA research, particularly the availability of administrative records and lists. As the EIA staff indicated, and the panel concurs, a sampling frame based on administrative records may have to completely or partially replace the second-stage area frame in the future because of the high costs associated with field listing. Although EIA has considered the use of more lists throughout the years, research on this should continue because the availability of sources of data is constantly evolving, particularly with more information becoming available on the Internet. For the 2011 CBECS, the most practical approach is to perform another round of updating of the sampling frame using the DSF. Even though unduplication proved to be a challenge when the DSF was first used in 2007, presumably the bulk of the work has now been done, and the 2011 frame can be updated by simply matching the new addresses to the address files used in 2007. We assume that a 2003-2007 cohort of listings is available for use in the 2011 sample based on the matching and updating performed in preparation for the 2007 data collection. As was done in previous years, this approach would have to be supplemented with lists from other sources to assure adequate representation of buildings of special interest. As an example of such a supplemental source, we recommend exploring the usefulness of local government databases that are available online, such as county property tax databases, some of which include information on square footage and heat source. Two available online databases of which we are aware are those of the Allegheny County Office of Property Assessments in Pennsylvania and of the King County Government in Washington. Although such databases are not universally available online, and their use would undoubtedly present some standardization challenges, their usefulness should be evaluated for two purposes: as a source for a sampling frame and for the possible use of some of the data that are now collected through interviews. Other possible data sources are discussed in the next section, although all require further research to evaluate them. In rural areas, the DSF often includes only rural route or post office box numbers and so tends not to be very useful. EIA should evaluate information available from vendors who specialize in providing address data to fill these types of gaps. If these sources are found to be inadequate, 3

OCR for page 1
field listing may still be necessary. Alternatively, half-open interval updating could be considered, if relisting is deemed too inefficient because of the scattered nature of rural areas. This technique involves updating only new or missed units within a small geographic area (an “interval” associated with an address in the sample). In areas where buildings are scattered in unusual ways, half-open interval updating may be difficult to carry out accurately, but the accuracy of the approach in this particular context could be evaluated as part of the 2011 CBECS. For example, in addition to performing half-open interval updating in the rural sample segments, relisting could also be carried out in a subset of these segments to compare the outcome of the two techniques in terms of the number of listings identified and the number that would end up being added to the 2011 CBECS. The relisting could be performed by experienced listers or supervisors to minimize the costs associated with training for these types of assignments. Further research is needed to understand the quality and future potential of the DSF. In addition to evaluating the performance of the DSF in comparison with other sources for a sampling frame, the panel recommends adding a question to the CBECS questionnaire to better understand the extent to which there is overlap between street addresses and the addresses where the building occupants receive their mail. For example, one challenge is that the DSF contains business-level entries, rather than building-level entries. Furthermore, some businesses have their mail delivered somewhere other than the street address (for example, to a post office box). SUPPLEMENTARY DATA SOURCES In addition to considering existing administrative records as an alternative source for a sampling frame, the panel recommends evaluating these records as potential sources for substantive data that could possibly replace an on-site interview at the building’s location or could provide additional data for modeling or to conduct new analyses. Relying on data from other sources may become more of a necessity as it becomes increasingly expensive to maintain high response rates, even if an ideal sampling frame of commercial buildings were available. Although gathering and combining data from a variety of administrative records can also be resource intensive, the costs may go down as such data become more widely available, especially online. Such other sources may also provide higher quality data. For example, EIA staff have expressed concerns regarding the difficulties associated with collecting data about the technical topics covered in the CBECS survey. Neither the typical interviewer nor the typical respondent is particularly knowledgeable about many of the items in the questionnaire, and this raises the question whether there are other sources that could provide better quality data. Because the CBECS is currently the most comprehensive data available on the energy consumption of commercial buildings in the United States, there is no “gold standard” against which the quality of the survey or other potential data sources can be evaluated. However, a variety of other sources exist and can provide at least partial data. Comparing the data from several of these sources will help EIA begin to understand the relative advantages associated with each and the optimal ways of combining information from different sources. Ultimately, conducting in-person interviews for at least a subset of the sample may be necessary for 4

OCR for page 1
validation purposes, if the research indicates that these interviews produce the highest quality data. In the rest of this section we discuss some additional sources of data that should be explored and validated at this stage, even if none of them, by itself, represents a realistic replacement for the building interviews. The panel is aware that EIA has considered the use of a variety of administrative records over the years. This option should be revisited periodically as the costs and data quality benefits associated with integrating these data sources evolve, and the 2011 CBECS presents an opportunity to carry out this research. Although different sources may be available for different types of buildings, for the purposes of evaluating them the overlap should be maximized to the extent possible. In other words, a subset of the buildings should be selected for this research and, for these buildings, data should be gathered from all of the available data sources being evaluated. The overlap will be particularly important with the building audits, as discussed below. Building Audits We understand that EIA has considered involving professional energy auditors to collect building data instead of relying on interviewers, but there has never been sufficient funding to implement this approach (except in the form of a small study in the past that was not conducted in conjunction with any of the CBECS data collections). We recommend testing the use of auditors on a small scale in the 2011 CBECS to better understand the costs and to collect data that can be used to assess the quality of other data sources. The data collected by the auditors would also be useful for evaluating some of the current back-end procedures, such as data editing, or the regression model used to identify outliers and to initiate a supplier follow-up survey. For a subset of the buildings we think it would be useful if the same data were collected by both interviewers and auditors to allow the evaluation of the differences between these two in-person data collection approaches, in addition to comparing them to information collected from other data sources. Even if geographic representation cannot be achieved due to cost considerations, to the extent possible the test should include buildings of different sizes and with different characteristics. The data collection should be performed by professional energy auditors, who would carry out their work around the same time as the other data collection efforts relevant to a particular building, and without knowledge of any data already collected or available about the buildings from other sources. Online Research The panel recommends selecting a small subset of the buildings in the 2011 CBECS sample and investigating the information that can be obtained about them solely from the Internet. This could be set up in the form of a pilot test involving a small number of buildings (for example, 10 large, 10 medium, and 10 small buildings). The results will provide EIA with a better understanding of what types of data are available online in terms of both quality and extent. If this research is scheduled before the beginning of the data collection, the insights gained could be useful in fine-tuning the data collection instruments and sample design for the 2011 CBECS, 5

OCR for page 1
but this type of research could be carried out at any time. Once the 2011 CBECS data are available, examining the consistency between the information available from a variety of Internet sources and the data collected through the current methods, as well as from building audits, will provide valuable information about data quality. LEED and Energy Star Certified Buildings Data quality can also be assessed by taking advantage of the information that is publicly available online about buildings that have received leadership in energy and environmental design (LEED) or energy star certification. Comparing the data collected through the building interviews about a subsample or all of the buildings that have such certification to the data submitted as part of the certification process for the same buildings can also contribute to a better understanding of possible data quality issues. Naturally, certified buildings are a specific subset of the CBECS sample, and their characteristics are not representative of the entire population of interest for the survey. However, examining any discrepancies in the data available about these buildings can improve EIA’s overall sense of the quality of the data and also identify potential areas of concern. The comparison could even be performed on data that have already been collected through a previous round of CBECS to inform the 2011 design. Data from Energy Suppliers The CBECS includes an energy supplier survey for about half of the CBECS buildings in the sample. The survey is initiated in cases where the energy usage and cost information cannot be obtained through a building interview or if the data obtained through the building interview are flagged as out of the expected range based on a regression model developed by EIA. To evaluate the quality of the data obtained from the building interviews, as well as the regression model used to identify out of range responses, the next round of the CBECS should include an energy supplier follow-up for more than just the problem cases currently included. For example, the 2011 CBECS could collect supplier data for a random sample of cases that provided usage and consumption data that were deemed valid during the building interview. Given the increasing interest in time-of-use, hourly, and real-time energy use data, the panel recommends collecting this type of information as well, where available. EIA could aim to collect hourly data or time-of-use data (along with rates) from a random sample of the suppliers contacted for a follow-up interview, all suppliers who are contacted for a follow-up interview, or a random sample of the suppliers for buildings for which interviews were also conducted. In addition, it is possible to specifically identify a few buildings with real-time metering and explore the data available from this source. The goal, again, would be to start establishing a framework for integrating this type of data into future surveys, understanding what type of information can be collected, and fine-tuning the procedures for working with a variety of (often reluctant) energy suppliers. 6

OCR for page 1
Digital Photographs EIA has considered the use of digital cameras in the past, and this idea should be revisited, at least as a one-time research effort. A test could be accomplished by either providing a small number of the 2011 CBECS interviewers with cameras or in a separate operation from the 2011 data collection, if the latter approach is deemed more cost-effective because of training and operational considerations. Photographs may provide valuable basic information about buildings. EIA’s definition of a building does not always correspond to a respondent’s definition of a building, and it is often left up to the interviewer to clarify the definition and come to an understanding with the respondent about what is meant by a building for the purposes of the interview. In addition, EIA staff indicated to the panel that the buildings of interest tend to use a wide range of specialized equipment related to building activity, and respondents’ abilities to describe the equipment vary. EIA should evaluate the extent to which the pictures of the buildings and critical equipment, including nameplates, are useful in the data cleaning and editing process (for example, for reconciling ambiguous or questionable entries) and whether their use could translate into cost savings in comparison to the current procedures. EIA should also investigate the privacy and confidentiality concerns and regulations that may be applicable to the potential use of cameras in this context, even if the pictures are only used for data cleaning and editing. Geocoding EIA should examine the costs and benefits of supplementing the data available about each case in the sample with the geographic coordinates of the building’s address. Geocodes could be added to the sample in house or during the data collection process. The former approach would probably be less precise, so capturing this information during the field work would be preferable if the interviewers can be equipped with the necessary devices at a reasonable cost. Adding geospatial information to each of the cases in the sample will enable researchers to conduct additional analyses of the CBECS data. For example, EIA currently integrates weather data from the National Oceanic and Atmospheric Administration (NOAA) into some of its analyses, and recording the building’s proximity to the closest weather station would expand the analytic possibilities. Again, EIA should conduct research on any potential confidentiality concerns related to the use of this type of data and whether there are ways of appending additional geographic information to the data while maintaining confidentiality. Other Data Sources The panel recommends evaluating other existing data sources that EIA has considered in the past, as well as the breadth and consistency of information that could be obtained from local governments. The availability of more funding for the 2011 CBECS than has been available in the past provides a unique opportunity to carry out research that can inform future decisions about the design of the survey. Even if the information available from the various data sources is limited in scope, the recommended research can provide valuable feedback about the quality of the self-reported building data and identify options for integrating a variety of data sources in the future. 7

OCR for page 1
DATA COLLECTION Process The panel understands that EIA staff participate in all interviewer training, but even more active involvement may be necessary to share the study’s goals and communicate how the quality of the data determines their usefulness. In addition, EIA staff members are also best qualified to conduct training on topics and concepts that are complicated, as a result of a long institutional history, such as the definition of a building and of a qualified respondent. Additional resources should be invested in analyzing the characteristics of the field operations and in identifying opportunities for increased efficiency. EIA should review any information available from the data collection contractor regarding the amount of time spent on cases of various types (such as buildings with different characteristics, respondents with different backgrounds, etc.). If the case level contact history is not recorded in sufficient detail, efforts should be made to capture this information in the future. In addition, EIA should ask the data collection contractor to schedule debriefings with the interviewers soon after the beginning of the field period, and EIA staff should attend these debriefings to better understand how interviewers spend their time in the field, what types of cases are presenting the biggest challenges, and why. A detailed analysis of the time allocation should reveal whether there are subsets of cases that require a disproportionately large amount of time to complete and whether the effort is justified in the context of data needs and statistical techniques available to compensate for missing information. EIA should also work closely with the data collection contractor to review the procedures used to select the best respondent for the building interviews and identify opportunities to streamline this process. Again, debriefings with interviewers can provide invaluable feedback that can help fine- tune the process and contribute to the development of new interviewer protocols. More efficient procedures for identifying a qualified respondent can not only reduce costs, but also address some of the concerns related to the technical nature of the questions. The qualitative feedback from the interviewers can then be further examined with an analysis of the quantitative responses by respondent type to identify possible differences in data quality. In other words, it is possible that most of the questions are not “too technical” if posed to the right respondent. Additional activities for which analyzing existing data could identify opportunities for increased efficiency include the handling of partial interviews, both in terms of the field operations and from the perspective of data editing. Given that the CBECS interview is relatively long (with an estimate of 30 to 45 minutes provided to respondents), interviewer debriefings could reveal new strategies for approaching buildings and asking for appointment times. It would also be useful to understand whether there is a pattern to at what point the partial interviews end and whether the order of the items in the questionnaire could be rearranged to make the partial interviews more useful to EIA for either weighting or imputation. Interviewers can be a good source of background and contextual information on questions that are difficult to administer, especially on whether particular questions are leading to partial interviews or possible data quality concerns. Discussions with the interviewers could represent 8

OCR for page 1
the beginning of a close examination of the questionnaire that has evolved with a face-to-face administration in mind and may need revising or simplifying to accommodate different future modes of data collection, as described in the next section. Modes Although EIA has considered the use of other modes of data collection, CBECS data are still collected primarily by in-person interviewing. In part because applying the CBECS definitions to determine the boundaries of a building is not always a straightforward task, as discussed above, EIA has continued to rely on face-to-face interviews. Identifying the most appropriate respondent is another task that is thought to benefit from the presence of an interviewer. Interviewers also carry hard-copy “show cards” that list the answer options for specific items and can be handed to the respondent to assist with answering questions that may otherwise be too difficult to remember if only read by an interviewer. In addition, one of the roles of the interviewers is to scan utility bills if they are available. To prevent declines in the response rates and to limit costs, EIA will have to revisit the use of other modes of data collection, particularly the possibility of a multimode approach, with at least a portion of the interviews being conducted online. Transitioning at least a subset of the buildings to the web will free up some resources in the long run, which then can be allocated to the more complex cases and possibly invested into increasing the sample size. Although collecting this type of data on the web will present some methodological challenges, the panel believes that these challenges can be addressed and that web data collection may also represent some methodological advantages, in addition to the likely cost savings. One possible approach that should be explored is to divide the sample into buildings that can be relatively easily transitioned to a web administration and buildings with more complicated characteristics that may benefit from interviewer administration. It may also be necessary to treat large buildings differently from smaller ones. The review of the case histories and the interviewer debriefings described above will be helpful in beginning to identify the building types for which data collection is fairly straightforward. Until reliable auxiliary data sources can be integrated into the data collection process, a first in- person visit to each building will still be useful. During this visit, interviewers should follow a protocol developed by EIA to determine whether a second in-person visit is necessary (as is currently done) or whether the building is a good candidate for a web interview. Given the concerns related to the definition of a building, the decision of whether a case can be transitioned to the web will likely have to depend in part on whether the definition seems straightforward, as it would be, for example, for a small, standalone building occupied by one business. Resources should be invested in testing ways of communicating the definition of a building through a self- administered format, in anticipation of possibly being able to transition more and more complex buildings to web administration in the future. The logistics of the best way to collect contact information for a web survey would have to be explored. Possible options include obtaining the information during the first visit or by telephone. Sometimes information on how to access a web survey is included in a hard-copy 9

OCR for page 1
advance letter mailed to respondents, even though this is less ideal than an e-mail invitation because it requires respondents to manually enter the web address of the survey and the login information. Since contacting respondents by mail may be the only option if an e-mail address cannot be obtained, investigating the extent to which building addresses and the mail delivery addresses overlap (as recommended above) will also be useful for this purpose. There is no question that identifying the best respondent for completing the interview is crucial in the case of the CBECS, but exploring ways of accomplishing this without involving an interviewer should be examined. It is possible that a web option could in fact contribute to more interviews being conducted with qualified respondents. In some cases, it may be easier to forward a questionnaire to the right person than to locate him or her in a building and arrange an interview. In-person interviewers may also have an incentive to complete an interview as quickly as possible by settling for a willing respondent rather than pursuing the most appropriate one. If a questionnaire is available on the web, it is also easier for several respondents to collaborate, each completing the sections he or she is most knowledgeable about. In addition, a web option could result in more complete data because it gives respondents the option to obtain information for questions they are not sure about and resume the survey later. Naturally, if respondents stop or forward a survey, there is a risk that they will not return to complete it, so an extensive follow- up effort is likely to be necessary. However, when a topic is too technical for many respondents, such as is the case of the CBECS, this kind of follow-up could make a significant difference in data quality. The CBECS questionnaire relies very heavily on show cards, which is another reason why the survey is administered face to face. However, the use of show cards raises the concern of order effects, especially because many of the show cards have a large number of answer options listed, making it difficult for respondents to focus equally on all of them. For example, the show card listing the answer options for the primary activity in the building contains 16 items. A respondent for a building with multiple activities may be tempted to select the first one that is applicable as the “primary” activity instead of carefully reviewing the entire list. A web questionnaire would make it easier to restructure these questions into layered sets of items, with fewer answer options, or to reduce the possibility of primacy effects with the use of innovative methods, such as the animated presentation of response choices or an eye-catching emphasis on the end of the list. The additional funding available for the 2011 CBECS represents an opportunity to test various ways of asking questions that EIA has identified as problematic because of their technical nature. For example, different approaches to obtaining the square footage information from respondents can be tested in the form of a split-sample experiment. Deconstructing this kind of an item into a series of questions would introduce complex skip patterns, but it would be easy to implement on the web without increasing the cognitive burden on respondents. Web administration can integrate various aids and tools for respondents, such as definitions or diagrams that can pop up if a respondent seems to be having trouble with a question or requests help. The interviewer debriefings described above will be useful in pinpointing specific questions that could benefit from a different approach and whether web administration is a promising option to pursue. 10

OCR for page 1
As is always the case with self-administered surveys, providing respondents with an e-mail address and toll-free telephone number they can use if they have questions may be valuable. The staff accessible through these means should be able to provide assistance related to the technical topics in the questionnaire, as well as answer to questions specific to the web administration. Finally, when evaluating the implications of transitioning to a mixed mode administration, options for collecting the utility bills that are currently collected during the interview should also be considered. Some respondents may be able to easily upload an electronic copy of their bills through the questionnaire website, and this possibility should be investigated. Asking respondents to mail a copy of their utility bills would probably not be cost-effective because extensive follow-up would likely be necessary. The options should be assessed in the context of the research conducted to evaluate the possibility of increased reliance on supplier data. The ideal time for beginning to explore the feasibility of transitioning some of the sample to web administration and conducting experiments on question wording is in parallel with the other data validation efforts, especially the involvement of the energy auditors. This timing will allow EIA to allocate some of the funding currently available to cover the cost of the transition, and it will provide an opportunity to take maximum advantage of the data collected from different sources. An analysis of the data collected from different sources can also guide decisions on whether the in-person interviews can be used in the future to calibrate the data collected through the web. All of the research should keep long-term plans in mind, such as the characteristics of the web as a data collection mode, even if the integration of web interviews is not realistic for the 2011 CBECS. DATA RELEASES The panel learned that the schedule of the data releases is a major concern to users who would like to see the lag between the data collection and release date reduced. EIA has been working on taking greater advantage of the Internet to facilitate data distribution.1 We note that introducing a web option during the data collection stage can, in the long run, reduce the time necessary for preparing the files for release by reducing data editing and cleaning time. The panel also recommends evaluating the possibility of eliminating some of the editing steps by reducing the number of editing rules or the number of variables edited or by focusing on cases that have the most impact on the estimation. Many data users will not only appreciate a shorter lag between data collection and release, but may also prefer access to data with fewer edits. SUMMARY The 2011 CBECS presents an opportunity to conduct research that can guide the redesign of the survey on the basis of empirical data about the most cost effective approach for collecting valid and reliable information about the energy consumption of commercial buildings. This letter 1 Energy Information Administration. (2009). State Energy Data Needs Assessment. Report SR-EMEU(2009)01. Washington, DC: U.S. Department of Energy.    11

OCR for page 1
report of the panel outlines a variety of research topics that seem most promising to pursue before or as part of the 2011 CBECS data collection. EIA should focus its efforts on (1) evaluating the availability and quality of alternative data sources that could assist with sampling frame development and potentially provide substantive data, and (2) developing a strategy for transitioning some of the interviews to a web-based data collection mode. This research will inform a possible future redesign of the sampling methodology and revisions to the data collection procedures that could be considered for subsequent rounds of the CBECS. We hope this letter and our recommendations are helpful to you in planning the 2011 CBECS. Sincerely, William F. Eddy Chair 12