Effective Tracking of Building Energy Use Improving the Commercial Buildings and Residential Energy Consumption Surveys (2012) / Chapter Skim
Currently Skimming:
5 Redesigning the Commercial Buildings Energy Consumption Survey
5 Redesigning the Commercial Buildings Energy Consumption Survey
Pages 39-72
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 39... ...
The major recommendations include implementing a rotating sample design with a longitudinal element for part of the sample and transitioning to a multimode data collection. The second half of this chapter offers additional suggestions for updating the CBECS with changes that are more incremental in nature.
|
From page 40... ...
For the purpose of discussion in the sections that follow, the panel assumes that the CBECS data collection will resume on at least a quadrennial schedule in fiscal year 2012. Rotating Panel Design for the CBECS One option for addressing data users' need to have quicker access to the data is to implement a rotating sample design, which would make more frequent data releases possible.
|
From page 41... ...
In some years, there will be some overlap in the reference period for the data collected by the three energy consumption surveys, which could also have some analytic advantages. Although the transition to a new sample design and new data collection operations will involve some temporarily increased costs compared to the typical start-up costs associated with the current design of the survey, the expectation is that, at least for the most straightforward implementation of the rotating design (Option 1 described below)
|
From page 42... ...
. Although the panel cannot fully assess these factors as they apply to the EIA energy consumption surveys, it is able to suggest options that should be explored using real data.
|
From page 43... ...
. Of course, it would still be possible to estimate θt using data from only Year t, but the estimate would be based on only one-quarter of the current sample size.
|
From page 44... ...
. In general, there would be 1 For further details on composite estimation, see Appendix E
|
From page 45... ...
before the rotation pattern stabilizes. That is, up until the Rth year, there will be some buildings that stay in the sample for fewer than R occasions.
|
From page 46... ...
Setting F = 2, for example, would offer a compromise between conducting the survey annually and having a fouryear gap between surveys. If buildings were surveyed every two years, the correlations for some key variables might still be high enough to yield good gains using composite estimation.
|
From page 47... ...
Recommendation CBECS-2: EIA should consider integrating a longitudinal element into the CBECS sample design to obtain better estimates of change. CBECS Multimode Data Collection Another strategy for increasing the timeliness of the data releases would be to change some of the interviews from face-to-face settings to collecting data via the web.
|
From page 48... ...
Although EIA has in the past considered using other modes of data collection, at this time CBECS data are still collected primarily by inperson interviewing. One reason is the initial cost of implementing a new data collection mode.
|
From page 49... ...
Once a web questionnaire has been built and tested, there are virtually no costs associated with the actual individual web interviews, in contrast to face-to-face interviewing, for which cost is directly related to the number of cases. Obviously, introducing a web data collection mode does not mean that follow-up, editing, and other data collection costs can be eliminated, even for the interviews that are conducted over the web.
|
From page 50... ...
may be especially good candidates to move to the web once one or more in-person visits have been completed and the boundaries have been ascertained. This strategy could be implemented regardless of whether a longitudinal element is introduced in the sample design.
|
From page 51... ...
However, when a topic is too technical for many respondents, such as is the case with the CBECS, this kind of follow-up could make a significant difference in data quality. Currently, the face-to-face CBECS interviews rely heavily on hard-copy show cards.
|
From page 52... ...
For example, the show card listing the answer options for the primary activity in the building contains 16 items, and a respondent for a building with multiple activities may be tempted to select the first one that is applicable as the "primary" activity instead of carefully reviewing the entire list. A web questionnaire would make it easier to restructure these questions into layered sets of items with fewer answer options, or to reduce the possibility of primacy effects by using various innovative methods, such as the animated presentation of response choices or an eye-catching emphasis on the end of the list.
|
From page 53... ...
Recommendation CBECS-3: Informed by a methodological research program, EIA should begin developing procedures for a multimode approach and should begin moving some of the CBECS data collec tion to the web. Revised Editing Procedures for the CBECS CBECS data undergo a complex series of edits to locate and correct errors in the recorded responses in order to increase accuracy.
|
From page 54... ...
Because the CAPI edits are built into the interview program, they are not as resource- or time-intensive as the postsurvey editing steps. While some of the postsurvey checks are automated, the editing is almost always based on manual review, and sometimes requires a callback to the building.
|
From page 55... ...
As discussed in the previous section, emphasis should be placed on a shift to editing procedures that can be implemented in a self-administered setting to facilitate the integration of a web data collection mode. This could include revising the warning and error messages that pop up during the interview to make them suitable for self-administration and evaluating whether more of the CAPI edits that currently involve warnings could be handled automatically.
|
From page 56... ...
In addition to a possible reduction in the time required to process the data, revised editing procedures could have the additional benefit of substantial reducing the staff time that is currently dedicated to this process. If some of the staff time could be freed up by eliminating some of the editing steps, EIA would be able to invest these resources into the start-up work required to implement a rotating sample design and web data collection.
|
From page 57... ...
It is especially difficult, using the data that are currently available, to perform multivariate analyses to assess the impact of policy decisions or to explain changes in energy consumption. A CBECS sample size of approximately 6,000 cases represents nearly 5 million commercial buildings.
|
From page 58... ...
Specific building types that are of interest to researchers include data centers, laboratories, convention centers, and arenas -- and there are probably others as well -- and these needs have to be evaluated and prioritized. One addition to the survey that could help data users better understand trends in the building sector would be to provide oversamples of new construction for use in assessing the degree of compliance with building energy codes.
|
From page 59... ...
However, this is an important factor in EIA's ability to meet data user needs. Recommendation CBECS-5: As part of its efforts to address the needs of data users, EIA should make it a priority to identify opportunities for increasing the sample size in order to enable the release of more of the CBECS data that are currently being collected.
|
From page 60... ...
. Recommendation CBECS-6: EIA should consider establishing a research data center or evaluate the option of using an existing RDC maintained by another organization to provide data users with secure access to CBECS data that are currently not publicly released.
|
From page 61... ...
• nformation about the efficiency ratings of end use equipment (rat I ings of fluorescent lighting of different sizes, such as T8 versus T12, energy efficiency ratio, seasonal energy efficiency ratio, annual fuel utilization efficiency ratings, and so on)
|
From page 62... ...
Some data might, for instance, be collected as part of a special study involving energy auditors, and it might be more efficient to collect information on rate structures from the energy utilities rather than from the sample buildings. Such options for collecting additional data will be discussed further in subsequent sections.
|
From page 63... ...
As a first step, EIA should contact suppliers that have smart metering in place to evaluate what level of data frequency is available for what customer groups, and to assess options for accessing the data. Options then could be evaluated for collecting smart meter data from a random sample of the suppliers contacted for a follow-up interview, all suppliers who are contacted for a follow-up interview, or from a random sample of the suppliers for buildings for which interviews were also conducted.
|
From page 64... ...
CBECS Sample Design As discussed above, the CBECS is based on an area probability sample of commercial buildings, supplemented with lists of special buildings. This is a thorough and well-respected method for developing a sampling frame, but it is expensive.
|
From page 65... ...
Because establishment surveys are more common than building surveys, the use of establishments as the basis of the sampling frame would allow the CBECS to benefit from procedures, lists, and data that already exist in connection with these data collections. As a first step in this process, EIA staff time would be required to evaluate the establishment-based list options that are available for use in sampling frame development and to assess their implications for the efficiency of the data collection.
|
From page 66... ...
The evaluation should determine whether there are centralized accounts that can provide all of the information necessary for the CBECS for all of their buildings in the sample, and assess how much variability can be expected between the accounts in terms of the data available centrally. Recommendation CBECS-11: To increase CBECS data collection efficiency, EIA should explore the possibility of contacting the head quarters of major centralized accounts and collecting data about all of the buildings in the sample from these centralized sources.
|
From page 67... ...
Another consideration is that establishing a framework for closer collaboration with utilities would also be useful in paving the way for greater reliance on smart meter data. Recommendation CBECS-12: EIA should evaluate whether working more closely with the energy suppliers of commercial buildings could lead to efficiencies in the data collection process.
|
From page 68... ...
A successful integration of auxiliary data sources into the energy consumption surveys could have some important advantages. Given the technical nature of the questions asked, for example, there are concerns about respondents' ability to provide accurate answers, which lead in turn to concerns about the accuracy and reliability of the data.
|
From page 69... ...
The 2011 CBECS data collection plan included a smallscale test with both auditors and interviewers collecting data from the same buildings in order to evaluate the differences between the two approaches, but the test was put on hold along with the survey. The panel considers this an important test to complete because it could have a large number of benefits.
|
From page 70... ...
The techniques used to evaluate the questions could involve examining the distribution of the responses to each of the questions to identify answer options that no longer reflect the distribution of building characteristics of interest as well as answer categories that may have become obsolete. Cognitive interviews could be conducted to better understand how respondents relate to the question and answer options and to evaluate whether updates are needed.
|
From page 71... ...
EIA should also ask the data collection contractor to schedule debriefings with the interviewers soon after the beginning of the field period, and EIA staff should attend these debriefings to better understand how interviewers spend their time in the field and what types of cases are presenting the biggest challenges and why. A detailed analysis of the time allocation should reveal whether there are subsets of cases that require a disproportionately large amount of time to complete and whether the effort is justified in the context of data needs and statistical techniques available to compensate for missing information.
|
From page 72... ...
Recommendation CBECS-16: Interviewer debriefings should become an integral part of the CBECS data collection process in order to iden tify problems with the questionnaires and procedures and to serve as a source of ideas for increased efficiencies.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.