Click for next page ( 224


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 223
Appendix D Statement of Work for CNSTAT Competitive Solicitation of Design Ideas RFP #DBASSE-004950-0001-031411 INTRODUCTION The Consumer Expenditure (CE) Quarterly Interview and Diary Sur- veys comprise a major program of the Bureau of Labor Statistics (BLS). The CE data are used to provide the market basket budget shares for one of the Nation’s most important statistics, the Consumer Price Index. The CE surveys’ unique and valuable data on the spending patterns of American consumers are used in a multitude of ways, including a new Supplemental Poverty Measure, IRS sales tax information, and economic research. The design of the CE surveys must be updated periodically to align their methodology with changes in society, technology, consumer prod- ucts, and consumer spending methods on survey estimates. Without such updates, the CE surveys will not be able to continue to fulfill their mission of producing high-quality expenditure estimates in support of their critical uses. All household surveys today face well-known challenges that include increasingly busy respondents, confidentiality and privacy concerns, many competing surveys, controlled-access residences, and non-English speak- ing households. In addition, the CE surveys face a unique challenge as the phenomena the surveys seek to measure have changed over the past 30 years, and continue to do so. Although the CE has made a number of im- provements to the survey design, such as transitioning to computer-assisted personal interviewing (CAPI), it has not implemented changes within a systematic, comprehensive framework to address this, and other, challenges. 223

OCR for page 223
224 MEASURING WHAT WE SPEND The CE surveys are faced with multiple issues that directly impact the quality of the data collected. These issues, presented in order of importance below, include (a) evidence of measurement error, most importantly under- reporting, in the survey data, (b) changes in the survey environment due to both new technology and consumer behaviors, and (c) the need for greater flexibility in the mode of data collection. (See “Background Mate­ ial” on r p. 227 for links to more information.) Measurement Error. Underreporting in the CE is evidenced by a grow- ing deviation from other data sources and by the results of several studies. Since 1984, the ratios of aggregate expenditure estimates from the CE compared with Personal Consumption Expenditures (PCE) data from the National Accounts have shown a declining trend for many expense categories. Internal methodological studies and the 2008 CE Program Review had similar findings, providing further evidence of a growing concern about the quality of reported data. Underreporting in the CE may result from respondent burden due to survey length and complexity, panel or questionnaire conditioning, increasing telephone administration of a survey originally designed for personal visit inter- views, proxy reporting by a single household member, recall effects stemming from a 3-month reference period, and/or other causes.  hanges in the Survey Environment. To remain effective, the CE sur- C veys must adapt to changes in purchasing behaviors. For example, respondents may purchase a variety of types of items in a single large store, such as Costco or Walmart, rather than buying a single type of item from a single store. The topic-specific design of the current survey instrument may not best aid respondent recall or reporting of this type of buying. It is equally unknown whether the current survey instrument is effective for capturing on-line purchases or automatic payments. Questionnaire design and data collection methods may have to be adapted to better account for these issues. Parallel to the changes in consumer behaviors, a transformation has been occurring in the tech- nology available for collecting data. The availability of new technology and software, such as a Web diary, a portable digital assistant (PDA) instrument, or financial software sheets, offers new opportunities to collect data. Finally, administrative sources of expenditure data (such as transaction databases built from credit/debit and from loyalty card use) now exist that provide potential alternatives to survey data.  lexible Mode of Data Collection. One size does not fit all, and the CE F needs greater flexibility in data collection modes for two main reasons: to be responsive to the needs of respondents and to allow faster imple-

OCR for page 223
APPENDIX D 225 mentation of changes to collection instruments. Regarding the first, a multimode design would allow data collection to be tailored to the needs and preferences of the respondent. For example, some respon- dents have very little time, others have difficulty keeping a diary, and still others do not want an interviewer in their home. Each respondent would have different ways of optimally reporting their data. The sec- ond motivation for greater flexibility is to allow for design changes to be made responsively as society or technology changes. Currently mak- ing changes to the CAPI instrument requires considerable lead time, and even minor changes often impact the whole instrument. A survey design utilizing multiple modes, modules, and/or technologies may al- low for changes to be made to a single mode without disrupting others. National Academy of Sciences The Bureau of Labor Statistics has undertaken a multiyear process to develop and implement a redesign of the CE surveys to address the above issues. As part of this process BLS has contracted with the National Acad- emy of Sciences Committee on National Statistics (CNSTAT) to conduct an independent review of the design options available, and to make specific recommendations for redesign. CNSTAT has organized a panel of experts (subsequently referred to as the panel) from across the country to carry out that task. (Panel membership is presented in Appendix A.) The panel is seeking proposals for design options and a discussion of the relative merits of the options, particularly from organizations with experience in designing complex data collection methods. It is in this context that the panel initiated this Request for Proposal through the National Academy of Sciences. A subcommittee of the panel will serve as the selection panel for this contract. Description, Scope, and Primary Tasks The Contractor will produce a research report laying out a design (including research studies to evaluate the proposed design) to collect information required by the primary users of the current CE. The design should be robust to recent and potential changes in the data collection and retail environment discussed above. The proposed design should focus on features that are proactive in collecting information such as the use of scan- ners, PDAs, or external data sources. A recall survey would be used only to supplement information not otherwise available. Thus, the proposed design would be fundamentally different from that of the current Quarterly CE survey. NAS anticipates that a team approach may be needed to meet the conditions of this contract, and the Contractor may consult with experts outside its own organization.

OCR for page 223
226 MEASURING WHAT WE SPEND The report will (1) summarize the Contractor’s evaluation that led them to reach the final proposal, (2) detail the proposed design, and (3) provide recommendations for evaluating the proposal in a CE-specific context. Each of these elements is specified below. 1.  The Contractor will develop a carefully balanced evaluation of the prospective benefits, costs and risks of their proposed methodological and technological options compared to the current CE surveys. This evaluation should be • d  eveloped and presented within the context defined by the primary user needs identified in the background material. •  ased on extensive and carefully balanced review of: b  literature and industry knowledge on household and establish- ment survey methodology and practice that is currently avail- able or likely to be available in practical form in the next five years.  data collection technologies and external sources of expendi- ture data currently available or likely to be available in practi- cal form with the next five years. Examples include, but are not limited to: bar code or receipt scanners, audio diaries, PDA instruments, and retail “loyalty card” or transaction-processor databases. •  eflective of the tradeoff between cost and improvement on mea- r surement error in terms of external benchmarks (e.g., CE to PCE comparisons) and other standard evaluations of data quality (e.g., subgroup comparisons). 2. Based on results from the evaluation described above, the report should detail a comprehensive proposal for a survey design, and/or other data acquisition process, which collects the required information. The pro- posal should describe all relevant elements for the development and implementation of the proposed design, including, but not limited to: • d  ata sources; • s  ample design, including size, weighting and precision implications; • d  ata collection mode(s) and/or description of alternative acquisition; • d  ata collection procedures; • e  stimates of the cost requirements for development, implementa- tion, and ongoing data collection; and • e  stimates of time requirements for development and implementa- tion of the design, as well as time required to collect and process data to create required estimates for publication.

OCR for page 223
APPENDIX D 227  the proposed design, the Contractor should address issues with the In current CE surveys: • U  nderreporting of expenditures. • F  undamental changes in the social environment for collection of survey data. • F  undamental changes in the retail environment (e.g., online spend- ing, automatic payments). • T  he potential availability of large amounts of expenditure data from a relatively small number of intermediaries such as credit card companies. • D  eclining response rates at the unit, wave, and item levels. The proposal may include a moderate number of methodological and technological options, (e.g., a mixture of household-based and estab- lishment/intermediary-based data collection) but should go beyond simply listing possible options. Instead the report should provide, to the extent possible, concrete and specific design elements supported by in-depth explanations and evidence from the Contractor’s evalu- ation. In other words, any proposed design should be practical and supported by evidence to demonstrate feasibility and have a rationale for expectations of improvement to the issues associated with the CE surveys. 3.  addition to proposing a new CE design, the report should include In clear recommendations for evaluating the proposal in a CE-specific context, such as a pilot study. The recommendation should include • t  he design of a study or studies to evaluate the feasibility and ef- fectiveness of the proposed design, • a  careful description of the ways in which the study or studies would provide practical insights into the proposed new design, and • e  stimated financial and time requirements for the recommended study or studies. Background Material The Contractor will review background material pertinent to the potential redesign of the CE. Many of these materials can be found at the Gemini Project website (see http://data.bls.gov/cgi-bin/print.pl/cex/geminimaterials. htm). These materials include, but are not limited to, the following:

OCR for page 223
228 MEASURING WHAT WE SPEND • G  emini Project Vision Document. • B  LS Handbook of Methods, Chapter 16, Consumer Expenditures and Income. • B  LS Statement of CE data priorities (available May 1, 2011). • C  PI Requirements of the CE. • C  E Data Quality Definition Report. • P  apers presented at the June 2010 CE Data Users’ Needs Forum. • P  apers presented at the Consumer Expenditure Survey Methods Workshop, held December 8-9, 2010 in Suitland, MD. • P  apers presented at the March 2010 Data Capture Technology Forum. Deliverables The following deliverables are required under this contract: • C  ontractor will participate in a kickoff meeting no later than four- teen (14) days following the award of this contract. • C  ontractor will produce monthly progress reports on the project. • C  ontractor will produce a draft report for NAS no later than ninety (90) days after the award of this contract. Draft report may be submitted electronically. • C  ontractor will revise the draft report based on NAS comments and provide 8 copies of the full report developed under this pro- posal and based on Description, Scope and Primary Tasks listed above. Final report is due 120 days from the contract award. Contractor will present the basic information in the full report to the panel at its upcoming Workshop on Redesign Options for the Consumer Expenditure Surveys, scheduled for October 26–27, 2011, and participate with the panel and other members of the workshop in a discussion of design options.