National Academies Press: OpenBook

Measuring What We Spend: Toward a New Consumer Expenditure Survey (2013)

Chapter: Appendix D: Statement of Work for CNSTAT Competitive Solicitation of Design Ideas

« Previous: Appendix C: Uses of the CE in Administering Federal Programs: Debriefing of Program Staff
Suggested Citation:"Appendix D: Statement of Work for CNSTAT Competitive Solicitation of Design Ideas." National Research Council. 2013. Measuring What We Spend: Toward a New Consumer Expenditure Survey. Washington, DC: The National Academies Press. doi: 10.17226/13520.
×

Appendix D

Statement of Work for CNSTAT Competitive Solicitation of Design Ideas

RFP #DBASSE-004950-0001-031411

INTRODUCTION

The Consumer Expenditure (CE) Quarterly Interview and Diary Surveys comprise a major program of the Bureau of Labor Statistics (BLS). The CE data are used to provide the market basket budget shares for one of the Nation’s most important statistics, the Consumer Price Index. The CE surveys’ unique and valuable data on the spending patterns of American consumers are used in a multitude of ways, including a new Supplemental Poverty Measure, IRS sales tax information, and economic research.

The design of the CE surveys must be updated periodically to align their methodology with changes in society, technology, consumer products, and consumer spending methods on survey estimates. Without such updates, the CE surveys will not be able to continue to fulfill their mission of producing high-quality expenditure estimates in support of their critical uses. All household surveys today face well-known challenges that include increasingly busy respondents, confidentiality and privacy concerns, many competing surveys, controlled-access residences, and non-English speaking households. In addition, the CE surveys face a unique challenge as the phenomena the surveys seek to measure have changed over the past 30 years, and continue to do so. Although the CE has made a number of improvements to the survey design, such as transitioning to computer-assisted personal interviewing (CAPI), it has not implemented changes within a systematic, comprehensive framework to address this, and other, challenges.

Suggested Citation:"Appendix D: Statement of Work for CNSTAT Competitive Solicitation of Design Ideas." National Research Council. 2013. Measuring What We Spend: Toward a New Consumer Expenditure Survey. Washington, DC: The National Academies Press. doi: 10.17226/13520.
×

The CE surveys are faced with multiple issues that directly impact the quality of the data collected. These issues, presented in order of importance below, include (a) evidence of measurement error, most importantly underreporting, in the survey data, (b) changes in the survey environment due to both new technology and consumer behaviors, and (c) the need for greater flexibility in the mode of data collection. (See “Background Material” on p. 227 for links to more information.)

Measurement Error. Underreporting in the CE is evidenced by a growing deviation from other data sources and by the results of several studies. Since 1984, the ratios of aggregate expenditure estimates from the CE compared with Personal Consumption Expenditures (PCE) data from the National Accounts have shown a declining trend for many expense categories. Internal methodological studies and the 2008 CE Program Review had similar findings, providing further evidence of a growing concern about the quality of reported data. Underreporting in the CE may result from respondent burden due to survey length and complexity, panel or questionnaire conditioning, increasing telephone administration of a survey originally designed for personal visit interviews, proxy reporting by a single household member, recall effects stemming from a 3-month reference period, and/or other causes.

Changes in the Survey Environment. To remain effective, the CE surveys must adapt to changes in purchasing behaviors. For example, respondents may purchase a variety of types of items in a single large store, such as Costco or Walmart, rather than buying a single type of item from a single store. The topic-specific design of the current survey instrument may not best aid respondent recall or reporting of this type of buying. It is equally unknown whether the current survey instrument is effective for capturing on-line purchases or automatic payments. Questionnaire design and data collection methods may have to be adapted to better account for these issues. Parallel to the changes in consumer behaviors, a transformation has been occurring in the technology available for collecting data. The availability of new technology and software, such as a Web diary, a portable digital assistant (PDA) instrument, or financial software sheets, offers new opportunities to collect data. Finally, administrative sources of expenditure data (such as transaction databases built from credit/debit and from loyalty card use) now exist that provide potential alternatives to survey data.

Flexible Mode of Data Collection. One size does not fit all, and the CE needs greater flexibility in data collection modes for two main reasons: to be responsive to the needs of respondents and to allow faster implementation

Suggested Citation:"Appendix D: Statement of Work for CNSTAT Competitive Solicitation of Design Ideas." National Research Council. 2013. Measuring What We Spend: Toward a New Consumer Expenditure Survey. Washington, DC: The National Academies Press. doi: 10.17226/13520.
×

of changes to collection instruments. Regarding the first, a multimode design would allow data collection to be tailored to the needs and preferences of the respondent. For example, some respondents have very little time, others have difficulty keeping a diary, and still others do not want an interviewer in their home. Each respondent would have different ways of optimally reporting their data. The second motivation for greater flexibility is to allow for design changes to be made responsively as society or technology changes. Currently making changes to the CAPI instrument requires considerable lead time, and even minor changes often impact the whole instrument. A survey design utilizing multiple modes, modules, and/or technologies may allow for changes to be made to a single mode without disrupting others.

National Academy of Sciences

The Bureau of Labor Statistics has undertaken a multiyear process to develop and implement a redesign of the CE surveys to address the above issues. As part of this process BLS has contracted with the National Academy of Sciences Committee on National Statistics (CNSTAT) to conduct an independent review of the design options available, and to make specific recommendations for redesign. CNSTAT has organized a panel of experts (subsequently referred to as the panel) from across the country to carry out that task. (Panel membership is presented in Appendix A.) The panel is seeking proposals for design options and a discussion of the relative merits of the options, particularly from organizations with experience in designing complex data collection methods. It is in this context that the panel initiated this Request for Proposal through the National Academy of Sciences. A subcommittee of the panel will serve as the selection panel for this contract.

Description, Scope, and Primary Tasks

The Contractor will produce a research report laying out a design (including research studies to evaluate the proposed design) to collect information required by the primary users of the current CE. The design should be robust to recent and potential changes in the data collection and retail environment discussed above. The proposed design should focus on features that are proactive in collecting information such as the use of scanners, PDAs, or external data sources. A recall survey would be used only to supplement information not otherwise available. Thus, the proposed design would be fundamentally different from that of the current Quarterly CE survey. NAS anticipates that a team approach may be needed to meet the conditions of this contract, and the Contractor may consult with experts outside its own organization.

Suggested Citation:"Appendix D: Statement of Work for CNSTAT Competitive Solicitation of Design Ideas." National Research Council. 2013. Measuring What We Spend: Toward a New Consumer Expenditure Survey. Washington, DC: The National Academies Press. doi: 10.17226/13520.
×

The report will (1) summarize the Contractor’s evaluation that led them to reach the final proposal, (2) detail the proposed design, and (3) provide recommendations for evaluating the proposal in a CE-specific context. Each of these elements is specified below.

1. The Contractor will develop a carefully balanced evaluation of the prospective benefits, costs and risks of their proposed methodological and technological options compared to the current CE surveys. This evaluation should be

• developed and presented within the context defined by the primary user needs identified in the background material.

• based on extensive and carefully balanced review of:

o literature and industry knowledge on household and establishment survey methodology and practice that is currently available or likely to be available in practical form in the next five years.

o data collection technologies and external sources of expenditure data currently available or likely to be available in practical form with the next five years. Examples include, but are not limited to: bar code or receipt scanners, audio diaries, PDA instruments, and retail “loyalty card” or transaction-processor databases.

• reflective of the tradeoff between cost and improvement on measurement error in terms of external benchmarks (e.g., CE to PCE comparisons) and other standard evaluations of data quality (e.g., subgroup comparisons).

2. Based on results from the evaluation described above, the report should detail a comprehensive proposal for a survey design, and/or other data acquisition process, which collects the required information. The proposal should describe all relevant elements for the development and implementation of the proposed design, including, but not limited to:

• data sources;

• sample design, including size, weighting and precision implications;

• data collection mode (s) and/or description of alternative acquisition;

• data collection procedures;

• estimates of the cost requirements for development, implementation, and ongoing data collection; and

• estimates of time requirements for development and implementation of the design, as well as time required to collect and process data to create required estimates for publication.

Suggested Citation:"Appendix D: Statement of Work for CNSTAT Competitive Solicitation of Design Ideas." National Research Council. 2013. Measuring What We Spend: Toward a New Consumer Expenditure Survey. Washington, DC: The National Academies Press. doi: 10.17226/13520.
×

In the proposed design, the Contractor should address issues with the current CE surveys:

• Underreporting of expenditures.

• Fundamental changes in the social environment for collection of survey data.

• Fundamental changes in the retail environment (e.g., online spending, automatic payments).

• The potential availability of large amounts of expenditure data from a relatively small number of intermediaries such as credit card companies.

• Declining response rates at the unit, wave, and item levels.

The proposal may include a moderate number of methodological and technological options, (e.g., a mixture of household-based and establishment/ intermediary-based data collection) but should go beyond simply listing possible options. Instead the report should provide, to the extent possible, concrete and specific design elements supported by in-depth explanations and evidence from the Contractor’s evaluation. In other words, any proposed design should be practical and supported by evidence to demonstrate feasibility and have a rationale for expectations of improvement to the issues associated with the CE surveys.

3. In addition to proposing a new CE design, the report should include clear recommendations for evaluating the proposal in a CE-specific context, such as a pilot study. The recommendation should include

• the design of a study or studies to evaluate the feasibility and effectiveness of the proposed design,

• a careful description of the ways in which the study or studies would provide practical insights into the proposed new design, and

• estimated financial and time requirements for the recommended study or studies.

Background Material

The Contractor will review background material pertinent to the potential redesign of the CE. Many of these materials can be found at the Gemini Project website (see http://data.bls.gov/cgi-bin/print.pl/cex/geminimaterials.htm). These materials include, but are not limited to, the following:

Suggested Citation:"Appendix D: Statement of Work for CNSTAT Competitive Solicitation of Design Ideas." National Research Council. 2013. Measuring What We Spend: Toward a New Consumer Expenditure Survey. Washington, DC: The National Academies Press. doi: 10.17226/13520.
×
  • Gemini Project Vision Document.
  • BLS Handbook of Methods, Chapter 16, Consumer Expenditures and Income.
  • BLS Statement of CE data priorities (available May 1, 2011).
  • CPI Requirements of the CE.
  • CE Data Quality Definition Report.
  • Papers presented at the June 2010 CE Data Users’ Needs Forum.
  • Papers presented at the Consumer Expenditure Survey Methods Workshop, held December 8-9, 2010 in Suitland, MD.
  • Papers presented at the March 2010 Data Capture Technology Forum.

Deliverables

The following deliverables are required under this contract:

  • Contractor will participate in a kickoff meeting no later than fourteen (14) days following the award of this contract.
  • Contractor will produce monthly progress reports on the project.
  • Contractor will produce a draft report for NAS no later than ninety (90) days after the award of this contract. Draft report may be submitted electronically.
  • Contractor will revise the draft report based on NAS comments and provide 8 copies of the full report developed under this proposal and based on Description, Scope and Primary Tasks listed above. Final report is due 120 days from the contract award.

Contractor will present the basic information in the full report to the panel at its upcoming Workshop on Redesign Options for the Consumer Expenditure Surveys, scheduled for October 26–27, 2011, and participate with the panel and other members of the workshop in a discussion of design options.

Suggested Citation:"Appendix D: Statement of Work for CNSTAT Competitive Solicitation of Design Ideas." National Research Council. 2013. Measuring What We Spend: Toward a New Consumer Expenditure Survey. Washington, DC: The National Academies Press. doi: 10.17226/13520.
×
Page 223
Suggested Citation:"Appendix D: Statement of Work for CNSTAT Competitive Solicitation of Design Ideas." National Research Council. 2013. Measuring What We Spend: Toward a New Consumer Expenditure Survey. Washington, DC: The National Academies Press. doi: 10.17226/13520.
×
Page 224
Suggested Citation:"Appendix D: Statement of Work for CNSTAT Competitive Solicitation of Design Ideas." National Research Council. 2013. Measuring What We Spend: Toward a New Consumer Expenditure Survey. Washington, DC: The National Academies Press. doi: 10.17226/13520.
×
Page 225
Suggested Citation:"Appendix D: Statement of Work for CNSTAT Competitive Solicitation of Design Ideas." National Research Council. 2013. Measuring What We Spend: Toward a New Consumer Expenditure Survey. Washington, DC: The National Academies Press. doi: 10.17226/13520.
×
Page 226
Suggested Citation:"Appendix D: Statement of Work for CNSTAT Competitive Solicitation of Design Ideas." National Research Council. 2013. Measuring What We Spend: Toward a New Consumer Expenditure Survey. Washington, DC: The National Academies Press. doi: 10.17226/13520.
×
Page 227
Suggested Citation:"Appendix D: Statement of Work for CNSTAT Competitive Solicitation of Design Ideas." National Research Council. 2013. Measuring What We Spend: Toward a New Consumer Expenditure Survey. Washington, DC: The National Academies Press. doi: 10.17226/13520.
×
Page 228
Next: Appendix E: Household Survey Producers Workshop Agenda »
Measuring What We Spend: Toward a New Consumer Expenditure Survey Get This Book
×
Buy Paperback | $50.00 Buy Ebook | $40.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The Consumer Expenditure (CE) surveys are the only source of information on the complete range of consumers' expenditures and incomes in the United States, as well as the characteristics of those consumers. The CE consists of two separate surveys: (1) a national sample of households interviewed five times at three-month intervals; and (2) a separate national sample of households that complete two consecutive one-week expenditure diaries. For more than 40 years, these surveys, the responsibility of the Bureau of Labor Statistics (BLS), have been the principal source of knowledge about changing patterns of consumer spending in the U.S. population.

In February 2009, BLS initiated the Gemini Project, the aim of which is to redesign the CE surveys to improve data quality through a verifiable reduction in measurement error with a particular focus on underreporting. The Gemini Project initiated a series of information-gathering meetings, conference sessions, forums, and workshops to identify appropriate strategies for improving CE data quality. As part of this effort, BLS requested the National Research Council's Committee on National Statistics (CNSTAT) to convene an expert panel to build on the Gemini Project by conducting further investigations and proposing redesign options for the CE surveys.

The charge to the Panel on Redesigning the BLS Consumer Expenditure Surveys includes reviewing the output of a Gemini-convened data user needs forum and methods workshop and convening its own household survey producers workshop to obtain further input. In addition, the panel was tasked to commission options from contractors for consideration in recommending possible redesigns. The panel was further asked by BLS to create potential redesigns that would put a greater emphasis on proactive data collection to improve the measurement of consumer expenditures. Measuring What We Spend summarizes the deliberations and activities of the panel, discusses the conclusions about the uses of the CE surveys and why a redesign is needed, as well as recommendations for the future.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!