Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 208
Appendix B
BLS Communication of Issues
ADDITIONAL COMMUNICATION TO PANEL VIA E-MAIL
FROM KATHY DOWNEY ON NOVEMBER 11, 2011
This is to followup on our conference call this afternoon. As Adam
and I stated, Mike Horrigan (OPLC associate commissioner), Jay Ryan (CE
division chief), and John Layng (CPI division chief) were concerned about
comments overheard at the 10/27 CNSTAT meeting regarding the difficulty
in designing a survey that meets all of the CPI requirements. After some
discussion, the program managers decided that the CE data requirements
document is sufficient.
Therefore, contrary to previous direction to the panel that both the CPI
Requirements of the CE (William Casey, June 17, 2010) and the CE Data
Requirements (Henderson, Passero, Rogers, Ryan, Safir, May 24, 2011) col-
lectively form the requirements for the survey, the program managers ask
that the panel members treat the CE Data Requirements as the mandatory
requirements for the survey. The CPI data requirements document is still
helpful in terms of providing larger context for data usage, but these are
not requirements that the panel’s recommendations needs to meet. We hope
that this relaxation of constraints provides the panel with greater flexibility
in considering their recommended design changes.
208
OCR for page 209
APPENDIX B 209
COMMUNICATION OF ISSUES PRESENTED BY
MICHAEL HORRIGAN
BLS CHARGE TO THE CNSTAT PANEL
CE PROGRAM STAFF
FEBRUARY 3, 2011
1. mission statement:
CE
a. The mission of the Consumer Expenditure Survey program (CE)
is to collect, produce, and disseminate information that presents
a statistical picture of consumer spending for the Consumer Price
Index, government agencies, and private data users. The mission
encompasses analyzing CE data to produce socio-economic studies
of consumer spending and providing CE data users with assistance,
education, and tools for working with the data. CE supports the
mission of the Bureau of Labor Statistics, and therefore CE data
must be of consistently high statistical quality, relevant, and timely,
and it must protect respondent confidentiality.
2. The technical aspects of CNSTAT’s task from the Summary Statement
of Work in the proposal are as follows:
a. The National Research Council, through its Committee on Na-
tional Statistics, will convene an Expert Panel to contribute to the
planned redesign of the Consumer Expenditure (CE) Surveys by the
U.S. Bureau of Labor Statistics (BLS).
b. The Panel will review the output of a data users’ needs forum and
a methods workshop, both convened by BLS.
c. The Panel will conduct a household survey data producer work-
shop to ascertain the experience of leading survey organizations in
dealing with the types of challenges faced by the CE surveys.
d. The Panel will conduct a workshop on redesign options for the CE
surveys.
e. The redesign options workshop will be based on papers on design
options the Panel commissions from one or more organizations.
f. Based on the workshops and its deliberations, the Panel will pro-
duce a consensus report at the conclusion of a 24-month study with
findings and recommendations for BLS to consider in determining
the characteristics of the redesigned CE surveys.
3. What CE expects from the report:
a. The report should synthesize information gathered through the
BLS data users’ needs forum, BLS methods workshop, CNSTAT
household survey data producer workshop, CNSTAT CE redesign
OCR for page 210
210 MEASURING WHAT WE SPEND
options workshop, and independent papers into multiple compre-
hensive design recommendations.
i. The design recommendations should include a menu of com-
prehensive design options with the highest potential, not one
specific all-or-nothing design.
ii. The design recommendations should be flexible to allow for
variation in program budget, staffing resources and skills, abil-
ity of the data collection contractors to implement, legal agree-
ments to be obtained (e.g., access to other data sources), etc.
b. The report will include recommendations about future research
that needs to be done, but that is not the focus. As much as pos-
sible, the focus should be on concrete design proposals that could
be implemented.
c. should focus on a comprehensive design, and include an approxi-
It
mate timeline for development, pilot testing, and implementation.
This timeline should not exceed 5 years for development and pilot
testing, and a new survey in the field within 10 years.
d. the recommendation, the Panel should focus special attention on
In
addressing issues with the current CE surveys:
i. Underreporting of expenditures
ii. Fundamental changes in the social environment for collection
of survey data
iii. Fundamental changes in the retail environment (e.g., online
spending, automatic payments)
iv. The potential availability of large amounts of expenditure data
from a relatively small number of intermediaries such as credit
card companies
v. Declining response rates at the unit, wave and item levels
e. The Panel should develop a carefully balanced evaluation of the
prospective benefits, costs, and risks of their proposed design rec-
ommendations compared to the current CE surveys. The evaluation
should include a consideration of the following factors:
i. The evaluation be based on extensive and carefully balanced
evaluation of literature and industry knowledge on method-
ology and practice that is currently available or likely to be
available in practical form in the next five years;
ii. Data collection technologies currently available or likely to be
available in practical form with the next five years;
iii.
Administrative record and external data sources and technolo-
gies currently available or likely to be available in practical
form with the next five years; and
iv. The evaluation should be reflective of the tradeoff between cost
and improvement on measurement error.
OCR for page 211
APPENDIX B 211
4. “Yes, there are two paths you can go by, but in the long run there’s still
time to change the road you’re on.” (Jimmy Page and Robert Plant,
Stairway to Heaven)
a. is pursuing two roads to the redesign:
CE
i. redesign from scratch, and
a
ii. changes within the current design
b. The focus of the Panel should be on the redesign from scratch
(4.a.i). In doing so, BLS would like the Panel to keep the following
considerations in mind:
i. The Panel should be aware of the research that CE is undertak-
ing to improve the current design.
ii. considering a new design options, CE is particularly in-
In
terested in approaches that focus on proactive approaches to
gathering expenditure data—whether they be from records,
receipts, etc. or by providing respondents the ability to easily
record purchases in real time. While retrieval of data from
memory in a standard reactive interview is appropriate for a
number of data elements, CE views a proactive data collection
methodology for expenditure data as a high priority.
c. mentioned, CE is currently researching or is planning to research
As
a number of ideas for improving the current design, including a
Web diary, individual diaries, streamlining the Interview survey,
reducing the length of the bounding interview, double placement
of diaries, reconciliation of expenditures and income/assets, etc.
(4.a.ii).
5. Constraints:
a. Maintain same budget.
b. Maintain value of the survey to taxpayers and data users.
6. What we know:
a. needs to support CPI needs.
CE
b. needs to support other data users as much as possible as long
CE
as the design to meet those needs meets the core CE mission.
c. What makes CE unique is the complete picture of spending, in all
categories, at the household level, with household income, assets,
and demographics.
7. What we don’t know:
a. The final level of expenditure detail needed to support CPI’s needs
after redesign.
i. has a very detailed set of current technical requirements
CE
from CPI (http://www.bls.gov/cex/duf2010casey2.pdf).
OCR for page 212
212 MEASURING WHAT WE SPEND
ii. cases where CE does not provide enough detail to meet CPI’s
In
needs, CPI adopts alternative approaches.
1. For example, there are cases where the level of detail in
the CE is not sufficient for CPI, such as for gasoline, food
away from home, and medical care procedures.
2. Also, there are cases where the CE sample size is not suf-
ficient for CPI’s purposes such as in calculating Entry Level
Index selection probabilities at the PSU level or calculating
base period weights using annual calendar periods.
iii. is currently looking anew at its own data requirements and
CE
in that process will attempt to clearly state where it can and
cannot meet CPI’s needs in terms of CPI’s current detailed tech-
nical requirements. A report will be completed by the end of
April, in advance of the award of the contract for the redesign
option RFP.
iv. the redesign process develops it is critical that ongoing dia-
As
log be maintained between CE and CPI in terms of how the
redesign options would affect/change the CPI’s current detailed
technical requirements.
v. particular, CPI will need to make assessments as to the
In
efficacy of the inputs received from CE, along with possible
alternative approaches, to meet its technical requirements.
vi. BLS views this dialog as an iterative process that must accom-
pany the evaluation of redesign options.
b. What importance should we place on possible future CPI informa-
tion needs that could be provided by a redesigned CE?
i. Rob Cage’s presentation, along with the document in 7.a.i
above, will outline some possible future CPI information needs
that could be provided possibly by a redesigned CE.
ii. Please note: These possible future CPI information needs are
not requirements of the redesigned CE. CE views these future
information needs as ones to be evaluated in terms of the fol-
lowing prioritized goals:
1. Does the redesign meet the data needs of CE?
2. Does the redesign meet the current requirements of CPI, an
assessment of which includes an evaluation by CPI of the
efficacy of alternative approaches in the cases where the
redesigned CE does not meet its current technical detailed
requirements?
3. Within the framework of the redesigned CE, is there suf-
ficient flexibility, especially with respect to time and cog-
nitive burden, to collect additional data from respondents
that could meet possible future information needs of CPI?
OCR for page 213
APPENDIX B 213
c. Possible administrative data sources that could be used to replace
some of the data CE collects, or could be used to model data.
d. of the feasible technological solutions for data collection.
All
e. Data users’ reaction to collecting less than the complete picture
of spending and using more imputed/modeled data to create that
missing data.
i. That is, would they find it acceptable to collect fewer data,
either as part of a multiple matrix design, or because there are
some expenses we won’t collect, either because they are too
hard to collect (like tolls on trips) or because they are such a
small percentage of total spending (like reading materials)?
ii. Whether an approach to impute/model for a much larger
amount of missing data is feasible depends on the reaction of
data users and issues related to staffing and implementing a
much larger statistical modeling system into production.
iii. Would a split sample and data collection design be feasible—
one that is based on a smaller sample for which all expendi-
tures are collected and a larger sample that takes advantage of
matrix sampling and greatly reduces the burden of any given
interview.
8. Consensus on the design so far:
a. needs to publish a complete picture of spending, but we do not
CE
need to collect all of those data directly from respondents.
b. reduce burden and improve data quality, CE is interested in
To
moving away from a retrospective recall-based design to one that
is more proactive.
i. The current Interview design calls for collecting almost all
categories of spending from all households (the Diary is used
to collect some small frequently purchased items, food, and
clothing). For the most part, this collection is done through a
three-month recall.
ii. The proposed design should not be based on a retrospective
recall survey, but instead should focus on features that are
proactive in collecting information from respondents or other
sources. These design elements would be fundamentally dif-
ferent from those of the current CE surveys, and potentially
include innovative features such as the use of mobile devices
(e.g., smart phones, PDAs, tablets), financial software, elec-
tronic purchase records, receipt scanning, and auxiliary data.
iii. Retrospective recall may be incorporated into the proposed
design as a method of “filling in gaps” or collecting informa-
tion not otherwise provided.
OCR for page 214
214 MEASURING WHAT WE SPEND
c. The constraint of maintaining the current budget needs to be con-
sidered, particularly since moving data capture from a respondent
recall–based approach to one involving greater use of technology
and data extraction from receipts, scanners, and administrative
sources has the potential to increase collection costs.
d. The CE program produces two main data products: published
tables and microdata files. The current design is based on the idea
that two surveys are needed to get a complete picture of spending—
the Interview for large or regular purchases and the Diary for small
or difficult-to-recall items. In the CE production process, data from
the Interview and Diary are integrated at an aggregate level for
publication tables; they are not integrated for the microdata files.
(It may be possible to create synthetic households from the two
surveys at the micro-level, but the CE program has not attempted
this due to the difficulty of the project, limited resources, and the
fact that historically the microdata were viewed as secondary to
the publications.) This presents a problem for microdata users
and means that usually they will use data from one survey or the
other, but not both. With the possibility that a redesigned CE may
capture data from many sources (e.g., scanners, receipts, diaries,
recall interviews, administrative sources, etc.) this problem may
be exacerbated. It is important that the CE program continue to
make available quality microdata files to the public. These data files
may include synthetic data that account for missing data in order
to give a complete picture of spending at the household level. The
redesigned CE must allow for a straightforward integration of the
various data sources into one complete picture of spending at the
microdata level. (Note: this is a new requirement which is not met
by the current design/processing system.) As with any work that
involves imputation and synthetic data, practical implementation
of this approach will require a complex balance of multiple fac-
tors, including (a) implicit or explicit modeling assumptions; (b)
the extent to which those assumptions are consistent with the data
for specific subpopulations and expenditure types; (c) bias and
variance effects arising from (a) and (b); (d) costs and complexity
for the statistical agency; and (e) costs and complexity for the final
data user.
