The Workshop on the Future of Federal Household Surveys was designed to address the increasing concern among many members of the federal statistical system that federal household data collections in their current form are unsustainable. The workshop was held at the request of the U.S. Census Bureau. Other statistical agencies that helped sponsor the workshop through the core grant to the Committee on National Statistics from the National Science Foundation’s Methodology, Measurement, and Statistics Program include the Bureau of Justice Statistics, the Bureau of Labor Statistics, the Bureau of Transportation Statistics, the National Center for Education Statistics, the National Center for Health Statistics, the National Center for Science and Engineering Statistics, and the Social Security Administration.
Although no consensus recommendations were generated, the workshop was structured to bring together leaders in the statistical community and to facilitate a discussion of opportunities for enhancing the relevance, quality, and cost-effectiveness of household surveys sponsored by the federal statistical system.
Federal household surveys today face significant challenges: (1) increasing costs of data collection, (2) declining response rates, (3) perceptions of increasing response burden, (4) inadequate timeliness of estimates, (5) discrepant estimates of key indicators, (6) inefficient and considerable duplication of some survey content, and (7) instances of gaps in needed research and analysis (e.g., lack of information on institutional populations). The more recent American Community Survey (ACS) can possibly be leveraged to help cope with these
challenges, and the workshop considered options for better integrating it into the federal household survey portfolio.
Although moving forward with a coordinated strategy will require many more conversations on the issues covered at the workshop, if solutions are not developed in a comprehensive and timely manner, the challenges put at risk the ability of the federal statistical system to provide important policy-relevant information. The goal of the workshop was to begin and to facilitate the much-needed discussion on solutions that range from methodological approaches, such as the use of administrative data, to emphasis on interagency cooperative efforts.
The goal of and charge to the steering committee were to develop a workshop aimed at enhancing the household surveys sponsored by the federal statistical system. As part of his welcoming remarks, the steering committee chair, Hal Stern (University of California, Irvine) noted three guidelines for participants to keep in mind. First, the workshop was to provide a picture of the system as it is, including an overview of the many current challenges. And although such issues as nonresponse and increasing cost are of great interest, these challenges would be used to set the context for discussion rather than being the focus of discussion themselves, he said, because a number of other recent meetings have focused on these topics extensively.
Second, an important cross-cutting issue was how a large continuous survey, such as the ACS, could be useful to the household survey system. The questions were what could be done with a survey like this and how could it best be used. This issue came with a caution, however, not to get bogged down in the details at this stage of the conversation.
The final caution made by Stern was to avoid the trap of focusing on what cannot be done, which would be the wrong kind of discussion for this workshop. Instead, he emphasized that workshop participants should keep open minds and consider where innovation and experimentation might happen. He said that this was not just a presentation workshop; it was meant to inspire and encourage participation from those present.
Stern said that this point was worth reinforcing: this workshop was intended to be about ideas. It is ever more critical that the statistical community consider ways to make the household survey system better and more efficient. In that spirit, he encouraged the participants to consider some challenging questions. Is the model of data collections centered around individual surveys outdated? How can new data collection modes and analysis techniques be integrated most efficiently? Can the resources invested in maintaining and updating address files be streamlined and perhaps directed toward developing a universal address file?
The workshop began with a look at the U.S. household survey system and where it stands, followed by overviews of household survey systems from several other countries: the United Kingdom, the Netherlands, and Canada. These countries are facing many of the same issues as the United States. Although what works in one country may not work in another, it is important not to rule any ideas out in the course of these discussions.
The workshop then focused on topic areas in which promising research is being done and there is also room for additional discussion and perhaps some experimentation. One of these topics is sampling frames: Can large surveys serve as first-phase samples for smaller surveys? Can the statistical community work together to make the development and maintenance of sampling frames more efficient? There was also a general discussion of methodology—for example, modes of data collection and the use of administrative records.
The agenda then shifted to a discussion of estimation challenges and the boundaries between direct estimation and model-based small-area estimation. This was followed by a discussion of survey content, particularly instances of multiple measures of the same concept, when this is desirable, when it is not, and what can be done about it. This session included thoughts on the potential future role of the ACS and of the U.S. Office of Management and Budget.
This summary of a workshop is intended to describe the presentations of the workshop and the discussions that followed each session topic, as outlined in the agenda (see appendix). Following this introduction, Chapter 2 represents the first session of the workshop with an overview of the U.S. federal household survey system at a crossroads. It also presents models of household surveys in other countries in contrast to those in the United States. Chapter 3 covers the session on sampling frames and new ideas on how to use them. Chapter 4 addresses various methods of collection of household data, including the use of administrative records. Chapter 5 summarizes the discussions that took place at the end of the first day’s presentations. Chapter 6 covers the topic of small-area estimation, how this methodology is used now, and other ways that it might be used in federal surveys. Chapter 7 focuses on survey content, discussing standardized measures of the same concept used across different surveys (e.g., disability) and instances when the use of different measures is more appropriate (e.g., poverty or income). The chapter also addresses the topic of official statistics. Finally, Chapter 8 summarizes the floor discussion that took place at the workshop’s close.
It is important to note that the nature of this report is that of a factual summary of the presentations and related discussions that transpired during the workshop. Therefore, all views presented herein are those solely of the
workshop participants. The presentation topics and content reflect the areas of expertise of the presenters and are not intended to be an exhaustive discourse on the future of federal household surveys. Furthermore, this workshop was not designed to produce either conclusions or policy recommendations. Rather, the intent of the workshop was to open a dialogue on the subject, encourage further research, and share new ideas about improving the system of household surveys.