Another issue is that it has become increasingly difficult to reach respondents. It is also increasingly difficult, once people are reached, to convince them to grant an interview. Finally, increasing concerns about privacy and confidentiality have exacted a toll on survey participation.

Coverage patterns in many federal household surveys are evidence that survey frames are not always adequate to reach a representative sample of the population. As Abraham noted, coverage ratios for personal visit surveys tend to be lower for black respondents than for nonblack ones; they are lower for men than women; and they vary systematically by age. Despite coverage ratios that generally trended downward from 2000 to 2008, coverage ratios for the American Community Survey (ACS) have, by contrast, been higher and more stable than those of other Census Bureau surveys. To help combat the coverage problem, the Census Bureau, in its 2010 survey redesign process, decided to use the continually updated Master Address File (MAF)—the frame the ACS uses—as the frame for its other current surveys. The use of the MAF will begin with the 2014 surveys.

Another problem creating challenges in the survey environment is the increasing difficulty of contact with survey respondents. Gated communities restrict access to respondents for in-person interviews and nonresponse followup. The use of voicemail and caller ID helps respondents avoid contact with an interviewer in telephone surveys: they can let calls go to voicemail or not answer calls from numbers they do not recognize on their caller ID display. The number of cell-phone-only households has risen sharply in the past 10 years and continues on an upward trend, thus making an initial contact through a telephone frame more difficult in the case of these households.

Obtaining respondent cooperation has become increasingly difficult. Abraham explained that increasing demands on respondents’ time, such as long commute times and increasing numbers of telephone solicitations, make respondents less likely to cooperate with an interview request. Furthermore, survey requests, such as from the federal government, compete with multiple other surveys and sales solicitations for the already limited time and interest of potential respondents. Finally, pervasive concerns about privacy and confidentiality among many in U.S. society hinder survey participation. It is not only the federal government and its data collection contractors that suffer from an increasingly unfriendly and costly climate for surveys; other survey research organizations are also encountering similar problems.

In addition to an increasing unwillingness to participate in surveys, there is also evidence of rising item nonresponse within surveys. As an example, Abraham cited a study by Bollinger and Hirsch (2006) showing that item nonresponse has increased on the Current Population Survey’s usual weekly earnings question. Increased item nonresponse is further evidenced by increasing imputation rates on questions of wages and salaries. By 2000-2004, imputation rates for weekly earnings were up to about 30 percent for survey respondents.

The National Academies of Sciences, Engineering, and Medicine
500 Fifth St. N.W. | Washington, D.C. 20001

Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement