Nonresponse in Social Science Surveys A Research Agenda (2013) / Chapter Skim
Currently Skimming:
4 Approaches to Improving Survey Response
4 Approaches to Improving Survey Response
Pages 61-100
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 61... ...
Approaches to improving survey response must take these factors into account. One possibility suggested by researchers is that the decline in response rates reflects a corresponding increase in the overall level of burden that surveys place on sample populations.
|
From page 62... ...
We then discuss several approaches that are being taken or have been proposed to increase survey response rates. The first group of approaches involve sampling procedures -- respondent-driven sampling (RDS)
|
From page 63... ...
Progress in understanding burden and its impact on survey response must begin with an analysis of the concept, its dimensions, and how it is operationalized. Unfortunately, there is little research to show conclusively that there is a causal relationship between measured burden and propensity to respond.
|
From page 64... ...
Bogen reviewed both non-experimental and experimental studies that were available in the mid-1990s. She concluded that "the non-experimental literature paints a picture about the relationship between interview length and response rates that is not uniform" (p.
|
From page 65... ...
task force on including cell phones in telephone surveys (American Association for Public Opinion Research, 2010a)
|
From page 66... ...
. Mail Surveys Low response rates have long been considered a major problem for mail surveys, so much so that much of the early research on improving response rates focused on mail surveys.
|
From page 67... ...
, in which the basic elements of survey design and implementation are shaped further for particular populations, sponsorship, and content. Despite these advances in understanding the determinants of high response rates in mail surveys, which are grounded in research covering more than a quarter of a century, the challenges continue.
|
From page 68... ...
They also coined the term "address-based sampling." Link and his colleagues (2008) compared mail surveys based on ABS with telephone surveys based on RDD using the BRFSS questionnaire in six low-response rate states (California, Illinois, New Jersey, North Carolina, Texas, and Washington)
|
From page 69... ...
In conjunction with mail data collection, it appears to produce higher response rates than telephone interviewing and RDD sampling produce. However, it has been pointed out that when eligibility rates fall below a certain point, it is no longer cost-effective (Amaya and Ward, 2011)
|
From page 70... ...
Subsequently, recruitment of both blacks and Hispanics was more successful, and the reported network converged with the composition of the recruited sample. Comparing self-reported network composition and peer recruitment patterns provided a qualitative measure of representativeness even though it could not be expressed in a traditional response rate.1 Another approach is to ask those who are responsible for recruiting respondents about those who refused to be recruited.
|
From page 71... ...
Matrix Sampling While the goal of RDS is to identify and maximize responses from a hard-to-reach population at a reasonable cost, the goal of matrix sampling is to reduce any particular respondent's burden and thereby improve survey response rates. Matrix sampling is a procedure in which a questionnaire is split into sections of questions, and each section is then administered to subsamples of the main sample.
|
From page 72... ...
. To the extent that this advantage holds, survey administrators should be able to use matrix sampling to achieve higher response rates with lower costs.
|
From page 73... ...
Cell Phone Surveys The explosive growth in cell phone usage has created challenges for survey managers even as it has opened new possibilities for survey operations. A recent AAPOR task force report on cell phone survey techniques (American Association for Public Opinion Research, 2010a)
|
From page 74... ...
Internet Panel Surveys Many survey researchers see increased use of the Web as the key to controlling escalating data collection costs in surveys. In the committee's workshop, Reg Baker, chair of the AAPOR panel on online surveys, summarized the results of the AAPOR panel's study of these surveys (American Association for Public Opinion Research, 2010b)
|
From page 75... ...
The mode that is selected for the self-administered questionnaire makes a difference in eliciting survey responses.
|
From page 76... ...
The thinking is that if surveys that rely on a single mode have unacceptably low response rates, then combining modes may take advantage of different modes to increase response rates and potentially reduce nonresponse bias. In a presentation to the panel, Mick Couper (2011a)
|
From page 77... ...
The mail-only groups responded at higher rates than the Web panels, but both achieved higher response rates than might be expected with only an RDD telephone survey. Yet despite this and other research on Web-mail surveys, Messer and Dillman conclude that it "remains unclear as to what procedures are most effective in using the DSF with mail and the Internet survey modes to obtain acceptable levels of non-response" (2010, p.
|
From page 78... ...
American Community Survey: A Sequential Mixed-Mode Case Study4 The most ambitious use of a mixed-mode approach to improve survey response rates is the approach in the American Community Survey (ACS)
|
From page 79... ...
For example, sample members from households in less economically advantaged areas and ethnic enclaves are less likely to respond to the mail surveys than are other households. ACS 2006 data also showed that individuals not in the labor force were more likely than those who were employed to respond to the mail mode, while those with no high school education had low response to the initial mail questionnaire and were more likely to participate by telephone or personal interview.
|
From page 80... ...
These mode changes do not seem to have affected response rates, likely because respondents in longitudinal studies have already made a commitment to the survey and have already had some experience with the interview process. (Refer back to Tables 1-6 through 1-9 in Chapter 1 for response rate information for the NLSY79, NLSY97, PSID, and HRS.)
|
From page 81... ...
make Web instruments more complex and demanding. Needed Research As more researchers turn to mixed-mode designs in an effort to maintain response rates, it is increasingly important to conduct research on mode effects, not only on response rates but also on measurement errors.
|
From page 82... ...
. In addition, some of what appears to be interviewer variance in survey responses may be due to the effects of the interviewer on participation rather than on measurement (West and Olson, 2010)
|
From page 83... ...
. Although it is clear from this discussion that interviewer and sample person characteristics do play a role in survey response, it is not apparent that matching those characteristics necessarily results in improved response rates.
|
From page 84... ...
Therefore, the available literature does not offer a clear picture of the mechanism connecting an interviewer's unobservable characteristics and the survey participation that he or she achieves. Experience, however, is important.
|
From page 85... ...
The training improved the response rates of interviewers, especially for those who had lower response rates before the training. Relatedly, Dijkstra and Smit (2002)
|
From page 86... ...
The medium version included the elements of the short version and the reason for calling. The long version included elements that in theory and sometimes in research increase response rates, such as (a)
|
From page 87... ...
. However, the response propensity that the sample member brings to the contact with the interviewer might be modified over the course of the encounter and may affect the leverage that a feature of the survey design has with a respondent.
|
From page 88... ...
Where possible, experiments should be done to identify the most effective techniques. Concluding Remarks on the Role of Interviewers In summary, interviewers play a valuable role in obtaining survey responses.
|
From page 89... ...
All theories of survey response emphasize the role of incentives in motivating behavior, though these need not be monetary incentives. Singer noted that results from responses to open-ended questions suggest that there are three main reasons for responding to surveys: altruistic reasons (e.g., wanting to be helpful)
|
From page 90... ...
The message had no effect on either screener or survey response rates, and there were no interaction effects. Incentives in Longitudinal Studies Longitudinal surveys have special issues, because incentives are usually part of a larger motivational package designed to retain respondents.
|
From page 91... ...
(2007) studied the results of the incentive experiments and concluded that incentives moderately increased response rates and had a greater impact on those respondents who did not participate in the previous round relative to those who did participate.
|
From page 92... ...
The exception is Jaeckle and Lynn (2008) , who found that incentives increased item nonresponse.
|
From page 93... ...
• Monetary incentives increase response rates more than gifts do, and prepaid incentives increase them more than promised incentives or lotteries do. • There is no good evidence for how large an incentive should be.
|
From page 94... ...
. As discussed in Chapter 3, paradata are used for many purposes: to monitor the status of field collection, to confirm that fieldwork has been carried out according to instructions, to compute response rates, to identify reasons for nonresponse, to implement responsive design strategies, and to adjust for nonresponse bias.
|
From page 95... ...
. The power of employing paradata and auxiliary data for improving response rates is now becoming recognized.
|
From page 96... ...
1) , responsive designs "pre-identify a set of design features potentially affecting costs and errors of survey statistics, identify a set of indicators of the cost and error properties of those features, monitor those indicators in initial phases of data collection, alter the active features of the survey in subsequent phases based on cost/error tradeoff decision rules, and combine data from separate design phases into a single estimator." Responsive design is a flexible menu of design approaches that can be employed in real time to ameliorate the damage caused by reduced response rates to surveys.
|
From page 97... ...
Although administrative records hold promise for helping to improve survey operations, in Czajka's judgment, it is unlikely that they can be substituted for survey reports. Reasons include that the set of survey items for which there is a high quality administrative records alternative is small and largely limited to federal records; the concepts underlying administrative records may differ from survey concepts (e.g., tax versus survey income)
|
From page 98... ...
sur vey data collections. OTHER MEANS OF COLLECTING SOCIAL SCIENCE DATA The problems with obtaining cooperation with social science surveys do not mean that probability-based sampling should be abandoned.
|
From page 99... ...
The BPP is said to have closely tracked the Consumer Price Index. • Google Price Index.
|
From page 100... ...
official survey statistics.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.