randomly administered incentives increase bias, but not enough is known about the effect to use them effectively to reduce bias.
Whether incentives have an effect on Internet surveys is still unknown. Singer (2011) pointed out that research in this area is limited. Much of the published experimental research has been done by Göritz (2006), who finds that incentives increase the proportion of invitees starting a survey and the proportion completing it over a no-incentive group. Lotteries are the incentives most often used. Her literature review concluded that specific tests of lotteries against other types of incentives or against no incentives show that lotteries are no more effective in Web surveys than in other kinds of surveys. In most tests, lotteries did not significantly increase response rates over a no-incentive or alternative incentive group.
Incentives are sometimes differential: different amounts are offered primarily to convert refusals, often on the basis that differential incentives are more economical than prepaid incentives and that they are more effective in reducing bias. But there is also a question of fairness; many sample members are likely to consider differential incentives to be unfair. Nonetheless, Singer et al. (1999) found that respondents would be willing to participate in a new survey by the same organization even when told that differential incentives would be paid. When differential incentives are used to reduce bias, they are commonly paired with a small prepaid incentive to all possible participants, which serves to increase the sample size and helps to satisfy the fairness criterion.
Whether or not there are long-term incentive effects is not yet known. Singer (2011) said that there is no evidence of long-term effects thus far, but studies have been done only over short intervals.
Conclusions on Incentives
Singer (2011) drew the following conclusions regarding incentives:
• Incentives increase response rates to surveys in all modes and in cross-sectional as well as panel studies.
• Monetary incentives increase response rates more than gifts do, and prepaid incentives increase them more than promised incentives or lotteries do.
• There is no good evidence for how large an incentive should be. In general, although response rates increase as the size of the incentive increases, they do so at a declining rate. Also, there may be ceiling effects to the extent that people come to expect incentives in all surveys.
• Few studies have evaluated the effect of incentives on the quality of response; most of these have found no effects.