In 2010, six of the eight largest categories of expenditures available in both the Interview and Diary surveys show greater underreporting in the Diary survey (two other large categories of expenditures are not collected in the Diary survey). The weighted average of Diary totals to national income account totals for categories that line up with the national accounts is 17 percentage points lower than the ratio for the Interview survey, though this partly reflects the different coverage of the two surveys.
For the 37 categories that can be compared in the Interview and Diary, the weighted average of the coefficient of variation is 58 percent higher in the Diary, with 33 of the 37 categories subject to greater variation even after accounting for differences in sample size (Bee, Meyer, and Sullivan, 2012). The higher variability of responses from the Diary survey mean that 2.5 diary responses are needed to equal the precision of one interview response, an issue that is not reflected in any of the cost accounting in the report.
Response rate differences between the surveys and the frequency of reports of no expenditures are also consistent with higher Interview survey data quality. The recent experience of Canada is instructive: a switch from interview to diary led to 9–14 percent more under-reporting on average (Dubreuil et al., 2011). This evidence is mentioned in the report, but is not deemed to have important implications for a redesign. Additional evidence of problems with diary surveys can be found in Crossley and Winter (2012). While these findings refer to existing surveys, they raise important concerns about the collection of expenditure data that relies heavily on diary methods.
The suggested redesigns, of course, aim to improve upon existing methods, although there is not a great deal of evidence to support that the effects will be dramatic. For example, monetary incentives for respondents are emphasized, but the evidence on such incentives from randomized trials suggests small effects that are even smaller for high-income households (e.g., Clark and Mack, 2009). The differential effects by income are particularly important since one of the most significant problems with the CE is lower response and under-reporting among high-income households (Sabelhaus et al., 2012).