Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
IMPLICATIONS FOR DATA COLLECTORS 18 QUESTIONNAIRE DESIGN AND ADMINISTRATION Workshop participants underscored the need for federal statistical agencies to conduct methodological research on such issues as the appropriate design of questionnaires and wording of question content to elicit accurate survey reports of participation and benefits in an era of welfare program devolution and experimentation. Cognitive research and such techniques as focus group analysis will likely be needed to develop questionnaires that respondents can understand and respond to appropriately (e.g., indicate their participation in a welfare program when benefits take such forms as wage or day care subsidies and not cash). Another topic for research is that of optimum questionnaire length, balancing data needs against survey costs and burden on respondents. The need for more information about children, both to accurately characterize program eligibility (especially for child-only cases) and to provide adequate outcome measures for children, raises issues of questionnaire length because most household surveys do not now obtain as much information about children as about adults. This need also raises issues of how to obtain accurate information for children who do not report for themselves. Finally, research will be needed to develop appropriate training methods and materials for interviewers so that they are able to elicit accurate responses to the extent possible. Coordination among federal statistical agencies for all of these kinds of methodological research would be beneficial so that research can be conducted in a cost-effective manner and research findings can inform improvements in all of the major national household surveys. DATA VALIDATION Monitoring and assessment of health and social welfare programs in the new environment will require accurate measures of program participation and benefits, of characteristics that are needed to estimate program eligibility, and of outcome variables. Maintaining current levels of accuracy, let alone improving on them, will be difficult given the likelihood of increased variation in program provisions across states and localities and over time.13 Survey response rates may also fall if questionnaires are lengthened to accommodate new data needs. Validation of survey responses will be essential, not only to establish the quality of the data collected, but also to suggest changes to questionnaires and survey procedures that could improve data quality. Workshop participants suggested a number of specific methods for validating survey questionnaires, including matches of survey and administrative data for cross-validation, cognitive research designed to ensure that respondents understand survey inquiries, and experiments embedded in surveys to test the sensitivity of results to different ways of asking questions about new program features. Workshop participants also suggested collecting information about programs from respondents, as a validation technique. Thus, inquiries about program names, eligibility criteria, and benefit levels may not only measure awareness, but also help to establish the accuracy of responses. Participants noted that proper training and informing of interviewers about the details of state and local programs may reduce respondent confusion and reporting errors. 13 Not only are program provisions likely to vary across jurisdictions, but caseworkers may have greater flexibility to tailor program provisions to the circumstances of individual participants (e.g., in determining what activities meet the work requirements).
