Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
IMPLICATIONS FOR DATA COLLECTORS 19 FLEXIBILITY Flexibility in designing and implementing survey questions emerged as an important issue in the workshop discussions. Since states may now experiment with different programs with different names, participants suggested that the need to make last-minute modifications to survey questionnaires may be common. Surveys that ask respondents whether they participated in program X will be useless if the program is now called Y. Although computer-aided surveying techniques, such as computer-assisted personal interviewing (CAPI) and computer-assisted telephone interviewing (CATI), have provided such benefits as reducing inconsistent responses (inconsistencies can be resolved at the time of the interview instead of in a subsequent editing process), statistical agencies have not found it easy to modify CAPI/CATI questionnaires in a timely manner. One suggested short-term solution is to supplement the computer-aided survey with a small sample that is interviewed with a paper questionnaire that can be modified as needed. SAMPLE DESIGN The need for statistical agencies to carefully evaluate appropriate sampling schemes for the major national household surveys in light of increased data needs for program analysis was a recurring theme of the workshop. Participants discussed the possible benefits of expanding survey sample sizes to allow for subnational analyses and of developing new stratification schemes to overrepresent certain vulnerable populations. Taking account of attrition in longitudinal surveys and, if necessary, developing refreshment samples were also stressed as important sampling design considerations. The existing federal household surveys arc designed for national and sometimes regional level analyses, but they are generally unable to provide reliable statistics on specific subnational areas. Given program devolution, the ability to track and monitor outcomes by stateâand even by countyâmay become increasingly important. Thus, one suggested modification to sampling schemes is to supplement the existing samples so that the surveys are representative of individual states. None of the major surveys can now be used to provide reliable estimates for each state, although the CPS sample design is representative of all states and is designed to provide reliable estimates for the largest states. (The NHIS sample design is also state representative.)14 Several workshop participants also suggested the value of modifying the existing sampling schemes to overrepresent vulnerable populations. PRWORA imposes strict new eligibility requirements that affect specific groups of people, including disabled children, legal immigrants, unemployed people, and teenage parents. Often, these groups are small segments of both the total population and most national samples of the population, so that the current national surveys are often unable to provide reliable estimates for them,15 yet such groups may be the ones that are most affected by the new legislation. Redesigning national household survey samples to provide more reliable data for analysis of states or small population groups could be very costly. Workshop participants stressed the need to search for sampling schemes that balance costs and benefits. For example, one possibility is to supplement national samples each year with added sample for selected states, on a rotating basis. 14See Appendix C for additional details on the sampling schemes used in CPS, SIPP, NHIS, and MEPS. 15 The existing surveys are not only unable to provide precise estimates for these groups, but they are also often unable to even identify some of these subgroups, such as immigrants and disabled children (see Appendix C).
IMPLICATIONS FOR DATA COLLECTORS 20 Also, it may be possible to identify groups of states with sufficiently similar programs to permit pooling their samples, thereby reducing the added sample size that might otherwise be required. Workshop participants expressed a wide variety of views in the extensive discussion about modifying the sampling schemes for the major national household surveys. Some participants argued that the sample designs should be completely revised, others argued to make no changes, and still others suggested some middle ground. Perhaps the most consistent theme was the need for a more careful examination of these issues in light of long- term data requirements and existing budget constraints. In addition, participants stressed the need to carefully consider the statistical issues involved in supplementing existing samples while maintaining a nationally representative design.
