What Information Is Needed?

Recent changes to social welfare programs have devolved substantial program responsibility to the states, ended an entitlement to cash assistance, and tied benefit receipt to work effort and other behaviors. Coupled with the on-going transformation of the health care system, the changes pose substantial new challenges to national household surveys to provide relevant data for program monitoring and assessment.

Much of the workshop discussion focused on the new or revised information needs that result from the changes. Participants agreed that current federal data are inadequate for tracking social welfare programs, but they did not agree about the scope and magnitude of the changes that are needed to surveys. Some participants anticipated that, under the new legislation, welfare programs will differ substantially across and even within states, in which case it may be difficult for national surveys to provide accurate data even for such basic measures as program participation and benefits. Other workshop participants predicted that there would be more convergence among programs as states adopt provisions that appear to work well from other states, so that it will be easier for existing surveys to provide needed data. Whatever the ultimate outcome in terms of program differences among states, national household surveys will clearly need to change to accommodate new information needs.

Survey Measures of Program Eligibility, Participation, and Benefits

Several national surveys track and evaluate the effects of participation in health and social welfare programs. Three major surveys for which the current content and design will require modification to serve these purposes adequately in the future are the March Income Supplement of the Current Population Survey (CPS), the Survey of Income and Program Participation (SIPP), and the National Health Interview Survey (NHIS). A fourth survey, the Medical Expenditure Panel Survey (MEPS), may also require modification to provide adequate data for monitoring health care programs. (Descriptions of these surveys are provided in Appendix C.)

Although participants differed in their views as to how extensively these major surveys should be revised, most of them stressed that changes to the existing surveys should "do no harm" (Carlson, 1996)—that is, to the extent possible, questionnaire content and other features should remain intact in order to facilitate time-series comparisons. Maintaining the consistency of the surveys is especially important because devolution will probably interrupt some administrative data time series.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 8
--> What Information Is Needed? Recent changes to social welfare programs have devolved substantial program responsibility to the states, ended an entitlement to cash assistance, and tied benefit receipt to work effort and other behaviors. Coupled with the on-going transformation of the health care system, the changes pose substantial new challenges to national household surveys to provide relevant data for program monitoring and assessment. Much of the workshop discussion focused on the new or revised information needs that result from the changes. Participants agreed that current federal data are inadequate for tracking social welfare programs, but they did not agree about the scope and magnitude of the changes that are needed to surveys. Some participants anticipated that, under the new legislation, welfare programs will differ substantially across and even within states, in which case it may be difficult for national surveys to provide accurate data even for such basic measures as program participation and benefits. Other workshop participants predicted that there would be more convergence among programs as states adopt provisions that appear to work well from other states, so that it will be easier for existing surveys to provide needed data. Whatever the ultimate outcome in terms of program differences among states, national household surveys will clearly need to change to accommodate new information needs. Survey Measures of Program Eligibility, Participation, and Benefits Several national surveys track and evaluate the effects of participation in health and social welfare programs. Three major surveys for which the current content and design will require modification to serve these purposes adequately in the future are the March Income Supplement of the Current Population Survey (CPS), the Survey of Income and Program Participation (SIPP), and the National Health Interview Survey (NHIS). A fourth survey, the Medical Expenditure Panel Survey (MEPS), may also require modification to provide adequate data for monitoring health care programs. (Descriptions of these surveys are provided in Appendix C.) Although participants differed in their views as to how extensively these major surveys should be revised, most of them stressed that changes to the existing surveys should "do no harm" (Carlson, 1996)—that is, to the extent possible, questionnaire content and other features should remain intact in order to facilitate time-series comparisons. Maintaining the consistency of the surveys is especially important because devolution will probably interrupt some administrative data time series.

OCR for page 8
--> Some changes to the existing surveys in order to track health and social welfare program participation are obvious. At the most basic level, questions that ask whether a respondent received AFDC are now obsolete. Other changes will be needed to obtain complete and accurate measures of program participation, benefits, and characteristics that pertain to program eligibility. Some of the kinds of information that will be required that are not covered sufficiently, if at all, in existing surveys are reviewed below (see also Adler, 1996; Carlson, 1996; Harvey, 1996; Zedlewski, 1996, especially Table 4). Welfare Program Participation And Benefits Such surveys as the March CPS income supplement and SIPP have for many years provided information about social welfare program participation and benefits. However, more extensive information will be required than in the past to provide a comprehensive picture of the more diverse programs and benefits that now exist. Information will be required on: (1) the type and amount of benefits provided, including not only regular cash payments, but also one-time cash payments, vouchers for such expenses as transportation to work and child care, wage subsidies, and other services (e.g., marriage counseling or pregnancy prevention counseling); (2) who in the household receives the benefits (e.g., only the children may receive some benefits); (3) current work activity, including unpaid jobs, education, and training; (4) duration of benefits, including duration of the current spell and other spells; and (5) whether a household's benefits are temporarily reduced because one or more adults fails to meet the legislated work requirements. Workshop participants expressed concerns about obtaining accurate reports from recipients of program participation and benefits in the new environment. Recipients of benefits may not even be the best source of information when benefits are provided, not in cash, but in such forms as wage subsidies, vouchers, or services. Indeed, they may not even be aware that they are receiving "welfare" if, for example, their wages are subsidized by TANF funds, especially if they also receive other types of assistance. Also, given the likelihood of greater variation than in the past in program names, benefit types, and other features among and even within states, it may be more difficult to keep survey questionnaires up to date with respect to such variations, which, in turn, may make it more difficult for respondents to provide accurate reports. Another concern is that survey respondents may not be able to indicate the source of funding for benefits, specifically, whether they are from a TANF block grant or from an independent state program. Since the federal government no longer provides matching funds yet does impose stringent new eligibility criteria, states may have incentives to develop independent programs. For instance, states may choose to assist adults beyond the 5-year time limit or to provide assistance under the TANF block grant to legal immigrants.7 Finally, the need to have information on the length of current and previous spells of program participation presents challenges for surveys. Accurate retrospective information on spell lengths may be difficult to obtain. A longitudinal survey, such as SIPP, can track spells that begin during the period of observation, but it faces the same problem as other surveys of obtaining accurate information on prior spells and spells already in progress when a household respondent is first interviewed. 7   To receive the entire TANF grant, a state must continue to spend at least 80 percent of its fiscal year 1994 AFDC budget. These state funds must be used either for programs associated with the block grant or to fund certain related state and local programs. How much freedom states will have to use these maintenance-of-effort funds to support programs not funded by the TANF grant is unclear.

OCR for page 8
--> Welfare Program Eligibility In addition to information about current and past participation and benefits, household surveys need to collect data to enable analysts to estimate program eligibility. Measures of program eligibility are needed to calculate program participation rates, which are important to monitor to determine if, for example, an increase in the number of participants is more likely due to an expansion in the pool of people who are eligible for benefits or to increased applications due to such factors as increased outreach by program officials. Program eligibility measures are also needed to simulate the effects of proposed changes to programs (e.g., increases in benefits that might increase participation among currently eligible people or changes in eligibility provisions themselves that might increase or decrease caseloads and costs). Finally, program eligibility measures are needed to evaluate the accuracy of reported program participation (e.g., some people who report benefits may not appear to be eligible). Even in the past, eligibility for such programs as AFDC was difficult to estimate with available survey data: for example, the data often did not clearly distinguish subsets of family members who were eligible for benefits within a household that was not eligible for benefits as a whole. In the new program environment, the data requirements for accurate estimation of eligibility are even greater. A range of information is needed, including: residency history (e.g., whether a person recently moved from another state); household living arrangements (e.g., whether a teenage parent lives at home); health and disability status; immigrant status; time spent receiving assistance; and detailed work experience, including the types of jobs and hours worked. Improved measures of health and disability status for both adults and children will be particularly important to collect for accurate simulation of eligibility for SSI and other assistance that is tied to work-based programs (see Adler, 1996). Data are also needed to permit accounting for the various exemptions to eligibility requirements, which may affect a substantial fraction of the caseload. For instance, 20 percent of the adult caseload and all child-only cases are not subject to the 5-year time limit on benefits under the TANF block grant, and some categories of immigrants are eligible for TANF assistance (e.g., refugees are eligible for assistance for at least 5 years, while lawful permanent residents are eligible with 40 qualifying quarters of work).8 Workshop participants expressed concern that devolution will complicate the measurement of program eligibility from national household surveys. Not only are there new federally mandated restrictions on program eligibility, but states may also impose added eligibility criteria, which may differ across programs and substate areas and may change over time. Child Care Because a prime goal of PRWORA is to move welfare recipients into the workforce, child care subsidies are likely to be a major component of state welfare programs. Although such surveys as SIPP have regularly collected information on child care arrangements and their costs to households, more information will be required under PRWORA. Specificially, information is needed on the sources of financing, including not only households' out-of-pocket costs, but also such sources as vouchers and direct state payments to providers. It would also be useful to have some indicators of quality of care (e.g., type of facility, family involvement, adult-child ratio). 8   The detailed information for immigrants, such as legal status or refugee status, that is necessary to determine program eligibility for TANF and other programs may be difficult to obtain in a household survey.

OCR for page 8
--> Unemployment Compensation Unemployment benefits may emerge as an important safety net program for people who previously relied on social welfare program benefits (e.g., people who lose their jobs and are not eligible for TANF because they have already received benefits for the maximum allowed period or do not meet other eligibility requirements). Household surveys have regularly collected information on receipt of unemployment insurance benefits, but additional information will be needed with which to simulate eligibility for such benefits, including a person's work activity in the year prior to the survey and characteristics of the employer. Medicaid Evaluations of the Medicaid program in an era of devolution will require more information from household surveys than has been collected. Accurate data will be needed on eligibility criteria, the type of plan for Medicaid participants (some plans may be available to nonparticipants), and plan provisions that affect costs to the recipient (e.g., copayment requirements). Currently, states are experimenting with all of these features of Medicaid, but the existing surveys do not distinguish among the many payment and plan options that are offered by state Medicaid programs. More generally, existing surveys do not provide information about the health care system in the United States, which is characterized by rapidly changing relationships between patients, providers, and insurance companies. The March CPS and SIPP have only very general questions about health insurance coverage. The NHIS asks more detailed questions about insurance plans using such traditional categories as fee-for-service, HMO, PPO, and IPA, but these categories no longer adequately describe how insurance risk is shared between provider and health plan, the degree of enrollee choice between and within plans, and other important features of health care systems. Obtaining detailed information about insurance coverage and payment plans will likely require obtaining responses from patients, providers, insurers, and employers. Such linkages are now carried out in MEPS, which draws its sample from NHIS, but MEPS has a relatively small sample size. Outcome Measures in Surveys The need to develop a more comprehensive set of outcome and activity measures to evaluate the impact of recent changes to health and social welfare programs emerged as a common theme throughout the workshop. Not only is eligibility now linked to certain activities (e.g., work), but programs have much broader objectives than in the past. For instance, PRWORA specifically encourages states to reduce non-marital teenage pregnancies. Thus, in addition to standard indicators of income, poverty, and labor force attachment, a number of other outcome measures may be desired from national household surveys, such as measures of teenage out-of-wedlock births, detailed labor force participation and work activities, food and shelter insecurity, overall consumption levels, and mental and physical health status.9 Several workshop participants noted that existing surveys include many outcome measures, but others will need to be developed, particularly measures that indicate changes in the well-being of 9   Some measures may need to be obtained by analyzing other household surveys than the four discussed in this report. For example, the Consumer Expenditure Survey provides detailed measures of household consumption, although it currently has a small sample size (5,000 consumer units) that limits analysis for population groups and geographic areas.

OCR for page 8
--> children. The new legislation emphasizes desired changes in the behavior of adults, but a very important question is the effects on children. In that regard, one participant spoke of the need for "canaries," that is, outcome measures that can provide early warning signals of problems for children (e.g., children in very poor families in deep poverty), and "crocuses," that is, outcome measures that signal positive trends (e.g., children in families that are no longer very poor). Workshop participants also noted the need to measure outcomes for both children and adults in families who would have been eligible for benefits under the old program rules but are no longer eligible. Issues of concern include whether previously eligible families will be able to maintain their own households and how they are affected when there is an economic recession. Finally, some participants stressed the importance of improved measures of health and disability status for a range of purposes, such as monitoring the effects of changes to SSI program eligibility, the transformation of public assistance into a work-based program, and, more generally, the aging of the population (see Adler, 1996). Program Rules Almost all participants noted the importance of collecting detailed and consistent information on program eligibility criteria and benefit provisions from the states. Given the devolution of responsibility for health and social welfare programs from the federal to state and local governments, program rules with regard to eligibility and benefits will almost certainly differ across states. Program rules may also differ within states, and they may change over time. In the absence of reliable and timely information on these rules, it will not be possible to use national household survey data to accurately simulate program eligibility, calculate participation rates among those who are eligible, or evaluate the accuracy of reporting of participation and benefits. Also, without regularly updated, reliable information about state (and local) program rules, it will not be possible to develop survey questionnaires that ask appropriate questions for respondents in different parts of the country. Thus, a comprehensive national database that provides detailed, consistent, and timely information about how state and county programs are intended to work needs to be developed (see Zedlewski, 1996). A number of organizations, including the Congressional Research Service, the Urban Institute, and the American Public Welfare Association, are attempting to monitor changes to state welfare program rules; see Table 1. However, these efforts are generally limited in coverage of states or programs (or both). Workshop participants stressed the need for a central source for collection and dissemination of information about program rules. Administrative Data On Program Recipients In addition to discussing changes needed to national household surveys, workshop participants discussed the value of administrative data on current program beneficiaries. Historically, such data have been useful for tracking program participation, simulating the likely effects of program changes that decrease benefits or restrict eligibility, and validating survey-based reports of program participation and benefits. However, administrative data cannot be used to simulate program changes that increase benefits or expand eligibility because they contain no information on nonparticipants. Also, their use for behavioral research is limited because they do not cover people who stop receiving benefits, have limited information for current participants, and, as just noted, have no information for nonparticipants. With PRWORA now in place, administrative data tracking the AFDC system will be eliminated and replaced with a system that tracks participation in state and local programs funded by the

OCR for page 8
--> TABLE 1 Ongoing Efforts to Monitor and Track State Welfare Program Rules Group Topic(s) Methods/Notes Congressional Research Service (CRS), U.S. Congress TANF Will conduct state (mail back) surveys to document a key set of program rules. Center for Law and Social Policy (CLASP) and Center for Budget and Policy Priorities (CBPP), Washington, D.C. TANF; Food stamps; Medicaid; Employment related to TANF Set up a network of legal services groups to report program rules for all 50 states; verification of prepublished summaries by states. Funded by the Annie E. Casey Foundation. Office of the Assistant Secretary for Planning and Evaluation, U.S. Department of Health and Human Services (ASPE/DHHS) TANF Current focus is summarization of state submitted TANF plans and waivers. Health Systems Research, Washington, D.C. Food stamp waivers Telephone survey of states during the fall of 1997. Up to 15 states will be visited in person to understand implementation issues. Funded by the Food and Nutrition Service, U.S. Department of Agriculture (USDA). Mathematica Policy Research, Inc., Princeton, N.J. Simplified food stamp plans Assessment of cost neutrality for states choosing this option. Funded by the Food and Nutrition Service, USDA. Institute for Research on Poverty, University of Wisconsin TANF, local area variants Development of a prototype project to characterize local area variation in welfare programs that could be linked to SIPP at the primary sampling unit (PSU) level. Funded by the Census Bureau. American Public Welfare Association (APWA), Washington, D.C. TANF Simplified food stamp plans Monthly surveys of states on different topics related to reforms. Urban Institute, Washington, D.C. TANF Food stamps Child care Workforce development Medicaid General Assistance Understanding policies for 13 states in depth through site visits, local collaborators. Information for remaining states (in less detail) will be collected through available published documents from the states, APWA, CRS, and CLASP, and augmented through annual telephone surveys to document implementation and special issues. Some programs will be covered through biannual telephone surveys. Funded by Annie E. Casey, Kellogg, Kaiser, and other foundations.   SOURCE: Zedlewski (1996:Table 3); information as of December 1996. Used by permission of the author.

OCR for page 8
--> TANF grant. Thus, time series of administrative data for AFDC will end. Still, administrative data may be useful to evaluate and track caseloads in each state. In fact, PRWORA requires states to file quarterly reports detailing demographic information, employment status, earnings, and forms of assistance provided for a sample of TANF recipients. However, given that current federal regulations provide no matching money for the development of the quarterly reports (the federal government previously shared in the costs of program administration and reporting) and the lack of a quality review process, workshop participants raised questions about how reliable those data will be. Workshop participants noted that administrative and quality control data on the Food Stamp Program may prove useful for some kinds of analyses of welfare program trends and outcomes. For many food stamp recipients, a time series of administrative data will continue. The usefulness of administrative data for evaluating health care services, costs, and quality has been affected by changes in the health care industry, particularly the rise in managed care. Traditionally, public health insurance programs have used fee-for-service payment plans that create a paper trail linking payments to services that can be analyzed, and claims data for the Medicaid and Medicare programs have been used to evaluate health care services funded by these programs. However, the growing use of managed care plans, which often charge a flat annual payment for services, means that there are no claims data documenting services and costs for increasing numbers of beneficiaries (see Harvey, 1996).