National Academies Press: OpenBook

Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report (1999)

Chapter: 3 ASPE Leaver Studies and Other Current Research on Welfare Reform

« Previous: 2 Framework, Principles, and Designs for Evaluation
Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×

3
ASPE Leaver Studies and Other Current Research on Welfare Reform

Part of the charge to the panel is to assess the designs of 14 state-level 1 welfare ''leaver'' studies funded by the Office of the Assistant Secretary for Planning and Evaluation (ASPE) in the Department of Health and Human Services (DHHS). This chapter reports such an assessment. These studies are designed to track those who leave welfare programs, through the development of administrative data, survey data, and linkages of these two. The first part of the chapter briefly summarizes the proposed study plans. The second part discusses some conceptual and technical issues raised by the proposed studies, including issues raised during a workshop held with researchers and administrators from the states and counties. Here we also use the principles of good evaluation design reviewed in Chapter 2 to draw lessons for leaver studies.

The final section of the chapter goes briefly beyond these leaver studies. A large number of leaver studies around the country have been completed already, and in the third part of this chapter we summarize some of these other leaver studies. In addition, DHHS is sponsoring other major welfare reform evaluation studies, and many other government agencies, private institutes and researchers are funding and conducting such studies as well. The fourth part of this chapter briefly reviews some of the other major studies evaluating the effects of welfare reform.

1  

 These 14 studies include two counties and one group of counties (see below), but, for simplicity, we often refer to all of them as state-level studies throughout the report.

Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×

ASPE-FUNDED LEAVER STUDIES

In an effort to provide support to states who are conducting studies of families who leave welfare programs, ASPE in May 1998 invited states to submit proposals to study the outcomes of at least one of three groups: those individuals and families who once participated in TANF but who stopped (leavers), those who applied for TANF but were diverted from participating, or those who appeared to be eligible for TANF but did not apply to receive it. In September 1998, 14 jurisdictions—10 states, the District of Columbia, and 3 counties or groups of counties—were awarded a total of $2.9 million to conduct studies.2 The average grant award was just over $250,000; the studies will last from 1 to 1 1/2 years. All grantees must produce interim reports in the first year of the grant period and final reports at the end of the grant period. The grantees are: Arizona, the District of Columbia, Florida, Georgia, Illinois, Massachusetts, Missouri, New York, South Carolina, Washington, Wisconsin, Cuyahoga County, Ohio (Cleveland), Los Angeles County, and a group of counties in Northern California (Santa Clara, Santa Cruz, and San Mateo counties—the San Mateo county group).3Table A-1 in Appendix A summarizes key study components for the 14 jurisdictions. As that table shows, and as we discuss further below, most grantees plan to study the first group mentioned by ASPE—leavers.

At this writing, not all state grantees' interim reports were finalized. Our summary and review of the issues in these studies are therefore based on two sources: the proposals and, when possible, the finalized interim reports submitted by the grantees, and a workshop held with the grantees and the panel in November 1998. This summary discussion of the state plans considers the outcomes proposed for study, the study populations, the proposed methodologies, and the data sources, in turn.

Outcomes

ASPE gave the grantees much discretion in choosing the outcomes to be studied, except that one requirement of all grantees was to study employment and earnings outcomes. Table A-2 in Appendix A summarizes the outcomes studied by the grantees and the sources of data for the outcomes. Employment outcomes

2  

 Additional funding for these studies was provided by the Department of Agriculture, the Department of Labor, and the Administration for Children and Families in the Department of Health and Human Services. The award to one of the states, South Carolina, was made under a previous grant announcement.

3  

 ASPE issued a second Request for Applications in spring 1999 but the grant recipients have not yet been announced. This request called for studies of the status of applicants and potential applicants to TANF and families entering TANF, in addition to further studies of leavers, including, perhaps, extensions of some of the 14 studies.

Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×

that states and counties proposed to study include: employment status, hourly wage, hours worked, quarterly earnings, attachment to the labor force, types of jobs held, receipt of fringe benefits by those who are employed, and receipt of education and training. Other adult and family outcomes the studies plan to investigate (not all studies plan to measure all of these outcomes, but rather combinations of some of these outcomes) include: economic well-being (overall economic status, food security, housing security, child support receipt, other sources of support such as support from emergency shelters or support from other family members, and health insurance coverage); program participation (reason for leaving TANF, recidivism, reason for returning to TANF, and other assistance program participation such as food stamps or public housing); family structure, family formation and family functioning; barriers to self-sufficiency (lack of child care, mental health barriers, drug usage or illiteracy); and, finally, attitudes and awareness of public assistance programs. Child outcomes are also of interest to the grantees, including: availability, usage, and quality of child care; health; school attendance; behavior patterns; living arrangements; and abuse and neglect.

Study Populations

The major population of interest to the grantees are families who once received TANF but have stopped receiving it because their earnings were too high, they were sanctioned for not complying with requirements, they reached the end of their time limit, they voluntarily withdrew from the program, their children reached the age limit, they thought they were no longer eligible for the program, or for any other reason. This is the so-called leavers population. What it means to leave TANF varies across jurisdictions, but the majority of states count families who do not receive cash assistance for at least 2 consecutive months as leavers. For all states, to be eligible to leave TANF, cases only had to be open for 1 month. The 2-month criterion was used to avoid administrative "churning" and to avoid the inclusion of cases that were erroneously classified as having left for 1 month, which would presumably be correctly reported after 2 months.

Within the group defined as leavers, further distinctions are made. First, some states have partial case sanctions, which often means that a client's benefit is reduced (usually for failing to meet work requirements). In some states these benefit reductions are small. In others, the reductions fully sanction the adult(s) in the case so that the remaining grant theoretically covers only the client's child(ren). Another distinction is made between these "partially sanctioned" child-only cases and other child-only cases, which are often cases in which the child lives with foster parents or grandparents or has immigrant parents who may not be eligible for assistance. The adults in the other child-only cases are not subject to work requirements; in some states, the adult in the partially sanctioned child-only case is still subject to work requirements and may face a more severe

Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×

benefit reduction for continued noncompliance.4 In the 14 leaver studies, five states have proposed to count the partially sanctioned child-only cases as leavers (Georgia, Illinois, Missouri, New York, and the San Mateo County group5). Only Georgia and New York have decided to include all types of child-only closed cases (child-only for any reason, including foster care or immigrant status) as leavers. New York will follow this type of child-only case with administrative data only.

The unit of observation in all of the state and county proposals is the closed case, including children and other family members who are living in the same household as the client at the time of closure. For child outcomes, some states have proposed tracking the outcomes of all children in a recipient's family, while other states plan to follow one randomly selected child per family.

Several of the grantees also proposed to study families who were diverted from enrolling in the TANF program (or its predecessor, AFDC) through a formal diversion program or who were informally diverted, which some states have defined as families who begin the application process but withdraw the application before it is completed or complete the application but never enroll in the program. Washington has a formal diversion program and will be studying those who participate in it. In Jackson County (Kansas City), Missouri, those who apply for TANF and who are participating in Work First, a job placement program, will also be studied. Florida, the San Mateo County group, and South Carolina will study families who started the application process but never finished or who finished the application process but then voluntarily withdrew without receiving any cash assistance. The San Mateo County group will also study families who applied but whose applications were denied for nonmonetary reasons. Florida, South Carolina, and Washington also plan to study the outcomes of families who appear to be eligible for, but are not receiving, cash assistance. The proposed methodology to reach this population is to identify those who received food stamps or Medicaid (and so met means-tested requirements for these programs) but who did not apply to receive TANF grants. A similar effort is being undertaken in Milwaukee, Wisconsin, as part of Wisconsin's welfare leaver study grant. This part of the study, being conducted by the Institute for Research on Poverty at the University of Wisconsin-Madison, is examining three types of applicants for Wisconsin Works (W-2) in Milwaukee: families who apply and later receive assistance, families who apply for assistance and are determined to be ineligible for assistance, and families who apply and appear to be eligible for assistance but do not participate.

4  

 Eventually, with increasingly severe benefit reductions, some states completely close a case, while other states fully sanction only the adults in a case, but still provide benefit coverage for the child (see Gallagher et al., 1998, for state-by-state sanction rules).

5  

 Cuyahoga County will do some analysis of child-only cases.

Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×

Study Methods: Definition of Cohorts and Comparison Groups

The majority of states followed ASPE's suggestion for defining two separate cohorts of families who were once receiving cash assistance but have since stopped. Most of the plans define the first, pre-PRWORA, cohort as those who left cash assistance in the last quarter of 1996 or the first two quarters of 1997. The second, post-PRWORA, cohort of leavers is typically defined as those who left in a quarter as early as 1997 or as late as 1999. Although most of the studies plan to simply monitor the outcomes of these cohorts, some jurisdictions have proposed to compare outcomes of the two cohorts of welfare leavers. (See below for a discussion of the complications of such a comparison.) A couple of states have also proposed to compare the outcomes of the sample of leavers to a sample of families who stayed in the program. Several subgroup comparisons were proposed by states and counties; Table A-1 in Appendix A lists the subgroups that each grantee hopes to examine.

Some states and counties are conducting qualitative analyses from data collected through focus groups or ethnographic studies. Both the Cuyahoga County, Ohio, and Los Angeles County projects are part of Manpower Demonstration Research Corporation's Project on Devolution and Urban Change, which includes ethnographic studies. The District of Columbia will be conducting focus group interviews of the first cohort of welfare leavers in its study. South Carolina is conducting "semi-structured" case-study interviews (in-depth personal stories) with 40 family heads who will also be interviewed in the survey component of its study.

Data Sources

All the grantees proposed to use administrative data supplemented with survey data for their studies. Most states have proposed using only administrative data to track the first cohort of welfare leavers and to use both administrative data and survey data to track the second cohort. In addition, the four studies that are collecting qualitative data through in-depth ethnographic studies or focus groups will use these data to complement the quantitative data.

Administrative data will provide an inexpensive way for states to collect data on program participation, earnings, and employment outcomes. All the states proposed linking administrative data on earnings and employment to the records of closed cases. Most of them proposed using employment and earnings records from unemployment insurance (UI) reporting by employers to the states. A couple of them proposed using state Department of Revenue administrative records to track a case's employment, earnings, and income status. In order to obtain data on other well-being measures, such as income security, food security, housing security, mental health, and child support, and to track dependency of

Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×

cash assistance leavers on other social welfare programs, states plan to link administrative records from programs in housing, education, child welfare services, Medicaid, food stamps, Supplemental Security Income, and child support.

The period for which most jurisdictions plan to track leavers with administrative data varies widely. Some states have extensive data and can track individual cases as far back as 1988. Most proposed to track leavers for at least 1 year after their program participation ended. Some states proposed using administrative data for all leavers defined in a quarter, while others proposed using administrative data for only a subsample of the leavers in a quarter.

In order to obtain information about outcomes not available in administrative records, all the states proposed conducting surveys of a subsample of the leavers, most often a subsample of the second cohort of leavers. Surveys will cover questions not typically available on administrative records, such as living arrangements and marital status, earnings and income support not on administrative records, fringe benefits covered by employers, health insurance coverage, use of child care, barriers to self-sufficiency, food insecurity, housing insecurity, rent and other expenses, use of emergency services, and, finally, child well-being.

Not all the states plan a stratified sample, but most surveys will stratify the subsample of closed cases in a particular quarter of the year. The characteristics used for stratification are reason for leaving assistance, urban/rural, and region of state or county. Most of the states will use mixed mode surveys—telephone interviews followed by in-person interviews for those who do not respond to the telephone survey. There is a wide range in the sample sizes of the surveys: the smallest proposed sample is 350, while the largest is 15,000. All the studies hope to achieve a response rate of around 70 percent; many of the states will provide incentives for completing the survey.

Three states have proposed to interview a case more than once after leaving welfare, but most proposed only one interview with each case. The proposed timing of the surveys varies: most states plan to interview the individuals between 6 months to 1 year after leaving TANF, although the time frames for interviews range from the month the client left the rolls up to almost 2 years after leaving.

Each project plans to link all the administrative data sets. Most states proposed using a common identification code across all records in order to link the data (most often the client's Social Security number). A few states proposed using probabilistic matching to do the linking. The linked administrative data sets will also be linked to the survey data collected on each individual. The grants require that the data be made available for public use, so the states also have plans for how access will be provided and how confidentiality of the cases will be maintained.

Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×

RESEARCH ISSUES FOR THE ASPE LEAVER STUDIES

The 14 leaver studies funded by ASPE have many strengths, and they show considerable promise for providing new information relevant to welfare reform when their final reports are completed. The panel believes their most important strength is their contribution to the building of data infrastructure at the state level, which will provide many benefits in the future as the study of welfare reform continues. The studies will also be of generally higher quality and have more cross-state comparability than welfare leaver studies that have been completed to date. At the same time, using the general principles of welfare reform evaluations we outline in Chapter 2, there are several issues raised by the grantees' plans that the panel believes should be addressed for the studies to yield their maximum potential.

Cross-Study Comparability

One issue that cuts across all the individual topic areas (outcome variables, populations, etc.) is that of cross-study comparability. Such comparability is not the sole objective of the studies, for a goal of each study is an adequate description of its own leaver population and the special characteristics of that population in terms of the outcomes most important to that state. Nevertheless, cross-study comparability would assist the states in comparing their own programs to those of other states, and it would assist Congress and the administration, which will need some national-level assessment of the effect of welfare reform. In the absence of randomized trials or well-designed and credible nonexperimental analyses within each individual state, cross-state comparisons of outcomes and policies are very important.

ASPE has made some attempts at cross-state comparability among the studies. However, many data barriers to cross-state comparability in the studies remain. Although a consensus was reached on the length of time off the rolls required in order to be considered a leaver (a notable achievement),6 there is still variation in defining leavers in the presence of partial sanctions, child-only cases, and transitional benefits besides cash assistance. These differences arise naturally because different states officially terminate cases at different points in the process—sometimes immediately after a sanction has been imposed, sometimes only after an appeal process has been exhausted, sometimes only after the family has had a certain length of time to come into compliance, and so on. For administrative data, there are considerable differences across states in concepts used for the definition of a case, types of cases delineated, and the universe of administrative

6  

 Consensus among the grantees on the length of time for which a case is considered a leaver was achieved primarily through discussion at a technical assistance workshop that ASPE held with grantees and through the ASPE-sponsored Internet-based Welfare Reform Outcomes Grantees list serve.

Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×

data sets. For example, one state (Wisconsin) defines a welfare leaver as someone who leaves all assistance programs (both cash and noncash benefits), while other states define those who leave cash assistance only as leavers, meaning the recipients could still be receiving other benefits.7 In principle, survey data should be capable of standardization across states. Because new surveys are being undertaken in the leaver studies, comparability would be possible, but it requires attention to detail in making survey questions and constructs similar and in making accounting and tracking time periods the same.

Another aspect of comparability across states (which relates to in-state issues, as well) is the need for a characterization of the recipient population in terms of the key variables discussed in Chapter 2. Documenting the distribution of cases across long-termers, cyclers, and short-termers in each state and reporting the findings on outcomes separately for the three groups would allow a crude, but useful means of standardization across states and go a long way toward eliminating differences among the compositions of caseloads. Similarly, stratification of the state samples by household type, education, health, and other fundamental determinants of well-being would assist in comparability. For example, individuals with more education or who are in better health when they leave the program may have very different outcomes than those who left with less education or bad health.

Stratifying the analysis by groups defined by these types of characteristics does not need to be justified by cross-state comparability, however, but rather simply by the importance of subgroup analysis. The states have plans for some subgroup analyses (see Appendix A, Table A-1), but the groups are largely defined by programmatic status (time-limit leavers, sanctioned cases) or by race or number of children. These subgroup categories are important because time limits differ and are enforced differently across states. This is an obvious reason for variability in exit rates and the outcomes of leavers. However, characterizing leavers by these subgroup categories does not capture the basic characteristics of individual earnings capacity or the capability for self-sufficiency. As noted in Chapter 2, characterizing women by their work history, welfare history, and their history of health and related social problems should be important in predicting the likelihood of success or lack of success after leaving TANF. To do so clearly has implications for data collection and subsequent analysis.

Another analytic approach that would both be informative to a state and assist cross-state comparability is a comparison of the characteristics of families who are leavers with the characteristics of families who are not leavers ("stayers"). With such an analysis included in all state reports, an assessment could be made about whether differences in the characteristics of leavers across states is a result

7  

 Wisconsin will present results defining a leaver in a manner the other state's do, as well as under its own definition of a welfare leaver.

Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×

of differences in the types of families that leave the rolls or of differences in the types of families on the rolls. It would also be helpful to compare the characteristics of some of the families in the in-between categories mentioned above (partial-sanctions, child-only cases, transitional cases, etc.) to the characteristics of both stayer and leaver families. Such comparisons require that data be collected on stayers as well as leavers, but many states do not plan their analyses in that way.8 However, administrative data could be used to compare characteristics of stayers to characteristics of leavers with relatively low time and cost efforts.

Monitoring Versus Evaluation

A second issue raised by the state studies is the relative importance of monitoring and evaluation. The studies, as they have evolved thus far, have been primarily focused on monitoring. This is proper and the best use of resources in the short term. Setting up state data bases to be able to track recipients and nonrecipients over time, for example, is a valuable exercise in data building and data infrastructure that is needed for long-term efforts to study the effects of welfare reform. Determining the employment rates of leavers and, more generally, their well-being is also important, as is documenting the percentage of leavers who can be characterized as in distress. An understanding of particular barriers to self-sufficiency faced by particular kinds of families can be gained from the monitoring studies as well. Monitoring can also help states understand what transitional services former recipients are using, the extent to which they are using them, and what other social programs the recipients use (such as food stamps), in order to plan future programs and policies and to allocate resources. For such an assessment, it is important to know the characteristics of those leaving TANF, especially their past history of welfare receipt and employment, and their education level, since those who are relatively better off in terms of these characteristics may not need as many transitional services after they leave TANF as those who are relatively worse off.

As was stressed in Chapter 2, however, a monitoring study is not capable of generating an estimate of the effect of the policy change by itself, because no comparison group or counterfactual is present. Determining the employment rates, earnings, and general well-being outcomes of leavers is of interest. However, unless it can be determined whether those outcomes are different than the outcomes that would have occurred in the absence of the policy change, it cannot be known whether the actual outcomes are a result of the policy change.

The states are collecting data on a pre-PRWORA cohort implicitly for evalu-

8  

 More generally, it would be helpful for cross-state comparability if states describe the characteristics (education, work history, recipiency history, demographic structure, etc.) of the caseload as a whole, not just for leavers.

Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×

ation purposes. In either their current or future reports, they will be analyzing the two cohorts and comparing their outcomes along dimensions for which data are available in both periods. Unfortunately, many outcomes (such as those obtainable only through household surveys) will generally not be available on the pre-PWRORA cohorts. When this stage of the analysis is reached, all the important issues discussed in Chapter 2 will need to be addressed: that is, the possible threats to the cohort design will have to be studied. In particular, states will have to assess whether the pre-PRWORA cohorts are similar to the post-PRWORA cohorts in their characteristics. For example, a key issue in all states will be the effect of the socioeconomic environment on outcomes because the economy improved over the period of the two cohorts in most states. This improvement will lead the outcomes for leavers in the post-PRWORA cohort to be more favorable than those in the pre-PRWORA cohort, even if there had been no policy change. To address that threat to the study designs, states will have to make an estimate of the effect of the economy on the outcomes of leavers and "net out" that effect when arriving at an estimate of the effect of their policies.

Yet, as also discussed in Chapter 2, declining caseloads may alter the composition of the caseload, leaving a disproportionate number of long-termers on the rolls, with corresponding low leaving rates and worse outcomes when they leave (some of whom will be forced to leave because of the time limit provisions). This effect works in the opposite direction to the effect of the economy, because in this case it could appear that outcomes of leavers have worsened over time. However, this change in leaver outcomes would be a result entirely of the caseload composition, not an effect of the legislation. To address that threat, states will need to characterize both study cohorts in terms of their status (long-termers, cycler, and short-termers), their work histories and general job skills, and other characteristics related to potential success off the rolls.

Both of these threats will also affect a comparison of the rates of reentry of leavers in the pre-PRWORA and post-PRWORA cohorts. The improvement in the economy will tend to make reentry rates lower in the post-PRWORA cohorts, while the change in the composition of the caseload will tend to make reentry rates higher in those cohorts. Both of these effects are independent of a state's policies.

A major barrier to assessing comparability of the pre-PRWORA and post-PRWORA cohorts is, once again, data availability. Although almost all states are linking welfare recipiency data to other administrative data sets, few have proposed to track welfare recipiency history for more than 1 year prior to the dates for which the cohorts are defined (see Appendix A, Table A-1). Yet data on welfare recipiency history over longer periods of time are needed to assess changes in caseload composition. It might also be helpful to understand the use of other social programs (such as food stamps, Welfare to Work, Medicaid, and job training programs) by the pre-PRWORA cohort, which must necessarily rely on administrative rather than survey data. Because of the lack of survey data, it

Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×

will be difficult to measure income, family structure, and other characteristics and outcomes for the pre-PRWORA cohort.

It is not clear whether the states have data collection and analysis plans that can assess the effects of the economy on outcomes for leavers or characterize other changes in the programmatic and policy environment (changes in child care, child support, Medicaid, and other programs) that may have changed between the pre-PRWORA and post-PRWORA cohorts. In fact, if a state had a waiver policy at the time of the pre-PRWORA cohort, the programmatic environment would not be AFDC in its pre-reform structure. In such a case, it may be necessary to use time-series caseload modeling, which imposes a separate and very different set of data requirements on the analysis, or comparisons of different counties in the state with different unemployment rates. In the absence of any formal empirical analysis of the effect of the economy and other programs, states should carefully document what those changes have been, to at least allow policy makers and analysts to judge the likelihood that they changed outcomes in a particular direction.

The use of a cohort comparison design implies that the relevant counterfactual is what went before PRWORA, which will generally be the AFDC program. Most states had Section 1115 waivers at the time their pre-PRWORA cohorts were drawn. These waivers had been implemented at various times, from 1993 until right before PRWORA was passed. Some waivers covered only a few counties or an experimental group in a state, while some covered an entire state except for a small control group. 9 This variation of rules in the pre-PRWORA cohort complicates any conclusions drawn from the comparison.

As noted in Chapter 2, future evaluations may wish to examine as a counter-factual an incremental altering of welfare reform provisions within the general framework of a state's PRWORA program. Changes in time-limit policies that affect particular groups of recipients who are thought to be in particular need, provisions of extra services to the hard-to-serve population, testing the effects of alternative child care reimbursement policies, and other provisions might be important in the not-so-distant future. While estimating the total effect of a program bundle rightly deserves first priority, states should begin thinking of ways in which modifications and incremental reforms could be tested and evaluated as well.

TANF Entry Rates

A third issue raised by the state grantee studies is the need for an examination of the TANF entry rates of those who apply and, more broadly, of the entire

9  

 Nine states are continuing waiver plans unmodified under PRWORA. Twelve states are modifying components of their waivers (including some of the nine states with unmodified waivers that had more than one waiver).

Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×

population of eligible persons.10 Divertees and discouraged applicants constitute the other part of the caseload flow that, with leavers, determine how the caseload changes over time and how well-off the nonrecipient population is. The major difficulty in studying entry is the difficulty of obtaining data on divertees and on nonapplicants. Outcome information on divertees may require surveys from the applicant pool if administrative data are not collected on divertees. Other administrative records, such as those from food stamp, Medicaid, and UI wage records, may also provide data on at least some of the nonrecipient population. The best source of data on nonrecipient households in general is the in-person, block-based household survey, which can be very expensive to conduct. States will need to be creative in seeking low-cost methods of fielding such surveys. Piggy-backing on national surveys or other surveys in the state is one avenue that may be available in some areas.

Data Collection and Availability

A fourth set of issues raised by the state grantee studies relates to data collection and availability, although many of these issues have been raised above for administrative data. The state grantees are starting to collect data sets from a large variety of sources (see Appendix A, Table A-2), but more progress will need to be made before these data sets can fulfill their potential. The quality and reliability of the information in many of the data sets is still mostly unknown, largely because they were generally designed for administrative, not research, purposes. States should be able to learn a great deal in their initial grant periods in this respect.

The possibilities for data matching across state agencies, or between state and federal agencies, are often unknown, and approaches and methodologies have not been standardized. Many barriers to interagency and intergovernmental cooperation exist, and more systematized confidentiality and disclosure agreements are needed. Obtaining administrative data sufficiently far back in time to construct recipiency histories and to obtain information on pre-PRWORA cohorts is a problem in many states. Nevertheless, this is a direction that should be pursued vigorously given the high payoff that matched administrative data records can provide. There is a research community with considerable experience in developing linked administrative data sets that is a potential resource for state grantees.

The states face challenges in their surveys as well. Relatively small sample sizes, largely resulting from inadequate financial resources to conduct large surveys even if by telephone, are a problem in many of the surveys. The response

10  

 Five of the current state grantees have begun to address entry rates and ASPE plans to fund studies of entry in the next round of grants.

Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×

rates by telephone for the population remain to be assessed, and methods for locating families and maintaining contact with them are also only beginning to be studied by the states. For states that use telephone surveys in combination with in-person follow-up for nonresponse, there is potential for a mode bias11 that states will need to assess and report. Statistical methods for adjusting for nonresponse and methods for assessing the degree of bias from nonresponse need to be fully understood by the state grantees.

Another issue related to response rates is the seemingly simple issue of defining response rates the same way across studies. Unfortunately, response rates can be defined in a variety of ways that can produce noncomparable figures across studies. The American Association for Public Opinion Research (AAPOR) (1998) has published a study on the different types of response rates that can be calculated that offers guidance on preferred measures. The AAPOR report defines four groups of response cases: interviewed, eligible nonresponse, unknown eligibility and nonresponse, and not eligible. The standardization of calculating and reporting response rates will aid in cross-state comparability of the studies, and, at a minimum, grantees should be encouraged to report the number of cases in each of these four groups to aid comparability.

The accuracy of long retrospective questions in surveys and the difficulty of respondent recall are issues of concern for grantees, as are many issues about the more general accuracy of survey responses concerning income and welfare receipt. In addition, how to balance the tradeoff between expenditures to improve the quality of the data from a given set of interviews and the number of interviews itself, given a fixed budget constraint, is a difficult issue.

The survey research community has addressed many of these issues for the last 3 or 4 decades, and a substantial body of knowledge has been accumulated on which the states could capitalize. Much is known about strategies for encouraging response by households, the effects of incentive payments on response rates, statistical methods for adjusting for nonresponse, the accuracy of recall in retrospective questions, and the accuracy of income and welfare receipt reporting (see Andranovich and Riposa, 1993; Fowler, 1993; Groves and Couper, 1998; and Lavarakas, 1993, for guidance). Formal or informal mechanisms for making contact with the larger survey research community is a possible avenue that state grantees should explore. However, much of the survey research community has studied general populations, and it is not clear which of the lessons learned from those populations apply, or to what degree, to the populations of interest for welfare reform studies.

Finally, as noted in Chapter 2, data that explicitly document the program rules in force at the time of the two cohorts used in the analysis are necessary and should be provided by the state grantees in their reports. Such an effort should

11  

 A bias due to survey mode is a nonrandom difference in survey responses by the mode of the survey given to respondents.

Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×

include documentation not only of TANF rules, but also of rules of other programs providing services for those on the caseload (such as Welfare to Work services and job training and job search programs) because the services recipients receive may affect their outcomes. While this is largely a matter of routine and good research practice, such documentation often requires a separate effort and devotion of some resources, even for analysts who are located in the state where the policies have been enacted. It is also important that states make an effort to document the degree of implementation of the programs within the state and to give a qualitative assessment of their effects on the operation of the welfare program offices. A formal process evaluation is one method of obtaining that information but it may not be feasible for all states to conduct such an evaluation in this round of grants. At the least, a description of implementation of a state's policies should be included in all state grantee reports and analyses.

OTHER WELFARE LEAVER STUDIES

The 14 ASPE-funded welfare leaver studies will add to a growing list of such studies conducted around the country.12 There have been a number of reviews of leaver studies to date (Brauner and Loprest, 1999; U.S. General Accounting Office, 1997, 1999; National Governor's Association et al., 1998b; Parrott, 1998; U.S. Department of Health and Human Services, 1998), but the number of leaver studies being conducted and reported is growing rapidly, and these reviews will be out of date quite soon.

The most recent reviews of the results of these leavers studies are those by Brauner and Loprest (1999) and the U.S. General Accounting Office (GAO) (1999). The Brauner and Loprest review (subsequently referred to as B&L) considered 11 studies in 10 states and the GAO review considered 8 studies in 7 states. Seven of the studies in the GAO review are also covered in the B&L review. B&L and the GAO both restricted their reviews to studies which had adequate response rates in their surveys and other data collection mechanisms, and thus do not cover all leaver studies that have been completed.

According to these two reviews, the leaver studies show that over half the former recipients were working at the time the data were collected and employment outcomes were measured, which was usually several months to a year after exit. The studies in the B&L review indicated that between 51 and 75 percent of former recipients were working at the data collection point and that between 68 and 88 percent were ever employed over the course of the study periods. The studies reviewed by the GAO indicate that 61 to 71 percent of leavers were

12  

 A summary of leaver studies in all states has been prepared and is being updated continually in a joint effort by the National Governor's Association, National Conference of State Legislatures, and the American Public Human Services Association (1998a).

Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×

employed at the time the data were collected and that 63 to 87 percent had been employed at some time during the study periods.

Few of the reviewed studies reported how many of the welfare leavers returned to cash assistance. The three studies that did report on recidivism found that between 19 to 30 percent of leavers returned to welfare. Many studies reported the rates at which leavers used other non-cash benefits (most often Medicaid and food stamp usage were reported). Both the GAO and the B&L reviews indicate that between 44 to 83 percent of former recipients received Medicaid benefits after leaving welfare. Food stamp usage by former recipients also varied widely. The studies reviewed by the GAO show that between 38 to 60 percent of former recipients received food stamps while the studies reviewed by B&L show that between 31 and 66 percent received food stamps after leaving cash assistance.

Both of the reviews note a number of difficulties in comparing results across the studies. For example, one difficulty is that some states restricted their studies to those who had been off welfare for at least 6 months or more, thus excluding welfare leavers who had not been successful and had returned fairly quickly to the rolls. The B&L review indicates that studies that included all leavers, instead of only including those who did not return to the rolls, showed lower employment rates, generally between 51 to 69 percent at the time of data collection. The employment rates for studies that included only those who stayed off of welfare were generally higher, between 55 and 75 percent at the time of data collection. One problem common to many leaver studies that was discussed at length in the GAO review is that leaver studies usually have such small and specialized samples that their results cannot be generalized to the entire state in which the studies took place. Both reviews noted many other differences in methodologies, definitions, data used, and calendar time periods covered. Some studies included only sanctioned leavers, while others included all closed cases. Some included only single-parent families, while others included two-parent and other types of families.

In addition to these issues of cross-study comparability, the leaver studies reviewed by B&L and the GAO fail to address most of the other issues we noted in our discussion of the 14 ASPE leaver studies. Very little subgroup analysis according to short-termer, cycler, or long-termer categories has been conducted; consequently, only averages across all types of leavers are generally provided. The studies are almost entirely monitoring in nature, making no attempt at estimating the effects of welfare reform. This lack is apparent as none of the leaver studies has a comparison group and none has attempted to determine whether the employment rates of leavers are greater or less than they would have been in the absence of welfare reform. Finally, none of the studies seriously examined the other groups of interest besides leavers—those still on the rolls (stayers), divertees, and eligible nonapplicants.

To illustrate some of these methodological, definitional, and data differences and their implications for cross-state comparisons in more detail, we take a

Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×

closer look at the leavers studies in four of the states covered by the B&L and GAO reviews: Indiana, Iowa, Maryland, and South Carolina (two studies). Detailed descriptions of the studies and results are shown in Table 3-1.13

In these five studies, between 53 and 88 percent of former recipients had worked since leaving cash assistance. For the studies that reported the number of leavers who were working at the time of data collection, results range between 64 percent in Indiana to 70 percent in the first South Carolina cohort and 62 percent in the second South Carolina cohort. Some leavers' incomes increased after departing, but not all. The majority of leavers received food stamps or Medicaid after leaving, except for food stamp recipiency in Indiana. Finally, only one study reported the number of leavers who returned to welfare: Maryland found that 19 percent of leavers returned to welfare within 3 months of leaving, and 23 percent returned within 12 months of leaving.

These studies show in more detail how the populations studied and definitions used to define leavers vary across the studies. For example, the Iowa study considers only sanctioned leavers, while the other four consider leavers of any type. The Indiana study considers leavers at one time, although the leavers could have returned to welfare and subsequently left welfare again by the time the outcomes were measured. The South Carolina studies do not include leavers who return to welfare in the study population, which makes it difficult to compare outcomes with those studies that do include those who return to welfare, since it is likely that those who did not return to cash assistance are doing better than those who did.14

The range of employment rates for leavers—from 53 percent to 88 percent—is quite wide across the five studies. This is cause for concern because it cannot be determined if the outcome differences are caused (in part or in whole) by the differences in populations studied and caseload dynamics at the time of the study or the economic conditions in the states at the time of the study, rather than the policies in effect during the study period. In addition, in only one study (Indiana) is there an attempt to characterize who was a leaver and who was a stayer to guard against the possibility that the characteristics of the overall caseload are the same across states but that different types of families left the rolls in one state versus another. There is little information on how the outcomes varied for long-termers

13  

 Three of these studies (Iowa, Indiana, and Maryland) were cited in the First Annual Report to Congress on the TANF Program (U.S. Department of Health and Human Services, 1998). Neither of the two South Carolina studies were cited in this report, but results from an earlier cohort of leavers in South Carolina were cited.

14  

 The Iowa study does not include welfare returners either, but the Limited Benefit Plan Program has a mandatory 6-month period during which sanctioned leavers cannot return to welfare, after which they may return.

Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×

TABLE 3-1 Summary of Methods and Results from Selected Recent Research on Welfare Leavers

Study

Iowa:Limited Benefit Plan Study (LBP)

Indiana

Maryland

South Carolina 2nd Quarter 1997 Cohort

South Carolina 3nd Quarter 1997 Cohort

Description of Program Environment

Pre-TANF waiver;a Study of PROMISE JOBS program in 3 service delivery areas

Pre-TANF waiver;b experimental design

Post-TANF implementationc

Post-TANF implementation (which occurred October 1996)d

Post-TANF implementation (which occurred October 1996)d

Populations Covered

Cases sanctioned to LBP 11/95-1/96

All cases 5/95-5/96 (both control and experiment programs)

All closed cases exiting 10/96-9/97

Closed cases 4/97-6/97e

Closed cases 7/97-9/97e

Does Study Include Those Who Return to Welfare?

No

Yesf

Yes

No

No

When Are Outcomes Measured?

2-6 months after benefits terminated

Early 1997

Up to 12 months after exit

Not available

8 to 13 months after case closed

Study Methodology

Descriptive; before and after benefit loss

Descriptive

Monitoring

Monitoring

Monitoring

Data Collection Method

Telephone survey with in-person follow-up; 85% response rate; n = 137; some administrative data also used

CAPI survey with in-person follow-up; stratified random sample (half from control group, half from experimental group); 71 % response rate; n = 1,593

Matched administrative data, 5% sample of cases closing each month; n = 2,156

Telephone survey with in-person follow-up; random sample; 76% response rate; n = 391

Telephone survey with in-person follow-up; random sample; 76% response rate; n = 403

Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×

Study

Iowa:Limited Benefit Plan Study (LBP)

Indiana

Maryland

South Carolina 2nd Quarter 1997 Cohort

South Carolina 3nd Quarter 1997 Cohort

Reason for Closure

All cases sanctioned from cash assistance and reached a time limit of LBP

Not available

19% income ineligible; 32% did not complete redetermination process or provide eligibility information; remainder for various reasons.

54% earned income; 19% sanctioned; 13% procedural; 8% voluntary withdraw; 4% ineligible

51% earned income; 22% sanctioned; 9% procedural; 9% voluntary withdraw; 7% ineligible

Reason for Sanctions

Not available

Not available

7% of all closings sanctions: 6% nonwork compliant; 1% noncompliance with child support enforcement

Not available

Not available

Ever Worked After Leaving

53 percent

84 percent

58 percent

88 percent

86 percent

Working at Time of Data Collection

Not available

64 percent

Not available

70 percent

62 percent

Received Food Stamp Benefits After Leaving

64 percent

38 percent

Not available

57 percent

60 percent

Received Medicaid After Leaving

66 percent

53 percent

Not available

79 percent

80 percent

Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×

Study

Iowa:Limited Benefit Plan Study (LBP)

Indiana

Maryland

South Carolina 2nd Quarter 1997 Cohort

South Carolina 3nd Quarter 1997 Cohort

Income After Leaving Welfare

40% had increase; 49% had decline; 11% no change

43% of all leavers had income >> $1,000/month; 40% had income between $500 and $1,000

For those working, first quarter earnings after leaving averaged $2,384

66% said income was up

59% said income was up

Percent Returned to Welfare

Not available

Not available

19% within 3 months; 23% within 12 months

Not available

Not available

a Waiver provisions include: removal of 100-hour work cap, higher earnings and asset disregards, must participate in PROMISE JOBS employment and training program, if not, assigned to LBP. For the cases in this study period, LBP recipients received 3 months of full benefits, 3 months of limited benefits, and then no benefits for a period of 6 months, after which they are eligible to receive benefits again.

b Waiver provisions for experimental group include: signing a Personal Responsibility Agreement, children must attend school and be immunized, family benefit cap, disregard of child support and earnings in first 6 months, work or training requirements, time limit of 24 months if subject to work requirements.

c No further policy details provided.

d TANF policy includes: 24-month time limit on economic assistance over 10 years and 60 months over lifetime, full family sanction for not meeting requirements, spouses eligible to participate in program.

e Only closed cases that met the following three requirements were included in the study: (1) household received at least one check for cash assistance; (2) household received no subsequent cash assistance; (3) at least one household member was required to seek work or voluntarily sought work.

f A leaver is defined as someone on welfare during the study period, but who was not on welfare at the time of the survey. A leaver may have left welfare, returned to welfare, and left again before the survey was conducted. All results are reported for those not on welfare at the time of the survey.

SOURCES: Data from U.S. Department of Health and Human Services (1998); Fraker, et al. (1997); Fein (1997); Fein, et al. (1997); Maryland School of Social Work (1998); South Carolina Department of Social Services (1998).

Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×

and short-termers and whether different caseload compositions in this respect across the states might explain some of the observed differences.15

In addition, there is no information on whether the leaving rates measured in these studies were higher or lower than at any time in the past, nor whether the improvement in the economy affected the leaving rates.16 The studies did not report the unemployment rates in the states during the period of study. However, the time periods covered by these studies range from May 1995 to as late as September 1997, and the state of the economies across these time periods and across these areas could be quite different.

A final reason that the studies are difficult to compare is because the policies in effect in each state differ. Both the Iowa and Indiana studies were conducted before PRWORA and the policies in effect in both states were waiver policies. Both of these studies were part of a larger evaluation that included policy implementation studies which provided details on the policies in place during the study periods. Although these studies had some similar policy components, such as work requirements and asset and earnings disregards, each study had policy components that differed significantly, making comparisons difficult. The Iowa study focuses only on those cases that are sanctioned for not participating in an employment and training program, and so are assigned to the Limited Benefit Program, which gives the recipient a full benefit for 3 months, then a limited benefit for 3 months, and, finally, no benefit for 6 months. After these 6 months, the recipient may be eligible for benefits again. In the Indiana study, half of the sample of leavers were part of an experimental group that was subject to family caps, time limits, and work requirements, among other things, and half the sample consisted of those in a control group that were subject to the old AFDC rules. These differences make it difficult to compare and interpret outcomes from the two studies. The South Carolina and Maryland studies were conducted after TANF was implemented in each state. While the policies in these states may be similar to each other and may even be similar to the pre-PRWORA policies for the Iowa and Indiana studies, neither study documents the policies in place in sufficient detail to make such a comparison possible. The South Carolina study reported a few components of the state's TANF policies, and the Maryland study provided no detail on its policies. Because the populations studied, the economic conditions, and the policy regimes in place across the states in these studies likely differ so much, what can be learned from these studies is quite modest.

15  

 The Maryland study did characterize the welfare recipiency history of leavers and whether or not leavers had past work experience. However, the only outcome that was compared across these characteristics of leavers was whether or not the leaver returned to welfare. The Indiana study also compared leavers to stayers by past welfare receipt and work experience histories of recipients, but did not compare employment outcomes or any other outcomes of leavers by past welfare receipt or work history characteristics.

16  

 A separate component of the Indiana study did show trends in AFDC caseloads and state unemployment rates for several years before and after the study.

Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×

There is also considerable variation in the time that the outcomes are measured compared to when the leavers stopped receiving cash assistance, ranging from 2 months to almost 2 years. Recidivism is one outcome that is difficult to assess since the number of leavers who eventually return to welfare can only increase with time.

The GAO and B&L reviews both only considered studies that achieved an adequate response rate. Several of the leaver studies conducted by states were eliminated because they had such low response rates. Four of the studies in Table 3-1 use survey data (Iowa, Indiana, and the two cohorts from South Carolina). Each of these studies achieved good response rates. Both the Iowa and South Carolina studies describe nonresponse in their surveys and how they defined their response rates. These two studies also compared demographic characteristics of respondents and nonrespondents. The Iowa study showed few differences across respondent status. The South Carolina second-quarter cohort survey showed that those who left because their earned income levels were too high were more likely to respond to the survey, and the third-quarter cohort survey showed that sanctioned leavers were more likely to not respond to the survey than other types of leavers. This may be cause for concern since sanctioned leavers may be worse off than those leavers who left for other reasons, and those with higher earned incomes may be better off, meaning the survey results may not reflect the worse outcomes. Neither studies supplemented their surveys with administrative data to attempt to characterize responders and nonresponders by outcomes of interest, such as employment and wage status. The brief report of the Indiana study we used to gather information about this study did not show how the response rate of the survey was defined or if responders differed from nonresponders.

The Maryland study uses only administrative data, which, as we discussed in Chapter 2, does not cover the entire population of interest, as many welfare leavers may not show up in any administrative data sets. Such an exclusion is potentially biasing because leavers who are missing may be those who have moved out of state or who are not participating in any other program in the state with an available administrative database. This study does not provide detailed information on whether any observations had missing data for the outcome measures or whether those with missing data were dropped from the study. Such reporting should be standard practice in reporting results.

Although the range of employment rates in the leaver studies is between 53 and 88 percent, one could view this range as narrow instead of broad and a range that adds up to a fairly consistent story, given the differences in methods and lack of comparability across the studies in Table 3-1 and in the B&L and GAO reviews that we have noted. Moreover, subgroup analysis, which the leaver studies have mostly neglected, may show a narrower range of employment outcomes. That the range of employment rates for leavers is not wider in these studies may suggest that the differences in policies, in socioeconomic environments, and in data comparability are not that great. Or it may merely reflect a

Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×

strong national economy with low unemployment so that economy wide effects dominate all other effects. Thus, much more progress in leaver studies is needed before one can be confident of their general conclusions.

OTHER MAJOR WELFARE REFORM PROJECTS

There are a large number of other welfare reform studies under way around the country. In this section we review some of the other efforts to evaluate welfare reform that are being conducted by the ASPE and other offices in DHHS, as well as the efforts of researchers in academia and at private research organizations.

We first summarize several projects whose researchers addressed the panel during its initial meeting in September 1998. There are many more evaluations under way as well; a more comprehensive list is included in Table B-1 in Appendix B.

Office of the Assistant Secretary for Planning and Evaluation

In addition to the grants to the 14 jurisdictions to study welfare leavers, ASPE has several other research projects it is funding or conducting internally. Among the external projects, the largest study is an evaluation of the welfare-to-work grantees that the Department of Labor funded to provide services for welfare recipients and noncustodial parents of children on TANF who are the hardest to employ. ASPE has contracted with a team of researchers from the Urban Institute and Mathematica Policy Research to evaluate the net impact and cost-effectiveness of the program at up to 10 sites and to conduct a process evaluation of how the projects are being implemented at several sites. To the extent possible, net impacts will be assessed using a random assignment design.

External evaluations of child and youth well-being funded by ASPE include projects on abstinence education programs, child welfare, child support enforcement, and child care policies for low-income families. ASPE and the Administration for Children and Families are jointly sponsoring a project, Measurement of the Impacts on Children in Evaluations of State Welfare Reforms, which seeks to improve states' abilities to measure child outcomes for welfare reform evaluations. ASPE is also funding several smaller studies that focus on special populations of welfare recipients, such as Native Americans, individuals with disabilities, individuals who are victims of domestic violence, and child-only welfare cases.

ASPE will also be funding a second round of grants to states and large counties to examine the diversion of applicants and potential applicants for cash assistance. Applicants who eventually do not enroll because they are ineligible for nonfinancial reasons, participate in formal diversion programs, participate in job search activities prior to enrolling, or never finish the application process are

Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×

of particular interest. ASPE will also consider funding projects that focus on individuals and families entering or leaving TANF.

One major internal ASPE project is creating an historical baseline of data on welfare receipt, starting with AFDC and including TANF. Other internal projects include: analyzing administrative data from the Food Stamp Program to learn more about low-income individuals not receiving welfare, assessing the research capabilities of the New Hires Database, matching data from the Social Security Administration with national-level survey data, and working on strategies for ensuring that former welfare recipients and persons diverted from TANF are enrolled in Medicaid when they are eligible. Finally, ASPE is working with the Census Bureau in developing the Survey of Program Dynamics, which is summarized below.

Administration for Children and Families

The Administration for Children and Families (ACF) in DHHS also funds and conducts research on welfare reform. One of ACF's major projects is the evaluation of nine states with Track 1 welfare reform demonstration waivers, which were implemented before TANF and have not been modified or have been only slightly modified since. These studies, almost all of which use an experimental design, compare the outcomes of individuals in a control group subject to the old set of AFDC rules to the outcomes of individuals in the experimental group, who were subject to the demonstration rules, which had many TANF-like components. Five of these states were also given grants to do comprehensive and systematic measurements of family processes and child outcomes.

ACF is also funding 13 Track 2, or modified state welfare reform demonstration evaluations in 12 states that modified or replaced their welfare reform demonstrations implemented prior to TANF. Some of the 13 sites use an experimental design; some of the projects include process analyses on the implementation of policies; and some of the projects include impact analyses.

Another study funded by ACF is the Employment Retention and Advancement Initiative, which involves planning grants to states to examine job retention and job advancement program development and subsequent outcomes. There are two phases to this project: a planning and design phase and the provision of technical assistance through a contract with the Lewin Group. The purpose of this contract is to help states refine their program interventions and develop evaluation designs.

In conjunction with ASPE, ACF is also finishing evaluations of the former JOBS program, now called the National Evaluation of Welfare-to-Work Strategies. The evaluations began in 1989 and will end in 2000. There are 7 sites and 11 different programs.

Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×

Survey of Program Dynamics

The Survey of Program Dynamics (SPD) is being conducted by the Census Bureau under a requirement of the 1996 PRWORA legislation (see Weinberg et al., 1998). The purpose of the survey is to collect longitudinal data on the demographic, social, and economic characteristics of a nationally representative sample of the U.S. population so that overall evaluations of welfare reforms can be conducted. Congress mandated that the 1992 and 1993 panels of the Survey of Income and Program Participation (SIPP) continue to be followed so that the pre-reform characteristics and well-being of families would be understood. The data from the 1992 and 1993 SIPP panels give 3 years of a longitudinal baseline before the reforms in 1996 (1992–1995 for half the sample and 1993–1995 for the other half). SPD will follow the 1992 and 1993 panels of SIPP participants over the years 1996–2001, meaning that, combined, SIPP and SPD provide 10 years of panel data. The 1997 SPD Bridge Survey attempted to interview all sample persons in the 38,000 households that completed all waves of the 1992 and 1993 SIPP panels (76% of the original sample). The Census Bureau interviewed 82 percent of the households in the Bridge Survey (about 30,000), using a modified version of the March 1997 Current Population Survey (CPS) questionnaire. A new core SPD questionnaire was developed for the 1998 survey (with the assistance of Child Trends, Inc.). This survey included a self-administered adolescent questionnaire and retrospective questions on the core topics of jobs, income, and program participation for all persons over the age of 15. The sample size for the 1998 SPD was reduced to approximately 18,500 because of budget constraints. The 1998 SPD interviewed 89 percent of eligible households. The 2001 SPD will repeat the 1998 adolescent questionnaire. A topical module on child well-being will be included in the 1999 and 2001 SPD, and a children's residential history module will be included in 2000. The 1998 SPD data were released in February 1999 as a research file. The 1998 SPD data are expected to be released by the end of summer 1999.

One criticism of the SPD is the cumulative attrition rate, which, from the beginning of the 1992 and 1993 SIPP panels through the 1998 SPD is approaching 50 percent. The Census Bureau is taking steps to address this attrition problem by investigating the use of incentives to induce former SIPP respondents to return to the SPD sample to offset additional attrition expected. Plans to link Social Security Administration earnings records to SPD households to assess any effects of attrition and to look at employer–side variables have also been made. ASPE is contributing funds for the Social Security records and SPD/SIPP analysis.

Urban Institute: Assessing the New Federalism Project

The Assessing the New Federalism Project (ANF) of the Urban Institute has a broad focus, based on the assumption that the shift of responsibility for social

Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×

welfare programs from the federal to the state level will affect many families in the low-income population, not just those receiving welfare (Kondratas, Weil, and Goldstein 1998). The project is not specifically designed as a welfare reform evaluation; rather, it is a study of all social support systems: health, income, housing, food, and cash assistance. The project looks at the effect of changes in the nation's social safety net on the well being of families. Intensive analysis is under way in 13 states that cover half the population of the United States. The 13 states are geographically diverse, vary in their fiscal capacities, and have different policy approaches to social and health care services. In addition, the project has established a state database with more than 900 variables covering the program rules and policies for all 50 states. The ANF project is primarily a monitoring project, although it also includes policy analysis.

The primary source of individual-level data (the unit of analysis is families) is through the Urban Institute's National Survey of America's Families (NSAF). The initial survey of a cross-section of families in the 13 states, as well as in the nation as a whole, was conducted between February and September 1997. A follow-up survey of another cross-section is currently (mid-1999) being fielded. The survey interviewed over 48,000 households: about 28,000 parents with children under 18 and 20,000 households without children. Half of the sample is of households with incomes below 200 percent of the official poverty line. The survey was conducted by telephone. The survey of the first cross-section achieved a 65 percent response rate for interviews about children and a 62 percent response rate for interviews about nonelderly adults (Urban Institute, 1999). Topics covered in the survey include economic security, health and health care, child education and cognitive development, child social and positive development, child behavior problems, family environment, and community environment. The Urban Institute has made public-use data files from the survey available on-line at its Web site.

Manpower Demonstration Research Corporation: Project on Devolution and Urban Change

The Project on Devolution and Urban Change is a study of welfare reform in four urban counties: Philadelphia County, Cuyahoga County (Cleveland), Ohio, Dade County (Miami), Florida, and Los Angeles County (see Manpower Demonstration Research Council, 1998, for a full account of the project). The study has five components: an individual-level impact study, a neighborhood indicators study, an implementation study, an ethnographic study, and an institutional study.

The individual-level study includes, for each county, a sample of households that used food stamps, AFDC/TANF, or, possibly, Medicaid at any time between 1992 and 1999 and whose head is under age 65. From this universe, cohorts of AFDC/TANF recipients and nonrecipients will be selected and tracked over time with administrative data from unemployment insurance records and survey data.

Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×

The surveys will be conducted in person and last approximately 90 minutes. The goal is to obtain 1,000 completed surveys in each county. Two cohorts will be surveyed: single mothers between the ages of 18 and 55 who received AFDC benefits in May 1995 and single mothers who received TANF during May 1997. The two cohorts in each area will be compared in order to estimate the effects of the changes in welfare programs.

The implementation component involves visiting three welfare offices in each county several times a year to study how the policies are being implemented. The ethnographic study consists of a series of in-depth interviews with 30–40 families in each of the four counties over several years. The institutional component includes interviews with nonprofit and for-profit community institutions that provide general services and emergency services for the poor. Each of these components of the studies will provide descriptive assessments of outcomes. The neighborhood indicators study will collect data on the social and economic conditions of the neighborhoods in the counties that are being studied. The characteristics of these neighborhoods will be described over time, and neighborhoods will also be compared with one another at a given time. Changes in neighborhood-level measures will be compared among poor neighborhoods and between poor and nonpoor neighborhoods.

Welfare, Children, and Families: A Three-City Study

The Three-City Study covers Boston, Chicago, and San Antonio and is being conducted by researchers at Harvard University, Johns Hopkins University, Northwestern University, Pennsylvania State University, and the University of Texas (Angel et al., 1998). The study has three major components. The first is a survey that plans to sample 2,800 low-income, primarily single-mother families from poor and moderate-income blocks in the central urban areas of each of the three cities beginning in 1999. Half of the families interviewed will be TANF recipients and half will not. Only families with children between birth and age 4 or between the ages of 4 and 14 will be included, reflecting a focus on the effects of welfare reform on young children and adolescents as well as adults. The longitudinal survey will interview families and collect information on adult and family well-being, employment and welfare outcomes, and child outcomes and the home environment, three times over a span of 4 years. Over the study period, information about children of almost all ages will be collected as the children grow into older age ranges.

The second component is an embedded developmental study, which designates 800 families with children aged 2 to 4 for whom more detailed parent and child interaction data will be gathered. Parent and child interactions will be videotaped and coded for further analysis. This part of the study will also interview out-of-home child care providers as well as the child's father. A time diary of the child's activities will also be collected. The 800 families will be drawn

Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×

from the 2,800 families in the main survey, and will be interviewed on the same schedule over 4 years.

The third study component involves ethnographic studies of 170 families in the three cities. It will track how welfare policies affect the daily lives and neighborhood resources of poor families. In-depth interviews will be conducted over the course of 2 years and will cover such topics as the respondent's life history and daily routines of life. This component also includes diary studies and observations of the participant when she goes to a social services office for assistance. Neighborhood data and indicators will also be collected under this component of the study.

In the third year of the project, a second cohort of families will be drawn from the same cities, neighborhoods, and population strata as the first cohort. Both TANF recipients and nonrecipients will again be included and the sampling frame will be identical in other respects to that used to draw the first cohort sample. The second cohort will be compared to the first to assess how the progress of welfare reform over time affects succeeding cohorts of families.

Fragile Families and Child Well-Being Study

The Fragile Families and Child Well-Being Study is a study of parents and newborns being conducted by a team of researchers from Columbia University and Princeton University (see McLanahan and Garfinkel, 1999). The study's focus is on unwed parents of newborns, although a sample of married parents of newborns will also be drawn. The purpose of the study is to understand the capabilities and situations of the fathers of the children, the dynamics of the relationships between the fathers and mothers, the well-being of the children in early life, and the effects of welfare and family policy changes on each of these areas.

The study is drawing a cohort sample of 4,000 children born to unwed parents and 1,000 children born to married parents. These samples are being drawn from 20 cities randomly selected from all cities with populations over 200,000. Mothers of the children are interviewed in the hospital after the births. If a father is not in the hospital for a child's birth, the interviewers ask the mother how the father can be contacted. The interviews are 30 minutes long for the mother and 40 minutes long for the father. Follow-up interviews are planned for each parent every year for the next 4 years. Interviews in the first 3 years will be conducted by telephone and will last 45 minutes. The fourth-year interview will be conducted in person.

Other Studies

There are many other studies of welfare reform in progress around the country which we have not separately described (see Table B-1 in Appendix B).

Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×

Many, if not most, are studies at the state level and represent some type of evaluation of a local program. Many are directly conducted by the state, while others are being conducted by national research organizations (Abt Associates, Manpower Demonstration Research Corporation, Mathematica Policy Research, Inc., etc.). Not all are focused directly on PRWORA or related waiver programs, however. Some are focused on programs now defunct (the JOBS evaluation), while others are evaluations of different types of reform programs that have not been implemented on a large scale (New Hope).

Assessment

The number and variety of welfare reform studies under way around the country is large. The majority of studies have welfare recipients, current or former, as their main population of interest, but a few have included nonrecipients as well (SPD, ANF-NSAF, Three-City Study, Fragile Families). Most studies use traditional household survey data for their databases, but some have significant ethnographic components in addition (New Hope, Three-City Study, Urban Change). Some studies have a strong focus on children, (ASPE Child Impacts State Studies, Three-City Study, SPD topical supplements), while others are focused on adults and their barriers to employment and self-sufficiency (the Michigan Women's Employment Study). Most are focused on individual families, but a few have neighborhoods and communities as their major unit of analysis and focus of interest (RAND Community Survey). Most studies are focused on single-parent families, but one has as its major goal the inclusion of absent fathers (Fragile Families). Almost all are focused on individual cities, counties, or states; only one is completely national in scope (SPD).

With respect to the general principles for welfare reform evaluation discussed in Chapter 2, these studies must be judged to be only partially successful in fulfilling those principles. All of the studies necessarily study the effects of a single bundled set of program services; there are none which have, as a major focus, the identification of the effects of the individual provisions in the reform bundle. Most studies that have welfare recipients as their population of interest necessarily do not address issues of diversion and entry; only a few of the studies incorporate those effects.

For the key issue of evaluation methodology, only a small number of evaluations have an experimental design (New Hope, the ACF studies); the rest are nonexperimental. In the latter category, only a few have explicitly outlined comparison groups (Urban Change, Three-City Study), and those studies will need to assess the threats to cohort comparison designs that we have discussed. Other studies (ANF-NSAF, Fragile Families, SPD) will have sufficient cross-state variation to permit an evaluation based on cross-sectional or cross-sectional-panel designs, and these studies will face both the problem of measuring what the different state policies are—a fundamental data problem we identified in

Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×

Chapter 2 and above—as well as controlling for other state differences that might confound the effects of welfare policy.

To a great extent, the importance of these issues will be clear only when the results of the studies are available. If the diverse populations, cities, and evaluation designs yield a reasonably coherent and consistent set of findings that add up to a credible overall picture of the consequences of welfare reform, the issues and potential difficulties we have noted will be of only modest importance. In this case, the great diversity of the studies will work to the advantage of the overall national evaluation effort by demonstrating the robustness and consistency of estimates across a wide range of approaches and across different populations for study. If, on the other hand, the studies yield results that vary greatly and in ways that are not easily explainable by obvious differences, a mixed message will result and some assessment of the relative reliability of the different approaches will have to be undertaken. This could be a formidable challenge.

CONCLUSIONS

The state grantee studies funded by ASPE have made a good start in what is expected to be a longer term effort to assess the effects of welfare reform policies on the low-income population. To a great extent, the grantee studies should be viewed as initial investments in building data capacity at the state level, leading to improved administrative and survey data. However, much remains to be done for these data sets to fulfill their potential. As monitoring studies, the ASPE grantee designs have significant interest and should yield interesting findings on the outcomes of welfare leavers. While the cross-state comparability in these 14 studies will be much greater than among the welfare leaver studies conducted previously, thus strengthening their findings as a whole, many improvements in such comparability need to be made. The grantee designs also have thus far not proposed to identify the most important subgroups in the welfare recipient population, namely, long-term and short-term recipients and those with strong and weak work histories. As evaluation studies, the ASPE grantee designs face considerable challenges in demonstrating the credibility of a recipient-based cohort comparison design. Finally, the grantee designs thus far concentrate almost exclusively on welfare leavers; they do not include an examination of nonapplicants and divertees or of families who are still on the rolls (stayers).

The other welfare leavers studies that have been completed around the country have the same strengths and weaknesses, and the ASPE grantee studies promise to be of higher general quality. There are many other welfare reform projects under way around the country as well, both within and outside of government, and these will, when completed, yield a large number of assessments of welfare reform using different approaches and different data. Most of these projects also face significant evaluation challenges, which we have identified. Whether the set of findings that will emerge from these diverse studies will yield a consistent and coherent picture of the effects of welfare reform remains to be seen.

Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×
Page 50
Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×
Page 51
Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×
Page 52
Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×
Page 53
Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×
Page 54
Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×
Page 55
Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×
Page 56
Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×
Page 57
Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×
Page 58
Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×
Page 59
Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×
Page 60
Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×
Page 61
Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×
Page 62
Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×
Page 63
Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×
Page 64
Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×
Page 65
Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×
Page 66
Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×
Page 67
Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×
Page 68
Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×
Page 69
Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×
Page 70
Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×
Page 71
Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×
Page 72
Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×
Page 73
Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×
Page 74
Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×
Page 75
Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×
Page 76
Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×
Page 77
Suggested Citation:"3 ASPE Leaver Studies and Other Current Research on Welfare Reform." National Research Council. 1999. Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/9672.
×
Page 78
Next: 4 Recommendations »
Evaluating Welfare Reform: A Framework and Review of Current Work, Interim Report Get This Book
×
Buy Paperback | $68.00 Buy Ebook | $54.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA) of 1996 fundamentally changed the nation's social welfare system, replacing a federal entitlement program for low-income families, called Aid to Families with Dependent Children (AFDC), with state-administered block grants, the Temporary Assistance for Needy Families (TANF) program. PRWORA furthered a trend started earlier in the decade under so called "waiver" programs-state experiments with different types of AFDC rules-toward devolution of design and control of social welfare programs from the federal government to the states. The legislation imposed several new, major requirements on state use of federal welfare funds but otherwise freed states to reconfigure their programs as they want. The underlying goal of the legislation is to decrease dependence on welfare and increase the self-sufficiency of poor families in the United States.

In summer 1998, the Office of the Assistant Secretary for Planning and Evaluation (ASPE) of the Department of Health and Human Services (DHHS) asked the Committee on National Statistics of the National Research Council to convene a Panel on Data and Methods for Measuring the Effects of Changes in Social Welfare Programs. The panel's overall charge is to study and make recommendations on the best strategies for evaluating the effects of PRWORA and other welfare reforms and to make recommendations on data needs for conducting useful evaluations. This interim report presents the panel's initial conclusions and recommendations. Given the short length of time the panel has been in existence, this report necessarily treats many issues in much less depth than they will be treated in the final report. The report has an immediate short-run goal of providing DHHS-ASPE with recommendations regarding some of its current projects, particularly those recently funded to study "welfare leavers"-former welfare recipients who have left the welfare rolls as part of the recent decline in welfare caseloads.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!