support standards, unmarried fathers could provide almost half of the welfare payments that female-headed families expect to receive. The early results from the pilot studies of the Parents' Fair Share Program in nine sites around the nation showed that two-thirds of noncustodial fathers subject to requirements participate in an employment or training or peer-support activity, and many found jobs on their own (Bloom and Sherwood, 1994).
Horn called attention to the great differences in the emphases of social science research on coresident fathers and on absent fathers: there is interest in whether the coresident fathers help with child care and how well, whereas absent fathers are interesting only insofar as they have money and transfer some of it to the mother. The message seems to be "If you're in the home, pay attention; if you're out of it, just pay." Given the large and growing number of absent fathers, Horn believes more research is needed on what they do to help with childrearing and how programs and policy could encourage a larger contribution. We do not know, for example, how the father's involvement is associated with payment of financial support. It is also very important to learn more about the effects of paternal involvement on the child at different ages. From the limited evidence available, it appears that unmarried fathers who do get involved with their children do so only for the first few years of their lives. There may not be much net benefit for the child in having such involvement followed by early abandonment.
Bumpass added that little is known about such basics as whether unwed fathers who do get married are marrying the mothers of their children. O'Neill called for more research on the effects of welfare and nonmarital fatherhood on men's lives: if they are not expected to take care of their own children, does that lessen social pressures for other types of responsible behavior?
AFDC rules about child support pass-through payments may have discouraged provision of direct financial support by absent fathers and encouraged substitution of informal sources of support. Provisions of the AFDC-Unemployed Parent (AFDC-UP) program may have discouraged marriage or coresidence. Several workshop participants proposed that design of new welfare programs should encourage child support payments by treating them comparably to the mother's earnings and not reducing the welfare benefits by their full amount.
RESEARCH AND EVALUATION NEEDS
Evaluations of State Reforms Under AFDC Waivers
In Isabel Sawhill's view, AFDC waivers were sought and granted in the 1990s, not so much to inform national policy as to get around it. Beginning in the last years of the Bush administration and increasingly during the Clinton administration, waiver applications were motivated less by the desire to learn from controlled experiments and more by a desire to "bring devolution in by the back
door." Previously, waivers had usually allowed testing of new rules on a small subset of a state's caseload, with the majority subject to the old AFDC rules and serving as a comparison group. In recent years, waivers have allowed states to change several rules at once for almost their entire caseload. The 1996 act allowed existing waivers to continue and gave states a great deal of freedom to change rules on their own. It is far from clear whether they will use this freedom to test innovations in some counties for comparison to others and to evaluate the effects as they were required to do under the old waivers, or whether they will simply adopt new programs statewide and see what happens.
Thomas Corbett presented the early results of a review of what can be learned from evaluations of the demographic effects of state waivers to inform the design of new programs (Maynard et al., 1997). The most direct attempts to influence fertility decisions, family cap provisions, have become common only in the last few years. By January 1996, 20 states had such waivers approved, but in all but 5 cases the waiver was less than 36 months old, so there is as yet less experience to evaluate than is the case with work and training requirements and other modifications in eligibility rules.
A second common type of waiver with potential demographic effects identified by Corbett and his colleagues is that reforming the AFDC-UP program, which offers benefits to two-parent families. Many states reduced the 100-hour rule restricting work levels of earners in AFDC-UP families, and other states extended eligibility to families with more spotty employment histories than had heretofore been the case. These reforms are aimed at increasing provision of benefits to married couples and hence indirectly have demographic effects. Like the family cap waivers, there are no evaluation results to date of these waivers.
Although presumably evaluations of these waivers will continue under the Personal Responsibility and Work Opportunity Reconciliation Act, Corbett et al. caution that evaluation will not be easy. Family caps are embedded in complex packages of reforms. For example, the typical waiver entailed 10 potentially significant changes in AFDC rules; Wisconsin has 15 separate demonstrations going on at once. This complexity makes it very difficult to get a "clean" test of the effects of family caps or of other provisions on behavior, because the effects of any one provision cannot be separated from the others. Quick evaluations are unlikely to be worth much, in Corbett's view. Researchers need to construct and update databases that will allow identification of exactly what benefits persons with particular characteristics are eligible for at different times; they then need to translate program rules into a context of incentives.
A further complexity is introduced by the fact that social policy reforms both reflect and shape public attitudes. Rolston commented on the potential "contamination" in the early evaluations of the New Jersey family cap. The new provision was widely announced, but it was a complex change in already complex rules, and the fact that it applied in only a few counties was probably not clear, so
behavior in comparison areas could have been affected at least as much as behavior in treatment areas.3 O'Neill described problems in getting the data on births for the New Jersey evaluation. Changes in eligibility that are intended to affect demographic behavior may also affect the incentives of both clients and program staff to report events, as well as the ability of the system simply to process reports. With complicated program reforms, a great load is put on administrative data systems, which may become less reliable as sources of data for evaluations.
AFDC waivers required either random assignment of individuals to programs or else a quasi-experimental evaluation model, with changes introduced in one or two counties, to be compared with other counties where no changes had been introduced. Moffitt felt these quasi-experimental designs were ineffective; too many other differences could exist between treatment and comparison areas, or local economic conditions affecting both could swamp any effects of the program changes in short-term evaluations. The great variation across states and over time, as described by Corbett, could be used to good effect by researchers, if they could link accurate program descriptors to good-quality, household-level data.
Does Research Matter?
Several times during the workshop, participants (both those engaged in producing research and those engaged in producing laws and regulations) raised the question whether research on these issues really matters for policy. The prevailing view was that research matters, eventually; it is worth figuring out why studies have disagreed, evaluating the new program changes, and improving data and methods. The welfare reform debate does not end with the passage of federal legislation, nor even with the first round of changes at the state level.
Moffitt distinguished between the academic studies (typically analyses of individual-level data from panel surveys that compare sample households in different states over time) and the evaluations of specific programs. The former can test a wider range of variation of benefits and rules, controlling for varying economic conditions and social and political background, thus presumably allowing more confident statements about the likely effects of changes being considered by a state. But this potential cannot be fully exploited now, according to Moffitt, because the results of academic studies are so mixed, and no one can say with confidence why the results are mixed. Studies differ in the datasets analyzed and in the models used to overcome confounding effects. Far too little invest
ment has been made in what he called the "three R's": replications (of a particular result), studies of robustness (reanalyzing data to see if different reasonable choices about the model lead to very different results), and reconciliation studies (reanalyzing the data used by one study with the models used by others, to see how much of the difference in results can be explained by differences in models).
Another problem in conducting research on welfare is the difficulty in determining what policies are in place. It is exceptionally difficult to figure out exactly what AFDC rules prevailed in different states (and counties within states, since the waivers provide for within-state variation) at what times. Lacking that information, analysts cannot use the national survey data effectively for analyses of impact in an era of ever-widening policy variation. There are various lists of the number, timing, and type of AFDC waivers that have been granted, but these are not kept in a form that would allow researchers to map them onto the location codes in datasets such as the National Survey of Families and Households, the National Longitudinal Surveys (NLS), the Panel Study of Income Dynamics, the Survey of Income and Program Participation, etc. Trying to figure out exactly what rules prevailed in a particular location at a given time is an arduous task, as Bumpass could testify, and he urged the U.S. Department of Health and Human Services to maintain a county-level database available to all as a way to facilitate useful research.
This problem of uncertainty in matching survey data with the welfare rules applying at a particular place and time will only become worse as state programs diverge even more under the 1996 act. Since reforms can now be introduced by states without any requirement to collect data and report to the U.S. Department of Health and Human Services, there will be no federal agency from which, in theory, researchers could collect the information needed for a geocoded dataset of welfare rules.
Currie noted that newly available data from the National Longitudinal Survey Child-Mother File will allow long-term studies of effects of childhood participation in AFDC, food stamps, Medicaid, and WIC, with representative samples, if continued waves of the panel study are funded. Another approach to data collection at a national level is adding supplementary questions to panel studies that contain information about welfare participation from previous rounds, as was done for a 1995 supplement to the Panel Study of Income Dynamics.
Several participants called for more research and evaluation on effects of the welfare system on children. There was some discussion about the need to agree on sets of outcome measures for children, although Daniel Lichter argued that the child development literature already provides a good set of outcome variables with well-studied psychometric properties, which the welfare evaluation literature has not used. Ron Haskins listed some of the provisions for monitoring and evaluation included in welfare reform bills. These include efforts to improve state collection of data from child protective services systems and expansions of the Survey of Income and Program Participation sample for state-level estimates.