10
Administrative Data on the Well-Being of Children On and Off Welfare

Richard Barth, Eleanor Locklin-Brown, Stephanie Cuccaro-Alamin, and Barbara Needell

The Personal Responsibility and Work Opportunity Reconciliation Act of 1996 (PRWORA) significantly altered the way the United States provides assistance to its neediest citizens. The act eliminated the federal entitlement to services that existed under the Aid to Families with Dependent Children (AFDC) program and replaced it with the block grant program Temporary Assistance for Needy Families (TANF). TANF provides temporary financial assistance while recipients make the mandatory transition from welfare to work. These efforts to move adult recipients toward self-sufficiency may have consequences for the well-being of their children (Collins and Aber, 1996; Zaslow et al., 1998). Potential implications include changes in children’s health, safety, education, and social competence.

Whether these consequences are positive or negative depends on how reforms impact family income, parenting practices or parental stress, and access to services (Collins and Aber, 1996; Zaslow et al., 1998). For example, economic hardship related to loss of benefits or other supports may complicate families’ efforts to provide supportive environments for their children (Knitzer and Bernard, 1997). Increased parental stress related to economic, employment, or child care difficulties also may lead to poor parent/child interactions or exacerbate existing mental health conditions such as depression or substance abuse, thereby increasing the risk of negative outcomes (Knitzer and Bernard, 1997; Zaslow et al., 1998). Child health and safety also might be compromised if TANF alters access to non-TANF services such as health and childcare.

In contrast, positive changes in these areas may be beneficial to children and



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 316
Studies of Welfare Populations: Data Collection and Research Issues 10 Administrative Data on the Well-Being of Children On and Off Welfare Richard Barth, Eleanor Locklin-Brown, Stephanie Cuccaro-Alamin, and Barbara Needell The Personal Responsibility and Work Opportunity Reconciliation Act of 1996 (PRWORA) significantly altered the way the United States provides assistance to its neediest citizens. The act eliminated the federal entitlement to services that existed under the Aid to Families with Dependent Children (AFDC) program and replaced it with the block grant program Temporary Assistance for Needy Families (TANF). TANF provides temporary financial assistance while recipients make the mandatory transition from welfare to work. These efforts to move adult recipients toward self-sufficiency may have consequences for the well-being of their children (Collins and Aber, 1996; Zaslow et al., 1998). Potential implications include changes in children’s health, safety, education, and social competence. Whether these consequences are positive or negative depends on how reforms impact family income, parenting practices or parental stress, and access to services (Collins and Aber, 1996; Zaslow et al., 1998). For example, economic hardship related to loss of benefits or other supports may complicate families’ efforts to provide supportive environments for their children (Knitzer and Bernard, 1997). Increased parental stress related to economic, employment, or child care difficulties also may lead to poor parent/child interactions or exacerbate existing mental health conditions such as depression or substance abuse, thereby increasing the risk of negative outcomes (Knitzer and Bernard, 1997; Zaslow et al., 1998). Child health and safety also might be compromised if TANF alters access to non-TANF services such as health and childcare. In contrast, positive changes in these areas may be beneficial to children and

OCR for page 316
Studies of Welfare Populations: Data Collection and Research Issues families (Collins and Aber, 1996; Zaslow et al., 1998). Specifically, policy changes might lead to improved outcomes for children whose parents become employed successfully. Children might benefit from access to more comprehensive health care, opportunities to observe parents coping effectively with work demands, higher educational aspirations and achievement, and exposure to parental peers who are engaged in more prosocial activities. PRWORA currently is being praised by some, and criticized by others, for moving nearly 1.7 million recipients from welfare to work. However, until the impact of these reforms on child well-being is known, such celebrations are premature. Even if reforms succeed in moving mothers from welfare to work, if this in turn has negative consequences for children, its effectiveness will need to be reevaluated in light of these costs. Prior to passage of welfare-to-work legislation, more than 40 states received waivers to experiment with welfare-to-work programs. Experimental evaluations of these initiatives now under way will provide valuable information about possible effects of certain aspects of PRWORA. Most, however, focus primarily on adult outcomes such as changes in income, employment, family formation, and attitude, and cover only a limited number of child outcomes (Research Forum on Children, Families, and the New Federalism, 1999; Yaffe, 1998). Additionally, those child outcomes “typically lack depth and uniformity” (Yaffe, 1998). Several states (e.g., Connecticut, Florida, Iowa, and Minnesota) are looking at child outcomes resulting from parental participation in AFDC waiver conditions that approximate TANF and that eventually will yield child-level outcome data. The Administration for Children and Families (ACF), Office of the Assistant Secretary for Planning and Evaluation (ASPE) in the Department of Health and Human Services has implemented the Project on State-Level Child Outcomes to assist waiver states in using administrative data to expand these child outcome measures and make them comparable across states. Although this uniformity will allow for the assessment of different state models, their utility is limited by their small sample sizes. In particular, small sample sizes make subgroup comparisons difficult and prohibit evaluation of rare events such as foster care placement or child mortality. Current evaluations of state welfare-to-work initiatives under PRWORA suffer from similar limitations. Given these limitations, researchers increasingly are turning to administrative data to try to gauge the relationship between receiving income assistance services under TANF and child well-being. The purpose of this paper is to assist researchers in addressing the following questions: What are the key policy issues and related domains of child well-being associated with the impact of PRWORA? What are the opportunities and challenges in using administrative data to measure the impact of PRWORA on children?

OCR for page 316
Studies of Welfare Populations: Data Collection and Research Issues KEY POLICY ISSUES AND CHILD WELL-BEING Data about social welfare programs for children are fragmented and incomplete and lack a cohesive framework or policy ownership. Given this, it is important to identify the priority of policy issues and the specific research questions about PRWORA and child well-being when research is conducted. These early steps will assist in then defining the population of interest, the appropriate methodology, and the data sources available (National Research Council, 1999). Key Policy Issues Several key policy issues have been identified for states examining the impact of welfare reform (Child Trends, 2000a; National Conference of State Legislators, 1999). First is the need to understand the specific components of an individual state’s welfare reform program. This is especially important given the diversity across states in the implementation of PRWORA and corresponding use of TANF funds. Specific components left up to the states include mandated work, time limits, and sanctions. The implementation of these components may impact the outcomes for families and children; for example, mandated work without childcare may lead to cases of child neglect. Also needed is an examination of the interaction between welfare reform—that is, the change from AFDC to TANF— and other social welfare programs such as Medicaid, Children’s Health Insurance Program (CHIP), the Food Stamps Program, Special Supplemental Nutrition Program for Women, Infants, and Children (WIC), Child Protective Services, and foster care, which also may be undergoing changes. Although efforts over the past two decades have sought to delink AFDC to other social welfare programs, in practice and for individual families these programs remain interwoven. The eligibility, availability, and accessibility of these supplemental services can impact family outcomes in conjunction with, or separate from, TANF enrollment, exit, and reentry. Understanding TANF’s impact on children requires thoughtful specification of the child well-being outcomes TANF might influence so that investigators can fashion data collection efforts to maximize measurement of the predicted impact. Research about child outcomes can guide questions about current implementation and future programming using the flexibility of TANF funds. Domains of Child Well-Being Child Trends, a nonprofit, nonpartisan research center, along with colleagues at the federal and state levels participating in the Project on State-Level Child Outcomes, have offered a conceptual framework that organizes and clarifies the pathways through which welfare reform can impact children. This framework is displayed in Figure 10–1.

OCR for page 316
Studies of Welfare Populations: Data Collection and Research Issues FIGURE 10–1 Child Trends, Inc. conceptual framework for child well-being—how welfare policies might affect children. SOURCE: Child Trends (1999).

OCR for page 316
Studies of Welfare Populations: Data Collection and Research Issues In this conceptual framework, state policies regarding PRWORA will directly effect family income, employment, family formation outcomes such as marriage or out-of-wedlock births, and attitudes of the adults in the case, such as self-esteem or feelings of being in control of one’s life. Changes in these four areas will further influence the family and child’s psychological well-being, the stability of the child’s home, involvement of the absent parent (through child support enforcement), use of health and human services, and consumption of goods and services spent on the child. As a single mother copes with job training, new employment, transportation problems, childcare dilemmas, and confusion about eligibility for supplemental services, the child’s physical home environment, relationship to the parent, and time with the parent can be expected to change. Such changes in the capacity and day-to-day schedule of maternal care-taking very likely will, in turn, impact the child’s educational experiences, health and safety, and social and emotional adjustment. This conceptual framework evinces the multifaceted ways that welfare programs can influence a range of child outcomes. The framework offers a critical organizing logic about how welfare reform impacts children, one that can be used to shape research agendas and frame research questions. The framework was developed to guide survey researchers studying about welfare, but it also serves as a useful reference for the inclusion of administrative data in research initiatives. Administrative data are far more useful for estimating some components of this model than others. Specifically, administrative data rarely contain in-depth information about parental psychological well-being or home environment, parenting practices, or social and emotional adjustment of children. Survey research is much better suited to these components. Administrative data can, however, be used to estimate family stability and turbulence through the inclusion of information about household changes and movement in and out of foster care. Administrative data also can help explain the changes in the utilization of supplemental health and human services—especially Medicaid, CHIP, Food Stamps, or WIC—which could be an indirect result of welfare changes. Finally, administrative data have the potential to measure health and safety outcomes, including abuse and neglect, injury, and mortality. The remaining sections of this paper are developed to assist researchers in defining indicators for domains of child well-being and to clarify substantive issues that must be considered in applying these indicators toward addressing key evaluation questions about the impact of welfare reform. The paper is divided into sections roughly corresponding to the constructs and domains of child well-being offered by Child Trends (1999): (1) health, (2) safety (child welfare), (3) education, and (4) social and emotional adjustment (juvenile justice). For each domain we will identify information that directly describes or reflects on child well-being. Each section also includes a description of key data that are available to inform us about children’s outcomes. For each domain we discuss exemplary

OCR for page 316
Studies of Welfare Populations: Data Collection and Research Issues efforts to use these data for evaluation of welfare reform. Finally, we conclude by discussing some of the scientific sensibilities that should be respected in the use of such data during research on welfare reform, including a discussion of linkages between population surveys and administrative data. CHILD HEALTH Access to health care services is a central consideration in the assessment of welfare reform, as these reforms change existing relationships among income, employment, and insurance of health care services for poor families and children (Child Trends, 2000a; Darnell and Rosenbaum, 1997; Moffitt and Slade, 1997; Schorr, 1997). Measures of child health typically emphasize access to care as an important measure, recognizing that health care is necessary but not sufficient for positive child health outcomes (Gortmaker and Walker, 1984; Margolis et al., 1997; Andrulis, 1998). Parents access health care for their children through several paths. Many children receive health insurance provided by their parent’s employer. However, some children of working parents may not have employer-sponsored health plans, and children of nonworking parents certainly do not have this benefit. These children of low-income or nonworking parents are eligible for services paid by publicly funded programs such as Medicaid or the new CHIP. The PRWORA legislation did not significantly alter Medicaid eligibility, and CHIP is designed to reach more of these uninsured children. Yet in July 1999 an estimated 4.7 million uninsured children were eligible for Medicaid but not enrolled (Families USA, 1999). Many states are beginning to track children’s enrollment in Medicaid and CHIP, implement outreach efforts to increase CHIP enrollment, and expand Medicaid and CHIP income-level guidelines (Families USA, 1999; Children’s Defense Fund, 1998). The actual health services the child receives are also major determining factors in child health status. Examples of services may include (1) preventive care such as immunizations or dental care; (2) diagnostic screening such as vision and hearing screening, or weight for height measures; and (3) treatment for chronic conditions and disability, with corresponding risk of secondary disability. State policies about welfare reform have the potential to change, positively or negatively, the family environment where health behaviors and health decisions are carried out (Willis and Kleigman, 1997; O’Campo and Rojas-Smith, 1998; Brauner and Loprest, 1999). For example, even if a child is enrolled in Medicaid or CHIP, PRWORA work requirements may constrain a parent’s ability to access health care. When access to health care services is limited, either through limited availability or limited utilization of services, children’s health could suffer. Alternatively, the work requirements could encourage the parent to secure a job that includes health insurance (gaining access to health care), which may mean the family is able to utilize more services.

OCR for page 316
Studies of Welfare Populations: Data Collection and Research Issues Access and utilization of services are interesting for evaluation purposes because they are believed to contribute to the actual health of the child. However, direct measures of child health outcomes are also needed to measure the effects of welfare reform on children. Direct measures of child health outcomes are scarce, however. Often researchers have to rely on indicators of health status. Recent discussions about welfare reform and health suggest some indicators to measure child health status. Children in poverty are more likely to be undernourished, iron deficient, or lead exposed (Geltman et al., 1996). Several measures such as infant mortality, injury, and the use of preventive medical services can be good indicators of child health status (Pappas, 1998). Starfield’s Child Health and Illness Profile (Starfield et al., 1993) combines several of these indicators into a bio psycho social developmental assessment but is not found in administrative data sets. Even in survey research, questions about child health status may be limited to asking parents to rate their child’s health from excellent to poor (Child Trends, 2000b). Thus, when using administrative data about child health status, it is often necessary to use measures of health services as markers for positive outcomes such as immunizations, enrollment in health plans, or preventive screening, along with indicators of actual outcomes such as infant mortality, low birthweight, blood lead levels, or adolescent substance abuse. Our purpose here is to identify a reasonably comprehensive set of child health indicators available in at least some administrative data that are relevant to changes in welfare policy because they address health access or status of children. Healthy People 2000, an initiative begun in 1990 by the U.S. Department of Health and Human Services, set health objectives for the nation, including child health status objectives (National Center for Health Statistics, 1996). Over the years, the initiative has prompted state and local communities to develop their own similar objectives and indicators of progress toward achieving them. As a result, the Healthy People 2000 effort has created a set of fairly common measurements of child health across a range of public and private health programs. For example, one of the Healthy People objectives is to reduce infant mortality. This supports the inclusion of infant mortality reduction as part of most state health objectives, and as part of many state and local programs targeted toward women and children. At the federal level, the Maternal and Child Health Bureau (MCHB) identified 18 of the Healthy People 2000 objectives that specifically relate to women and children. Of these, 15 are child health status indicators that can be used to measure impact of welfare reform (Maternal and Child Health Bureau, 1996). Table 10–1 presents these indicators, along with several others, as recommendations for measuring utilization of health services as well as child health status. For each indicator, we describe whether data generally are available at the individual level or aggregated to some larger population. We also identify suggested data sources for these indicators. Many of these data sources are being used in current research about child health (Vermont Agency of Human Services, 2000; Child Trends, 1999).

OCR for page 316
Studies of Welfare Populations: Data Collection and Research Issues TABLE 10–1 Suggested Child Health Indicators Indicator Level Data Sources Medicaid eligibility/enrollment/services Individual Medicaid data files CHIP eligibility/enrollment/services Individual CHIP data files Number/percent uninsured Population State dept. of insurance SSI benefits Individual SSA data Infant mortality Individual Vital statistics Low birth weight Individual Vital statistics HIV infection among women with live births Individual Vital statistics Prenatal care Individual Vital statistics Newborn screening Individual Vital statistics Birth defects registry State data system for newborn screening Early and Periodic Screening, Diagnosis, and Treatment (EPSDT) Population Medicaid services/payment data Identification of hearing impairments Individual State data system for newborn screening   Population Program evaluation data Immunizations Individual Medicaid, state immunization registry   Population Program evaluation data Blood lead levels Individual EPSDT, clinic record   Population Program evaluation data Dental caries Individual Medicaid     Public health department Unintentional injuries Individual Vital statistics, hospital discharge     School-based health centers Child homicide Individual Vital statistics Adolescent suicides Individual Vital statistics Adolescent substance use rates Individual Vital statistics   Population Hospital discharge/health department     School-based health centers     Program evaluation data STD rates among youth Population Hospital discharge     Program evaluation data     School-based health centers Adolescent pregnancy rates Individual Vital statistics Of the data sources identified in Table 10–1, the core indicators come from Medicaid and vital statistics. The following two sections discuss these two sources of data, how they can be used in studies of welfare reform outcomes on children, and some methodological issues in their use.

OCR for page 316
Studies of Welfare Populations: Data Collection and Research Issues Medicaid Data Data from Medicaid eligibility, enrollment, and claims and the new state CHIP can be linked to provide longitudinal tracking of a child or family’s health care services or lack of services. For example, a state could track the Medicaid or CHIP enrollment of a child whose mother left AFDC. Since it is unlikely that many families leaving TANF will promptly go to jobs with sufficient health benefits or wages above the Medicaid and CHIP guidelines, that is, 200 percent of the federal poverty level in most states but 350 percent of the federal poverty level in some Medicaid expansion programs, lack of Medicaid coverage of a child in an AFDC/TANF leaver family may indicate that the child is at risk of having no health care coverage. If Early and Periodic Screening and Diagnostic Testing (EPSDT) services also are recorded in the Medicaid files, similar linkages with welfare data will allow tracking of the utilization of preventive services for these low-income children. Linked administrative data from AFDC/TANF and Medicaid have also been used as the sample frame for complementary survey research, which can gather indicators of health status or measures of health care utilization and provide more in-depth measures. For example, the Next Generation, a project conducted by Manpower Demonstration Research Corporation (2000), will use survey data from 10 studies to obtain a more comprehensive perspective about the effects of welfare reform on health outcomes. Variables about health will be measured through survey questions, but the project also will include the existing administrative data used in each of the 10 studies. Using administrative data from Medicaid and CHIP (or other health-related supplemental services such as the Food Stamps Program or WIC) requires attention to a variety of considerations. One must consider the populations in the data sets in relation to the population of interest for the study. Specifically: Determining what cases are to be included in the population of study. Study populations that can be drawn from Medicaid or CHIP files include: applicants, eligible cases, open cases, closed cases, cases closed with high risk, timelimited or sanctioned cases, or reentry cases. Within the group of eligible children are several subgroups that might be of interest. One group for Medicaid is those children actually enrolled. This subgroup of enrolled children includes a second subgroup of children receiving services. This group is not representative of all children enrolled, or all children eligible, or all low-income children in need of health care. Medicaid data can be used to extend the analysis of the impact of welfare reform beyond the TANF population because the Medicaid eligibility pool is larger than the TANF eligibility pool. For example, California uses data files on Medicaid recipients as the core of its data sharing/data integration initiatives (National Conference of State Legislators, 1999). This strategy can allow evalu-

OCR for page 316
Studies of Welfare Populations: Data Collection and Research Issues ators to track service provided across time and programs to low-income children and families. However, Medicaid administrative data can provide data on some of these populations, but not all (i.e. Medicaid administrative data do not represent the entire population of children eligible for Medicaid). Public services data tend to overrepresent families at greatest risk. Findings must be interpreted with this in mind. If a family or child leaves TANF and does not appear on Medicaid enrollment files, this does not necessarily mean the child does not have access to health care as they could be covered by private insurance (Child Trends, 2000a). Beyond data on eligibility and enrollment, the actual Medicaid or CHIP benefits within a state also should be considered part of the evaluation. State CHIP programs can vary by age, geographic area, disability status, or calculation of income. A thorough understanding of the administrative data being used is necessary. One consideration is whether historical AFDC or Medicaid data were defined the same way across the years. In a cross-state context, one must consider possible differences in programs, definitions of data, caseload characteristics, and take-up rates in each state. Within state differences in each of these are also possible. The dynamics of changing caseloads to determine whether changes are due to differences in entries to health services or differences in lengths of stay in those services need to be clarified (Greenberg, 1998). Administrative data systems for Medicaid often are inadequately automated, even though provision of Medicaid benefits to needy families and children are highly dependent on automated systems. These systems may erroneously terminate a family from Medicaid. Also, eligibility systems typically are not part of the Medicaid division’s information system, but reside elsewhere in state government. Because current technology dollars are being spent on TANF automated data systems, there may be some migration away from more archaic Medicaid data (Ellwood, 1999). When designing research about children’s access to health care services, it is important to remember that a family or adult parent can be dropped from Medicaid but the child can remain eligible. Data linkage and confidentiality issues also arise: How cases in the two files are linked requires the establishment of clear decision rules that are appropriate to specific research questions. There are inherent challenges to linking welfare data to Medicaid data because welfare data are

OCR for page 316
Studies of Welfare Populations: Data Collection and Research Issues case based (and can include a family or group of siblings) and Medicaid data are individual based (Ellwood, 1999). It is useful to maximize the use of common health identifiers. In some states, such as North Carolina, a common health identifying number is used across a range of data sets, from vital statistics to disease registries (North Carolina State Center for Health Statistics, 1997). Where the health identifying number and social services number can be linked together, one can evaluate a child’s experiences and outcomes with both health and social service programs. Examining claims data under Medicaid or CHIP requires that issues of confidentiality are responsibly addressed. Many states, such as California, Maryland, Kentucky, and Tennessee, are already addressing these concerns through data sharing and data warehousing projects: (National Conference of State Legislators, 1999). In addition to these concerns about administrative data, identification of the relevant research questions is critical in guiding the analysis plan and selection of relevant data sets. The question of whether regulations make health care services available to all children who need them could be answered with eligibility data. The question of whether children leaving TANF continue to get needed health services cannot be answered with eligibility or enrollment data. That question only can be answered with service utilization data. The question of whether children exiting TANF are continuing to get timely immunizations could be answered by Medicaid services data or by separate immunization registries within a state (Child Trends, 1999). Another relevant research question to include would be whether the population of cases had changed since PRWORA was enacted. Will you study AFDC populations before PRWORA, or just those TANF cases after the legislation was implemented? This would require including AFDC and TANF cases in the research. Beyond analysis of the data about AFDC/TANF and Medicaid, the Food Stamps Program, or WIC, research should include questions about barriers to supplemental services for families exiting welfare. One possible barrier is the continued linking of welfare to these supplemental services, despite efforts over the past decade to delink regulations about the programs. In practice, and for individual families, these programs remain interwoven. Another barrier is the complicated eligibility rules for services to support families leaving TANF and the media about the program that might affect whether families think they are eligible or not (Ellwood, 1999). Finally, a research question of interest would be “Upon exiting TANF, do families drop supplemental services, add supplemental services, or maintain existing levels?”

OCR for page 316
Studies of Welfare Populations: Data Collection and Research Issues pressured to bring new resources into the household or to, at least, find resources that would allow them to be less dependent on their families for food, clothing, and entertainment. The most common approach to assessing criminal justice involvement is to study “arrest records.” This is the device used in most studies of the transition from child welfare programs to juvenile justice involvement (e.g., Widom, 1991; English and Widom, 1999). The potential drawback of arrest records is that they reflect the combined behaviors of juveniles and criminal justice systems. This is counterbalanced by the fact that they are generally considered to be more useful than conviction records because convictions or incarcerations are determined by so many other factors—especially for less violent crimes. Still, convictions or incarcerations can be used if the theoretical relationship between welfare participation and crime suggests there would be higher rates of major crimes. Incarcerations in state training programs have been shown to be sensitive enough to pick up differences between groups that did and did not obtain ongoing child welfare services following a child abuse investigation (Jonson-Reid and Barth, 2000). Juvenile justice data also can be obtained from a variety of settings, depending on the geographic locus of the study. At the local level, youth often are remanded to juvenile detention and county camps and ranches. At the state level, they may attend a training school or youth authority program. In more populous counties, they generally have greater capacity to hold more youth who commit more serious offenses at the local level, whereas more rural areas may use the statewide facilities to a greater extent. Statewide facilities often have their own databases, which include substantial additional information collected about the child at intake. This makes such information particularly useful in trying to explain exit patterns and the path of services once in the training program. Some juveniles are tried as adults and others may have their records sealed for a variety of offenses. Still, these remain the exception and they are unlikely to bias study results or affect interjurisdictional comparisons as long as reasonable sample sizes are maintained. Although these authors were unable to identify any studies that have directly tested the relationship between parents’ welfare participation and children’s juvenile justice involvement, one important study matched juvenile justice data with survey data from the Moving to Opportunity (MTO) experiment. In the MTO, a total of 614 families living in high-poverty Baltimore neighborhoods were assigned into three different “treatment groups”: experimental group families received housing subsidies, counseling, and search assistance to move to private-market housing in low-poverty census tracts (poverty rates under 10 percent); Section 8-only group families received private-market housing subsidies with no constraints on relocation choices; and a control group received no special assistance under the MTO. The impact of this “treatment” on juvenile arrests was then assessed (Ludwig et al., 1999). (The authors also tested models that used convictions instead of arrests and found similar results.)

OCR for page 316
Studies of Welfare Populations: Data Collection and Research Issues The study also cast light on the utility of a variety of indicators of juvenile justice involvement, finding that false arrests are likely to be crime specific and disproportionately involve charges such as disorderly conduct, resisting arrest, and assaulting a police officer. Second, they replicated their analysis using convictions instead of arrests, assuming that these show less variation across neighborhoods in false convictions than arrests because juvenile prosecutions are handled at the county level and arrests are made by local police. LINKING WELFARE AND CHILD WELL-BEING DATA Despite the benefits of using linked longitudinal administrative data, the work is complex and the level of effort and skill required is easily underestimated. Linking across data systems poses many challenges. Linking is accomplished by matching unique identifying information such as Social Security numbers across data systems of interest. Even when “unique” identifiers exist in the data sources to be linked, probabilistic-matching software should be employed to link records across data systems to reduce matching errors. Readers should consult Lee and Goerge (1999) for an in-depth review of the advantages of probabilistic matching even when Social Security numbers are available in both data sets. In addition to the complex logistics of linking files, new issues are posed by TANF reforms themselves. In particular, a model that thoroughly investigates the relationship between parental welfare paths and child well-being requires not only data on the timing of welfare receipt, but also an indication of the reason that aid ceased. Without an explanation of the reason for termination, it is difficult to distinguish between parents who left aid for gainful employment and those who were dropped from the rolls due to a sanction and/or failure to comply with regulations. In many cases, this information is lacking. Therefore, researchers may try to link welfare and child well-being data to parental employment data in an attempt to understand which families are leaving welfare for “positive” reasons. Finally, most current evaluation efforts typically focus on examining the relationship between parental welfare careers and outcomes for children. Under TANF, however, children’s and parents’ welfare careers must be considered separately. In some states, such as California, sanctions and time limits will result in a decrease in only the parental portion of the welfare grant, with the child’s portion maintained. Children might, then, move to another household assistance unit where the parent figure gets full benefits. Identifying and successfully tracking these parents and children may involve record linkage across cases and incorporate case flow dynamics that are quite complicated. Beyond receipt of TANF assistance, children’s participation in other important programs such as Medicaid, the Food Stamp Program, and WIC also must be evaluated if we are to gain a comprehensive understanding of the impacts that reforms have on child well-being.

OCR for page 316
Studies of Welfare Populations: Data Collection and Research Issues CONCLUSIONS Using administrative data to evaluate welfare reform presents challenges and opportunities within each of the domains of child well-being. Child abuse and neglect data generally are available to the evaluation of welfare reform because both child welfare records and TANF data sets typically reside within the same governmental department at the local and state levels. However, developing appropriate measures of child well-being from administrative child welfare requires intricate programming of longitudinal data files: understanding of the differences between the child’s experience and the system performance indicators; expertise with a range of sophisticated analysis methods; and understanding of many interpretations that administrative data might allow. In contrast, health measures of child well-being—for example, birthweight or immunization completeness—are more uniformly defined and there is more agreement about their implications. However, these data are less available to study welfare reform because they typically reside within government entities separate from departments where welfare data reside. When these data sources differ, issues of compatibility of data formats and definitions, linking of data, confidentiality, and ownership of data files call for collaborative efforts to evaluate welfare reform. Evaluation of impacts within juvenile justice and education include particularly acute challenges of data availability, as well as the need to create valid and reliable measures. The authors of this paper have endeavored to increase readers’ familiarity with needed indicators of child well-being and the administrative data sets that contain them. A secondary goal has been to alert readers to the ways that existing policies hamper access to the data necessary to make informed decisions. Obtaining permission to use administrative data for evaluation purposes is harder than it needs to be. Without substantial convergence around the purposes of using administrative data, this emerging technology is going to be a partial, piecemeal, and ephemeral aid to government. The technical solutions for linking are increasing (storage is more affordable, processing times are shorter, and matching software is better), but public support has not been built to encourage this linking. Issues of data access and confidentiality present the greatest barriers to full utilization of this resource. Although the federal government is demanding more accountability from the states, and the states from the counties, there is little outcry from public officials to permit the broader use of administrative data to generate the information required to track the performance of human service agencies. Scandinavian countries are generating invaluable research using linked data across generations to understand, for example, the transmission of schizophrenia across generations and the likelihood that children born with birth defects will give birth to children with birth defects. Similarly, program participation data have been combined with information from driving records, educational attainment, military service,

OCR for page 316
Studies of Welfare Populations: Data Collection and Research Issues and marriage certificates to understand lifetime outcomes of family recomposition and participation in service programs. Researcher access to administrative data is beneficial, and more open access will permit individuals to educate themselves about what is contained in such databases, to use the information within those databases to conduct research for multiple purposes, and to reassure the public about the feasibility of using already gathered information for the public good. Concerns over confidentiality continue to present a major barrier to linking administrative data to evaluate the effects of welfare reform on child well-being. Perhaps nowhere is there as much sensitivity concerning privacy and confidentiality as with records containing information about vulnerable children and parents who have been accused of violating social norms by abusing or neglecting their children. At the same time, electronic availability of information on individuals permits sophisticated research that was simply impossible in the past. How do we reconcile the need to provide privacy and confidentiality to individual patients while enabling public health researchers and policy makers to use available information to make the best decisions? Although privacy and confidentiality of records about children’s well-being are important, we suggest there are already adequate protections, incentives and disincentives, and policies and procedures, to preserve individual privacy. We already trust millions of individuals in our society to respect the confidentiality of information they encounter each day in the human services, child welfare, health care, law enforcement, juvenile justice, and education sysem, to name a few. We trust the individuals conducting research within each of these systems to maintain the confidentiality of records. Most of these data are collected without any explicit discussion of whether or how they will be used for research that might inform administration of the program. Yet we have generated the expectation that individuals not working for those institutions who obtain data from them in order to advance services research through data linking represent a risk to the confidentiality concerns of service recipients. The expectation that there is likely to be even a minimal risk of mishandling data lacks an evidentiary base. In our 10 years of experience using administrative data of the most sensitive kinds (including child abuse reports and juvenile justice records), we know of no violations of individual rights of persons in those data sets. Nor do we have any stories to tell about exceptional procedures we instituted to prevent such misuse. The handling of that information was simply very routine. Perhaps we need a more systematic effort to determine what real and imagined threats to confidentiality exist in datalinking efforts. Until we have evidence to the contrary, we should continue to maintain databases with adequate identifying information to support future research projects, and we should advocate for change in unwisely broad legislative or regulatory language that adversely affects interorganizational research. We believe it is appropriate and indeed necessary to maintain personal identifying information on public health and child well-being databases, and that those identifiers should be available to facilitate linkage of electronic health

OCR for page 316
Studies of Welfare Populations: Data Collection and Research Issues databases to support research to improve the health of our population as well as to enhance the health of individuals. At the same time, we emphasize that availability of such identifiers is quite different from license to invade the privacy of individuals or disregard the need for strict confidentiality of the information held within medical records. We believe it is possible to reconcile all these goals. We need to encourage constant conversation between investigators specializing in administrative data and those designing surveys so that the surveys can be used to help inform the interpretation of the administrative data. Survey researchers generally are not familiar with the needs for researchers to be provided with data that have adequate variables for matching. For example, data that tell us about the reasons why clients change service use patterns can be combined with information from administrative data about how often and when these service use patterns change. Furthermore, we must develop better strategies for making survey data available for linking with administrative data. A serious threat to this possibility is the assumption that if it is possible for the confidentiality of a data set to be compromised, it will. This leads to counterproductive strategies such as making it impossible to accurately match samples to their communities or counties of origin (thus obviating the possibility of exploring neighborhood or county effects). Whereas linked administrative data can provide important information on the impact of welfare reform on child well-being, it is not a panacea and will not provide us with all the information we need to monitor welfare reform. We must be wary of the conclusions we draw from linked data because we often cannot determine whether an individual did not experience the outcome, was recorded as experiencing the outcome but could not be matched across data systems (e.g., if they moved across jurisdictional lines), or experienced the outcome but was not recorded as such. Even when the data are accurate, at best they help us monitor who appears to be affected by welfare reform, when those impacts occur, and where the impact is greatest or least. Sometimes we do not even know the direction of that change. For example, if more children per capita are reported for abuse and neglect under TANF than were reported under JOBS, this could mean that the smaller TANF caseloads have resulted in more opportunities for home visiting and better early identification of child abuse and neglect. As to why welfare reform affects children and families differentially, administrative data can only guide us as to the best places to look for those answers. Carefully designed representative samples can be drawn and subjected to other methods (e.g., surveys) that can build on the framework that a comprehensive administrative data analysis provides.

OCR for page 316
Studies of Welfare Populations: Data Collection and Research Issues REFERENCES Aber, J.L., J.Brooks-Gunn, and R.Maynard 1995 The effects of welfare reform on teenage parents and their children. The Future of Children 5:53–71. American Humane Association 1984 Highlights of Official Child Neglect and Abuse Reporting: 1982. Denver. American Humane Association. Ammerman, R.J., J.E.Cassissi, M.Hersen, and V.R.Van Hasselt 1986 Consequences of physical abuse and neglect in children. Clinical Psychology Review 6:291–310. Andrulis, D.P. 1998 Access to care is the centerpiece in the elimination of socioeconomic disparities in health. Annuals of Internal Medicine 129(5):412–416. Barnett, W.S. 1998 Long-term cognitive and academic effects of early childhood education on children in poverty. Preventive Medicine 27:204–207. Barth, R.P. and D.Blackwell 1998 Death rates among California’s foster care and former foster care population. Children and Youth Services Review 20:577–604. Berrick, J.D., B.Needell, and M.Minkler 1999 The policy implications of welfare reform for older caregivers, kinship care, and family configuration. Children and Youth Services Review 21(9/10):843–864. Boots, S.W., and R.Green 1999 Family care or foster care? How state policies affect kinship caregivers. In New Federalism Issues and Options for States, Series A (A-34). Washington, DC: The Urban Institute. Boudett, K.P., and D.Friedlander 1997 Does mandatory basic education improve achievement test scores of AFDC recipients? A reanalysis of data from California’s GAIN program. Evaluation Review 21:568–588. Brauner, S., and P.Loprest 1999 Where Are They Now? What States’ Studies of People Who Left Welfare Tell Us. New Federalism: Issues and Options for States. Series A. No. A-32. Washington, DC: The Urban Institute. Card, D., and A.B.Krueger 1998 School resources and student outcomes. Annals of The American Academy of Political and Social Science 559:39–53. Chapin Hall Center for Children at the University of Chicago, Center for Social Services Research, University of California at Berkeley, University of North Carolina at Chapel Hill School of Social Work, and American Institutes for Research 2000 Dynamics of Children’s Movement Among the AFDC, Medicaid, and Foster Care Programs Prior to Welfare Reform: 1995–1996. Washington, DC: Office of the Secretary for Planning and Evaluation, U.S. Department of Health and Human Services. Available: http://aspe.hhs.gov/hsp/movement00/index.htm. Child Trends 1999 Indicators of Children’s Well-Being: From Construction to Application. Washington, DC: Child Trends. 2000a Children and Welfare Reform: A Guide to Evaluating the Effects of State Welfare Policies on Children. Washington, DC: Child Trends. 2000b Indicators of Child and Family Well-Being: A Selected Inventory of Existing Projects. Available: http://childtrends.org. [September 30, 2000]

OCR for page 316
Studies of Welfare Populations: Data Collection and Research Issues Children’s Defense Fund 1998 Welfare to What? Early Findings on Family Hardship and Well-Being. Washington, DC: Children’s Defense Fund. Collins, Anne 1997 Children and Welfare Reform Issue Brief 2: Anticipating the Effects of Federal and State Welfare Changes on Systems that Serve Children. New York: National Center for Children in Poverty, School of Public Health, Columbia University. Available: http://cpmcnet.columbia.edu/dept/nccp/main10.html Collins, Anne, and Lawrence Aber 1996 State Welfare Waiver Evaluations: Will They Increase Our Understanding of the Impact of Welfare Reform on Children? Working paper prepared for the National Center for Children in Poverty, School of Public Health, Columbia University, New York. Available: http://cpmcnet.columbia.edu/dept/nccp/main10.html Couch, J.L., and J.S.Milner 1993 Effects of child neglect on children. Criminal Justice and Behavior 20(1):49–65. Coulton, C., and R.Hollister 1999 Measuring comprehensive community initiative outcomes using data available for small areas. In New Approaches to Evaluating Community Initiatives. Volume 2: Theory, Measurement, and Analysis. The Aspen Institute Roundtable on Comprehensive Community Initiatives. Available: http://www.aspenroundtable.org. Coulton, C.J., J.E.Korbin, M.Su, and J.Chow 1995 Community level factors and child maltreatment rates. Child Development 66:1262–1276. Currie, J., and T.Duncan 1995 Could Subsequent School Quality Affect the Long Term Gains from Head Start. Cambridge MA: National Bureau of Economic Research. Darnell, J., and S.Rosenbaum 1997 Welfare reform: Unanticipated but inevitable consequences for health insurance coverage for the poor. Nutrition 13(5):490–491. Ellwood, M. 1999 The Medicaid Eligibility Maze: Coverage Expands, But Enrollment Problems Persist: Findings from a Five State Study. Kaiser Commission on Medicaid and the Uninsured. Available: http://www.kff.org [November 30, 1999] Families USA 1999 One Step Forward, One Step Back, Families USA. Available: http://www.familiesusa.org. English, D., and C.S.Widom 1999 Childhood Victimization and Delinquency, Adult Criminality, and Violent Criminal Behavior. Unpublished paper presented at the Child Welfare League of America National Conference on Research in Child Welfare. Seattle. Gabor, V., A.Ben-Avi, and I.Hill 1997 Developing Performance Measures and Performance-Based Contracts for State and Local MCH Activities. Washington, DC: Health Systems Research, Inc. Under contract to Health Resources and Services Administration, Maternal and Child Health Bureau. Garfinkel, I., and S.S.McLanahan 1986 Single mothers and their children: Summary and recommendations. In Single Mothers and Their Children, I. Garfinkel and S.S.McLanahan, eds. Washington, DC: The Urban Institute. Geen, R., and S.Waters 1997 The Impact of Welfare Reform on Child Welfare Financing. Washington, DC: The Urban Institute.

OCR for page 316
Studies of Welfare Populations: Data Collection and Research Issues Gelles, R.J. 1992 Poverty and violence toward children. American Behavioral Scientist 35(3):258–274. Geltman, P.L., A.F.Meyers, J.Greenberg, and B.Zuckerman 1996 Welfare reform and children’s health. Archives of Pediatric and Adolescent Medicine 150:384–389. Gil, D.G. 1971 Violence against children. Journal of Marriage and the Family 33(4):637–648. Giovannoni, J.M., and A.Billingsley 1970 Child neglect among the poor: A study of parental adequacy in families of three ethnic groups. Child Welfare 49(4):196–204. Gortmaker, S.L., and D.K.Walker 1984 Monitoring child health in communities. In Monitoring Child Health in the United States, D.K.Walker and J.B.Richmond, eds. Cambridge, MA: Harvard University Press. Grason, H., and N.E.Nachbar 1997 Developing MCH Systems Measures: A Process and Strategy in Kansas. Washington, DC: Health Systems Research, Inc. Under contract to the Health Resources and Services Administration, Maternal and Child Health Bureau. Gray, E. 1988 The link between child abuse and juvenile delinquency: What we know and recommendations for policy and research. In Family Abuse and its Consequences: New Directions in Research, G.T.Hotaling, D.Finkelhor, J.T.Kirpatrick, and M.A.Straus, eds. Newbury Park, CA: Sage. Greenberg, M. 1998 Participation in Welfare and Medicaid Enrollment. Kaiser Commission on Medicaid and the Uninsured. Available: http://www.kff.org. Haskins, R. 1995 Losing ground or moving ahead? Welfare reform and children. Pp. 63–86 in Escape from Poverty: What Makes a Difference for Children, P.Chase-Lansdale and J.Brooks-Gunn, eds. New York: Cambridge University Press. Jonson-Reid, M., and R.P.Barth 2000 From maltreatment report to juvenile incarceration: The role of child welfare services. Child Abuse and Neglect 24:505–520. Knitzer, J., and S.Bernard 1997 Children and Welfare Reform, Issue Brief I: The New Welfare Law and Vulnerable Families: Implications for Child Welfare and Child Protection Systems. New York: National Center for Children in Poverty. Koppelman, J., and J.G.Lear 1998 The new child health insurance expansions: How will school-based health centers fit in? Journal of School Health 68(10):441–446. Lee, B.J., & R.M.Goerge 1999 Poverty, early childbearing, and child maltreatment: A multinomial analysis. Children and Youth Services Review 21(9/10):755–780. Levinson, P. 1969 The next generation: A study of children in AFDC families. Welfare in Review 7:1–9. Ludwig, J., G.J.Duncan, and P.Hirschfield 1999 Urban poverty and juvenile crime. Evidence from a randomized housing-mobility experiment. American Journal of Epidemiology 49(11):203–218. Magura, S., and B.S.Moses 1986 Outcome Measures for Child Welfare Services: Theory and Applications. Washington, DC: Child Welfare League of America.

OCR for page 316
Studies of Welfare Populations: Data Collection and Research Issues Maine Department of Human Services 1983 Children’s Deaths in Maine: 1976–1980 Final Report. Augusta: Maine Department of Human Services. Manpower Demonstration Research Corporation 2000 The Next Generation. Available: http://www.mdrc.org/Families&Children/NextGeneration.htm [July 19, 2000] Margolis, L.H., G.P.Cole, and J.B.Kotch 1997 Historical foundations in maternal and children’s health. Chapter 2 in Maternal and Child Health: Programs, Problems and Policy in Public Health, Jonathan B.Kotch, ed. Gaithersburg, MD: Aspen Publishers, Inc. Maternal and Child Health Bureau 1996 Maternal and Child Health Strategy Statement Data Utilization and Enhancement. Washington, DC: U.S. Department of Health and Human Services. Meezan, W., and J.Giovannoni 1995 The current threat to protective services and the child welfare system. Children and Youth Services Review 17(4):567–574. Moffitt, R.A., and E.P.Slade 1997 Health care coverage for children who are on and off welfare. Future of Children 7(1):87– 98. National Center for Health Statistics 1996 Healthy People 2000 Review, 1995–96. Hyattsville, MD: Public Health Service. National Conference of State Legislators 1999 Tracking Welfare Reform: Designing Follow-up Studies of Recipients Who Leave Welfare, National Conference of State Legislators. Available: http://www.ncsl.org/statefed/welfare/trackbrf.html. [November 1, 1999] National Research Council 1999 Evaluating Welfare Reform: A Framework and Review of Current Work. Interim Report, Robert A.Moffitt and Michele Ver Ploeg, eds. Committee on National Statistics, Panel on Data and Methods for Measuring the Effects of Changes in Social Welfare Programs , Washington, DC: National Academy Press. Needell, B., S.Cuccaro-Alamin, A.Brookhart, and S.Lee 1999 Transitions from AFDC to child welfare in California. Children and Youth Services Review 21(9/10):815–841. O’Campo, P., and L.Rojas-Smith 1998 Welfare reform and women’s health: Review of the literature and implications for state policy. Journal of Public Health Policy 19(4):420–446. Orthner, D.K., and K.A.Randolph 1999 Welfare reform and high school dropout patterns for children. Children and Youth Services Review 21:881–900. Pappas, G. 1998 Monitoring the health consequences of welfare reform. International Journal of Health Services 23(4):703–713. Philips, D.P., N.Christenfeld, and N.M.Ryan 1999 An increase in the number of deaths in the United States in the first week of the month: An association with substance abuse and other causes of death. New England Journal of Medicine 341(2):93–98. Reiss, J. 1999 Does Your State’s Title XXI SCHIP Plan Promote the Development and Maintenance of Quality Systems of Care for Children with Special Health Needs, Issues and Criteria for SCHIP Plan Review and Analysis. Available: http://www.ichp.edu/schip/materials/893253826.html

OCR for page 316
Studies of Welfare Populations: Data Collection and Research Issues Research Forum on Children, Families, and the New Federalism 1999 Evaluating the effects of state welfare policies on children: The project on state level outcomes. The Forum 2(2):4–5. Schorr, L.B. 1997 Common Purpose: Strengthening Families and Neighborhoods to Rebuild America. New York: Doubleday. Shenkman, E. 1999 Analysis of Pending Accounts in the Florida Healthy Kids Program: Executive Summary. Prepared for the Healthy Kids Corporation. Available: http://www.ichp.edu/rmanaged/materials/896969477.html. [October 30, 1999] Shook, K. 1998 Assessing the consequences of welfare reform for child welfare. Poverty Research News 11(1):8–12. South Carolina Department of Social Services 1999 Welfare Reform/Child Well-Being Administrative Data Linking. South Carolina CHILD LINK. Available: http://aspe.hhs.gov/hsp/adminlink/sc/scfinal.htm. [November 13, 1999] Starfield, B., M.Bergner, M.Ensminger, A.Riley, S.Ryan, B.Green, P.McGauhey, A.Skinner, and S.Kim 1993 Adolescent health status measurement: Development of the child health and illness profile. Pediatrics 91(2):430–435. UC Data Archive and Technical Assistance 1999 An Inventory of Research Uses of Administrative Data in Social Services Programs in the United States 1998. Berkeley, CA: University of California. Available: http://ucdata.berkeley.edu/new_web/inventory/intro.html. [December 1, 1999] Vermont Agency of Human Services 2000 Community Profiles. Available: http://www.ahs.state.vt.us. Wells, K., and S.Guo in press Reunification of foster children under conditions of welfare reform. Children and Youth Services Review. Widom, C.S. 1991 Avoidance of criminality in abused and neglected children. Psychiatry 54:162–174. Willis, E., and R.M.Kleigman 1997 Wisconsin’s welfare reform and its potential effects on the health of children. Journal of Health Care for the Poor and Underserved 8(1):25–35. Wilson, J.B., D.T.Ellwood, and J.Brooks-Gunn 1995 Welfare-to-work through the eyes of children. Pp. 63–86 in Escape from Poverty: What Makes a Difference for Children, P.Chase-Lansdale and J.Brooks-Gunn, eds. New York: Cambridge University Press. Wolock, I., and S.Magura 1996 Parental substance abuse as a predictor of child maltreatment re-reports. Child Abuse & Neglect 20(12):1183–1193. Wulczyn, F.H., K.Brunner, and R.M.Goerge 1999 A Report from the Multistate Foster Care Data Archive: Foster Care Dynamics, 1983– 1997. Chicago: Chapin Hall Center for Children at the University of Chicago. Yaffe, A. 1998 Examining the Effects of Welfare Reform on Children: The project on state level outcomes. Poverty Research News 2(1), 1, 6. Yoshikawa, H. 1999 Welfare dynamics, support services, mothers’ earnings, and child cognitive development: Implications for contemporary welfare reform. Child Development 70(3):779–801.

OCR for page 316
Studies of Welfare Populations: Data Collection and Research Issues Zaslow, M., K.A.Moore, D.R.Morrison, and M.J.Coiro 1995 The family support act and children: Potential pathways of influence. Children and Youth Services Review 17(1,2):231–249. Zaslow, M., K.Tout, C.Botsko, and K.Moore 1998 Welfare reform and children: Potential implications. In New Federalism: Issues and Options for States, Series A, No. A-23. Washington, DC: The Urban Institute. Zeller, D.E. 1999 Welfare Reform Impacts on Child Welfare Caseloads: A Research Agenda. Troy, NY: Hornby Zeller Associates, Inc. Zuravin, S.J., and F.A.DiBlasio 1996 The correlates of child physical abuse and neglect by adolescent mothers. Journal of Family Violence 11(2):149–166.