Evaluating WELFARE REFORM in an Era of Transition

Panel on Data and Methods for Measuring the Effects of Changes in Social Welfare Programs

Robert A.Moffitt and Michele Ver Ploeg, Editors

Committee on National Statistics

Division of Behavioral and Social Sciences and Education

National Research Council

NATIONAL ACADEMY PRESS

Washington, DC



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page R1
Evaluating Welfare Reform in an Era of Transition Evaluating WELFARE REFORM in an Era of Transition Panel on Data and Methods for Measuring the Effects of Changes in Social Welfare Programs Robert A.Moffitt and Michele Ver Ploeg, Editors Committee on National Statistics Division of Behavioral and Social Sciences and Education National Research Council NATIONAL ACADEMY PRESS Washington, DC

OCR for page R1
Evaluating Welfare Reform in an Era of Transition NATIONAL ACADEMY PRESS 2101 Constitution Avenue, N.W. Washington, DC 20418 NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance. This study was supported by Contract No. HHS-100–98–0011 between the National Academy of Sciences and the U.S. Department of Health and Human Services. Support of the work of the Committee on National Statistics is provided by a consortium of federal agencies through a grant from the National Science Foundation (Number SBR-9709489). Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the organizations or agencies that provided support for the project. Library of Congress Cataloging-in-Publication Data Evaluating welfare reform in an era of transition/Panel on Data and Methods for Measuring the Effects of Changes in Social Welfare Programs ; Robert A.Moffitt and Michele Ver Ploeg, editors. p. cm. “Committee on National Statistics, Division of Behavioral and Social Sciences and Education, National Research Council.” Includes bibliographical references and index. ISBN 0-309-07274-3 (hardcover) 1. Public welfare—United States—Evaluation. 2. Public welfare administration—United States—Evaluation. 3. Social surveys—United States—Evaluation. I. Moffitt, Robert. II. Ver Ploeg, Michele. III. National Research Council (U.S.). Panel on Data and Methods for Measuring the Effects of Changes in Social Welfare Programs. HV95 .E958 2001 361.973–dc21 2001002534 Additional copies of this report are available from National Academy Press, 2101 Constitution Avenue, N.W., Lockbox 285, Washington, D.C. 20055; (800) 624–6242 or (202) 334–3313 (in the Washington metropolitan area); Internet, http://www.nap.edu Printed in the United States of America Copyright 2001 by the National Academy of Sciences. All rights reserved. Suggested citation: National Research Council (2001) Evaluating Welfare Reform in an Era of Transition. Panel on Data and Methods for Measuring the Effects of Changes in Social Welfare Programs, Robert A.Moffitt and Michele Ver Ploeg, Editors. Committee on National Statistics, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.

OCR for page R1
Evaluating Welfare Reform in an Era of Transition THE NATIONAL ACADEMIES National Academy of Sciences National Academy of Engineering Institute of Medicine National Research Council The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Bruce M.Alberts is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. William A.Wulf is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Kenneth I.Shine is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Bruce M.Alberts and Dr. William A.Wulf are chairman and vice chairman, respectively, of the National Research Council.

OCR for page R1
Evaluating Welfare Reform in an Era of Transition This page in the original is blank.

OCR for page R1
Evaluating Welfare Reform in an Era of Transition PANEL ON DATA AND METHODS FOR MEASURING THE EFFECTS OF CHANGES IN SOCIAL WELFARE PROGRAMS ROBERT A.MOFFITT (Chair), Department of Economics, Johns Hopkins University JOHN L.ADAMS, RAND, Santa Monica, California THOMAS CORBETT, Institute for Research on Poverty, University of Wisconsin, Madison JOHN L.CZAJKA, Mathematica Policy Research, Inc., Washington, D.C. KATHRYN EDIN, Institute for Policy Research, Northwestern University IRWIN GARFINKEL, School of Social Work, Columbia University ROBERT M.GOERGE, Chapin Hall Center for Children, University of Chicago ERIC A.HANUSHEK, Hoover Institution, Stanford University V.JOSEPH HOTZ, Departments of Economics and Policy Studies, University of California, Los Angeles RICHARD A.KULKA, Statistics, Health, and Social Policy, Research Triangle Institute, Research Triangle Park, NC REBECCA A.MAYNARD, Graduate School of Education, University of Pennsylvania SUZANNE M.RANDOLPH, Department of Family Studies, University of Maryland WERNER SCHINK, California Department of Social Services, Sacramento MICHELE VER PLOEG, Study Director CONSTANCE F.CITRO, Senior Program Officer JAMIE CASEY, Senior Project Assistant

OCR for page R1
Evaluating Welfare Reform in an Era of Transition COMMITTEE ON NATIONAL STATISTICS 2000–2001 JOHN E.ROLPH (Chair), Marshall School of Business, University of Southern California JOSEPH G.ALTONJI, Department of Economics, Northwestern University LAWRENCE D.BROWN, Department of Statistics, University of Pennsylvania JULIE DAVANZO, RAND, Santa Monica, California ROBERT M.GROVES, Joint Program in Survey Methodology, University of Maryland, College Park JOEL HOROWITZ, Department of Economics, University of Iowa HERMANN HABERMANN, Statistics Division, United Nations WILLIAM KALSBEEK, Department of Biostatistics, University of North Carolina RODERICK J.A.LITTLE, School of Public Health, University of Michigan THOMAS A.LOUIS, RAND, Arlington, VA DARYL PREGIBON, AT&T Laboratories-Research, Florham Park, NJ FRANCISCO J.SAMANIEGO, Division of Statistics, University of California, Davis RICHARD L.SCHMALENSEE, Sloan School of Management, Massachusetts Institute of Technology ANDREW A.WHITE, Director

OCR for page R1
Evaluating Welfare Reform in an Era of Transition Acknowledgments The Panel on Data and Methods for Measuring the Effects of Changes in Social Welfare Programs wishes to thank the many people who contributed to the preparation of this report. The study was sponsored by the Office of the Assistant Secretary for Planning and Evaluation in the U.S. Department of Health and Human Services (ASPE-DHHS) through a congressional appropriation. Throughout the study, the panel was assisted by the dedicated staff of the agency. In particular, we thank Patricia Ruggles, Susan Hauan, Julie Isaacs, and Don Oellerich for briefing the panel and providing background information. The panel also thanks Howard Rolston of the Administration for Children and Families (ACF) in DHHS and his staff for meeting with panel members to discuss the reporting requirement data for the Temporary Assistance for Needy Family (TANF) program and efforts to catalogue state welfare programs. The panel is greatly indebted to those who presented and discussed papers at the Workshop on Data Collection for Low-Income and Welfare Populations sponsored by the panel. The papers comprehensively discussed the current state of knowledge for surveying low-income populations; preparation and use of and access to welfare program-relevant administrative data systems; and measuring important outcomes for welfare studies. The panel would also like to acknowledge Kristin Moore of Child Trends, Inc., and Janet Norwood of the Urban Institute for their remarks at the workshop. The panel is also grateful to Greg Acs and Pamela Loprest of the Urban Institute, who prepared a paper summarizing welfare leavers studies and to Leyla Mohadjer and Hussain Choudhry of Westat, who prepared a paper on adjusting for missing data on surveys. These papers will

OCR for page R1
Evaluating Welfare Reform in an Era of Transition appear in the panel’s companion volume entitled “Data Collection and Research Issues for Studies of Welfare Populations” to be published later this year. Many other individuals participated in panel meetings or briefed the panel on data and methodological issues that provided background information for the panel’s deliberations. Kenneth Bryson, Patricia Doyle, Enrique Llamas, Stephanie Shipp, and Edward Welniak, of the Census Bureau, briefed the panel on the Bureau’s low income and welfare related data sets. The panel is also grateful to participants of the panel’s Seminar on Evaluation Methods: Burt Barnow of Johns Hopkins University, Randy Brown of Mathematica Policy Research, Gary Burtless of the Brookings Institution, Robert LaLonde of the University of Chicago, Charles Metcalf of Mathematica Policy Research, Bruce Meyer of Northwestern University, Charles Michalopoulos of the Manpower Demonstration Research Corporation, and Robert Schoeni of RAND. The panel also thanks David Stevens of the University of Baltimore for preparing and presenting a paper and a follow-up memorandum on the composition of the Maryland caseload. We thank Sheila Zedlewski and Linda Giannarelli of the Urban Institute for preparing a paper on microsimulation methods and for briefing panel members and staff on the Welfare Rules Database. The panel also thanks Gary Cyphers and Kathleen Kinsella of the American Public Human Services Association for meeting with panel members and staff to discuss state efforts to develop administrative databases for use in welfare program research. Finally, we thank the Institute for Research on Poverty and its former director, Barbara Wolfe, for providing hospitality to Michele Ver Ploeg to conduct a study of Wisconsin welfare leavers. The panel is indebted to the efforts of the National Research Council (NRC) staff in the work of the panel and in the preparation of this report. Michele Ver Ploeg, the study director for the panel and coeditor of this report, provided essential support throughout the life of the panel and in drafting the report. She organized the panel’s discussions into coherent topics, identified issues and formulated questions and conclusions, and supervised the general activities of the panel by organizing meetings, gathering materials and interviewing outside experts, and serving as liaison to the sponsor and other outside groups. She also assisted the panel by drafting major sections of this report and by supervising the redrafting process as part of the NRC internal review process. Shelly also conducted an independent study of welfare leavers at the same time, which is included in the panel’s companion volume. Constance Citro, senior project officer, provided experienced and wise counsel throughout the life of the panel and on this report, and contributed important insights in panel discussions. Miron Straf, former Director of the Committee on National Statistics under whose auspices the panel operated, is acknowledged for his assistance in organizing the panel and working out the charge of the panel with the sponsor. The panel is very appreciative of the work of Jamie Casey, senior project assistant, for arranging the logistics of panel meetings and for her excellent work on the production of this

OCR for page R1
Evaluating Welfare Reform in an Era of Transition manuscript. We also thank Eugenia Grohman, associate director for reports of the Division of Behavioral and Social Sciences and Education, for her careful and thorough technical editing, her commentary on report drafts, and for overseeing the review process. This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the NRC’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making the published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We thank the following individuals for their participation in the review of this report: Kenneth Arrow, Department of Economics, Stanford University; Greg J.Duncan, Institute for Policy Research and School of Education and Social Policy, Northwestern University; David Illig, California Health and Human Services Agency, Sacramento; Michael Pergamit, National Opinion Research Center, Washington, DC; Frank Samaniego, Department of Statistics, University of California, Davis; Fritz Scheuren, Urban Institute, Washington, DC; and Deanna Schexnayder, Lyndon B.Johnson School of Public Affairs, University of Texas, Austin. Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations nor did they see the final draft of the report before its release. The review of this report was overseen by John Bailar, Department of Health Studies, University of Chicago (emeritus), and Robert Michael, Harris Graduate School of Public Policy, University of Chicago. Appointed by the National Research Council, they were responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the authoring panel and the institution. Finally, I thank my fellow panel members for giving their time and expertise so generously towards the completion of this report. Every member of the panel brought a critical perspective and expertise to the discussions, without which the report would have been missing important elements. Their hard work and self-less dedication in the service of a public benefit was exemplary. Robert Moffitt, Chair Panel on Data and Methods for Measuring the Effects of Changes in Social Welfare Programs

OCR for page R1
Evaluating Welfare Reform in an Era of Transition This page in the original is blank.

OCR for page R1
Evaluating Welfare Reform in an Era of Transition Contents     EXECUTIVE SUMMARY   1     The Panel,   1     Findings,   2     Key Questions of Interest,   3     Evaluation Methods for the Questions of Interest,   7     Data for Monitoring and Evaluating Social Welfare Programs,   8 1   INTRODUCTION   13     The Panel,   14     Policy Background,   16     Structure of the Report,   23 2   WELFARE REFORM MONITORING AND EVALUATION: THE CURRENT LANDSCAPE   24     Descriptive and Monitoring Studies,   26     Studies of Welfare Leavers and Related Groups,   28     Randomized Experiments   30     Caseload and Other Econometric Models,   31     Process, Implementation, and Qualitative Studies,   33     Other Welfare Reform Studies,   34     Studies on Topics Related to Welfare Reform,   34 3   RESEARCH QUESTIONS AND POPULATIONS OF INTEREST   36     Populations of Interest,   37

OCR for page R1
Evaluating Welfare Reform in an Era of Transition     Outcomes of Interest,   39     Questions of Interest,   41     Nationwide Versus Individual State Assessments,   49     Assessment,   50 4   EVALUATION METHODS AND ISSUES   54     Overview of Evaluation Methods,   55     Evaluation Methods for the Questions of Interest,   63     Issues in Evaluation Methodology,   73     Assessment of Current Evaluation Efforts,   93     Next Steps,   99 5   DATA NEEDS AND ISSUES   102     Survey Data,   104     Administrative Data,   128     Data Access and Confidentiality,   138     Program Description Data,   141     Qualitative Data,   144     Summary and Assessment of the Current Data Infrastructure,   146 6   ADMINISTRATIVE ISSUES FOR MAINTAINING THE DATA INFRASTRUCTURE   149     The Current System,   150     A Proposed System,   154     REFERENCES   161     APPENDICES     A   Major Current Welfare-Related Research Sponsored by the U.S. Department of Health and Human Services   171 B   Summary of Current Welfare Reform Projects   187 C   The Statistical Power of National Data to Evaluate Welfare Reform John Adams and V.Joseph Hotz   209 D   Summaries of National-Level Survey Data Sets Relevant to Welfare Monitoring and Evaluation   221     BIOGRAPHICAL SKETCHES OF PANEL MEMBERS AND STAFF   235     INDEX   241

OCR for page R1
Evaluating Welfare Reform in an Era of Transition Evaluating WELFARE REFORM

OCR for page R1
Evaluating Welfare Reform in an Era of Transition This page in the original is blank.