Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
The Future of the - SURVEY OF INCOME AND PROGRAM PARTICI RATION Constance F. Citro and Graham Kalton, Editors Panel to Evaluate the Survey of Income and Program Participation Committee on National Statistics Commission on Behavioral and Social Sciences and Education National Research Council National Academy Press Washington, D.C. 1993
NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council' whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance. This report has been reviewed by a group other than the authors according to procedures approved by a Report Review Committee consisting of members of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The National Academy of Sciences is a private, nonprofit, self-perpetuating society of dis- tinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Frank Press is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Robert M. White is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Kenneth I. Shine is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy's purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies detained by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communi- ties. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Frank Press and Dr. Robert M. White are chairman and vice chairman, respectively, of the National Research Council. The project that is the subject of this report was supported by funds from the Bureau of the Census, U.S. Department of Commerce. Copyright 1993 by the National Academy of Sciences. All rights reserved. Library of Congress Catalog Card No. 92-62584 International Standard Book Number 0-309-04795-1 Additional copies of this report are available from: National Academy Press 2101 Constitution Avenue, NW Washington, DC 20418 B044 Printed in the United States of America
PANEL TO EVALUATE THE SURVEY OF INCOME AND PROGRAM PARTICIPATION GRAHAM KALTON (Chair), Westat, Inc., Rockville, Md. PAUL P. BIEMER, Research Triangle Institute, Research Triangle Park, N.C. GORDON J. BRACKSTONE, Statistics Canada, Ottawa CLIFFORD C. CLOGG, Departments of Sociology and Statistics, Pennsylvania State University MARTIN H. DAVID, Department of Economics, University of Wisconsin GREG J. DUNCAN, Institute for Social Research, University of Michigan RALPH E. FOLSOM, Research Triangle Institute, Research Triangle Park, N.C. ROBERT M. HAWSER, Department of Sociology, University of Wisconsin V. JOSEPH HOTZ, Harris Graduate School of Public Policy Studies, University of Chicago RANDALL J. OLSEN, Center for Human Resource Research, Ohio State University PATRICIA RUGGLES, The Urban Institute, Washington, D.C. TERENCE P. SPEED, Department of Statistics, University of California, Berkeley CONSTANCE F. CITRO, Study Director AGNES E. GASKIN, Senior Project Assistant *Served until December 1991 . . .
COMMITTEE ON NATIONAL STATISTICS 1991-1992 BURTON lI. SINGER (Chair), Department of Epidemiology and Public Health, Yale University NORMAN M. BRADBURN, National Opinion Research Center, University of Chicago MARTIN H. DAVID, Department of Economics, University of Wisconsin ANGUS S. DEATON, Woodrow Wilson School of Public and Intemational Affairs, Princeton University NOREEN GOLDMAN, Office of Population Research, Princeton University LOUIS GORDON, Department of Mathematics, University of Southern California JOEL B. GREENHOUSE, Department of Statistics, Carnegie Mellon University ROBERT M. HAWSER, Department of Sociology, University of Wisconsin GRAHAM KALTON, Westat, Inc., Rockville, Md. WILLIAM A. MORRILL, Mathtech, Inc., Princeton, N.J. JANET L. NORWOOD, The Urban Institute, Washington, D.C. DOROTHY P. RICE, Department of Social and Behavioral Sciences, School of Nursing, University of California, San Francisco JOHN E. ROLPH, The RAND Corporation, Santa Monica, Calif. DONALD B. RUBIN, Department of Statistics, Harvard University MIRON L. STRAP, Director SUSANNA McFARLAND, Administrative Assistant V
Acknowledgments The Panel to Evaluate the Survey of Income and Program Participation (SIPP) wishes to thank the many people who made possible the preparation of this report. The staff of the Bureau of the Census were extremely helpful in provid- ing information about SIPP, in briefing the panel about developments at the Census Bureau in a wide range of areas, and in responding to numerous queries and comments from the panel. We particularly thank Daniel Kasprzyk, formerly coordinator of SIPP research at the Census Bureau and now with the National Center for Education Statistics, who served as the panel's project officer from spring 1990 to spring 1991 and contributed most generously of his time and knowledge in every aspect of the panel's work, and John McNeil and Enrique Lamas of the Census Bureau's Housing and Household Economic Statistics Division, who subsequently served as liaisons to the panel and were most helpful in responding to requests for information. There are many other Census Bureau staff members who made informa- tive presentations to the panel or met with panel members and staff to share their knowledge: William Butz, Associate Director for Demographic Pro- grams; staff of the Housing and Household Economic Statistics Division, including Daniel Weinberg, chief, Charles Nelson, assistant chief, Gordon Green, former assistant chief, and John Coder, special assistant; staff of the Demographic Surveys Division, including Sherry Courtland, chief, Barry Fink, assistant chief, Chester Bowie, former income surveys branch chief, and Donald Fischer, income surveys branch chief; staff of the Demographic v
Vl ACKNOWLEDGMENTS Statistical Methods Division, including Preston Jay Waite, chief, Rajendra Singh, income branch chief, and Vicki Huggins, now with the Economic Statistical Methods Division; staff of the Population Division, including Arthur Norton, assistant chief, and David McMillen, now on the staff of the Senate Committee on Governmental Affairs; Gerald Gates of the Program and Policy Development Office; and Kent Marquis of the Center for Survey Methods Research. Policy analysts in federal agencies that use SIPP shared their knowl- edge and insights with the panel about SIPP's role: Christine Schmidt- Bayne, formerly with the Food and Nutrition Service, U.S. Department of Agriculture, and now with the U.S. Department of Health and Human Ser- vices; Michele Adler and Joan Turek-Brezina of the Office of the Assistant Secretary for Planning and Evaluation, U.S. Department of Health and Hu- man Services; Susan Grad and Denton Vaughan of the Office of Research and Statistics of the Social Security Administration; and Roberton Williams of the Congressional Budget Office. The panel also benefited from the views expressed in interviews conducted with a large number of analysts in federal agencies as part of the Committee on National Statistics's interim assessment of SIPP (see Committee on National Statistics, 1989:App. A). The participants in the Conference on the Future of SIPP convened by the panel in Washington, D.C., in April 1991 contributed a wealth of knowl- edge about SIPP's usefulness for research and analysis and suggestions to improve its relevance and utility in the future. Appendix B lists the topics covered at the conference and the paper authors and invited discussants, who included social science researchers and policy analysts. The panel greatly appreciates the work of Timothy Smeeding of Syra- cuse University, who prepared a stimulating paper on priorities for improv- ing income data from surveys and administrative records. Courtenay Slater of Slater-Hall Information Products and Harold Watts of Columbia Univer- sity contributed useful commentaries on Smeeding's paper. The panel also benefited from the deliberations and views of other advisory bodies for SIPP, including the Working Group on Technical As- pects of SIPP of the Survey Research Methods Section of the American Statistical Association and the SIPP Committee of the Association of Public Data Users (APDU). The papers from the conferences sponsored by the SIPP Committee of the Social Science Research Council (see David, 1983, 1985b; Bureau of the Census, 1988a) provided useful material for evaluat- ing SIPP and considering ways to enhance its usefulness in the future. Other individuals who made helpful presentations or provided useful comments include Pat Doyle, chair of the APDU SIPP committee, formerly with Mathematica Policy Research, Inc., and now with the Agency for Health Care Policy and Research of the U.S. Department of Health and Human Services; Thomas Jabine, statistical consultant and principal author of the
ACKNOWLEDGMENTS . . Vll SIPP Quality Profile; David Cantor, principal investigator for cognitive re- search on the SIPP questionnaire at Westat, Inc.; Alan Tupek, formerly with the Bureau of Labor Statistics and now with the Science Resources Studies Division of the National Science Foundation; James Lepkowski of the Sur- vey Research Center of the University of Michigan; and James Smith and Joseph Waksberg of Westat, Inc. The panel is grateful to Eugenia Grohman, Associate Director for Re- ports of the Commission on Behavioral and Social Sciences and Education, for her fine technical editorial work, which contributed greatly to the read- ability of this report. We also thank members of the Committee-on National Statistics who proffered valuable comments on a draft of the panel's report. Agnes Gaskin served very ably as the senior project assistant for the panel. She made excellent logistical arrangements for the large number of plenary and working group meetings held by the panel and for the Confer- ence on the Future of SIPP. She also performed admirably in preparing the final manuscript of the report. A special acknowledgment is due to Connie Citro for her excellent work as the panel's study director. She had primary responsibility for orga- nizing the panel's activities and drafting the final report. She demonstrated exceptional skills in synthesizing the panel's views and producing a coher- ent document, she readily appreciated different viewpoints on an issue, and she patiently and good-humoredly rewrote parts of the draft to meet con- cerns of panel members. In addition to all of this, she made many signifi- cant contributions to the panel's discussions on the basis of her extensive knowledge of SIPP. On behalf of the panel and personally, I thank Connie for her unstinting efforts and outstanding work. She contributed not only to the quality of the final report, but also to panel members' enjoyment and satisfaction with the enterprise. Finally, I thank the panel members for their generous contributions of time and expert knowledge. Several of them participated in working ses- sions that developed detailed specifications for tabulations on income and program participation from SIPP. Others conducted on-site evaluations of data collection and processing systems and the proposed plan to oversample low-income groups in SIPP. Many of them prepared useful background papers or sets of working notes. Overall, this was an exceptionally hard- working group of people, who conducted a wide-ranging and very thorough assessment of the SIPP program. It has been a genuine pleasure to work with them. Graham Kalton, Chair Panel to Evaluate the Survey of Income and Program Participation
Contents TABLES AND FIGURES SUMMARY Goals, 2 Survey Design, 3 Data Collection and Processing, 4 Data Products and Their Use, 4 Methodological Research and Evaluation, 5 Management and Oversight, 6 Recommendations, 6 1 INTRODUCTION SIPP in Brief, 13 SIPP to Date, 16 Looking, to the Future, 22 GOALS FOR SIPP The Development of SIPP and Its Goals, 26 Views About SIPP's Goals, 33 Recommendation, 41 3 ACHIEVING SIPP's GOALS Improving Data on Income and Other Economic Resources, 43 IX . . X11 13 26 43
x CONTENTS Improving Data on Program Eligibility and Participation, 73 Topical Modules, 80 SIPP and the March CPS, 85 SIPP and Administrative Records, 87 SURVEY DESIGN Major Design Elements and Alternatives, 91 Oversampling in SIPP, 1 19 Following Rules, 128 DATA COLLECTION AND PROCESSING Current Operations, 131 Computer-Assisted Interviewing, 138 Database Management, 147 Investing in the Data Processing Staff, 152 Transition to a CAPI/Database Management System for SIPP, 153 6 DATA PRODUCTS AND THEIR USE Publications, 158 Measurement Issues for Core Statistics, 169 Microdata Products, 189 Documentation and Services for Users, 195 7 METHODOLOGICAL RESEARCH AND EVALUATION Research to Date, 203 Redesign of SIPP, 206 Continuous Error Monitoring, 217 Cognitive Research, 220 8 MANAGEMENT AND OVERSIGHT Management Structure, 228 Oversight, 236 REFERENCES APPENDIX A: USE OF SIPP FOR RESEARCH AND POLICY ANALYSIS APPENDIX B: CONFERENCE ON THE FUTURE OF SIPP: TOPICS AND PARTICIPANTS APPENDIX C: BIOGRAPHICAL SKETCHES OF PANEL MEMBERS AND STAFF INDEX 91 131 158 202 227 245 265 268 270 277
Tables and Figures TABLES 3-7 3-1 Nonasset Income Sources in SIPP, 46 3-2 Assets and Asset Income Sources in SIPP, 49 3-3 Nonresponse Rates for Selected SIPP Core Items, by Panel, 51 3-4 Item Nonresponse Rates in the 1984 SIPP and March 1985 CPS, by Selected Income Types, 52 3-5 Item Nonresponse Rates in the 1985 SIPP and March 1986 CPS, by Selected Income Types, 53 3-6 1984 SIPP and March 1985 CPS Estimates of Income Aggregates, for 1984 by Type, 54 1984 SIPP and March 1984 CPS Estimates as Percentages of Independent Estimates of Income Recipients and Amounts for Selected Income Types, 55 3-8 Comparisons of SIPP 1984 State Unemployment Compensation Estimates with Estimates Derived from Independent Sources, 57 3-9 Item Nonresponse Rates for Asset Amounts in SIPP and the Income Survey Development Program (ISDP) 1979 Research Panel, 58 3-10 1984 SIPP, March CPS and Independent Estimates of Aggregate Interest and Dividend Income, 1983- 1984 and Calendar 1984, 59 3-11 Asset and Liability Estimates from SIPP and the Federal Reserve Board (FRB) Balance Sheet Data for the Household Sector, for 1984 and 1988, 60 Xl
. . xI! TABLES AND FIGURES Coverage Ratios for SIPP and CPS Samples: March 1984 and March 1986, 62 Topical Modules in SIPP, 1984-1990 Panels, 81 Cumulative Household Noninterview and Sample Loss Rates, 1984 1988 and 1990 SIPP Panels, 100 4-2 Estimated Minimum Sample Sizes for Subgroups of Policy Interest from SIPP Panels of Different Sizes, 113 4-3 Reference Periods for Rotation Groups for SIPP Redesign, 115 SIPP Budget, by Major Function, Fiscal 1992, 132 Suggested Schedule for Implementing the SIPP Redesign arid Use of CAPI/DBMS Technology, Including a Large Dress Rehearsal Panel in 1995, 155 6-1 SIPP Reports Published in P-70 Series Through 1991 by U.S. Bureau of the Census, 160 A-1 Papers Produced from SIPP and NLS Data Over a 6-Year Period, 266 A-2 Dissertations Produced from SIPP, PSID, and NLS Over a 6-Year Period, 267 A-3 SIPP Papers Issued in 1989-1990 by Topic Areas, 267 FIGURES 6-1 Contents of SIPP Quarterly Reports, Series P-70, Nos. 1-6, 162 SIPP Reports in P-70 Series Published in 1992, 167 Census Bureau Organization Chart, 229