NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.
This report has been reviewed by a group other than the authors according to procedures approved by a Report Review Committee consisting of members of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine.
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Frank Press is president of the National Academy of Sciences.
The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Robert M. White is president of the National Academy of Engineering.
The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Samuel O. Thier is president of the Institute of Medicine.
The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy's purposes of furthering knowledge and advising the federal government Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Frank Press and Dr. Robert M. White are chairman and vice chairman, respectively, of the National Research Council.
The project that is the subject of this report was supported by funds from the Office of the Assistant Secretary for Planning and Evaluation, U.S. Department of Health and Human Services, and the Food and Nutrition Service of the U.S. Department of Agriculture.
Library of Congress Catalog Card No. 91-62261
International Standard Book Number 0-309-04541-X
Additional copies of this report are available from:
National Academy Press
2101 Constitution Avenue, NW Washington, DC 20418
S400
Printed in the United States of America
PANEL TO EVALUATE MICROSIMULATION MODELS FOR SOCIAL WELFARE PROGRAMS
ERIC A. HANUSHEK (Chair),
Department of Economics, University of Rochester
DAVID M. BETSON,
Department of Economics, University of Notre Dame
LYNNE BILLARD,
Department of Statistics, University of Georgia
SHELDON DANZIGER,
Institute of Public Policy Studies, University of Michigan
EUGENE P. ERICKSEN,
Department of Sociology, Temple University
THOMAS J. ESPENSHADE,
Office of Population Research, Princeton University
HARVEY GALPER,
KPMG Peat Marwick, Washington, D.C.
LOUIS GORDON,
Department of Mathematics, University of Southern California
KEVIN M. HOLLENBECK,
W.E. Upjohn Institute for Employment Research, Kalamazoo, Mich.
GORDON H. LEWIS,
School of Urban and Public Affairs, Carnegie Mellon University
ROBERT MOFFITT,
Department of Economics, Brown University
GAIL R. WILENSKY,
Health Care Financing Administration, U.S. Department of Health and Human Services*
MICHAEL C. WOLFSON,
Analytical Studies Branch, Statistics Canada
CONSTANCE F. CITRO, Study Director
MICHAEL L. COHEN, Consultant
CHRISTINE M. ROSS, Research Associate
AGNES E. GASKIN, Administrative Secretary
COMMITTEE ON NATIONAL STATISTICS
BURTON H. SINGER (Chair),
Department of Epidemiology and Public Health, Yale University
NORMAN M. BRADBURN,
National Opinion Research Center, University of Chicago
RONALD S. BROOKMEYER,
Department of Biostatistics, Johns Hopkins University
MARTIN H. DAVID,
Department of Economics, University of Wisconsin
ANGUS S. DEATON,
Woodrow Wilson School of Public and International Affairs, Princeton University
CLAUDIA D. GOLDIN,
Department of Economics, Harvard University
LOUIS GORDON,
Department of Mathematics, University of Southern California
ROBERT M. HAUSER,
Department of Sociology, University of Wisconsin
GRAHAM KALTON,
Survey Research Center, Institute for Social Research, University of Michigan
WILLIAM A. MORRILL,
Mathtech, Inc., Princeton, New Jersey
DOROTHY P. RICE,
Department of Social and Behavioral Sciences, School of Nursing, University of California, San Francisco
JOHN E. ROLPH,
The RAND Corporation, Santa Monica, California
DONALD B. RUBIN,
Department of Statistics, Harvard University
KENNETH W. WACHTER,
Department of Statistics, University of California, Berkeley
MIRON L. STRAF, Director
FLORENCE E. WOLF, Administrative Assistant
Contents Volume II: Technical Papers
|
||||
|
||||
Databases for Microsimulation: A Comparison of the March CPS and SIPP |
|
|||
Statistical Matching and Microsimulation Models |
|
|||
|
||||
Alternative Model Designs: Program Participation Functions and the Allocation of Annual to Monthly Values in TRIM2, MATH, and HITSM |
|
|||
DYNASIM and PRISM: Examples of Dynamic Modeling |
|
|||
|
||||
Future Computing Environments for Microsimulation Modeling |
|
|
||||
Variance Estimation of Microsimulation Models Through Sample Reuse |
|
|||
Evaluations of Microsimulation Models: Literature Review |
|
|||
A Validation Experiment With TRIM2 |
|
|||
Evaluating the Accuracy of U.S. Population Projection Models |
|
|||
|
||||
Documentation for Microsimulation Models: TRIM2, MATH, and HITSM |
|
Acknowledgments
At the outset, I wish to thank the members of the Panel to Evaluate Microsimulation Models for Social Welfare Programs for their generous contributions of time and expert knowledge. Several panel members served ably as chairs of working groups on broad issues. Other members participated extensively in the analysis of the panel's Transfer Income Model 2 (TRIM2) validation experiment. Still others contributed papers (in Volume II) or sets of notes on specific topics that greatly helped in preparation of the panel's report. Overall this was an exceptionally hard-working panel that strove unstintingly to grapple with a complex set of problems covering a very broad terrain of policy concerns, models, and data. The members thought and worked hard to develop recommendations that would enhance the quality of policy analysis tools and thereby the quality of national debate about important social welfare issues. It has been a genuine pleasure to work with them.
Of course, anybody who has ever had contact with an effort such as this realizes that the quality of the report—to say nothing about the enjoyment and satisfaction of participating in the project—depends directly on the study director. In this case, speaking both personally and for the panel, I can say absolutely that Constance Citro was the irreplaceable input to our effort. There is little doubt that we would have produced a very different report, one not nearly so strong, if Connie had not been involved. Her substantive knowledge, her prodigious feats of writing at the word processor, and her calm, easygoing nature each contributed to the activity.
The panel is also indebted to Michael Cohen, who served as the panel's
statistical consultant throughout the project. He prepared papers reviewing statistical matching techniques for developing microsimulation databases, sample reuse techniques for estimating the variance in microsimulation model estimates, the previous literature of microsimulation model validation studies, and (in collaboration with several members of the panel) the results of the panel's TRIM2 validation experiment. Mike participated actively in the panel's deliberations and contributed materially to the quality and depth of the panel's review of model validation issues.
Christine Ross served as research associate for the panel during its first year and made extremely valuable contributions to the study. She prepared materials describing the characteristics of a number of static and dynamic microsimulation models and a background paper comparing two dynamic models. She conducted a number of interviews with agency staff on their use of microsimulation models. Overall, Chris contributed enormously to helping the panel organize its work and gather the information it needed to carry out the study.
The staffs of our sponsors, the Office of the Assistant Secretary for Planning and Evaluation (ASPE) of the U.S. Department of Health and Human Services (HHS) and the Food and Nutrition Service (FNS) of the U.S. Department of Agriculture, were extremely helpful in providing information about their use of microsimulation models for policy analysis and in making available to us the expertise and resources of their modeling contractors. We would like to thank particularly certain individuals for their assistance. Joan Turek-Brezina, Director for Technical and Computer Support for ASPE and the HHS project officer for the panel, contributed generously of her time and support in every aspect of the panel's work. Robert Dalrymple, the FNS project officer for the panel, also devoted time and energy to the project. ASPE staff, including Paul Gayer, M. Eugene Moyer, William Prosser, Reuben Snipper, and Daniel Weinberg (now with the Census Bureau), presented informative case studies of the use of microsimulation models at the panel's first meeting, as did Steven Carlson, Director of Analysis for the Office of Analysis and Evaluation in FNS. Reuben Snipper also provided detailed information about the techniques used to produce estimates for the policy debate that led to the Family Support Act of 1988.
We would also like to thank staff members of the Congressional Budget Office who shared their knowledge and insights about policy analysis techniques. In particular, we owe thanks to Janice Peskin, who provided detailed information about the procedures used to develop estimates for the Family Support Act, and to Nancy Gordon and Sandra Christensen who commented on models for health care policy issues. We would also like to thank James Cilke of the Office of Tax Analysis of the U.S. Department of the Treasury, who provided useful information on microsimulation models for tax policy.
Staff members of firms involved in microsimulation modeling provided invaluable assistance to the panel through briefings and sharing documentation
and other materials about their models with the panel. We would like to acknowledge in this regard the valuable contributions of Harold Beebout and Pat Doyle of Mathematica Policy Research, Inc., and of David Kennell and John Sheils of Lewin/ICF, Inc. We would like to thank particularly Richard Michel, Director of the Income and Benefits Policy Center at the Urban Institute, and his staff, including Sheila Zedlewski, Linda Giannarelli Paul Johnson, and Margaret Moore. Dick and Sheila not only made a number of presentations to the panel on static and dynamic models, but helped the panel design a validation experiment with TRIM2 that was a centerpiece of the panel's work, providing a wealth of information for the panel's deliberations. The experiment was ably carried out by Linda, Margaret, and Paul.
We are indebted to many other people who made presentations to the panel or prepared background papers at our behest. These include Gary Burtless of the Brookings Institution, Thomas Grannemann of the University of Colorado, and Robert Strauss of Carnegie Mellon University, each of whom prepared a paper on modeling behavioral effects of program changes; Steven Caldwell of Cornell University, who prepared a paper on techniques of aging microsimulation model databases; Deborah Chollet of Georgia State University, who prepared a paper on microsimulation models for health care policy issues; Paul Cotton of Fulcrum Technologies, Inc., and George Sadowsky of New York University, who collaborated on a paper about future computing environments for microsimulation models; Donald Fullerton of the University of Virginia, who briefed the panel on computable general equilibrium models; and Barry Bluestone of the University of Massachusetts-Amherst and John Havens and Alan Clayton-Matthews, both of Boston College, who briefed the panel on the Multi-Regional Policy Impact Simulation (MRPIS) model.
The panel also benefited greatly from a dinner meeting and discussion with congressional staff members who work with and interpret the information developed by policy analysts for the legislative debate, including William Hoagland, minority staff director for the Senate Budget Committee; Wendell Primus, chief economist of the House Ways and Means Committee; and Randall Weiss, former chief economist of the Joint Committee on Taxation and now with Deloitte and Touche.
The panel is very grateful to Eugenia Grohman, Associate Director for Reports of the Commission on Behavioral and Social Sciences and Education, for invaluable assistance in helping the panel organize a large volume of technical material into a coherent and readable report, as well as for her fine editorial work. We would also like to thank members of the Commission on Behavioral and Social Sciences and Education and the Committee on National Statistics who reviewed the report and proffered valuable comments.
Agnes Gaskin served ably as the administrative secretary for the panel. She made admirable logistical arrangements for the large number of meetings held by the panel—nine plenary and six working group sessions over a two-year
period. She also coped cheerfully and competently with multiple rounds of revisions to sections of this report and to the papers comprising Volume II.
Overall, the panel was ably assisted in its endeavors at every stage of the process by many, many people, whom we are honored to thank.
Eric A. Hanushek, Chair
Panel to Evaluate Microsimulation Models for Social Welfare Programs