Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page R1
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations
Improving Information for Social Policy Decisions
The Uses of Microsimulation Modeling
VOLUME I
REVIEW AND RECOMMENDATIONS
Constance F. Citro and Eric A. Hanushek, Editors
Panel to Evaluate Microsimulation Models for Social Welfare Programs
Committee on National Statistics
Commission on Behavioral and Social Sciences and Education
National Research Council
National Academy Press
Washington, D.C.
1991
OCR for page R2
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations
NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.
This report has been reviewed by a group other than the authors according to procedures approved by a Report Review Committee consisting of members of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine.
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Frank Press is president of the National Academy of Sciences.
The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Robert M. White is president of the National Academy of Engineering.
The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Samuel O. Thier is president of the Institute of Medicine.
The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy's purposes of furthering knowledge and advising the federal government Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Frank Press and Dr. Robert M. White are chairman and vice chairman, respectively, of the National Research Council.
The project that is the subject of this report was supported by funds from the Office of the Assistant Secretary for Planning and Evaluation, U.S. Department of Health and Human Services, and the Food and Nutrition Service of the U.S. Department of Agriculture.
Library of Congress Catalog Card No. 91-62261
International Standard Book Number 0-309-04541-X
Additional copies of this report are available from:
National Academy Press
2101 Constitution Avenue, NW Washington, DC 20418
S400
Printed in the United States of America
OCR for page R3
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations
PANEL TO EVALUATE MICROSIMULATION MODELS FOR SOCIAL WELFARE PROGRAMS
ERIC A. HANUSHEK (Chair),
Department of Economics, University of Rochester
DAVID M. BETSON,
Department of Economics, University of Notre Dame
LYNNE BILLARD,
Department of Statistics, University of Georgia
SHELDON DANZIGER,
Institute of Public Policy Studies, University of Michigan
EUGENE P. ERICKSEN,
Department of Sociology, Temple University
THOMAS J. ESPENSHADE,
Office of Population Research, Princeton University
HARVEY GALPER,
KPMG Peat Marwick, Washington, D.C.
LOUIS GORDON,
Department of Mathematics, University of Southern California
KEVIN M. HOLLENBECK,
W.E. Upjohn Institute for Employment Research, Kalamazoo, Mich.
GORDON H. LEWIS,
School of Urban and Public Affairs, Carnegie Mellon University
ROBERT MOFFITT,
Department of Economics, Brown University
GAIL R. WILENSKY,
Health Care Financing Administration, U.S. Department of Health and Human Services*
MICHAEL C. WOLFSON,
Analytical Studies Branch, Statistics Canada
CONSTANCE F. CITRO, Study Director
MICHAEL L. COHEN, Consultant
CHRISTINE M. ROSS, Research Associate
AGNES E. GASKIN, Administrative Secretary
*
Served until January 1990
OCR for page R4
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations
COMMITTEE ON NATIONAL STATISTICS
BURTON H. SINGER (Chair),
Department of Epidemiology and Public Health, Yale University
NORMAN M. BRADBURN,
National Opinion Research Center, University of Chicago
RONALD S. BROOKMEYER,
Department of Biostatistics, Johns Hopkins University
MARTIN H. DAVID,
Department of Economics, University of Wisconsin
ANGUS S. DEATON,
Woodrow Wilson School of Public and International Affairs, Princeton University
CLAUDIA D. GOLDIN,
Department of Economics, Harvard University
LOUIS GORDON,
Department of Mathematics, University of Southern California
ROBERT M. HAUSER,
Department of Sociology, University of Wisconsin
GRAHAM KALTON,
Survey Research Center, Institute for Social Research, University of Michigan
WILLIAM A. MORRILL,
Mathtech, Inc., Princeton, New Jersey
DOROTHY P. RICE,
Department of Social and Behavioral Sciences, School of Nursing, University of California, San Francisco
JOHN E. ROLPH,
The RAND Corporation, Santa Monica, California
DONALD B. RUBIN,
Department of Statistics, Harvard University
KENNETH W. WACHTER,
Department of Statistics, University of California, Berkeley
MIRON L. STRAF, Director
FLORENCE E. WOLF, Administrative Assistant
OCR for page R5
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations
Contents
CONTENTS OF VOLUME II
ix
ACKNOWLEDGMENTS
xi
SUMMARY
1
Improving the Tools of Policy Analysis: Investment Priorities,
2
The Role of Microsimulation as a Policy Analysis Tool,
8
Recommendations for Improving Policy Analysis,
13
Recommendations for Microsimulation Models,
15
1
INTRODUCTION
21
A Scene in Washington, D.C.,
21
The Tools of Policy Analysis,
24
The Panel Study,
26
PART I
INFORMATION FOR SOCIAL WELFARE POLICY: TOWARD A SECOND REVOLUTION
29
2
THE SEARCH FOR USEFUL INFORMATION
33
The First Information Revolution,
35
Policy Analysis: Between Social Science Research and Politics,
38
A Case Study of Policy Analysis: The Family Support Act of 1988,
41
OCR for page R6
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations
3
IMPROVING THE TOOLS AND USES OF POLICY ANALYSIS
52
A Strategy for Investment,
52
Data Quality and Availability,
55
Validation,
72
Documentation and Communication of the Results of Policy Analysis,
84
APPENDIX: MODELS, UNCERTAINTY, AND CONFIDENCE INTERVALS
89
Models,
89
Uncertainty of an Estimate,
90
Conditional Versus Unconditional Confidence Intervals,
93
An Illustrative Diagram,
95
PART II
THE ROLE OF MICROSIMULATION AS A POLICY ANALYSIS TOOL
97
4
MICROSIMULATION MODELS: THEN AND NOW
101
Basic Elements of Microsimulation Models,
101
Development of Microsimulation Modeling for Policy Analysis,
107
Role and Current Status of Microsimulation: Findings,
114
5
DATABASES FOR MICROSIMULATION
123
Data Quality: The March CPS,
125
Strategies for Treating Missing and Erroneous Data,
132
The Promise of SIPP,
137
Recommendations for Improving Data Quality,
139
6
MODEL DESIGN AND DEVELOPMENT
153
Model Design Principles and Practices,
155
Current Microsimulation Model Design,
161
Strategic Directions for Microsimulation Model Development,
164
7
COMPUTING TECHNOLOGY AND MICROSIMULATION
182
The Evolution of Microsimulation Computing Platforms,
183
New Developments in Computing Technology,
185
Future Directions for Computing in Microsimulation,
191
8
MICROSIMULATION MODELING OF HEALTH CARE, RETIREMENT INCOME, AND TAX POLICIES
194
Health Care Policies,
195
Retirement Income Policies,
212
Tax Policies,
219
OCR for page R7
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations
9
VALIDATION
231
Corroboration as a Stand-in for Validation,
233
Techniques of Model Validation,
235
Review of Validation Studies,
241
A Validation Study of TRIM2: The Panel's Experiment,
248
Strategies for Validating Microsimulation Models: Recommendations,
259
10
DOCUMENTATION AND ARCHIVING
265
Standards for Model Documentation,
266
A Documentation Case Study,
268
Recommendations,
270
11
THE MICROSIMULATION MODELING COMMUNITY
273
Relationships Among Federal Agencies,
274
Policy Analysis Agencies and Their Suppliers,
278
Policy Analysis Agencies and Decision Makers' Staffs,
280
The Role of Research,
281
APPENDIX: MICROSIMULATION MODELS, DATABASES, AND MODELING TERMS
290
Models,
290
Databases,
298
Modeling Terms,
302
GLOSSARY OF ACRONYMS
311
REFERENCES
314
BIOGRAPHICAL SKETCHES OF PANEL MEMBERS AND STAFF
331
OCR for page R8
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations
This page in the original is blank.
OCR for page R9
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations
Contents Volume II: Technical Papers
INTRODUCTION
DATABASES AND METHODS OF DATA ENHANCEMENT
1
Databases for Microsimulation: A Comparison of the March CPS and SIPP
Constance F. Citro
2
Statistical Matching and Microsimulation Models
Michael L. Cohen
MODEL DESIGN
3
Alternative Model Designs: Program Participation Functions and the Allocation of Annual to Monthly Values in TRIM2, MATH, and HITSM
Constance F. Citro and Christine M. Ross
4
DYNASIM and PRISM: Examples of Dynamic Modeling
Christine M. Ross
COMPUTING TECHNOLOGY
5
Future Computing Environments for Microsimulation Modeling
Paul Cotton and George Sadowsky
OCR for page R10
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations
MODEL VALIDATION
6
Variance Estimation of Microsimulation Models Through Sample Reuse
Michael L. Cohen
7
Evaluations of Microsimulation Models: Literature Review
Michael L. Cohen
8
A Validation Experiment With TRIM2
Michael L. Cohen, Lynne Billard, David M. Betson, and Eugene P. Ericksen
9
Evaluating the Accuracy of U.S. Population Projection Models
Laurence Grummer-Strawn and Thomas J. Espenshade
MODEL DOCUMENTATION
10
Documentation for Microsimulation Models: TRIM2, MATH, and HITSM
Kevin M. Hollenbeck
OCR for page R11
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations
Acknowledgments
At the outset, I wish to thank the members of the Panel to Evaluate Microsimulation Models for Social Welfare Programs for their generous contributions of time and expert knowledge. Several panel members served ably as chairs of working groups on broad issues. Other members participated extensively in the analysis of the panel's Transfer Income Model 2 (TRIM2) validation experiment. Still others contributed papers (in Volume II) or sets of notes on specific topics that greatly helped in preparation of the panel's report. Overall this was an exceptionally hard-working panel that strove unstintingly to grapple with a complex set of problems covering a very broad terrain of policy concerns, models, and data. The members thought and worked hard to develop recommendations that would enhance the quality of policy analysis tools and thereby the quality of national debate about important social welfare issues. It has been a genuine pleasure to work with them.
Of course, anybody who has ever had contact with an effort such as this realizes that the quality of the report—to say nothing about the enjoyment and satisfaction of participating in the project—depends directly on the study director. In this case, speaking both personally and for the panel, I can say absolutely that Constance Citro was the irreplaceable input to our effort. There is little doubt that we would have produced a very different report, one not nearly so strong, if Connie had not been involved. Her substantive knowledge, her prodigious feats of writing at the word processor, and her calm, easygoing nature each contributed to the activity.
The panel is also indebted to Michael Cohen, who served as the panel's
OCR for page R12
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations
statistical consultant throughout the project. He prepared papers reviewing statistical matching techniques for developing microsimulation databases, sample reuse techniques for estimating the variance in microsimulation model estimates, the previous literature of microsimulation model validation studies, and (in collaboration with several members of the panel) the results of the panel's TRIM2 validation experiment. Mike participated actively in the panel's deliberations and contributed materially to the quality and depth of the panel's review of model validation issues.
Christine Ross served as research associate for the panel during its first year and made extremely valuable contributions to the study. She prepared materials describing the characteristics of a number of static and dynamic microsimulation models and a background paper comparing two dynamic models. She conducted a number of interviews with agency staff on their use of microsimulation models. Overall, Chris contributed enormously to helping the panel organize its work and gather the information it needed to carry out the study.
The staffs of our sponsors, the Office of the Assistant Secretary for Planning and Evaluation (ASPE) of the U.S. Department of Health and Human Services (HHS) and the Food and Nutrition Service (FNS) of the U.S. Department of Agriculture, were extremely helpful in providing information about their use of microsimulation models for policy analysis and in making available to us the expertise and resources of their modeling contractors. We would like to thank particularly certain individuals for their assistance. Joan Turek-Brezina, Director for Technical and Computer Support for ASPE and the HHS project officer for the panel, contributed generously of her time and support in every aspect of the panel's work. Robert Dalrymple, the FNS project officer for the panel, also devoted time and energy to the project. ASPE staff, including Paul Gayer, M. Eugene Moyer, William Prosser, Reuben Snipper, and Daniel Weinberg (now with the Census Bureau), presented informative case studies of the use of microsimulation models at the panel's first meeting, as did Steven Carlson, Director of Analysis for the Office of Analysis and Evaluation in FNS. Reuben Snipper also provided detailed information about the techniques used to produce estimates for the policy debate that led to the Family Support Act of 1988.
We would also like to thank staff members of the Congressional Budget Office who shared their knowledge and insights about policy analysis techniques. In particular, we owe thanks to Janice Peskin, who provided detailed information about the procedures used to develop estimates for the Family Support Act, and to Nancy Gordon and Sandra Christensen who commented on models for health care policy issues. We would also like to thank James Cilke of the Office of Tax Analysis of the U.S. Department of the Treasury, who provided useful information on microsimulation models for tax policy.
Staff members of firms involved in microsimulation modeling provided invaluable assistance to the panel through briefings and sharing documentation
OCR for page R13
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations
and other materials about their models with the panel. We would like to acknowledge in this regard the valuable contributions of Harold Beebout and Pat Doyle of Mathematica Policy Research, Inc., and of David Kennell and John Sheils of Lewin/ICF, Inc. We would like to thank particularly Richard Michel, Director of the Income and Benefits Policy Center at the Urban Institute, and his staff, including Sheila Zedlewski, Linda Giannarelli Paul Johnson, and Margaret Moore. Dick and Sheila not only made a number of presentations to the panel on static and dynamic models, but helped the panel design a validation experiment with TRIM2 that was a centerpiece of the panel's work, providing a wealth of information for the panel's deliberations. The experiment was ably carried out by Linda, Margaret, and Paul.
We are indebted to many other people who made presentations to the panel or prepared background papers at our behest. These include Gary Burtless of the Brookings Institution, Thomas Grannemann of the University of Colorado, and Robert Strauss of Carnegie Mellon University, each of whom prepared a paper on modeling behavioral effects of program changes; Steven Caldwell of Cornell University, who prepared a paper on techniques of aging microsimulation model databases; Deborah Chollet of Georgia State University, who prepared a paper on microsimulation models for health care policy issues; Paul Cotton of Fulcrum Technologies, Inc., and George Sadowsky of New York University, who collaborated on a paper about future computing environments for microsimulation models; Donald Fullerton of the University of Virginia, who briefed the panel on computable general equilibrium models; and Barry Bluestone of the University of Massachusetts-Amherst and John Havens and Alan Clayton-Matthews, both of Boston College, who briefed the panel on the Multi-Regional Policy Impact Simulation (MRPIS) model.
The panel also benefited greatly from a dinner meeting and discussion with congressional staff members who work with and interpret the information developed by policy analysts for the legislative debate, including William Hoagland, minority staff director for the Senate Budget Committee; Wendell Primus, chief economist of the House Ways and Means Committee; and Randall Weiss, former chief economist of the Joint Committee on Taxation and now with Deloitte and Touche.
The panel is very grateful to Eugenia Grohman, Associate Director for Reports of the Commission on Behavioral and Social Sciences and Education, for invaluable assistance in helping the panel organize a large volume of technical material into a coherent and readable report, as well as for her fine editorial work. We would also like to thank members of the Commission on Behavioral and Social Sciences and Education and the Committee on National Statistics who reviewed the report and proffered valuable comments.
Agnes Gaskin served ably as the administrative secretary for the panel. She made admirable logistical arrangements for the large number of meetings held by the panel—nine plenary and six working group sessions over a two-year
OCR for page R14
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations
period. She also coped cheerfully and competently with multiple rounds of revisions to sections of this report and to the papers comprising Volume II.
Overall, the panel was ably assisted in its endeavors at every stage of the process by many, many people, whom we are honored to thank.
Eric A. Hanushek, Chair
Panel to Evaluate Microsimulation Models for Social Welfare Programs
