Improving Information for Social Policy Decisions

The Uses of Microsimulation Modeling

VOLUME I

REVIEW AND RECOMMENDATIONS

Constance F. Citro and Eric A. Hanushek, Editors

Panel to Evaluate Microsimulation Models for Social Welfare Programs

Committee on National Statistics

Commission on Behavioral and Social Sciences and Education

National Research Council

National Academy Press
Washington, D.C.
1991



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page R1
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations Improving Information for Social Policy Decisions The Uses of Microsimulation Modeling VOLUME I REVIEW AND RECOMMENDATIONS Constance F. Citro and Eric A. Hanushek, Editors Panel to Evaluate Microsimulation Models for Social Welfare Programs Committee on National Statistics Commission on Behavioral and Social Sciences and Education National Research Council National Academy Press Washington, D.C. 1991

OCR for page R1
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance. This report has been reviewed by a group other than the authors according to procedures approved by a Report Review Committee consisting of members of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Frank Press is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Robert M. White is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Samuel O. Thier is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy's purposes of furthering knowledge and advising the federal government Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Frank Press and Dr. Robert M. White are chairman and vice chairman, respectively, of the National Research Council. The project that is the subject of this report was supported by funds from the Office of the Assistant Secretary for Planning and Evaluation, U.S. Department of Health and Human Services, and the Food and Nutrition Service of the U.S. Department of Agriculture. Library of Congress Catalog Card No. 91-62261 International Standard Book Number 0-309-04541-X Additional copies of this report are available from: National Academy Press 2101 Constitution Avenue, NW Washington, DC 20418 S400 Printed in the United States of America

OCR for page R1
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations PANEL TO EVALUATE MICROSIMULATION MODELS FOR SOCIAL WELFARE PROGRAMS ERIC A. HANUSHEK (Chair), Department of Economics, University of Rochester DAVID M. BETSON, Department of Economics, University of Notre Dame LYNNE BILLARD, Department of Statistics, University of Georgia SHELDON DANZIGER, Institute of Public Policy Studies, University of Michigan EUGENE P. ERICKSEN, Department of Sociology, Temple University THOMAS J. ESPENSHADE, Office of Population Research, Princeton University HARVEY GALPER, KPMG Peat Marwick, Washington, D.C. LOUIS GORDON, Department of Mathematics, University of Southern California KEVIN M. HOLLENBECK, W.E. Upjohn Institute for Employment Research, Kalamazoo, Mich. GORDON H. LEWIS, School of Urban and Public Affairs, Carnegie Mellon University ROBERT MOFFITT, Department of Economics, Brown University GAIL R. WILENSKY, Health Care Financing Administration, U.S. Department of Health and Human Services* MICHAEL C. WOLFSON, Analytical Studies Branch, Statistics Canada CONSTANCE F. CITRO, Study Director MICHAEL L. COHEN, Consultant CHRISTINE M. ROSS, Research Associate AGNES E. GASKIN, Administrative Secretary *   Served until January 1990

OCR for page R1
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations COMMITTEE ON NATIONAL STATISTICS BURTON H. SINGER (Chair), Department of Epidemiology and Public Health, Yale University NORMAN M. BRADBURN, National Opinion Research Center, University of Chicago RONALD S. BROOKMEYER, Department of Biostatistics, Johns Hopkins University MARTIN H. DAVID, Department of Economics, University of Wisconsin ANGUS S. DEATON, Woodrow Wilson School of Public and International Affairs, Princeton University CLAUDIA D. GOLDIN, Department of Economics, Harvard University LOUIS GORDON, Department of Mathematics, University of Southern California ROBERT M. HAUSER, Department of Sociology, University of Wisconsin GRAHAM KALTON, Survey Research Center, Institute for Social Research, University of Michigan WILLIAM A. MORRILL, Mathtech, Inc., Princeton, New Jersey DOROTHY P. RICE, Department of Social and Behavioral Sciences, School of Nursing, University of California, San Francisco JOHN E. ROLPH, The RAND Corporation, Santa Monica, California DONALD B. RUBIN, Department of Statistics, Harvard University KENNETH W. WACHTER, Department of Statistics, University of California, Berkeley MIRON L. STRAF, Director FLORENCE E. WOLF, Administrative Assistant

OCR for page R1
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations Contents CONTENTS OF VOLUME II   ix ACKNOWLEDGMENTS   xi     SUMMARY   1     Improving the Tools of Policy Analysis: Investment Priorities,   2     The Role of Microsimulation as a Policy Analysis Tool,   8     Recommendations for Improving Policy Analysis,   13     Recommendations for Microsimulation Models,   15 1   INTRODUCTION   21     A Scene in Washington, D.C.,   21     The Tools of Policy Analysis,   24     The Panel Study,   26 PART I INFORMATION FOR SOCIAL WELFARE POLICY: TOWARD A SECOND REVOLUTION   29 2   THE SEARCH FOR USEFUL INFORMATION   33     The First Information Revolution,   35     Policy Analysis: Between Social Science Research and Politics,   38     A Case Study of Policy Analysis: The Family Support Act of 1988,   41

OCR for page R1
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations 3   IMPROVING THE TOOLS AND USES OF POLICY ANALYSIS   52     A Strategy for Investment,   52     Data Quality and Availability,   55     Validation,   72     Documentation and Communication of the Results of Policy Analysis,   84     APPENDIX: MODELS, UNCERTAINTY, AND CONFIDENCE INTERVALS   89     Models,   89     Uncertainty of an Estimate,   90     Conditional Versus Unconditional Confidence Intervals,   93     An Illustrative Diagram,   95 PART II THE ROLE OF MICROSIMULATION AS A POLICY ANALYSIS TOOL   97 4   MICROSIMULATION MODELS: THEN AND NOW   101     Basic Elements of Microsimulation Models,   101     Development of Microsimulation Modeling for Policy Analysis,   107     Role and Current Status of Microsimulation: Findings,   114 5   DATABASES FOR MICROSIMULATION   123     Data Quality: The March CPS,   125     Strategies for Treating Missing and Erroneous Data,   132     The Promise of SIPP,   137     Recommendations for Improving Data Quality,   139 6   MODEL DESIGN AND DEVELOPMENT   153     Model Design Principles and Practices,   155     Current Microsimulation Model Design,   161     Strategic Directions for Microsimulation Model Development,   164 7   COMPUTING TECHNOLOGY AND MICROSIMULATION   182     The Evolution of Microsimulation Computing Platforms,   183     New Developments in Computing Technology,   185     Future Directions for Computing in Microsimulation,   191 8   MICROSIMULATION MODELING OF HEALTH CARE, RETIREMENT INCOME, AND TAX POLICIES   194     Health Care Policies,   195     Retirement Income Policies,   212     Tax Policies,   219

OCR for page R1
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations 9   VALIDATION   231     Corroboration as a Stand-in for Validation,   233     Techniques of Model Validation,   235     Review of Validation Studies,   241     A Validation Study of TRIM2: The Panel's Experiment,   248     Strategies for Validating Microsimulation Models: Recommendations,   259 10   DOCUMENTATION AND ARCHIVING   265     Standards for Model Documentation,   266     A Documentation Case Study,   268     Recommendations,   270 11   THE MICROSIMULATION MODELING COMMUNITY   273     Relationships Among Federal Agencies,   274     Policy Analysis Agencies and Their Suppliers,   278     Policy Analysis Agencies and Decision Makers' Staffs,   280     The Role of Research,   281     APPENDIX: MICROSIMULATION MODELS, DATABASES, AND MODELING TERMS   290     Models,   290     Databases,   298     Modeling Terms,   302     GLOSSARY OF ACRONYMS   311     REFERENCES   314     BIOGRAPHICAL SKETCHES OF PANEL MEMBERS AND STAFF   331

OCR for page R1
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations This page in the original is blank.

OCR for page R1
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations Contents Volume II: Technical Papers INTRODUCTION     DATABASES AND METHODS OF DATA ENHANCEMENT     1   Databases for Microsimulation: A Comparison of the March CPS and SIPP Constance F. Citro     2   Statistical Matching and Microsimulation Models Michael L. Cohen     MODEL DESIGN     3   Alternative Model Designs: Program Participation Functions and the Allocation of Annual to Monthly Values in TRIM2, MATH, and HITSM Constance F. Citro and Christine M. Ross     4   DYNASIM and PRISM: Examples of Dynamic Modeling Christine M. Ross     COMPUTING TECHNOLOGY     5   Future Computing Environments for Microsimulation Modeling Paul Cotton and George Sadowsky    

OCR for page R1
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations MODEL VALIDATION     6   Variance Estimation of Microsimulation Models Through Sample Reuse Michael L. Cohen     7   Evaluations of Microsimulation Models: Literature Review Michael L. Cohen     8   A Validation Experiment With TRIM2 Michael L. Cohen, Lynne Billard, David M. Betson, and Eugene P. Ericksen     9   Evaluating the Accuracy of U.S. Population Projection Models Laurence Grummer-Strawn and Thomas J. Espenshade     MODEL DOCUMENTATION     10   Documentation for Microsimulation Models: TRIM2, MATH, and HITSM Kevin M. Hollenbeck    

OCR for page R1
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations Acknowledgments At the outset, I wish to thank the members of the Panel to Evaluate Microsimulation Models for Social Welfare Programs for their generous contributions of time and expert knowledge. Several panel members served ably as chairs of working groups on broad issues. Other members participated extensively in the analysis of the panel's Transfer Income Model 2 (TRIM2) validation experiment. Still others contributed papers (in Volume II) or sets of notes on specific topics that greatly helped in preparation of the panel's report. Overall this was an exceptionally hard-working panel that strove unstintingly to grapple with a complex set of problems covering a very broad terrain of policy concerns, models, and data. The members thought and worked hard to develop recommendations that would enhance the quality of policy analysis tools and thereby the quality of national debate about important social welfare issues. It has been a genuine pleasure to work with them. Of course, anybody who has ever had contact with an effort such as this realizes that the quality of the report—to say nothing about the enjoyment and satisfaction of participating in the project—depends directly on the study director. In this case, speaking both personally and for the panel, I can say absolutely that Constance Citro was the irreplaceable input to our effort. There is little doubt that we would have produced a very different report, one not nearly so strong, if Connie had not been involved. Her substantive knowledge, her prodigious feats of writing at the word processor, and her calm, easygoing nature each contributed to the activity. The panel is also indebted to Michael Cohen, who served as the panel's

OCR for page R1
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations statistical consultant throughout the project. He prepared papers reviewing statistical matching techniques for developing microsimulation databases, sample reuse techniques for estimating the variance in microsimulation model estimates, the previous literature of microsimulation model validation studies, and (in collaboration with several members of the panel) the results of the panel's TRIM2 validation experiment. Mike participated actively in the panel's deliberations and contributed materially to the quality and depth of the panel's review of model validation issues. Christine Ross served as research associate for the panel during its first year and made extremely valuable contributions to the study. She prepared materials describing the characteristics of a number of static and dynamic microsimulation models and a background paper comparing two dynamic models. She conducted a number of interviews with agency staff on their use of microsimulation models. Overall, Chris contributed enormously to helping the panel organize its work and gather the information it needed to carry out the study. The staffs of our sponsors, the Office of the Assistant Secretary for Planning and Evaluation (ASPE) of the U.S. Department of Health and Human Services (HHS) and the Food and Nutrition Service (FNS) of the U.S. Department of Agriculture, were extremely helpful in providing information about their use of microsimulation models for policy analysis and in making available to us the expertise and resources of their modeling contractors. We would like to thank particularly certain individuals for their assistance. Joan Turek-Brezina, Director for Technical and Computer Support for ASPE and the HHS project officer for the panel, contributed generously of her time and support in every aspect of the panel's work. Robert Dalrymple, the FNS project officer for the panel, also devoted time and energy to the project. ASPE staff, including Paul Gayer, M. Eugene Moyer, William Prosser, Reuben Snipper, and Daniel Weinberg (now with the Census Bureau), presented informative case studies of the use of microsimulation models at the panel's first meeting, as did Steven Carlson, Director of Analysis for the Office of Analysis and Evaluation in FNS. Reuben Snipper also provided detailed information about the techniques used to produce estimates for the policy debate that led to the Family Support Act of 1988. We would also like to thank staff members of the Congressional Budget Office who shared their knowledge and insights about policy analysis techniques. In particular, we owe thanks to Janice Peskin, who provided detailed information about the procedures used to develop estimates for the Family Support Act, and to Nancy Gordon and Sandra Christensen who commented on models for health care policy issues. We would also like to thank James Cilke of the Office of Tax Analysis of the U.S. Department of the Treasury, who provided useful information on microsimulation models for tax policy. Staff members of firms involved in microsimulation modeling provided invaluable assistance to the panel through briefings and sharing documentation

OCR for page R1
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations and other materials about their models with the panel. We would like to acknowledge in this regard the valuable contributions of Harold Beebout and Pat Doyle of Mathematica Policy Research, Inc., and of David Kennell and John Sheils of Lewin/ICF, Inc. We would like to thank particularly Richard Michel, Director of the Income and Benefits Policy Center at the Urban Institute, and his staff, including Sheila Zedlewski, Linda Giannarelli Paul Johnson, and Margaret Moore. Dick and Sheila not only made a number of presentations to the panel on static and dynamic models, but helped the panel design a validation experiment with TRIM2 that was a centerpiece of the panel's work, providing a wealth of information for the panel's deliberations. The experiment was ably carried out by Linda, Margaret, and Paul. We are indebted to many other people who made presentations to the panel or prepared background papers at our behest. These include Gary Burtless of the Brookings Institution, Thomas Grannemann of the University of Colorado, and Robert Strauss of Carnegie Mellon University, each of whom prepared a paper on modeling behavioral effects of program changes; Steven Caldwell of Cornell University, who prepared a paper on techniques of aging microsimulation model databases; Deborah Chollet of Georgia State University, who prepared a paper on microsimulation models for health care policy issues; Paul Cotton of Fulcrum Technologies, Inc., and George Sadowsky of New York University, who collaborated on a paper about future computing environments for microsimulation models; Donald Fullerton of the University of Virginia, who briefed the panel on computable general equilibrium models; and Barry Bluestone of the University of Massachusetts-Amherst and John Havens and Alan Clayton-Matthews, both of Boston College, who briefed the panel on the Multi-Regional Policy Impact Simulation (MRPIS) model. The panel also benefited greatly from a dinner meeting and discussion with congressional staff members who work with and interpret the information developed by policy analysts for the legislative debate, including William Hoagland, minority staff director for the Senate Budget Committee; Wendell Primus, chief economist of the House Ways and Means Committee; and Randall Weiss, former chief economist of the Joint Committee on Taxation and now with Deloitte and Touche. The panel is very grateful to Eugenia Grohman, Associate Director for Reports of the Commission on Behavioral and Social Sciences and Education, for invaluable assistance in helping the panel organize a large volume of technical material into a coherent and readable report, as well as for her fine editorial work. We would also like to thank members of the Commission on Behavioral and Social Sciences and Education and the Committee on National Statistics who reviewed the report and proffered valuable comments. Agnes Gaskin served ably as the administrative secretary for the panel. She made admirable logistical arrangements for the large number of meetings held by the panel—nine plenary and six working group sessions over a two-year

OCR for page R1
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations period. She also coped cheerfully and competently with multiple rounds of revisions to sections of this report and to the papers comprising Volume II. Overall, the panel was ably assisted in its endeavors at every stage of the process by many, many people, whom we are honored to thank. Eric A. Hanushek, Chair Panel to Evaluate Microsimulation Models for Social Welfare Programs