Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling Volume I, Review and Recommendations (1991) / Chapter Skim
Currently Skimming:
6 Model Design and Development
6 Model Design and Development
Pages 153-181
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 153... ...
We then turn to the substantive issues involved in determining the kinds of capabilities that might be added to existing microsimulation models or included in new models. We discuss the issues and tradeoffs involved in the choice of aging strategies for microsimulation models, in the decision of whether and how to incorporate first-round behavioral responses, and in the decision of whether and how to model second-round responses.
|
From page 154... ...
; · the ability to estimate, inside the model, the likely extent and effects of program-induced behavioral responses, such as the likelihood that people who are ineligible for income benefits will behave so as to become eligible (e.g., by quitting their jobs in order to obtain AFDC and Medicaid benefits) or that households will change their investment patterns in response to tax incentives; · the ability to estimate, inside the model or through links to other models, so-called second-round effects, that is, the longer run effects of program changes for example, the effects on wage rates of labor supply responses to welfare program changes, or the effects on health care providers of changes in the demand for health care services induced by mandated changes in the proportion of costs that patients have to pay themselves; and tin general, we also do not discuss the detailed approach of particular microsimulation models to specific aspects of the models' operations.
|
From page 155... ...
Yet, in simply modeling the complex provisions of the nation's income support programs or tax codes, today's microsimulation models are already very complicated and difficult for anyone other than experienced analysts to use or modify to respond to changing policy concerns. Design Principles and The complexity that reflects the real world of policies and individual circumstances is an inherent feature of microsimulation modeling, so it is especially important to minimize unnecessary complexities.
|
From page 156... ...
ought to simulate behavioral responses and second-round effects as well as the direct effects of policy changes (see Chapter 8~. Nonetheless, we argue strongly that model developers must make explicit choices about priorities.
|
From page 157... ...
For example, with such a design, a microsimulation model could feed results to a macroeconomic model in order to obtain a reading of second-round effects of proposed policy changes and, in turn, use outputs from He macroeconomic model in running further microsimulations. 2Indeed, enhanced computer technology may well make it possible for a good model design to be "integrated," that is, to require running all instead of just some of the model's component modules without incurring excessive computational costs.
|
From page 158... ...
We recommend that policy analysis agencies set standards for the design of future microsimulation models that include: · setting clear goals and priorities for the model; · using self-contained modules that can be readily added to (or deleted from) the model and that are constructed to facilitate documentation and validation, including the assessment of uncertainty through the use of sensitivity analysis and the application of sample reuse techniques to measure variance;
|
From page 159... ...
In our experience, model developers typically follow good practice with regard to validation in the development of individual components of modules. For example, standard procedure in developing a regression equation for imputing variables, such as child care deductions for the AFDC program, is to look at venous measures of goodness of fit before determining the final form of the equation.
|
From page 160... ...
We recommend that policy analysis agencies set standards of good practice for the development of future microsimulation models that require: · constructing prototypes and establishing milestones throughout the development process so that design flaws can be identified at an early stage and the agency provided with some analysis capability before the entire model is completed; · preparing fully adequate documentation on a timely basis for the model and its components; conducting validation studies of the model and its components, including estimates of variance and sensitivity analyses (the latter should be conducted for each new module, prior to full implementation, by examining its impact on the rest of the model in order to identify any unexpected or dysfunctional interactions or adverse effects on use) ; and · subjecting the model to a '~sunset" provision, whereby the model is periodically reevaluated, obsolete components are deleted, and other components are respecified to optimize the model's usefulness and efficiency.
|
From page 161... ...
In the history of microsimulation modeling, RIM (Reforms in Income Maintenance) and the KGB (Kasten, Greenberg, Betson)
|
From page 162... ...
For example, the federal and state income tax modules in TRIM2 were completely rewritten and expanded in the mid-1980s, as was the TRIM2 Medicaid module. This type of development increases the breadth of a microsimulation model in terms of the policy areas for which it is relevant.
|
From page 163... ...
An added factor in reinforcing emphasis on the accounting modules is the concern of the staffs in the policy analysis agencies to have credible models whose baseline results closely match available administrative data on program recipients, although, as noted above, the process of calibrating model results to control totals carries its own perils (see Chapter 5~. Last, but far from least, an important factor in the underemphasis on capabilities such as modeling behavioral response is the skimpy base of research knowledge on which to develop these kinds of capabilities.
|
From page 164... ...
STRATEGIC DIRECTIONS FOR MICROSIMULATION MODEL DEVELOPMENT The question we address in this section is what paths the policy analysis agencies should follow for the future of microsimulation model developments Clearly, a critical line of attack for building improved microsimulation models involves taking advantage of recent developments in computer hardware and software that will not only facilitate expansion of model capabilities at reduced costs, but also increase their flexibility and ease of use. (This topic merits a full discussion of its own, which we provide in the next chapter.)
|
From page 165... ...
The complex issues presented by aging in microsimulation models have led some policy analysis agencies to forgo aging inside the model and, instead, conduct simulations on the latest available data and then extrapolate the simulated values to develop future projections outside the model. The dimensions of a database most often of concern in aging analyses are its population proportions and its economic variables.
|
From page 166... ...
However, some users of microsimulation models do not use either type of aging capability. For example, analysts in ASPE ask the TRIM2 modelers to perform AFDC simulations on the most current database, which at best is 1-2 years out of date.
|
From page 167... ...
The major advantage of static aging is its comparative ease of implementation. Its theoretical weakness mirrors the strength of dynamic aging, namely, the failure to capture adequately the cohort effects in the variables on the file.
|
From page 168... ...
Only limited evaluations of microsimulation model aging techniques have been conducted to date. The panel, as part of its validation experiment using the TRIM2 model, varied the degree of static aging that was performed (see Chapter 9 and Cohen et al., in Volume II)
|
From page 169... ...
Recommendation 6-3. We recommend that policy analysis agencies sponsor an evaluation program to assess the quality of estimates from current static microsimulation models as a function of the aging technique that is used and that they further support such evaluations on a periodic basis for future models.
|
From page 170... ...
We recommend that policy analysis agencies require that future static microsimulation models build in an aging capability in a manner that facilitates evaluation and use of alternative aging assumptions and procedures. Incorporation of Behavioral Response Most of the policy measures simulated by the models that we reviewed have potential behavioral effects on the individuals and organizations affected by the policy.l3 Altering the structure of transfer programs for the low-income population, changing the tax code, or changing the form of health care benefit programs may all affect individual and organizational decisions in many ways: · Altering the level of cash or in-kind (e.g., Medicaid)
|
From page 171... ...
We consider initially only first-round behavioral effects and hence only partial responses to policy changes; later, we discuss the incorporation of second-round effects. Behavioral Responses in Current Models The documentation for some of the models that we reviewed left us in doubt as to the extent to which the model incorporated behavioral responses.
|
From page 172... ...
However, behavioral responses are rarely or only crudely incorporated in health care policy models at the present time. As we have noted, the one behavioral response that is incorporated in MATH, TRIM2, and other microsimulation models for income support programs is participation in such programs.
|
From page 173... ...
We have not found any systematic attempt in microsimulation modeling efforts to date to address the problem of measuring the degree of uncertainty in model outputs attributable to the use of particular behavioral parameters 1SMeta-analysis techniques, although still evolving and posing a number of thomy methodological issues, may be useful in helping to narrow the range of estimates of behavioral responses that are important to include in microsimulation models. Meta-analysis involves a systematic way to aggregate all available studies on a topic and, with the aid of statistical techniques, to determine the best estimate based on all of the studies, without conducting new research or secondary data analysis; see Wachter and Straf (1990)
|
From page 174... ...
Yet the assessment of uncertainty with regard to the estimation of behavioral response should not proceed without a similar assessment for the no-response estimates from microsimulation models that are obtained in the first place (see further discussion of this set of issues in Chapter 9~. The third major consideration concerns the problems involved in actually incorporating behavioral parameters in microsimulation models in an appropriate manner.
|
From page 175... ...
These discrepant results could not be attributed to the behavioral response function itself; rather, they must have been caused by differences in the two models' simulation of the baseline proreform policy environment. The fourth major consideration concerns the means by which behavioral responses are presented to policy makers—provided that reasonably good estimates are available, that they can be incorporated into a microsimulation model, and that they are of sufficient magnitude to be important for policy purposes.
|
From page 176... ...
In other cases, it may be that estimates of the behavioral effects of policy changes are best obtained within the structure of microsimulation models themselves. Or it may be that a dual strategy should be followed in which behavioral response estimates are obtained in both ways.
|
From page 177... ...
Hence, policy analysis agencies need to determine the kinds of behavioral responses to policy measures that could be important to consider. If behavioral responses are potentially large for some policy issues, and if it is decided to build some or all of those responses into microsimulation models, policy analysis agencies should attempt to reduce the extent of uncertainty surrounding existing statistical estimates of such responses.
|
From page 178... ...
Indeed, the development of macroeconomic models preceded the development of microsimulabon models, and early microsimulation modelers such as Guy Orcutt explicitly looked to macroeconomic models as sources of input. Thus, Orcutt 17The discussion in this section benefited greatly from presentations to the panel by Don Fullerton on computable general equilibrium (CGE)
|
From page 179... ...
Issues and liade-Of1s Many of the same issues and trade-offs discussed for the incorporation of behavioral response in microsimulation models apply to the incorporation of 18David (1991) reviews the general equilibrium modeling that was conducted by the U.S.
|
From page 180... ...
The second issue, that concerning uncertainty, is likely to be even more important for the modeling of second-round effects than for the modeling of first-round behavioral effects. For example, CGE models include simulation of behavioral effects in all markets on both supply and demand sides, and hence must rely on estimated elasticities for many different relationships.
|
From page 181... ...
We recommend that policy analysis agencies support research on second-round effects of policy changes that may be important to understand. We also recommend that the agencies require that future microsimulation models include entry and exit points that could facilitate linkages with second-round effects models.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.