Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume I, Review and Recommendations (1991)

We begin our consideration of the structure and design of the models themselves with a strong belief in the importance of following good design principles and practices in the development of policy analysis tools that are intended to be used many times in response to the changing requirements of the policy process. Skimping on resources for model design and testing in the development stage is likely to be a classic ''penny-wise, pound-foolish" decision that comes back to haunt the model users in the application stage, particularly for highly complex microsimulation models. A poorly designed model may not perform all of the functions that were intended, may entail higher-than-expected costs of operation, may be inordinately difficult to modify or to understand, or may experience other difficulties that require analysts to scale back their expectations or invest resources in an effort to patch up the problems after the fact.

We thus begin this chapter by outlining the design principles and practices that are appropriate for microsimulation models as a general class, and we briefly review the history and current status of microsimulation model development in the United States from the viewpoint of good and bad design practices. We then turn to the substantive issues involved in determining the kinds of capabilities that might be added to existing microsimulation models or included in new models. We discuss the issues and trade-offs involved in the choice of aging strategies for microsimulation models, in the decision of whether and how to incorporate first-round behavioral responses, and in the decision of whether and how to model second-round responses. We do not make specific recommendations about preferred approaches to the modeling task in any of these areas because there is an inadequate body of knowledge evaluating

the impact of particular design choices on the quality and utility of model outputs.¹ In the absence of such evidence, it would be foolhardy and arrogant to attempt to dictate the future course of microsimulation model development. Our recommendations, instead, lay out an agenda for research and evaluation that we hope will result in the information needed to make wise choices for future model development.

One concern that is common to all the issues we address is when a microsimulation model becomes "too big": When do the benefits from additional capabilities sink under the weight of added costs for model development and use—when costs include not only staff and computer resources but also reduced capability for timely response, reduced flexibility, and reduced accessibility of the model? The issue of how wide a range of program interactions to simulate poses this concern as well.

Historically, policy analysts have asked microsimulation models, as a bedrock capability, to estimate the direct effects of proposed program changes on individual decision units, that is, the effects if one assumes no changes in people's behavior in response to the program (other than to accept or reject the program's benefits). Analysts have also typically asked models to estimate interactions among closely related programs, such as SSI, AFDC, and food stamps. At times, analysts have asked models to provide additional capabilities, such as the following:

• the ability to project, inside the model, the cost and population effects of a program change when a new policy is expected to be implemented and for some number of years into the future (referred to as the process for "aging" the data);

• the ability to estimate, inside the model, the likely extent and effects of program-induced behavioral responses, such as the likelihood that people who are ineligible for income benefits will behave so as to become eligible (e.g., by quitting their jobs in order to obtain AFDC and Medicaid benefits) or that households will change their investment patterns in response to tax incentives;

• the ability to estimate, inside the model or through links to other models, so-called second-round effects, that is, the longer run effects of program changes—for example, the effects on wage rates of labor supply responses to welfare program changes, or the effects on health care providers of changes in the demand for health care services induced by mandated changes in the proportion of costs that patients have to pay themselves; and

• the ability to model interactions among a wide variety of programs across several issue areas, such as income support, taxation, retirement income, and health care benefits.

The majority of microsimulation models today incorporate some, but far from all, of these capabilities. For example, most models analyze interactions among multiple programs (although, typically, within just one or two issue areas), but few of them analyze second-round effects or, indeed, first-round behavioral effects other than program participation choices. Yet, in simply modeling the complex provisions of the nation's income support programs or tax codes, today's microsimulation models are already very complicated and difficult for anyone other than experienced analysts to use or modify to respond to changing policy concerns.

The complexity that reflects the real world of policies and individual circumstances is an inherent feature of microsimulation modeling, so it is especially important to minimize unnecessary complexities. We believe models should follow design strategies that enable them to meet the three criteria that we believe are essential for their cost-effective use in the policy process:

• flexibility and timeliness in responding to emerging policy needs;

• understandable outputs that can be assessed in terms of their quality; and

• tolerable costs of model development, use, and modification.

These requirements argue strongly for a design approach that has the following characteristics: clear analytic goals, self-contained modules, facilitated documentation and evaluation, linkages, computational efficiency, and accessibility to users.

It is important that the main analytic goals of a model are well thought-out and that development work gives priority to those goals, accepting limitations with respect to other possible goals. In other words, no one model can or should try to be all things to all people. Examples of models that foundered because the goals were far too ambitious, particularly given the restricted capabilities of the computer hardware and software technology available at the time, litter the history of microsimulation model development in the United States (see below).

We encourage broad conceptual thinking about model goals, and we support, when appropriate, model designs that are ambitious in their scope. For example, there are strong arguments that future model development in the health care policy world (whether for models of health care consumers or providers or both) ought to simulate behavioral responses and second-round effects as well as the direct effects of policy changes (see Chapter 8). Nonetheless, we argue strongly that model developers must make explicit choices about priorities. If they design a model that is broad in scope, then they should plan to develop only some of the model components in depth. To continue with the example, the developers of a behavioral health care policy model might pick one type of response to model in detail, while constructing a much cruder model of other responses. Conversely, a very detailed model should most likely restrict the range of issues or behaviors that it encompasses.

We are hopeful that current and expected future improvements in computer technology will make it possible to build more ambitious models that do not crumble under their own weight. Improvements in research knowledge and databases will also support this goal. However, we still see the need to trade off model breadth and depth. If such trade-offs are not made, the model development process is likely to abort or result in a model that is dysfunctional, wasting scarce resources.

Good design should be based on self-contained modules that can be readily added to (or deleted from) a model. As a corollary, the design should permit an analyst to specify all or only a subset of the existing modules to be used for an application. Furthermore, the modular principle applies not only to the components of the model that simulate programs such as AFDC or SSI, but also to the various steps that are used to generate a model database. Finally, each module should be highly "parameterized," that is, allow the user to change various features of the module by simply supplying a new value for one or more parameters instead of writing new code. For example, it should be possible to change the value of allowable deductions in the formula for calculating AFDC benefits or to change the value of the factor used to adjust income amounts for underreporting.

In our view, a well-specified modular structure is one of the most important means for enabling a model to respond in a timely and flexible manner to new policy issues, while minimizing development costs because the entire model does not need to be rebuilt. Similarly, a modular structure minimizes costs of model use because only those modules that are required for a particular application have to be used. Looking to the future, it may be that modular designs implemented with enhanced computer technology will enable microsimulation

models to become bigger in scope than current models without becoming "too big."²

A good design provides for features in each module that facilitate documentation and evaluation. Although many existing microsimulation models have been designed along modular lines, relatively few have sought ways to improve the production of model documentation, and none has paid sufficient attention to model validation.

Good documentation is essential to enable people other than a model's original developers to understand the model and modify it in a cost-effective manner and to evaluate the interactions of model components and their effects on the quality of the model output. Good model design should seek ways to reduce the costs and time of preparing complete documentation, such as providing for self-documenting program code in each module and automated audit trails that can track the effects of modifications in a module on other model components (see David, 1991; see also Chapter 10).

Good model design should also seek ways to facilitate the evaluation of model components, separately and collectively. To date, the task of model validation has been costly and time consuming and, partly for those reasons, has been generally ignored. Providing features such as the ready ability to use different procedures for constructing variables in the database could greatly facilitate the conduct of validation studies, including sensitivity analyses and estimation of variance in the model results through sample reuse techniques. Again, developments in computer technology should make possible great strides on these dimensions of good model design.

Good design provides for entry and exit points in the model that facilitate linkages with other models. In addition to a modular structure, another way to tackle the problem of providing enhanced capability without designing an overly elaborate and cumbersome model is to build in pointers that make it possible to use more than one model. For example, with such a design, a microsimulation model could feed results to a macroeconomic model in order to obtain a reading of second-round effects of proposed policy changes and, in turn, use outputs from the macroeconomic model in running further microsimulations.

Good design strives to attain a high degree of computational efficiency of a model and its components consonant with other important design objectives. It is obvious to assert that the design of computer-based models should strive to minimize computational costs, and, indeed, developers have historically given high priority to this aspect of model design. However, declining costs of computing mean that it may be feasible, and perhaps desirable, to give up some computational efficiency for other goals, such as ease of use and ease of modification of the model. In any case, in considering ways to reduce computing costs, developers need to consider not just the costs of a single run but the combined costs of all of the runs that are necessary for a particular application.

Good model design incorporates features that provide a high degree of accessibility of a model to analysts and other users who are not computer systems experts. At present, policy analysis agencies generally rely heavily on a few expert analysts, often affiliated with a contractor, to carry out microsimulation model applications. In turn, the analysts rely on programming staff to make changes in the model and even to set up model runs. Although, to our knowledge, the experts have met high performance standards, we believe that the cost-effectiveness of microsimulation modeling for the policy process would be improved by making the models accessible to more people.

This goal requires making the models more "user friendly" to people who are not computer buffs, as well as providing adequate documentation, training, and support services. Greater accessibility should reduce overall costs for model applications by making it possible for analysts to run the models themselves (at least for straightforward applications), should enhance the analysts' understanding of the properties of the model, and should encourage greater experimentation with the models and closer scrutiny of the quality of the model outputs.

Recommendation 6-1. We recommend that policy analysis agencies set standards for the design of future microsimulation models that include:

• setting clear goals and priorities for the model;

• using self-contained modules that can be readily added to (or deleted from) the model and that are constructed to facilitate documentation and validation, including the assessment of uncertainty through the use of sensitivity analysis and the application of sample reuse techniques to measure variance;

• providing for entry and exit points in the model that facilitate linkages with other models;

• attaining a high degree of computational efficiency of the model and its components consonant with other objectives such as ease of use; and

• attaining a high degree of accessibility of the model to analysts and other users who are not computer systems experts.

In addition to adopting good design principles, it is important to follow good practices in the actual implementation of a microsimulation model design. Such practices include careful development, full documentation, validation, and follow-up assessment.

The development process has to be carefully staged, with work directed toward intermediate goals or milestones, so that progress can be assessed and timely corrective action taken if there are undue delays or budget problems. In addition, the process should include the construction of prototypes of model components so that design flaws can be identified and corrected at an early stage. Prototypes or test versions of the model can also provide the sponsor agency with some analysis capability before the model is completed, an important consideration if the development process is expected to extend over several years.

Fully adequate documentation should be prepared on a timely basis for the model, its components, and its database. Not only is it important for the design to facilitate documentation, but the development process must assign sufficient priority and resources so that good documentation does, in fact, result. Ideally, documentation is developed on a flow basis, including an audit trail of all modifications, instead of being left to the end.

It is important to carry out validation studies of the model and its components, including sensitivity analyses as well as estimation of variance. It is particularly important to perform sensitivity analyses for each new module prior to full implementation, including studies of the impact on the rest of the model, in order to identify any unexpected or dysfunctional interactions or adverse effects on use. Again, the development process should assign sufficient priority and resources to this task.

In our experience, model developers typically follow good practice with regard to validation in the development of individual components of modules. For example, standard procedure in developing a regression equation for imputing variables, such as child care deductions for the AFDC program, is to look at various measures of goodness of fit before determining the final form of the equation. However, model developers are much less likely to look at the effect

of alternative forms of an equation on the operation of the module as a whole or of alternative modules on the operation of the entire model.³

Finally, once a model is in use, developers should periodically take stock of the model's capabilities and functioning. That is, they should conduct regular "sunset" or "zero-based" reviews, in which they reevaluate the structure of the model, determine components that have outlived their usefulness and need to be dropped, and identify other components that need to be rebuilt. Such zero-based reviews are important because models constantly undergo revision and adaptation to respond to users' changing requirements for estimates. Without regular reevaluations, it is all too likely that a once well-designed model will become increasingly less cost-effective over time. It is important that policy analysis agencies build into their budgets provision for reassessing and reoptimizing the microsimulation models they use for input to the policy process.

Recommendation 6-2. We recommend that policy analysis agencies set standards of good practice for the development of future microsimulation models that require:

• constructing prototypes and establishing milestones throughout the development process so that design flaws can be identified at an early stage and the agency provided with some analysis capability before the entire model is completed;

• preparing fully adequate documentation on a timely basis for the model and its components;

• conducting validation studies of the model and its components, including estimates of variance and sensitivity analyses (the latter should be conducted for each new module, prior to full implementation, by examining its impact on the rest of the model in order to identify any unexpected or dysfunctional interactions or adverse effects on use); and

• subjecting the model to a "sunset" provision, whereby the model is periodically reevaluated, obsolete components are deleted, and other components are respecified to optimize the model's usefulness and efficiency.

Many current microsimulation models have been constructed on the general principles that we advocate. But some modeling efforts have foundered because (in addition to whatever data and other problems occurred) the design was too grandiose, the structure did not provide for sufficient modularization, or the documentation was inadequate to permit easy updating or adaptation of model components. In addition, some models that were initially well designed became encumbered with ill thought-out additions or obsolete components and lost significant flexibility and ease of use.

In the history of microsimulation modeling, RIM (Reforms in Income Maintenance) and the KGB (Kasten, Greenberg, Betson) model represent notable examples of models that were developed under acute time pressures; asked to simulate more and more proposed policy changes; and then, having collapsed under their own weight, abandoned. Insufficient attention to documentation, modularity, and computing efficiency made them increasingly difficult for their own developers—let alone other analysts—to use. The developers of TRIM (Transfer Income Model) originally tried to rewrite RIM but gave up the attempt and instead developed an entirely new model; the KGB model wound up on the shelf after its developers left the originating agency, ASPE.

Other models have been constructed according to good design principles but, over the years, having become increasingly more cumbersome and difficult to access. Thus, a major impetus to the development of the KGB model was the perception that TRIM, as it then existed, could not readily be adapted to the requirements of simulating a combined income support and jobs program and that it would take too many resources to conduct such simulations with the MATH (Micro Analysis of Transfers to Households) model. The current TRIM2 model is the result of rewriting TRIM to achieve greater efficiency and modularity. The MATH model was also rewritten to improve computing efficiency.

Other modeling efforts have suffered from being overly ambitious, particularly given the limitations of the computing technology at the time of their development. The original DYNASIM (Dynamic Simulation of Income Model) was intended to model a wide range of human behaviors and policy issues and to be accessible to a broad research community. The goals were laudable, but the costs of running the model on more than a very small sample of households proved prohibitive. DYNASIM2 is a scaled-back version that was redesigned to maximize computational efficiency and to focus on policy issues surrounding the provision of adequate retirement income support. The MRPIS (Multi-Regional Policy Impact Simulation) model has also experienced high costs and long development times in its developers' effort to construct a complex system of fully linked models. The system includes a detailed microsimulation model of the household sector, together with input-output and cell-based models that

simulate the effects of changes in household disposable income on savings and consumption that, in turn, affect regional economic production and, ultimately, in a feedback loop, affect household employment. To date, MRPIS has had only limited application.

Overall, the history of microsimulation has exhibited a pattern of advances and retreats. Advances occurred as new capabilities of various kinds were added to the models to respond to policy needs. Invariably, there came a time when the added capability proved dysfunctional for the continued cost-effective operation of the model. Then came retreat. Historically, the response has been (if the model was not simply shelved) to redesign the model for maximum efficiency with the available computing technology. Another typical response has been to refocus the model on core modules: for example, current maintenance and development efforts for the MATH model are limited almost entirely to those modules that are necessary to simulate the food stamp program. Other modules have been largely shelved.

Today, the major microsimulation models are optimized for the mainframe, batch-oriented computing technology that prevailed in the late 1970s and early 1980s. Although this strategy has greatly reduced computing costs, the models are not well positioned to move toward adding new capabilities. Development as well as operation of the models can entail substantial overall costs because many applications require multiple runs and the intervention of skilled programmers to make necessary changes in the program code. Moreover, the linearity embedded in the models' hardware and software configuration imposes constraints, sometimes quite severe, on analysts' flexibility of use. For example, once a new TRIM2 or MATH baseline file has been constructed, it is very expensive to go back and alter its characteristics, say, to age the data forward in time if aging has not already been done or to construct different kinds of program filing units. This inflexibility also inhibits evaluation studies of models.

Today, the major models—particularly the cross-sectional "static" models such as TRIM2 and MATH—are focused on those capabilities that are central to simulating the direct effects of proposed program changes. The core of these models is the set of modules that mimic the detailed operation of government programs, and new model development in the past decade has generally focused on further elaborating these modules or developing modules for additional programs. For example, the federal and state income tax modules in TRIM2 were completely rewritten and expanded in the mid-1980s, as was the TRIM2 Medicaid module. This type of development increases the breadth of a microsimulation model in terms of the policy areas for which it is relevant. Another example of adding breadth, in this case to a dynamic model, is the recent development of a module for simulating long-term care financing programs in PRISM (Pension and Retirement Income Simulation Model).

In contrast, development of the kinds of capabilities that increase the depth of a microsimulation model in terms of simulating the effects of a proposed

policy change has either not taken place or resulted in less-than-satisfactory modules that were later mothballed. Examples of depth include functions for aging the data and simulating first-round and second-round behavioral responses inside a model. All of the major static models include aging modules; however, this capability in TRIM2 has not been used for policy purposes since the early 1980s. As we have noted, none of the static models currently includes modules for behavioral response (other than the participation decision) or for second-round effects.⁴ The modules that were developed for MATH in the mid-1970s to simulate work responses to welfare program changes have been dormant for many years.

The dynamic models, such as DYNASIM2 and PRISM, that are used to model long-range policy issues involving retirement income and other areas obviously include aging capabilities. Indeed, aging is the focus of these models. They also make heavy use of research knowledge about behavior to build their databases. However, they incorporate relatively few feedback effects of future policy changes on behavior, except for the decision to retire, which is the functional equivalent to the program participation decision in the static income support models.

Many factors have resulted in the situation we just described. The need to keep their models competitive in the policy analysis market has obviously motivated the model developers (and the analysts who use the results) to strive for reduction of computer costs and sufficient streamlining so as to facilitate their use.

In turn, the predilections of policy makers (and their staffs) have exerted a strong influence in the direction of emphasizing the models' accounting elements—that is, the detailed program rules—in preference to other capabilities. Particularly in the Gramm-Rudman-Hollings era, policy makers have often been more concerned about the immediate consequences of a proposed welfare or tax policy change for next year's budget deficit, for example, than about longer term behavioral effects. They have also been interested in obtaining results for very fine-grained changes in programs. An added factor in reinforcing emphasis on the accounting modules is the concern of the staffs in the policy analysis agencies to have credible models whose baseline results closely match available administrative data on program recipients, although, as noted above, the process of calibrating model results to control totals carries its own perils (see Chapter 5).

Last, but far from least, an important factor in the underemphasis on capabilities such as modeling behavioral response is the skimpy base of research knowledge on which to develop these kinds of capabilities. Social science is far from being able to predict the interactions of individuals with government policies and other social and economic forces with the precision that would be

desired for policy analysis purposes or in a form that readily adapts to models. Given the thin research knowledge base and the costs of implementing new model capabilities in the current microsimulation model computing framework, it is no wonder that the model developers have focused their attention on the accounting functions of their models.

The question we address in this section is what paths the policy analysis agencies should follow for the future of microsimulation model development.⁵ Clearly, a critical line of attack for building improved microsimulation models involves taking advantage of recent developments in computer hardware and software that will not only facilitate expansion of model capabilities at reduced costs, but also increase their flexibility and ease of use. (This topic merits a full discussion of its own, which we provide in the next chapter.) In this section we assume that a successful jump to a new computing environment will be made and, hence, that it is timely once again to consider expanding the capabilities of microsimulation models.

Three major types of capabilities might be incorporated or enhanced in future microsimulation models: (1) aging the data forward in time, (2) simulating behavioral response to proposed policy changes, and (3) simulating second-round effects of policy changes. It is important to note that our recommendations in each area do not prescribe a preferred path for microsimulation model development. In the absence of evaluation studies, there is simply not enough information—for example, about the merits of static versus dynamic aging techniques, of omitting or incorporating behavioral effects, or of limiting models to first-round effects or simulating second-round or later effects—to reach specific recommendations about preferred model capabilities. Even with greatly enhanced computing power at lower costs, it is important to consider carefully the addition of complex new capabilities for which the needed research knowledge may be very costly to obtain and to determine effective ways of implementing that knowledge in models. Thus, our recommendations concentrate on a research agenda with two major goals: to identify the kinds of enhancements to microsimulation models that are important because of their impact on the resulting policy estimates and to provide an improved knowledge base to support the effective implementation of key new model capabilities.

Microsimulation models share with other kinds of models the need for updating

or projecting available data so that they can be used to develop current or future-year impacts of existing and proposed new programs.⁶ The need for this process, commonly called ''aging," arises for several reasons:

• Needed data inevitably become available with a time lag, so that even current-year estimates require aging.

• Programs are never adopted instantaneously so that aging of the data is required to the year when a program change is expected to take effect.

• Policy makers typically want estimates of the costs and population impacts of proposed changes for a period of several years beyond the date when the change is to take effect.

• Policy makers sometimes want to see the impacts of past or proposed changes over much longer time spans, 30 or more years for programs for retirement income or health care of the elderly.

Although the need for aging databases forward in time confronts virtually all types of policy analysis tools, microsimulation models present particularly difficult problems due to the large number of variables they include. Moreover, the choice of aging strategies, whether static or dynamic, has far-reaching implications for the design and nature of the resulting model. The complex issues presented by aging in microsimulation models have led some policy analysis agencies to forgo aging inside the model and, instead, conduct simulations on the latest available data and then extrapolate the simulated values to develop future projections outside the model.

The dimensions of a database most often of concern in aging analyses are its population proportions and its economic variables. The age-race-sex and family type proportions in a population change over time, and these changes are likely to affect the aggregate simulated values of most policies under consideration in microsimulation models. Likewise, real growth as well as nominal growth in incomes and prices are both certain to occur over time and likely to affect simulated values. Another variable often considered for aging is the unemployment rate, because a cross-sectional database will by definition capture only one point in the business cycle, a point that may not be of interest to policy makers.

There are, roughly speaking, two main and very different aging strategies used in existing microsimulation models. First, many models (e.g., MATH) employ static aging of the population, a technique by which the individual weights for each record on a micro file are adjusted to meet a set of projected population

proportions; monetary variables are scaled upward in a separate exercise according to outside macro forecasts; and forecasted changes in unemployment are distributed across the micro units in some fashion in a third and yet separate exercise.⁷ Static aging is the technique most commonly used in microsimulation models because of its relative ease of implementation.

Second, some models (e.g., DYNASIM2, PRISM) use dynamic aging, a technique by which individual units on a micro file are aged directly (see Caldwell, 1989; Ross, in Volume II). For each year that a database is projected, the age of each individual is increased by 1; economic variables are scaled upward by individual-specific magnitudes; and many other variables, if not all, are moved forward on an individual-specific basis according to outside, econometrically estimated transition probabilities and dynamic micro equations.⁸ Older individuals are deleted from the sample according to estimated mortality rates, and hypothetical newborns are added to the sample according to estimated fertility rates.

However, some users of microsimulation models do not use either type of aging capability. For example, analysts in ASPE ask the TRIM2 modelers to perform AFDC simulations on the most current database, which at best is 1-2 years out of date. The analysts then apply the percentage differences estimated by TRIM2 for total costs and caseloads under current versus proposed law to projections of current-law total costs and caseloads developed through other means.⁹ This method avoids any controversy over use of a particular aging method, but it shifts the controversy to the method used for the out-of-model projections. With regard to population effects of proposed AFDC changes, such as "gainers" and "losers," ASPE uses no aging technique at all. Instead, the ASPE analysts use the unadjusted TRIM2 distributions, assuming that there will be no population changes of any import. ASPE adopted its out-of-model aging strategy in the early 1980s, after deciding that the cost and effort required to

perform static aging inside TRIM2 and the uncertainty regarding the quality of the resulting database were unacceptably high.

In the literature on in-model aging strategies, the central trade-offs between the different methods involve cost, theoretical desirability, and reliability. Theoretically, the preferred method is dynamic aging because it fully incorporates cohort effects and therefore properly preserves all the covariances in the data between age and other variables. However, the reliability of dynamic aging methods may be low if the estimated transition functions necessary to implement the aging have high variance or suffer from serious misspecification. Furthermore, it may be difficult to implement a dynamic technique in a partial manner. If it is implemented at all, it should be implemented for all age-related variables on a database. For example, if 30-year-old individuals on a file are increased in age by 1 year, all other variables for their income, labor force status, earnings, number of children, marital status, education, and so on should also be dynamically aged. Any variable not so aged but left at its age-30 value may provide a less accurate indicator of its age-31 value than the values of that variable for the age-31 sample at the initial time period. In other words, any variable not dynamically aged may be inferior to the values that would be obtained from static aging. However, aging all variables is a time-consuming process, requiring the estimation of many sets of dynamic equations on panel data, with unknown reliability.

The major advantage of static aging is its comparative ease of implementation. Its theoretical weakness mirrors the strength of dynamic aging, namely, the failure to capture adequately the cohort effects in the variables on the file. Simply changing the weight on 30-year-olds, for example, fails to account for the differences in the values of the other variables among 30-year-olds of different cohorts.

Another issue in aging is how a short-run time horizon, as in the MATH and TRIM2 models, or a long-run time horizon, as in DYNASIM2 and PRISM, affects the relative advantages of static and dynamic aging. Although dynamic aging has traditionally been used for long-run projections and static aging for short-run projections, in general either method could be used in either instance. Dynamic aging may be theoretically superior in the short run as well as the long run (see Caldwell, 1989); however, the degree of damage done by static aging is probably less for short-run projections than for long-run projections, because cohort effects are not likely to distort seriously the age-related covariances of the variables in a database over a 3- to 5-year period.

The out-of-model aging technique adopted by users of TRIM2 avoids these controversies at the cost of shifting problems to a later stage in the development of policy estimates. If a model is run on unaged data and the results of the

simulation extrapolated to future periods outside the model, the assumptions related to aging are not eliminated but only moved to a different place in the simulation procedure. The method of extrapolation then must be based upon static aging, dynamic aging, or some other concept.

Only limited evaluations of microsimulation model aging techniques have been conducted to date. The panel, as part of its validation experiment using the TRIM2 model, varied the degree of static aging that was performed (see Chapter 9 and Cohen et al., in Volume II). Model runs included no aging at all (in or out of the model), aging of population characteristics only, demographic aging together with adjusting for price changes, and full aging (that is, demographic aging, price adjustments, and labor force adjustments). To eliminate uncertainties with regard to the quality of the control totals, a 1983 database was aged to 1987 by using known values. The results showed significant differences in projected 1987 estimates of AFDC costs, caseloads, and characteristics of recipients across the four aging methods and between each set of estimates and administrative records data from the IQCS for 1987.

Doyle and Trippe (1989) carried out a more limited study of the quality of demographic aging and price adjustments in a 1980 MATH model database projected to 1984 using known control totals. This study found significant differences between the aged database and the actual 1984 database, for example, in the distribution of households by income as a percent of the poverty line. Overall, however, the aging process produced a population that more closely resembled the actual 1984 population than did the unaged base-year file.

Both Doyle and Trippe (1989) and the panel's study found that, even in the short run, demographic and economic changes occur and are important for estimating program costs and caseloads. They also found that current static aging routines capture some but not all of the changes.¹⁰ However, neither study compared alternative forms of the TRIM2 or MATH model aging procedures (both models use virtually the same aging routines), such as varying the assumptions used to distribute projected changes in unemployment rates to individual records. Nor did either study compare dynamic with static aging techniques.

Evaluations of dynamic models have examined their performance on specific dimensions, such as their ability to reproduce historical patterns of labor force participation over time, compared with actual data from panel surveys such as the Michigan Panel Study of Income Dynamics (see Cohen, Chapter 7 in Volume II). These studies in some cases led to modifications in the models' transition functions. Dynamic models are also routinely calibrated for agreement

with aggregate forecasts of macroeconomic models. Again, however, there has been no systematic evaluation of alternative forms of dynamic aging.

Given the need for some method of updating and projecting the data required for future-year estimates of program impacts, policy analysis agencies should devote more attention to the question of appropriate aging strategies for microsimulation models. We suggest that systematic evaluation begin with methods for aging the databases used by models with short time horizons, such as MATH and TRIM2, because the evaluation task seems less daunting for short-term than for long-term projections. We propose an evaluation that would be much more extensive than, but similar in design to, the panel's own experiment: the aging methods would use known control totals (so as to eliminate errors due to mistaken population and economic projections) and would be evaluated against targets from program administrative records, including characteristics that are and are not related to the control variables.

A principal focus of the evaluation of aging methods should be to determine how well a method performs, on average, when it is assessed for a number of time periods. Extensions of an evaluation would be to vary the length of time over which the aging is performed, for example, from 1 to 5 (or perhaps as many as 10) years, as well as the aging method itself. The techniques to be evaluated should include the current static aging routines in TRIM2 and MATH, as well as variants of those routines (e.g., modifying the scaling factors in the economic aging routine to reflect varying patterns of income growth by socioeconomic status or using raking techniques such as iterative proportional fitting to make simultaneous adjustments to multiple sets of control totals). An evaluation should also include the out-of-model aging technique currently employed with TRIM2, comparing the administrative data with the model estimates after ASPE analysts have made extrapolation adjustments. It should also be possible to generate aged populations by using a dynamic model such as DYNASIM2, to run the TRIM2 or MATH program modules on these populations, and to evaluate how well the output matches the program administrative data. The results should be useful for charting the future development of all models oriented to the short term, whether they be models of income support programs, taxation, or health care policies.

Recommendation 6-3. We recommend that policy analysis agencies sponsor an evaluation program to assess the quality of estimates from current static microsimulation models as a function of the aging technique that is used and that they further support such evaluations on a periodic basis for future models.

In conducting our experiment, we found that the cost and time of implementing the various aging scenarios in TRIM2 were high.¹¹ The aging must clearly be carried out at an early stage in the preparation of the database, which then necessitates rerunning many other database and program simulation modules each time a different aging method is used. These costs are one reason for the out-of-model aging strategy currently followed with TRIM2; they are also why new aged databases for the MATH model are developed only every 2 or 3 years (in contrast, new (unaged) TRIM2 databases are created annually from each March CPS). This situation clearly has a chilling effect on efforts to evaluate and experiment with alternative aging procedures. One goal for future model development or redevelopment should be to find ways to make it easier to add and experiment with alternative aging routines and control values.¹²

Recommendation 6-4. We recommend that policy analysis agencies require that future static microsimulation models build in an aging capability in a manner that facilitates evaluation and use of alternative aging assumptions and procedures.

Most of the policy measures simulated by the models that we reviewed have potential behavioral effects on the individuals and organizations affected by the policy.¹³ Altering the structure of transfer programs for the low-income population, changing the tax code, or changing the form of health care benefit programs may all affect individual and organizational decisions in many ways:

• Altering the level of cash or in-kind (e.g., Medicaid) benefits in the AFDC program may affect the work decisions of welfare recipients.

• Changing the tax rate on capital gains may affect decisions on the part of investors as to when to realize their gains.

• Mandating employer provision of health insurance may affect the demand for labor by small businesses.

More generally, changes in transfer policy, tax policy, and health care policy can be expected to affect participation by individuals in transfer and health care programs, levels of work effort among program recipients or the population as a whole, levels of health care expenditures and other health outcomes, and levels of investment and other variables affected by the policies under consideration.

Conceptually, one can distinguish two separate functions of microsimulation modeling: the accounting function of aggregating the effects of a policy change across the individual units in a given population and the function of predicting the behavioral response to a policy change. The first function is the more basic and must precede the second. This section considers the desirability and feasibility of incorporating behavioral responses into microsimulation models in addition to their accounting structures. We consider initially only first-round behavioral effects and hence only partial responses to policy changes; later, we discuss the incorporation of second-round effects.

The documentation for some of the models that we reviewed left us in doubt as to the extent to which the model incorporated behavioral responses. Nonetheless, it is clear that in the microsimulation models currently in operation, as well as those implemented in the past, the incorporation of behavioral response has been the exception rather than the rule.

MATH and TRIM2 do not currently have operational components for behavioral response in labor supply (or other dimensions) to policy changes in transfer programs; thus, the models almost exclusively serve the accounting function. Previously, however, labor supply modules were developed in both MATH and KGB to simulate the labor supply effects of the Carter welfare reform program, and a labor supply module is currently under development in TRIM2.

The dynamic microsimulation models of retirement income programs (e.g., DYNASIM2 and PRISM) make use of many behavioral parameters in developing their longitudinal databases, for example, using probabilities of job change as a function of variables such as age and previous job experience to build individual employment and earnings histories. However, they model only a limited range of the likely behavioral responses to proposed changes in retirement income programs. Thus, in neither model does the level or potential level of pension and social security income affect hours of work or earnings of retirees or nonretirees. Rather, changes in retirement income programs affect labor supply only by their effects, if any, on the decision to retire at a particular age (see Burtless, 1989:20-27; Ross, in Volume II).

For tax policy, most microsimulation models are tax calculators that simulate the revenue effects of changes in the tax code. Tax models typically include a function to decide whether or not taxpayers will itemize, on the basis

of the assumption that they will choose to minimize their tax liability. Other behavioral responses are also frequently taken into account, but not inside the models. Rather, the agencies and congressional committees using tax simulators adjust their accounting estimates for behavioral response outside the models, for example, adjusting the revenue estimates of a change in the capital gains tax rate to reflect assumptions about the effects on the timing of sales of capital assets (see Strauss, 1989).

For health care policy, strong behavioral responses have been documented in some cases: for example, hospitals responded when Medicare implemented a prospective payment system by reducing the average length of stay (see Grannemann, 1989). However, behavioral responses are rarely or only crudely incorporated in health care policy models at the present time.

As we have noted, the one behavioral response that is incorporated in MATH, TRIM2, and other microsimulation models for income support programs is participation in such programs. Given that some proportion of eligible households voluntarily choose not to apply for benefits, the models must incorporate the determinants of the participation decision. The analogous behavioral response function in models of retirement income programs is the decision to accept retirement benefits, which entails a corresponding decision to stop work or reduce hours.¹⁴

There are many issues that must be considered in an assessment of the desirability and feasibility of incorporating behavioral response in microsimulation models. First is the potential magnitude of behavioral responses that are thought to occur. Clearly, the greater the potential for a behavioral response, the more seriously it merits consideration for incorporation into models. The level of the potential response is likely to differ by the type of policy considered, the magnitude of the policy stimulus, the type of individual unit whose response is under consideration, and the type of behavior being examined. A related issue comes into play when the microsimulation modeling is designed to rank-order different policy alternatives rather than to obtain the best estimate of a single alternative. In this case, the relevant question is whether the potential magnitude of behavioral response across the alternatives being ranked is likely to affect their respective rankings.

A second major consideration is whether there are generally agreed-upon and reasonably reliable statistical estimates of the magnitude of a behavioral response in order to simulate it in the first place. To some extent, this issue

precedes the first one, because judgment of the potential magnitude of behavioral response presumes the existence of relatively good estimates. Nevertheless, it is often the case that the statistical estimates available from the research literature are sufficiently large to warrant concern over behavioral response but far from sufficiently reliable to give any degree of confidence to incorporating them in models.

Unfortunately, it appears that this situation is the rule rather than the exception for social welfare policy issues. For income support issues the variances of estimates of the work-effort effects of welfare programs are often large and, more important, the estimates obtained from different studies range widely (see Burtless, 1989). The same characterization applies to tax policy estimates, such as the effect of capital gains taxation (see Strauss, 1989). Estimates of other kinds of behavioral responses with important policy implications, including charitable giving, demand for health services, and demand for health insurance, also vary widely (Strauss, 1989:Tables 4-1 to 4-10). In many cases, it appears that research designed to replicate and reconcile the different findings from different studies could narrow considerably the range of estimates in the statistical literature. However, studies that attempt such a narrowing of the range of estimates are rarely conducted.¹⁵

Because of the problem of the unreliability of response estimates, there is a need to develop concrete and well-defined methods for assessing the uncertainty of behavioral response parameters if they are incorporated in microsimulation models. If a single estimated parameter is introduced in a microsimulation model, the variance of that parameter must be translated into variances of the aggregates simulated from the model. If multiple parameters are introduced, their covariance must be similarly incorporated; this task is often difficult because the different parameters are drawn from different studies. In addition, the uncertainty of parameters resulting from the range of estimates in the statistical literature must be assessed through some type of sensitivity analysis and the findings reflected in the caveats that accompany the output from a model. The degree of uncertainty from the range of estimates may often be much larger than that from the variance of any single estimated parameter.

We have not found any systematic attempt in microsimulation modeling efforts to date to address the problem of measuring the degree of uncertainty in model outputs attributable to the use of particular behavioral parameters

or to alternative specifications for those parameters.¹⁶ Indeed, there has been little attention to the development of scientifically based methods for measuring uncertainty in the accounting function estimates of the models. Yet the assessment of uncertainty with regard to the estimation of behavioral response should not proceed without a similar assessment for the no-response estimates from microsimulation models that are obtained in the first place (see further discussion of this set of issues in Chapter 9).

The third major consideration concerns the problems involved in actually incorporating behavioral parameters in microsimulation models in an appropriate manner. This task is not always straightforward (see Burtless, 1989; also Grossman and Watts, 1982). The database used to estimate a behavioral response is usually not the same as the database used by the microsimulation model. Thus, researchers have typically estimated labor supply and other responses on panel data sets such as the Michigan Panel Survey of Income Dynamics and the Retirement History Survey. However, these surveys, because of small sample size, limited populations, and other reasons, are rarely suitable as simulation databases. In contrast, large, regularly updated, nationally representative surveys that are heavily used for simulation, such as the March CPS income supplement, generally lack the richness of information desired for estimation. For example, the March CPS does not obtain direct measures of hourly wage rates, so that modelers must impute wage rates to the CPS records in order to simulate a labor supply response to a policy change. As Burtless (1989) suggests, a useful, but rarely performed, step to help assess the impact of basing the simulation on imputed values would be to compare the estimated response in the original data set using both actual and imputed variables.

A related concern is that the available behavioral specifications may reflect an inappropriate degree of aggregation in relation to the microsimulation model. Thus, a good deal of behavioral analysis is carried out in terms of a representative economic agent, although microsimulation models are designed to carry out simulations on diverse kinds of units. For example, using a specification that predicts a uniform decline in work hours in response to an increase in welfare benefits fails to take advantage of the capabilities of microsimulation models for detailed analysis of population subgroups. Moreover, use of such a specification may distort the results, if, in fact, the pattern of response is that some workers drop out of the labor force entirely while the rest maintain their work hours.

Furthermore, the interaction of a behavioral response module with other aspects of a microsimulation model may produce anomalous effects. Burtless (1989) reported a comparison of the labor supply effects of four negative income tax plans simulated by the MATH model and a variant of MATH developed

by SRI International (called TATSIM). The two models used the same labor supply functions, estimated from the Seattle and Denver Income Maintenance Experiments: MATH simulated that aggregate labor supply and earnings of single mothers would decline under all four plans; TATSIM simulated that labor supply and earnings for such women would rise. These discrepant results could not be attributed to the behavioral response function itself; rather, they must have been caused by differences in the two models' simulation of the baseline prereform policy environment.

The fourth major consideration concerns the means by which behavioral responses are presented to policy makers—provided that reasonably good estimates are available, that they can be incorporated into a microsimulation model, and that they are of sufficient magnitude to be important for policy purposes. One issue is whether simulations should be presented both with and without the behavioral response included or just including the response. In the case of cost estimates, it may be that the simulated values should be provided to policy makers with the behavioral response, because such response will generally affect costs. However, in the case of other estimates more directly related to the response variable in question, such as labor supply, health expenditures, or capital gains realizations, it may be that the simulated values should be presented both with and without response, because the magnitude of the response itself may be of interest to policy makers.

Another issue involving presentation is how to convey the uncertainty in the estimates to policy makers. Many policy analysts argue that only the best possible point estimates should be provided because policy makers will base their decisions on such point estimates and not on a range, even if it is presented. We argue throughout this report that policy makers should be made aware of the degree of uncertainty in the simulated values either directly or indirectly so that they can take uncertainty into account in their decisions.

Over the past decade there has been little investment in capabilities for simulating behavioral response inside microsimulation models. There has been even less investment in the kinds of research and evaluation studies that would permit charting a sensible course for the development of future behavioral components of such models.

Concern about the cost implications of major development work has been one important factor in creating this situation. As we noted above, the nature of the legislative process in this decade—in which policy makers have been most concerned about total costs and distributional effects only in the very short run—has limited the motivation to add behavioral responses to models. The absence of a body of research demonstrating the impact on model estimates of incorporating behavioral parameters has further dampened interest, while the absence of sufficient research to produce parameter estimates within a reasonably narrow range or in a form suitable for microsimulation modeling has made the modelers cautious about expending much effort in this direction. In short,

there has been a self-perpetuating cycle in which lack of investment leads to downplaying the need and also makes it more difficult to incorporate behavioral response in models, which, in turn, leads to continued lack of investment.

Yet, undoubtedly, there are important behavioral effects of proposed policy changes about which decision makers should be informed. Certainly, legislative initiatives such as the 1988 Family Support Act and the 1986 Tax Reform Act were predicated on the assumption that policy changes would in fact lead to behavioral changes—for example, that transitional Medicaid and day care benefits would make work more attractive to AFDC recipients and that lower tax rates would stimulate investment and economic growth. But these effects were not simulated, nor with current modeling capability could they have been simulated; instead, to the extent that the effects were treated at all, they were handled with ad hoc, out-of-model adjustments. Yet microsimulation modeling cannot continue to ignore these behavioral response issues and expect to remain intellectually strong or policy relevant in the future. (In Chapter 8 we note the particular importance of incorporating behavioral response effects for modeling policy changes related to health care. Such responses are also very important for modeling tax policy issues.) Thus, the goal must be to devise a strategy for development of enhanced modeling capabilities in the area of behavioral response that makes the best use of scarce investment resources.

Clearly, a balance is required between the two extremes of always and never developing in-model estimates of behavioral responses. Research is required to determine the appropriate mix. In some cases, the best method of obtaining estimates may be through behavioral response studies separate from microsimulation models, perhaps involving the application of microeconometric behavioral research results to aggregate estimates of key population subgroups. In other cases, it may be that estimates of the behavioral effects of policy changes are best obtained within the structure of microsimulation models themselves. Or it may be that a dual strategy should be followed in which behavioral response estimates are obtained in both ways. We note that, even where it appears more cost-effective to obtain estimates of behavioral responses outside microsimulation models, it is quite possible to use the models to generate estimates of the population groups that are most likely to have a response in order to obtain a rough assessment of the likely overall impact on program costs and caseloads.

Recommendation 6-5. We recommend that policy analysis agencies devote resources to studies of the relationship between behavioral research and microsimulation modeling, including studies of ways in which research and modeling can complement one another, as well as ways in which the two are alternative modes of deriving answers to policy questions.

An important criterion for deciding to spend the resources necessary to imbed behavioral response capabilities in microsimulation models is the likely magnitude of the impact. Hence, policy analysis agencies need to determine the kinds of behavioral responses to policy measures that could be important to consider. If behavioral responses are potentially large for some policy issues, and if it is decided to build some or all of those responses into microsimulation models, policy analysis agencies should attempt to reduce the extent of uncertainty surrounding existing statistical estimates of such responses. For many reasons, existing incentives for research, especially in academic circles, do not lead to the replication studies, additional data analysis, and sensitivity testing that are needed to reconcile the diversity of estimates from different analyses. Policy analysis agencies will need to provide those incentives by supporting such work.

Recommendation 6-6. We recommend that policy analysis agencies sponsor studies to determine when behavioral response effects are most likely to be important in different policy simulations and, hence, how investment in developing behavioral response capabilities in microsimulation should be concentrated. On the basis of such studies, policy analysis agencies should commission research to attempt to narrow the range of statistical estimates of behavioral parameters that may be of major importance to critical policy changes. Such research may require additional data analysis, replication studies, and multiple econometric analyses that use different data sets and analytic techniques.

One of the major issues regarding the incorporation of behavioral response parameters into microsimulation models concerns measurement of their uncertainty and the resulting impact on the uncertainty of the model outputs. After the analyses that we recommend above have been conducted, some uncertainty will still remain, both because the different results of varying studies will prove difficult to reconcile and because all behavioral estimates have variance. How to incorporate that uncertainty in measuring the total uncertainty of the estimates from a microsimulation model is far from obvious and requires the resolution of difficult technical issues (see further discussion in Chapter 9).

Recommendation 6-7. We recommend that policy analysis agencies commission methodological research to develop methods for systematically assessing the impact on microsimulation model estimates of the degree of uncertainty in the behavioral parameters that are used—both the uncertainty arising from the variance of specific parameters and that arising from the range of estimates from different behavioral studies. This work should be tied into the development of similar methods for assessing uncertainty of the estimates produced by microsimulation models without behavioral response.

As discussed above, the first function of a microsimulation model is the accounting, or aggregation, function; a second function is the simulation of behavioral response to a policy change. Here, we discuss a third possible function of microsimulation models: simulation of the second-round effects of a policy change.¹⁷ By second-round effects, we refer not to the immediate responses by individual economic units directly affected, but to effects that alter the nature of factor or product markets or the level and distribution of consumption, production, and employment in the economy or in a sector of it affected by the policy change.

Examples of potential second-round effects are many and pervasive:

• A change in a transfer program that alters labor supply may change the wage rate in the labor market and therefore further change labor supply. In addition, in the short run, prior to an equilibrating change in the wage rate, the unemployment rate may be affected and displacement or replacement effects may occur.

• Altering the coinsurance rate in Medicare or health care benefit programs in general will affect the demand for health care and therefore its price. If care is rationed because prices are inflexible, the policy change may affect the amount of health care in the market.

• Lowering the marginal tax rate in the federal income tax code will affect the tax price of all deductions and therefore affect the demand for the goods that provide such deductions, such as housing.

• Changes in transfer, health care, or tax policy will generally affect the level of consumption of different goods, which will affect employment in different industries, which, in turn, will affect the distribution of income in the population.

The incorporation of second-round effects has been studied for many years by economists and others in the field of macroeconomic policy modeling. Indeed, the development of macroeconomic models preceded the development of microsimulation models, and early microsimulation modelers such as Guy Orcutt explicitly looked to macroeconomic models as sources of input. Thus, Orcutt

proposed originally to build a macroeconomic component into DYNASIM that would interact with the microsimulation components (Zedlewski, 1990). In addition to this type of interaction, the development of input-output models has provided another avenue for linkage with microsimulation models, and input-output models have been incorporated in some microsimulation models even on a regional basis (Haveman et al., 1980). Inadequate documentation made it difficult for us to determine the detailed structure of many such models. To focus our efforts, we examined two types of second-round models: computable general equilibrium (CGE) models and the Multi-Regional Policy Impact Simulation model.

CGE models, such as those developed by Shoven and Whalley (1984), simulate the equilibration of a full set of interconnected markets in an economy and permit full long-run adjustment of prices to changes in supply and demand.¹⁸ CGE models are calibrated by a process of setting parameter values for supply and demand elasticities drawn from the econometric literature or picked to fit the available data on market prices and quantities. Although CGE models as generally implemented are not micro in nature, it is possible to develop disaggregated CGE models or to link them with microsimulation models by iterating back and forth between them until market equilibrium is reached (see, e.g., Berkovec and Fullerton, 1989; Slemrod, 1985).

The MRPIS model simulates shorter run responses (see Havens and Clayton-Matthews, 1989). Its focus is on modeling the impact of program changes on regional product mix and therefore on regional employment. Program changes, such as welfare reform or tax reform, are simulated first on a microlevel database with a microsimulation model, and marginal propensities to consume different goods are then applied to the changes in household income on the micro file. The implied region-specific changes in consumption demand are then applied to a regional input-output matrix to obtain regional employment demands by occupation. These changes in employment are then allocated across the individuals in the original micro file, which in turns generates income changes that are used to begin the process anew. The model iterates to convergence. All prices and wages are held fixed during the iteration.

Many of the same issues and trade-offs discussed for the incorporation of behavioral response in microsimulation models apply to the incorporation of

second-round effects. First, the potential empirical importance of second-round effects should be a primary determinant of whether attempts to incorporate such effects are undertaken. Second, the uncertainty surrounding the estimates must be an important concern. Third, the best manner to present estimates of second-round effects to policy makers—if the effects are of sufficient magnitude and the estimates of sufficient quality—is an important issue.

The second issue, that concerning uncertainty, is likely to be even more important for the modeling of second-round effects than for the modeling of first-round behavioral effects. For example, CGE models include simulation of behavioral effects in all markets on both supply and demand sides, and hence must rely on estimated elasticities for many different relationships. The range of uncertainty created by a simulation from such a large number of uncertain parameter values is likely to be larger than that of a first-round simulation, by one or more orders of magnitude, because the uncertainty is likely to increase multiplicatively rather than additively in the number of parameters. The MRPIS model also involves a large number of parameters, including detailed marginal propensities to consume, in order to project changes in consumption, and it also has a large number of parameters from a multiregional input-output matrix. The model uses estimated econometric equations to allocate changes in labor demand across the micro units as well. Although the literature on these models, particularly the CGE literature, has discussed issues of uncertainty, too little attention has been paid to the development of methods by which uncertainty can be quantified. This inattention reflects a more general lack of attention to the issue of model validation.

The addition of second-round effects to models also raises difficult issues relating to short-run versus long-run adjustment. For example, CGE models are best thought of as relatively long-run simulations, but such models rarely provide guidance about the time horizon for full adjustment or the dynamic path of the adjustment process. Policy makers will naturally be quite concerned with these issues.

A related issue is the nature of the counterfactual scenario that is being simulated by second-round models. For example, the MRPIS model simulates increases in transfer program benefits to increase national output and to lower the unemployment rate, and the simulated estimates reflect this change as well as the change in distribution of employment across different skill classes. Yet policy makers are unlikely to be directly interested in the fiscal stimulus provided by a particular change in a transfer program, because fiscal stimulus could be provided in many other ways as well. This issue has been discussed for some years in the economic theory of tax incidence; it is often proposed that programs be evaluated on the basis of their distributional effects, while total government expenditures are held constant.

Given these difficulties with second-round models, it is worth considering whether there is any middle ground between the polar extremes of completely

ignoring second-round effects and fully incorporating them into microsimulation models through a process of iterating with a second-round model. Such iteration is likely to be difficult to carry out in an appropriate manner with any degree of confidence in the results. The quality of the simulation depends critically on the proper allocation of outcomes from the aggregated second-round model across the individual units in the micro model.

A middle ground could be achieved by a single iteration in which the initial aggregated outcomes from a microsimulation model are put into a second-round model, from which second-round effects on prices and quantities are estimated. Such second-round estimates would provide a barometer of the importance of such effects as well as an indication of their sign (or direction). These outputs from the second-round model could then be presented in an appropriate way to policy makers as qualifiers to initial estimates.

We are less than sanguine about the cost-effectiveness of devoting investment resources to modeling second-round effects, given the difficult modeling issues involved and the high degree of uncertainty surrounding the outputs. However, it seems advisable for policy analysis agencies to support research to determine what kinds of second-round effects of policy changes are likely to be of large magnitude and hence important to understand. It also seems useful for the agencies to explore the issues involved in attempting to link microsimulation models with second-round effects models. At a minimum, the agencies should require that microsimulation models be designed to facilitate linkages with such models. As we discuss in Chapter 8, it may be particularly important to consider modeling second-round effects of policy changes for health care issues.

Recommendation 6-8. We recommend that policy analysis agencies support research on second-round effects of policy changes that may be important to understand. We also recommend that the agencies require that future microsimulation models include entry and exit points that could facilitate linkages with second-round effects models. However, except perhaps for health care issues, we do not recommend investment at this time in building second-round effects capabilities into microsimulation models.