Models

To obtain estimates of the kinds of outcomes described above for retirement-income-related policy changes requires models: analytical (or behavioral) models to estimate the probabilities of specific behavioral responses (e.g., increased savings, decreased work effort, increased employer contributions to benefit plans) as a function of the characteristics of the individual or the employer, policy parameters, and other factors (e.g., interest rates); and policy models to use the estimated probabilities and other information to project one or more outcomes for the retirement system of the continuation of current policies and alternative policies. (Estimation of behavioral responses and policy outcomes also requires data—see the next major section.)

The distinction between analytical and policy models is not hard and fast. For example, analytical models that are developed to understand various types of behavior may incorporate policy variables (e.g., levels of expected Social Security or employer pension benefits) as predictors. Conversely, policy models that are developed to inform decision makers may project outcomes under scenarios in which economic or demographic factors are varied but the policy regime is held constant. Also, while some kinds of policy models can use alternative behavioral specifications, other policy models are closely tied to a particular behavioral formulation. The key distinction, which has implications for desirable model features, concerns the intended use of the model —that is, whether its primary purpose is to explain relationships or to project future outcomes under different conditions.

For analytical purposes, there is a range of models from simple regressions of a dependent outcome variable on one or more independent variables to complex dynamic models that analyze behavioral decisions (e.g., when to retire) at successive points in time. The papers that we commissioned discussed various kinds of analytical models. We do not review these models in the interim report, but a priority task for the second phase of our work is to assess relevant analytical models, determine their usefulness for retirement-income-related policy modeling, and identify priority areas for further analytical work. A particular area for attention is how to incorporate risk factors and perceptions of relative risks into models so as to understand the decisions of workers and employers in the face of uncertainty.

There are also different kinds of policy models, among which are simple trend-line projection models, cell-based models, microsimulation models, macroeconomic models, and computable general equilibrium models (see Appendix A for examples). Trend-line projection mod-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 17
Toward Improved Modeling of Retirement Income Policies: Interim Report Models To obtain estimates of the kinds of outcomes described above for retirement-income-related policy changes requires models: analytical (or behavioral) models to estimate the probabilities of specific behavioral responses (e.g., increased savings, decreased work effort, increased employer contributions to benefit plans) as a function of the characteristics of the individual or the employer, policy parameters, and other factors (e.g., interest rates); and policy models to use the estimated probabilities and other information to project one or more outcomes for the retirement system of the continuation of current policies and alternative policies. (Estimation of behavioral responses and policy outcomes also requires data—see the next major section.) The distinction between analytical and policy models is not hard and fast. For example, analytical models that are developed to understand various types of behavior may incorporate policy variables (e.g., levels of expected Social Security or employer pension benefits) as predictors. Conversely, policy models that are developed to inform decision makers may project outcomes under scenarios in which economic or demographic factors are varied but the policy regime is held constant. Also, while some kinds of policy models can use alternative behavioral specifications, other policy models are closely tied to a particular behavioral formulation. The key distinction, which has implications for desirable model features, concerns the intended use of the model —that is, whether its primary purpose is to explain relationships or to project future outcomes under different conditions. For analytical purposes, there is a range of models from simple regressions of a dependent outcome variable on one or more independent variables to complex dynamic models that analyze behavioral decisions (e.g., when to retire) at successive points in time. The papers that we commissioned discussed various kinds of analytical models. We do not review these models in the interim report, but a priority task for the second phase of our work is to assess relevant analytical models, determine their usefulness for retirement-income-related policy modeling, and identify priority areas for further analytical work. A particular area for attention is how to incorporate risk factors and perceptions of relative risks into models so as to understand the decisions of workers and employers in the face of uncertainty. There are also different kinds of policy models, among which are simple trend-line projection models, cell-based models, microsimulation models, macroeconomic models, and computable general equilibrium models (see Appendix A for examples). Trend-line projection mod-

OCR for page 17
Toward Improved Modeling of Retirement Income Policies: Interim Report els appear to be of little use for issues of retirement income because of the long time horizon of most needed projections. The longer the projection period, the more hazardous it is to assume that past trends will continue unchanged into the future. Cell-based models divide the population into groups, such as 5-year age categories by sex, and develop separate projections for each group—for example, separate projections of life expectancies, early retirement probabilities, and so forth. Because cell-based models reflect differences among groups, the final outcomes they project may differ greatly from the outcome of a simple trend-line projection for the population as a whole. The SSA Office of the Actuary uses cell-based techniques to develop 75-year projections of the balance in the trust funds. Actuarial models of employer pension fund balances and models used by the Census Bureau, SSA, and others to develop population projections are also cell based. Microsimulation models operate on individual records for large samples of the population, estimating outcomes for each individual on the basis of that person's characteristics. Microsimulation models can estimate the effects of more detailed policy changes for a larger number of groups than is possible with cell-based models.1 In addition, Burtless (1995) argues that microsimulation provides an organizing framework. It forces the model developer and the users to think through all of the behavioral links and interactions, identifies gaps in research knowledge and data, and helps set priorities for filling in key gaps. However, microsimulation models have costs as well as benefits, which must be evaluated vis-à-vis other modeling approaches in the context of the user's information needs. In particular, they tend to be complex and difficult to use. Macroeconomic models look at the broader economy. Macroeconomic forecasting models use systems of simultaneous equations, estimated with historical time series, to forecast the effects of aggregate factors, such as rising inflation or changes in the federal budget deficit, on aggregate outcomes, such as gross national product or unemployment. Computable general equilibrium (CGE) models estimate the second-round behavioral effects of proposed policy changes on prices and quantities in markets, taking into account feedback effects between supply and demand. For example, an increase in marginal tax rates may decrease work effort, thereby leading to reduced labor supply, thereby leading to higher wage rates and increased labor supply until a new equilibrium is reached. As generally implemented, CGE models are macro rather than micro in nature, in that they are not usually disaggregated on many dimensions. However, unlike macroeconomic forecasting models, CGE models are based on the behavior of households and employers rather than on relationships among aggregates. Some models incorporate multiple approaches: for example, there are models that link a microsimulation-based model of the household sector and a macroeconomic-based model of the economy. Other models reflect primarily one approach but make use of the outputs of other kinds of techniques. In our final report, we will consider issues of modeling strategies for policy simulations. For example, one strategic issue is whether to have (1) one “fairly large” model that can evaluate a wide range of policy proposals on a wide range of outcomes together with some ancillary models for special purposes; (2) a large number of highly specialized models designed in such a way that they can be related in some fashion to provide a broader perspective; 1   Both cell-based and microsimulation models can be developed for organizations (e.g., employers) as well as for individuals. However, with the exception of the PBGC PIMS model (see Appendix A), there are no policy models known to us of retirement-income-related behavior of employers.

OCR for page 17
Toward Improved Modeling of Retirement Income Policies: Interim Report or (3) something in between.2 Other strategic issues are the type of modeling approach for a particular model (e.g., micro versus macro); the type of database (e.g., panel or cross-sectional data); and the type of projection or aging technique. We should note that the policy models (and types of models) we discuss are largely those developed by or for government agencies. Other developments have been proceeding in the academic community; the CGE models noted in Appendix A are one example. The models developed in the academic community pay much more attention to the underlying structure and conceptual basis of the simulations. They can also provide important insights, such as the adverse implications for intergenerational equity of differences in the size of cohorts (larger cohorts generally face lower living standards than smaller cohorts). However, as yet such models have not been well integrated into the policy process. They are not usually constructed with an eye toward the policy user who is interested in specific programmatic changes. An important part of the second phase of our work will be to consider how the insights and developments of the academic community can be better integrated into policy analysis. CRITERIA FOR POLICY MODELS We have identified a list of essential and desirable features of retirement-income-related policy models. In a later section, we use these criteria to assess, in broad terms, the adequacy of existing models. In the final report, we will use the criteria to develop recommendations about priorities for cost-effective modeling strategies. We also see the criteria as directly useful to agencies that want, now or in the future, to sponsor model development for particular policy analysis needs. Essential Features For a model to be considered for retirement-income-related policy analysis, it should first of all be relevant. Specifically, it should model one or more kinds of policy changes that we have identified as likely to be under debate in the near term, and it should provide estimates on one or more of the policy evaluation outcomes that we have identified (see section on “Policy Considerations” above). In addition, it should have the following three essential characteristics: the model provides accurate estimates of policy outcomes, in that the estimates approximate what we would observe if the proposed policy changes were implemented; the model output includes information on the uncertainty of the model estimates and their sensitivity to key assumptions; and the model incorporates best current professional judgment about the underlying behavior (e.g., best judgment about the appropriate functional form and parameter values with which to estimate changes in private savings in response to changes in Social Security or employer pensions). These are very stringent criteria. Indeed, for the first criterion, one might ask how we can 2   We do not believe it is feasible or appropriate to think in terms of a single very big model—the development costs would be enormous, the costs of using it would likely also be high, and, because the world does not stand still, some policy question would undoubtedly come along that even this model could not handle. However, there is merit to considering the use of an overall framework to guide the development of a mix of big, medium, and small models and to suggest ways to interrelate the individual models.

OCR for page 17
Toward Improved Modeling of Retirement Income Policies: Interim Report know whether a model's estimates are accurate when the policy change has not yet happened and may never happen. The report of the Panel to Evaluate Microsimulation Models for Social Welfare Programs discussed this issue at length (Citro and Hanushek, 1991:Chap. 9). Clearly, one cannot know at the time a model is estimating the effects of a proposed policy change that the estimates are, in fact, accurate. However, as the microsimulation panel pointed out, one can conduct a series of ex post external validation studies to establish a degree of confidence in a model's track record. Sometimes it is possible to compare a model's estimates with what actually occurred (projections by the Social Security actuary of trust fund balances under current law are often evaluated in this manner).3 More commonly, it is necessary to conduct “ex post forecasting” studies, in which one puts oneself in the place of the analyst who, say, 10 or 20 years ago, was asked to simulate a policy change to take effect in some future year. One chooses the “future” year to be in the recent past so that measures of what happened are available from administrative records or other sources. Correspondingly, one chooses the policy change to be the actual policy in effect during the comparison year. (An alternative approach is to use a current model to develop “backcasted” estimates of policies for earlier years.) Because differences between the model's estimates and what actually occurred may be due to economic or social changes that the model could not have been expected to forecast (e.g., an unanticipated recession or an unexpected increase or decrease in fertility rates), it will be helpful to conduct ex post forecasting (or backcasting) studies that use actual values for the factors that could not have been anticipated and focus on the accuracy of the model with regard to other elements of the simulation. Also, in many instances, the important criterion will be whether the model accurately projected the differences in the outcomes under two or more policy alternatives and not the more stringent criterion of whether it accurately projected the levels of outcomes for a specific alternative. In addition to studies that help establish confidence in a model's track record, it is important to develop estimates of the range of uncertainty of a model's estimates. On average, a model may produce unbiased estimates, but the results of a particular model run may have a wide band of uncertainty attached to them. Advances in statistical methodology that take advantage of modern computing power have made it possible to estimate the uncertainty in model estimates due to such sources as sampling variations in the input data. One such technique is called the bootstrap (see Cohen, 1991b). In addition, sensitivity analysis, which is carried out by developing and running one or more alternative versions of one or more model components, can help assess the uncertainty in model estimates due to the choice of a particular model structure or specification. For example, one would use sensitivity analysis to investigate the effects of a particular equation for simulating the labor supply response to a change in the retirement age for Social Security or employer pensions. In its simplest terms, sensitivity analysis is a diagnostic tool for ascertaining which parts of an overall model could have the largest impact on results and therefore are the most important to scrutinize for potential errors that could be reduced or eliminated (see Citro and Hanushek, 1991:Chap. 9). Sensitivity analysis is particularly important for retirement income modeling, in view of the long time horizon for most projections, which will compound any errors in model specification or assumptions. 3   Schieber and Shoven (1993) note that since the 1983 Social Security amendments were enacted to try to put the system on a sound footing (by such measures as raising the payroll tax and gradually reducing initial benefits), the actuary has for most years revised downward the estimates of the balance in the trust funds.

OCR for page 17
Toward Improved Modeling of Retirement Income Policies: Interim Report For various reasons, external validation studies, estimation of uncertainty, and sensitivity analysis have rarely been carried out for policy models. Historically, one reason was that such studies were very costly and time-consuming. However, if models are implemented in computing environments that provide ready, cost-effective access to users for multiple runs and model respecifications, then validation becomes very feasible. Another reason is that policy makers want point estimates and are not comfortable in dealing with uncertainty. In the final report, we will discuss useful ways to convey estimates of uncertainty so as to inform the policy debate (see also Citro and Hanushek, 1991:86-88). We note that policy makers may have more experience with estimates of uncertainty for retirement-income-related policy projections than for estimates in other areas. Thus, because of the long time horizon, the Social Security actuary routinely presents high-cost, intermediate, and low-cost scenarios for the balance in the Social Security trust funds under current law. However, there is no comparable record of regularly projecting the long-term funding status of employer pensions, even though this source of retirement income may be more uncertain than Social Security (see Burtless, 1993; and Schieber and Shoven, 1993, who develop such a projection for defined benefit and defined contribution plans). Finally, an essential feature of an acceptable policy model is that it incorporate the best professional judgment about relevant factors, particularly for aspects of the model to which estimates are sensitive. “Best” professional judgment is not always the newest theory or latest academic model, which may be too new to have been properly evaluated. Also, because in many areas there is a range of professional opinions, it will be important to conduct sensitivity analyses of alternative specifications before selecting a particular approach to use for a central or base case. Yet it is clear that when important changes have occurred—for example, in worker productivity or savings rates or mortality decline—a model that uses data or estimates of behavioral parameters that do not reflect these changes is not well placed to inform the policy debate. Also, given that relatively little work has been done to date to conduct external validation studies or sensitivity analyses of models (see below), the extent to which a model incorporates best professional judgment is often the only indicator of the likely level of model performance. Desirable Features There are many other features of models that are desirable—often highly desirable—but not necessarily essential for one or more policy applications. The emphasis that should be placed on incorporating one or another feature into a model necessarily rests on an assessment of the relative costs and benefits of that feature in the context of an agency's information needs. We list a range of desirable model features below without explicit consideration of the tradeoffs among them or of costs. An important component of the second phase of our work is to consider cost-effective strategies for building into models such desirable features as ease of use. Timely Response A generally desirable feature for a model is that it provide fast turnaround so that policy makers can request information about a range of options and receive information in a timely manner for the policy debate. Also, fast turnaround makes it easier to generate multiple runs for validation purposes. Broad Scope Other things equal, a broader model, in terms of the policy issues and outcomes it can address, is preferable to a narrower model. A broader model can simulate a broader range of options without the need to develop a new model, and it can provide more information about each policy option or combination of options. However, there will be a

OCR for page 17
Toward Improved Modeling of Retirement Income Policies: Interim Report point at which breadth becomes counterproductive in terms of the cost and time of model development and the difficulty of applying the model in a timely, cost-effective manner. Where to draw the line will heavily depend on an agency's policy needs versus its resources for model development and use. Disaggregation For many policy applications, a model that provides more disaggregated output, that is, which produces estimates of the distributions of outcomes, is preferable to a model that estimates average tendencies only. As an example, in considering a policy change that increases (or decreases) the extent of employer pension benefits, policy makers are likely to be interested in the effects not only for workers as a group but for workers in different industries, size-of-employer categories, age groups, and earnings categories. Detail For many policy applications, a more detailed model is preferable to a less detailed model. Thus, many kinds of proposed policy changes involve fine-tuning one or more aspects of complicated sets of program provisions. To evaluate such proposals requires a model that captures the details of current law and the proposed change. Ability to Link to Other Models At present, there is no all-encompassing retirement income model, and it is unlikely that such a model will be developed in the future. Hence, a very desirable feature of individual models is that they be designed to facilitate links with other models—either to supply their outputs as inputs to other models or to accept other models ' outputs as their inputs. Feedback Loops Policy changes typically have both first-order and second-order effects —that is, because of behavioral responses to the policy change, the net effect of a change is different from the initially estimated effect. (For example, changes designed to increase employer pension coverage, if they also increase employer costs, could lead employers to opt out of pension plans and hence reduce coverage in the longer run.) Modeling of feedback effects is quite difficult, both operationally and because of the uncertainty of behavioral responses to policy changes. A policy model is more useful if it can provide estimates of such effects (or if it can provide outputs to a second-round response model). However, such capability will require careful attention to ways to reduce the costs of model development and use through efficient model design. It will also require careful attention to adequate model validation and to ways to communicate to policy makers the additional uncertainty that is likely to attach to estimates of second-round effects. It may be that, for reasons of cost-effectiveness, different models will be required to address questions of detailed policy changes on the one hand and questions of second-order equilibrium effects on the other hand. Openness and Transparency Highly desirable features of a model are that it be open and transparent —in other words, that it be usable and understandable by people other than the designers. Such a model is more likely to be widely used for a variety of applications and by a variety of users. Hence, there will be many more sources of ideas to improve the model, and it is more likely to be rigorously evaluated. Requirements for an open and transparent model include that it be publicly available to other analysts and that it have good supporting documentation. Ease of use and portability also facilitate openness and transparency. Ease of Use Another highly desirable model feature is that it be easy to use. Requirements for an easy-to-use model are that it be well documented and designed to permit ready modification in order to simulate newly proposed policy options in a consistent manner and to facilitate multiple runs and specifications for model validation purposes. It should also have a user-friendly interface that allows people who are not programmers to operate the model. It could also be highly parameterized (i.e., the design could allow users to carry out many kinds of simulations simply by changing the values for such parameters as tax rates). In this case, to ensure that less experienced users are fully aware of the model's limitations, the

OCR for page 17
Toward Improved Modeling of Retirement Income Policies: Interim Report documentation and the model itself should have features that alert users to the importance of such validation exercises as sensitivity analyses and estimation of uncertainty. Portability A desirable model feature is that it can readily be transported from one computing hardware and software environment to another. Portability makes a model more open and works to encourage model use, improvement, and validation. We repeat that the decision about investing in one or another model feature—and in a technology or approach to attain that feature—should be made on the basis of a cost-benefit analysis in the context of an agency's current and likely future policy information needs. For example, our belief is that, other things equal, a model that is developed for a microcomputer or workstation environment is preferable to a model that operates on a mainframe or supercomputer. Microcomputing technology makes it much more likely that the model will have the desirable features of providing timely information and being open and transparent, portable, and easy to use. (In turn, these features make it more likely that the model can readily be evaluated.) However, a particular application may require computing power that is not available from microcomputers or workstations (e.g., CGE models may require supercomputers to attain the level of desired detail of inputs and outputs). In this case, the only option for obtaining needed answers to important policy questions is to invest in a model that has limited access.4 OVERALL ASSESSMENT OF EXISTING POLICY MODELS There are many policy models available today that relate in some way to retirement income security. Some of these models were developed for narrowly specific applications. Examples are actuarial cell-based models that are designed to project the balance of assets and liabilities for a particular employer pension plan. While these kinds of models serve important purposes, we have not reviewed them. We have primarily reviewed models developed for government use (such as those listed in Appendix A) that can address broader policy questions—for example, the effects of tax policy changes on contributions to employer pension plans in the aggregate.5 Our overall conclusions about existing models are mixed. Some models appear to have useful features for some kinds of applications; however, on balance, our assessment is that existing models are not adequate to inform the retirement income policy debate fully. As a group, existing models suffer from one or more of the following kinds of deficiencies: they have not been adequately validated; they are based on old data sets and do not reflect best current professional judgment about underlying behavior; their provision for feedback loops between policy changes and behavioral responses is weak; they generally have little provision for ready exchange of inputs and outputs with other models; and they are not open and transparent, portable, or easy to use. With regard to scope, disaggregation, and detail, there is considerable variation, with some models having more capabilities in these areas than others. However, the models that have a broader range of capabilities also tend to be the models that 4   An alternative is to implement a simpler version of the model on a microcomputer and a more elaborate version (or a version that will process larger numbers of sample cases) on a supercomputer. 5   Our review does not extend to health care financing policy models, although such models are very important to a full assessment of future retirement income security. See Office of Technology Assessment (1994) for an evaluation of models that were used in the recent health care reform policy debate. Also, to date, our review has not covered tax policy models as such (e.g., the revenue model used by the U.S. Treasury Department).

OCR for page 17
Toward Improved Modeling of Retirement Income Policies: Interim Report are less transparent and vice versa. Also, models that have been developed more recently, with improved capabilities for validation and access, tend to be more limited in scope. We have not attempted a detailed assessment of the pros and cons of particular models. Such an assessment is properly conducted by agencies with requirements for particular kinds of policy information. However, we note that such an assessment would be difficult for us (or others) in any case. The reason is the lack of adequate documentation and accessibility for many existing models. Indeed, it may be that some existing models have more capabilities than we assume, but the information available to us (and others) is not adequate to judge. Below we support our overall conclusions with some examples that are meant to be illustrative and not to single out particular models for criticism. We note two exceptions to our generally negative assessment. The first concerns CGE models that have been developed to answer such questions as the intergenerational benefits and costs of alternative Social Security financing systems. These models are impressive in their treatment of the interactions of the household, government, and private sectors of the economy and in their use of up-to-date information and research knowledge. However, these models for the most part deal with “big picture” issues in a highly stylized manner and primarily for the Social Security system, not the much more heterogeneous world of employer pensions. These models do not provide distributional detail; nor do they address the effects of very specific policy changes or combinations of changes. To date, there are no links between these models and cell-based or microsimulation policy models that would provide additional specificity. The second positive trend is that newer models are, by and large, following design principles that encourage use and validation. For example, the special-purpose model under development at the Pension Benefit Guaranty Corporation to estimate the risks to the PBGC insurance fund for workers in defined benefit pension plans is being implemented for personal computers with a design that facilitates model validation. The CORSIM model, which began as an adaptation of the DYNASIM2 model (see below) and has since been redesigned, operates on both PCs and supercomputers (using the latter for runs with large numbers of cases). This model provides detailed simulations of demographic histories. It also simulates housing and non-housing assets and some health-related characteristics; however, it has very limited capability for simulation of Social Security or employer pension policy changes. The SSA Model of the Social Security System The cell-based modeling techniques that the Social Security actuary uses illustrate some of the limitations of existing retirement-income-related policy models. By design, the SSA model is limited to projections of Social Security trust fund balances and related variables.6 It is not a general-purpose retirement income policy model; however, it is of special importance because of the key role of the Social Security system in supporting retirement income security in the United States today. Also, many other models use the SSA projections of such variables as population and wage growth to control their own simulations. The SSA model develops projections of payroll taxes and benefits for age-sex groups that take account of demographic and economic trends for such variables as life expectancy, retirement age, and worker productivity. Each year the model is run to estimate the balance in the trust funds over the next 75 years, assuming that current law continues unchanged. Typically, 6   We use the term “model,” although it appears that the SSA model is actually a series of models (e.g., for demographic projections, labor force projections).

OCR for page 17
Toward Improved Modeling of Retirement Income Policies: Interim Report estimates are prepared for low-cost, high-cost, and intermediate scenarios. The low-cost scenario (for the old-age and survivors' component) assumes higher fertility, less improvement in mortality, higher labor force participation, faster real wage growth, and higher interest rates than the intermediate scenario; while the high-cost scenario assumes the opposite. Hence, the low-cost (high-cost) scenario projects the lowest (highest) aggregate benefits paid out and the highest (lowest) aggregate contributions paid into the system.7 The SSA model can simulate a range of policy options for Social Security, such as changes in the payroll tax rate, the payroll tax base, the benefit calculation formula, the earnings test, the cost-of-living adjustment, and the retirement age. However, such policy options as means-testing benefits and increasing individual income tax rates on benefits are beyond the model's scope because they require information about other sources of income (e.g., private pension and asset income). Nor could the SSA model simulate a full or partial privatization of the Social Security system. The SSA model is limited in other ways. It does not provide distributions of outcomes, for example, for workers with high, low, or average levels of earnings. Also, it has no feedback loops—for example, no ready way to model the feedback effects on labor supply of raising the early retirement age. (Such effects could be built into the model by changing the equations that are used to estimate labor force participation, but such adjustments would necessarily be ad hoc.) An issue that has been raised about the SSA model concerns the way in which the assumptions are developed for the three scenarios. Burtless (1993) points out that the assumptions underlying the low-cost and high-cost scenarios are not chosen so as to bound the likely range of potential experience in any manner that approximates a statistical confidence interval. Hence, they are less useful to policy makers, analysts, and the general public because they do not indicate the probability of a best-case or worst-case outcome relative to the intermediate projection. Also, the assumptions may be internally inconsistent. For example, the growth rate of real wages (for which a higher value is a favorable outcome for the Social Security system) may be negatively correlated with real interest rates (for which a higher value is also a favorable outcome for the system). Finally, there is a concern that key assumptions of the central or intermediate scenario may not always reflect current best professional judgment. For example, projections of demographic experts suggest greater declines in mortality rates at older ages than are assumed in the SSA projections (see Lee and Skinner, 1995). Holmer (1995) has suggested that it would be useful if SSA were to use probabilistic Monte Carlo techniques to estimate the distribution of likely outcomes. With this approach, distributions of values would be generated for key variables (e.g., fertility rates, mortality rates), by using not only a mean value (representing, say, the Social Security actuary's intermediate scenario assumption) but also an assumed standard deviation and an assumed correlation with other variables. Multiple runs with different random draws from these distributions would generate a distribution of outcomes, which could provide a probability bound for the intermediate forecast. Finally, a key concern with the SSA model is that it is not well documented or publicly available.8 Considering the importance of policy choices about the Social Security system for 7   In addition to overall estimates for each scenario, SSA provides sensitivity analyses of the separate effects of key assumptions. In these analyses, the high-cost, intermediate, and low-cost values are used for one assumption, such as fertility, while the intermediate values are used for all other assumptions. 8   The annual board of trustees reports and other SSA studies document the assumptions that are used in the SSA model (e.g., the assumed fertility rate for each scenario), but not the workings of the model components themselves or how different components are integrated with each other.

OCR for page 17
Toward Improved Modeling of Retirement Income Policies: Interim Report U.S. workers and retirees, it is dismaying that the model on which the policy debate will rely for key estimates cannot be readily understood or evaluated by other analysts.9 Models of Social Security, Private Pensions, and Savings The DYNASIM2 and PRISM microsimulation models illustrate several key limitations of retirement income policy models. DYNASIM2 and PRISM are much broader than the SSA model of Social Security: they simulate employer pension and IRA contributions and benefits as well as Social Security taxes and benefits.10 They also simulate eligibility and benefits for SSI and individual income tax liabilities. They take some account of asset income (e.g., dividends and savings interest) but do not simulate wealth.11 Both DYNASIM2 and PRISM use dynamic microsimulation techniques and hence are able to provide highly disaggregated outputs for each year of the projection period (e.g., for population groups categorized by level of earnings and employment status). Both models can be obtained by others and are documented to some extent (the documentation for DYNASIM2 is more complete than that for PRISM). However, the accessibility of the models in actual practice is limited: they were designed well over a decade ago for mainframe (or minicomputer), batch-oriented computing environments and lack user-friendly design features. They have had relatively little formal validation, although some older validation studies for DYNASIM2 are available (e.g., Haveman and Lacker, 1984; Hendricks and Holden, 1976; Wertheimer et al., 1986). Key drawbacks of the two models are that they use old data and that they have not been revised to reflect best current professional judgment about underlying behavior. Both DYNASIM2 and PRISM take an initial database and age the records forward in time each year by means of a dynamic approach. These starting databases are exact-match files, which contain records for the members of about 60,000 households from the March Current Population Survey (CPS) income supplement, matched with their personal earnings histories from SSA administrative records. For DYNASIM2, the starting database is the March 1973 CPS-SSA exact match file; for PRISM, the starting database is the March 1978 CPS-SSA exact-match file, which, in turn, is matched with the March 1979 CPS and the May 1979 CPS pension supplement. The earnings histories in these files are essential for computing Social Security payroll contributions and benefit entitlements; they are also useful for simulating employer pension contributions and benefits. Then, for every year subsequent to the base year, DYNASIM2 and PRISM age the characteristics of the records in the file; they not only increment each person's age by 1 year but, on the basis of transition probabilities, also simulate whether or not people will go to school, marry, have a child, divorce, die, change employment status, change jobs, participate in a pension plan, retire, and so on. Both models use outside aggregates, such as the Social Security actuary's population projections, to control the simulation results for each projected year. However, many of the key transition probabilities—for example, estimates of labor force participation and retirement—are based on old analytical studies (e.g., the 1969-1979 9   We should note that Steve McKay in the SSA Office of the Actuary has made publicly available over the Internet his program (ANYPIA) for calculating Social Security primary insurance (benefit) amounts. This program accurately calculates Social Security benefits from input data supplied by the user (e.g., average indexed monthly earnings). 10   Indeed, these models have been used to simulate Social Security options that the SSA model could not handle—for example, provisions to credit homemaker spouses with a share of the employed spouse's earnings. 11   The long-term-care module that was later added to PRISM treats assets to some extent.

OCR for page 17
Toward Improved Modeling of Retirement Income Policies: Interim Report Retirement History Survey and early years of the Panel Study of Income Dynamics in DYNASIM2). With regard to employer-provided pensions, the models do not incorporate more recent knowledge about trends in pension plan coverage, such as the growth in defined contribution plans. Similarly, neither model reflects well the trend toward increasing heterogeneity of labor force behavior, in which an individual may “retire” from a succession of jobs. Also, neither model relates individuals ' decisions about labor supply and savings to expectations—for example, expectations about likely future policy changes or their own longevity. The reason that neither DYNASIM2 or PRISM have used newer initial databases is that, because of confidentiality concerns and resource constraints, government agencies have not been willing to prepare any new exact-match files (although this situation may be changing; see discussion in the section below on “Data”). The consequence is that both models must simulate more and more years of historical data before they can begin projecting into the future. Aggregate information, such as employment rates and total payroll contributions, can keep the simulations in line with actual historical experience for a handful of key variables for years up to the present, but there is no way to ensure that the models accurately reflect the interrelationships among all of the important variables for those years. The reason that neither model has been revised to update key transition probabilities or to incorporate more appropriate functional forms that reflect newer understanding of behavior has to do primarily with cost and time constraints imposed by their mainframe design. 12 Because the models are not easy to modify, it is not in the modelers ' interest to invest in redesign until substantial funding is available and there is a clear direction from the research community about appropriate functional forms and parameter estimates. But since the 1983 Social Security amendments were enacted, there has been relatively little demand for retirement-income-related modeling until now. Also, the decade of the 1980s saw important changes in labor markets and employee benefits that have made it difficult to determine the best professional judgment to incorporate in models (there is no capability in the models to experiment readily with alternative estimates). Another limitation of the models is their lack of a ready capability to estimate the feedback effects of policy changes on behavior. The reason again has to do with their design, which separates the construction of longitudinal year-by-year demographic and economic “histories” for each person in the database from policy simulations as such. Because creating longitudinal histories is expensive on a mainframe, the practice is to generate one set of histories (or a few sets under different reasonable assumptions) and then to run a larger number of program simulations on that longitudinal file. Hence, events or characteristics that are simulated after the creation of the longitudinal histories are affected by those histories but cannot in turn affect the sequence of demographic and labor force events. (Ross, 1991, indicates which variables in DYNASIM2 and PRISM are generated as part of a longitudinal history and which are simulated subsequently.) For example, in both models, changes in the Social Security system cannot lead to compensating changes in labor supply over an individual's work life unless the longitudinal histories are recreated with a new set of labor supply equations incorporating the expected response. While it could be possible for other models to simulate the second-round effects of policy changes that are initially simulated by DYNASIM2 or PRISM, neither model has a ready capability to link with second-round effects models. 12   The long-term-care subsystem of PRISM operates on a PC but uses the mainframe-developed longitudinal job and earnings histories from the main PRISM model. DYNASIM2 is currently being redesigned for a PC environment.