11
The Microsimulation Modeling Community

Throughout our report we single out aspects of the structure of the microsimulation modeling community that bear on the quality, relevance, and cost-effectiveness of the use of the models to generate estimates for the policy process. In this chapter, we bring together those scattered references and discuss other aspects of the community that merit attention.

By ''structure of the microsimulation modeling community," we mean the relationships among all of the organizational entities that are involved in developing microsimulation models and databases, applying them for policy analysis purposes, and using the resulting estimates to inform policy making. Hence, we are concerned about the relationships among many members of the community: federal policy analysis agencies that support model development and applications, and prepare estimates for decision makers; federal statistical agencies that produce needed input data; research and modeling contractors that conduct studies and analyses for policy analysis agencies; and the staffs of executive and legislative branch decision makers who receive the outputs from the models and interpret and use—or do not use—them in making policy. We are also concerned about the use of microsimulation techniques by the community of basic and applied social science researchers that, in turn, can benefit the development of improved microsimulation models for policy analysis purposes.

Model estimates do not spring forth unbidden as wisdom from the head of Zeus. They are developed, interpreted, and applied by people in diverse settings, including government agencies, contracting firms, congressional committee staffs, university departments, and others. Organizational characteristics



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 273
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations 11 The Microsimulation Modeling Community Throughout our report we single out aspects of the structure of the microsimulation modeling community that bear on the quality, relevance, and cost-effectiveness of the use of the models to generate estimates for the policy process. In this chapter, we bring together those scattered references and discuss other aspects of the community that merit attention. By ''structure of the microsimulation modeling community," we mean the relationships among all of the organizational entities that are involved in developing microsimulation models and databases, applying them for policy analysis purposes, and using the resulting estimates to inform policy making. Hence, we are concerned about the relationships among many members of the community: federal policy analysis agencies that support model development and applications, and prepare estimates for decision makers; federal statistical agencies that produce needed input data; research and modeling contractors that conduct studies and analyses for policy analysis agencies; and the staffs of executive and legislative branch decision makers who receive the outputs from the models and interpret and use—or do not use—them in making policy. We are also concerned about the use of microsimulation techniques by the community of basic and applied social science researchers that, in turn, can benefit the development of improved microsimulation models for policy analysis purposes. Model estimates do not spring forth unbidden as wisdom from the head of Zeus. They are developed, interpreted, and applied by people in diverse settings, including government agencies, contracting firms, congressional committee staffs, university departments, and others. Organizational characteristics

OCR for page 273
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations and modes of operation of those settings affect the ways in which the people in them act and interact. In turn, these interactions influence how well microsimulation modeling supports the policy analysis function and, ultimately, the legislative process. Sometimes, these interactions have positive results on the cost-effective use of microsimulation modeling; sometimes, their effects are detrimental. We necessarily focus on the negative aspects in order to indicate opportunities for improving the structure of the community from the viewpoint of facilitating the production and use of high-quality, relevant policy estimates from microsimulation models. RELATIONSHIPS AMONG FEDERAL AGENCIES Policy Analysis Agencies One of the themes throughout our report concerns the fragmented and decentralized character of data production and policy analysis in the federal government. In some ways, this decentralization is obviously appropriate and functional. For example, there is no need for the Food and Nutrition Service (FNS) to be particularly concerned about tax policy or the Office of Tax Analysis (OTA) to be concerned about the food stamp program. Each agency can focus on the data, research, and modeling that are most relevant to the policy mission of its department. Similarly, the various federal statistical agencies, such as the Bureau of Labor Statistics and the Statistics of Income Division of IRS, are well located to be responsive to the data requirements for the policy roles of their departments. But policy interests are not always easily separable. Many policies interact, and many policy analysis agencies have overlapping responsibilities and interests. One example is the number of executive and legislative agencies that are involved in tax policy analysis using microsimulation models: Office of Tax Analysis, Joint Committee on Taxation, CBO, ASPE, and others. Each of these agencies has a somewhat different perspective and takes a somewhat different approach to database construction and modeling of tax policies. Another example of more than one agency sharing policy interests concerns income support. FNS in the Department of Agriculture is responsible for monitoring the food stamp program, and its model of the program is more fully developed than the ASPE model. The U.S. Department of Health and Human Services (HHS) is responsible for overseeing the AFDC program, and ASPE, on behalf of HHS, has supported AFDC modeling capabilities that are more fully developed than those of FNS. This division of labor makes sense from the viewpoint of each department's primary policy interest; however, given program interactions, both ASPE and FNS need good models of all the major income support programs. To date, only limited efforts have been made to explore ways for each agency to borrow strengths from the other's model. Health care policy provides still another example of multiple agencies

OCR for page 273
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations with overlapping concerns and responsibilities. For example, the Health Care Financing Administration has responsibility for administering the Medicare and Medicaid programs. The Agency for Health Care Policy and Research has a broad responsibility for policy research in health and is specifically mandated to coordinate data and analysis programs related to research on the effectiveness of medical treatments. The National Institute on Aging has a mandate to assess health trends and policies from the perspective of the elderly population. ASPE has historically played and continues to play a role in health care policy analysis, for instance, in the recurring initiatives for some sort of national health insurance program or for mandated employer-provided health insurance.1 The fact that these agencies are lodged in the same department, HHS, has not in our assessment made the task of developing good interagency working relationships any easier. Indeed, for health care policy, it appears to us that communication and turf problems have hampered effective coordination of modeling, database construction, and behavioral research needed to support policy analysis. In general, the existence of multiple models and analyses in policy areas such as taxation and income support serves the positive function of providing a means of cross-checking to identify errors in computer codes and to highlight differences in key assumptions underlying model estimates produced by different agencies. Moreover, there is considerable potential for the differing perspectives of analysts in different agencies to contribute valuable insights for improvements in models and databases that could benefit a broad range of users. On the negative side, however, the involvement of multiple agencies adds costs because of duplication of effort. And differences in analytical approaches, models, and databases among the agencies make it difficult to determine the reasons for different outputs and, hence, difficult to pinpoint aspects of particular models or data sets that need improvement. Finally, barriers to interagency communication, including time pressures and the orientation of the analysts to the needs and viewpoints of their own agency and department, can all too easily nullify the potential gains from different perspectives.2 As we have noted repeatedly, microsimulation models and databases are usually large and complex, and thus likely to be large consumers of resources for development and use. This fact makes it particularly important for federal agencies to seek ways to improve cross-agency communication and coordination, both within and among departments and branches of government. Because there are frictional costs to any interagency collaboration, however, coordination efforts should not be allowed to become counterproductive. In particular, we are wary of joint model-building efforts that could result in a model that serves 1   The Department of Labor also has an interest in employer-provided health insurance and has funded some modeling efforts on this topic. 2   It appears to us that the barriers to effective communication are often particularly high between the policy analysis agencies in the executive and legislative branches.

OCR for page 273
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations none of its sponsor agencies' interests very well. Yet, there are many ways in which enhanced collaboration would be beneficial to the development of improved models and databases. For example, agencies could usefully work together to: specify common data elements for inclusion in surveys; consider mechanisms to facilitate data record linkages and access to linked files; develop model design standards to facilitate more ready interchange of model components, inputs, and outputs; sponsor evaluations of alternative model aging techniques and specifications; sponsor broad behavioral research; sponsor investigations of promising new computer technologies; and sponsor research on model validation methods. Recommendation 11-1. We recommend that executive and legislative branch policy analysis agencies expand their communications and undertake cooperative efforts to improve the quality of microsimulation models and associated databases through such means as cosponsoring research on model validation methods and other initiatives. (In Chapter 8, we specifically recommend that the Department of Health and Human Services establish a high-level steering group to coordinate development of microsimulation models, data, and research needed to improve the scope and quality of policy analysis in the vital health care policy area.) Statistical and Policy Analysis Agencies The fragmentation of authority and the overlapping responsibilities and interests among executive and legislative branch policy analysis agencies also characterize federal statistical agencies. Again, the area of health care is a prime example: not only the National Center for Health Statistics and the Census Bureau, but also the Agency for Health Care Policy and Research, the Health Care Financing Administration, and ASPE, have played a role in sponsoring important data collections that are needed for health care policy analysis. This fragmentation has had negative consequences. For example, health-related surveys have not always included variables that are important from the perspective of modeling and policy use, nor have they always become available in a timely fashion to users in other agencies. Another problem with the decentralized nature of federal statistical activities is that collection of administrative data is typically lodged in one set of agencies (for example, the IRS or the Social Security Administration), while other agencies, such as the Census Bureau or the Bureau of Labor Statistics,

OCR for page 273
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations have primary responsibility for data from household and other types of surveys. This situation exacerbates the problems in trying to link administrative and survey data; it is one reason that relatively few linked data files exist or are widely available, although, potentially, such files represent a cost-effective means of obtaining valuable data for statistical and research uses. The relationships between the statistical agencies, on one hand, and the policy analysis agencies, on the other, present some troubling aspects for cost-effective production of useful, high-quality databases for microsimulation modeling and other types of policy analysis. Because statistical agencies perceive a limited responsibility to add value to the data they collect, the users in the policy analysis agencies must spend considerable resources, often for duplicative work, to make various corrections and enhancements in order to have suitable databases for modeling. Moreover, differing perspectives often impair effective communication between statistical and policy analysis agencies. As a consequence, policy analysis needs are not always well reflected in statistical agency decisions about the design and content of surveys, and, conversely, user agencies do not always properly use or interpret the available data. In other chapters of the report we make several recommendations that bear directly or indirectly on the general relationships among statistical agencies and between those agencies and the user agencies. In summary, we recommend: Federal statistical agencies, generally, and the Census Bureau, particularly, should assume a more active role in adding value to databases for modeling and research purposes, through such means as adjusting responses for reporting errors. Federal data collection strategies should emphasize breadth of use and ability to respond to changing policy needs, including duplication of selected questions across surveys to the extent that such duplication enhances utility and facilitates evaluation of data quality. The federal statistical community should develop mechanisms to improve access, under appropriate circumstances, to administrative and survey microdata for statistical research and analysis purposes: in particular, the Census Bureau should conduct updated exact matches of social security earnings records with household survey data and develop ways to make them available for research use. The Census Bureau and the policy analysis agencies should work together to evaluate alternatives for short-term improvements to the data used for microsimulation modeling of income support and related social welfare programs.

OCR for page 273
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations POLICY ANALYSIS AGENCIES AND THEIR SUPPLIERS The extent to which policy analysis agencies are directly involved with the development and use of microsimulation models varies substantially. Some policy analysis agencies, such as the Office of Tax Analysis, develop, maintain, and operate their own models in-house. Other agencies, such as the Congressional Budget Office, do some modeling in-house but also rely on outside contractors. Still other agencies rely almost exclusively on contractors. Historically, ASPE has developed and used models in-house; currently, ASPE staff use the AFDC benefit-calculator model in-house but rely exclusively on contractors for model development and maintenance and for application of complex models, such as TRIM2. FNS has historically contracted for all microsimulation model services (although FNS staff now use a food stamp benefit-calculator model in-house). The past decade has seen increasing reliance on contractors. There are many reasons for that reliance. Cutbacks in agency staff and personnel ceilings have made it difficult for many agencies to recruit in-house modeling staff, particularly given competing needs for other kinds of staff capabilities. Furthermore, some agencies have preferred to develop long-term relationships with contractors whose staffs have developed in-depth expertise in the "care and feeding" of the models and databases and in working closely with the agency analysts. Microsimulation modeling contractors typically sell a package of services, not an off-the-shelf product. These services include routine maintenance, such as updating program parameters in the models and creating new baseline files as new survey data are released; new model development; and rapid response to agency requests for model runs. Underlying the attractiveness of the service package offered by contractors is the complexity of the microsimulation models, which places a premium on the knowledge and experience of staff. That same complexity limits the number of suppliers by putting high barriers in the way of new firms entering the field or competing successfully with established contractors. In fact, looking at microsimulation suppliers, a small handful of contracting firms are involved in providing the majority of modeling services to the policy analysis agencies. We have every reason to believe that microsimulation modeling contractors have ably fulfilled the terms of their contracts and, indeed, have often provided outstanding service to their agency clients. We also recognize that there are real costs to the agencies to try to increase the contractor pool, given that heightened competition may disrupt agency-contractor relationships that are beneficial for obtaining the most experienced and knowledgeable assistance in carrying out the policy analysis function. Yet we are disturbed that the small number of suppliers may well result in missed opportunities for new ideas and perspectives that could lead to improvements in models and model estimates.

OCR for page 273
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations Whether an agency runs microsimulation models in-house or uses contractors, we are disturbed about another aspect of the industry as currently structured, namely, that analysts almost never have hands-on experience with the models. Given the complexity of most current models and their reliance on batch-oriented mainframe technology, even straightforward model applications require intervention by skilled programming staff.3 This situation prevails not only at the agencies themselves but also at the contractors. Again, in our view it is likely that there are missed opportunities for new ideas and experimentation with alternative policy proposals and alternative model structures. Moreover, the distance of analysts from a model, which is particularly great for the staff of an agency that contracts for modeling services, is not conducive to the analysts' fully understanding the model or the resulting output. Such distance increases the opportunities for miscommunication, erroneous specifications, and erroneous interpretations of model results.4 We believe that it is critically important for agencies to expand the community of knowledgeable microsimulation model developers and users and, in particular, to bring the models closer to the analysts who have the ultimate responsibility for preparing estimates for the policy debate. There will undoubtedly be short-run costs for these efforts, but the long-run payoffs are vital for the continued ability of microsimulation models to support the policy analysis function in a cost-effective manner. As concrete means to these ends, following the design principles that we present in Chapter 6 should produce models that are more accessible to a broader range of users, not only for applications, but also for experimentation and validation efforts that, in turn, may well lead to improved models and model estimates. Similarly, implementing models with new computer hardware and software technologies, as discussed in Chapter 7, should be very effective in this regard. Providing more extensive model documentation, which includes both reference and instructional and informational materials (see Chapter 10), is another way of encouraging more extensive use and better understanding of the models by a broader community of users. Our recommendation that agencies let separate contracts for model validation and our suggestion that this approach be considered for documenting and archiving models (see Chapters 9 and 10) also offer important benefits from the perspective of model development and use. Such contracts can be expected to provide direct incentives for other firms and academic researchers to become involved with microsimulation modeling and, at least, to broaden the range of perspectives that address issues of model quality and utility. In this regard, it 3   The exceptions are the personal computer-based benefit-calculator models, which agency analysts often run themselves. 4   In Chapter 9, we discuss an example from the Family Support Act-for the unemployed-parent component of AFDC—that illustrates the problems for analysts' understanding of model outputs because of the models' complex nature.

OCR for page 273
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations is critically important that agencies lend their support only to those models that can readily be made available to all interested users. Recommendation 11-2. We recommend that policy analysis agencies have a strict policy that only public-use, nonproprietary microsimulation models—for which documentation, inputs, outputs, and programming code can be freely exchanged—will be considered for agency use. Finally, a goal that we encourage policy analysis agencies to adopt is to increase the in-house use of microsimulation models by their own analysts. Such a goal will confirm the agencies' commitment to improving models and databases through expanding the pool of knowledgeable users. Such a goal will also help motivate the agencies to take other needed actions. Specifically, effective use of models by agency analysts will necessarily require better design, computer implementation, and documentation of models, all of which are important for improving the flexibility of modeling tools and the quality and understandability of model estimates. Recommendation 11-3. We recommend that the policy analysis agencies set a goal of increasing the in-house use of microsimulation models by agency analysts, who have the ultimate responsibility of interpreting model results for policy makers. POLICY ANALYSIS AGENCIES AND DECISION MAKERS' STAFFS To this point we have considered relationships among organizational entities that are involved in the production of policy estimates. We now very briefly consider the relationships among organizational entities that are involved in the use of those estimates. Specifically, we are concerned with the interactions of the policy analysis agencies, which are responsible for supplying estimates to the policy debate, and the staffs of decision makers, who play a key role in interpreting the estimates and mediating their influence on policy choices. The single point that we want to stress is the need for policy analysis agencies to work with decision makers' staffs to improve understanding of the quality of the estimates and the uncertainty that inevitably surrounds them. We realize that, ultimately, decision makers will use point estimates and not the confidence intervals or other measures of uncertainty that accompany the estimates. Indeed, this is appropriate for such purposes as adding up cost estimates for various components of a legislative proposal and making the best assessment of the distributional effects of a proposed policy change. However, it is critical that decision makers' staffs receive, understand, and take cognizance of measures of uncertainty and other quality assessments

OCR for page 273
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations for the estimates produced by policy analysis agencies. The information that a particular cost estimate has an uncertainty range of +$50 billion to -$50 billion, for example, and another estimate has an uncertainty range of +$25 to +$35 billion, is vital input in assessing the weight that decision makers should accord to each of the two estimates in their deliberations. Furthermore, such information will help decision makers and their staffs make judgments about needed improvements in databases and analytical tools that promise to improve the quality of the estimates for future deliberations. We urge both policy analysis agencies and decision makers' staffs to give serious heed to our recommendation that quality measures be developed for policy estimates and used in the policy debate. THE ROLE OF RESEARCH Contributions of Research to Policy Analysis Another factor in the limited extent of the community involved in the use of microsimulation modeling for policy purposes has been the disassociation of much of academia from such analysis. Although the concepts of microsimulation originated in academia, relatively few academic researchers have been actively involved in the development and use of microsimulation models for policy applications over the past decade. In some areas of basic and applied social science research, notably the study of human populations by demographers and others (see further discussion below), academic researchers have a substantial tradition of development and use of microsimulation models. In other areas, however, such as economics, microsimulation techniques have been relatively little used. In our view, more direct involvement of academic researchers with microsimulation techniques would benefit the development of improved models for policy analysis as well as research purposes. Academic contributions to microsimulation modeling as a tool for policy analysis come not only from researchers' direct involvement with models, but also from their development of knowledge that provides needed input to model functions. Policy models can only be as good as the underlying base of knowledge about human behavior. In microsimulation, even those models that emphasize the accounting aspects of government programs must take into account decisions on the part of families and individuals to accept benefits, itemize deductions, and so on. Should the models attempt to simulate other kinds of behavioral responses, such as a work response to welfare reform, they need more information about behavioral relationships. Models that look at changing characteristics over time, such as the dynamic microsimulation models used to simulate retirement income programs, depend very heavily on basic knowledge about the factors that influence marriage, divorce, educational attainment, labor force participation, and other modeled characteristics.

OCR for page 273
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations Model developers rely on social science researchers for their knowledge base. Yet the interests and incentives of researchers, even those in applied research, are not necessarily directed to the topics of most concern for policy modeling. As we noted in Chapter 6, even ostensibly relevant research may not be carried out in a form that is readily usable for models. And, of course, all too often research results vary widely and hence leave model developers in a quandary about which estimate to use. Generally, social science researchers are heavily influenced by the requirements of academic publishing, which emphasize innovative work in highly specialized areas. In contrast, microsimulation modeling and other forms of policy analysis require well-established results from cross-disciplinary studies that incorporate replication or reanalysis of previous work. The results must be in a form that the models can readily implement and must, of course, relate to topics of policy concern. The issue for policy analysis agencies is how to obtain a higher payoff from social science research. An obvious answer is to increase contract research and grant funds for needed studies. Although additional funding may well be part of the answer, agency budgets may not be able to provide it. The funds that are available need to be carefully targeted to focus researchers' attention on the priority needs of the agencies. In other chapters of the report, we make several recommendations for priority research areas for agencies to support. In summary, we recommend that policy analysis agencies sponsor studies of the relationship between behavioral research and microsimulation modeling, including studies of ways in which research and modeling can complement one another, as well as ways in which the two are alternative modes of deriving answers to policy questions; studies to determine when behavioral response effects are most likely to be important in different policy simulations and to attempt to narrow the range of statistical estimates of those behavioral parameters that may be of major importance to critical policy changes; also, studies on second-round effects of policy changes that may be important to understand; studies to develop methods for systematically assessing the impact on microsimulation model estimates of the degree of uncertainty in the behavioral parameters that are used—both the uncertainty arising from the variance of specific parameters and that arising from the range of estimates from different behavioral studies; and studies to develop improved methods for validating microsimulation model output. We also believe that agencies may benefit from relatively low-cost activities designed to encourage and reward researchers' interest generally in policy analysis concerns that could benefit from a microanalytic perspective. Specifically, we suggest that the agencies inaugurate regular programs of conferences

OCR for page 273
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations that feature research results and methods that are pertinent to microsimulation modeling and other forms of microlevel policy analysis and highlight emerging policy issues on which research is needed. The resulting conference proceedings should be published and disseminated to a broad audience. We note that the Food and Nutrition Service has sponsored two such conferences in recent years (Food and Nutrition Service, 1986, 1990). The Statistics of Income Division of IRS has also sponsored conferences about challenges confronting microsimulation models related to tax policy, of both the household and the corporate sectors, that attracted academic interest (Internal Revenue Service, 1989; U.S. Department of the Treasury, 1988a, 1988b). We encourage other agencies to follow their lead. Recommendation 11-4. We recommend that policy analysis agencies encourage and support the involvement of social science researchers in work that is relevant to microsimulation modeling, and other microlevel policy analysis, through sponsoring regular research conferences. The conferences should highlight pertinent research results that can be used for models, with an emphasis on the synthesis of research findings and the reconciliation of conflicting results. These conferences should also work to develop research agendas to address emerging policy needs. The agencies should prepare and disseminate proceedings from all such conferences. The Promise of Microsimulation for Social Science Research Our report focuses on the use of microsimulation models for policy analysis purposes.5 Indeed, a primary motivation of Guy Orcutt, the economist who pioneered the development of microsimulation modeling more than three decades ago, was to find a technique that would bring social science research knowledge to bear on policy questions. Looking back 45 years later, Orcutt (1986:26) said, ''[I] shifted out of the study of physics into economics with the hope that my research might help to avoid a repeat of the international catastroph[y] referred to as the 'great depression'." He initially believed that macroeconomic time-series analysis held the key to understanding economic behavior and developing public policies to combat economic and social problems. However (p. 26), "By the middle of the 1950's it seemed evident that macroeconometrics, despite its many attractions, was a house built upon sand…I then came to think of microanalytic simulation modeling as a potentially promising alternative…I am still convinced of the correctness of this view." Orcutt believed that microsimulation techniques were necessary to trace through the potential 5   The discussion in this section benefited greatly from a set of notes prepared by Michael Wolfson, a member of the panel.

OCR for page 273
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations effects of alternative public policies on individual decision units and generate individual behavioral responses that, in turn, would have feedback effects on other sectors of the economy. Orcutt also had a vision that microsimulation models would make important contributions to social science research knowledge. He argued that the complexity of human behavior makes it very difficult for researchers to determine how interactions among social processes will play out over time. Dynamic microsimulation models, which can include multiple processes—such as the decisions to enter the labor force, marry, and have a child—and model them with a detailed population sample, provide the computational means for achieving this goal. Running a model may produce unexpected results that, in turn, lead a researcher to look for ways in which the underlying behavioral equations need to be improved. Stated another way, microsimulation models are vehicles for integrating the results of different strands of socioeconomic research within a framework that forces consistency and alerts a researcher to important gaps in data and knowledge about human behavior. Hence, Orcutt saw microsimulation modeling playing a prominent role in the development of knowledge about socioeconomic behavior. Orcutt's dream has never been fully realized. For many subjects of inquiry, microsimulation models have remained largely a tool of policy analysts and not of social science research. Moreover, the most heavily used policy models have tended to be static models that contain complex modules to determine program eligibility and benefits but incorporate very few of the dynamic behavioral processes that Orcutt saw as essential to the technique. Dynamic models such as DYNASIM2 and PRISM have had a much more limited analytic scope than Orcutt envisioned. There are many reasons for the failure of Orcutt's dream to excite the attention of academics in disciplines such as economics or to stimulate serious involvement on their part over the past two decades: most important, in our view, are the complexities and limited access to many current models. There is a high entry cost for academic researchers (as well as other would-be users) to learn enough about large-scale microsimulation models such as DYNASIM2 or TRIM2 to be able to use them effectively within the confines of a limited-duration research grant. Inadequate documentation and instructional tools compound this problem. The highly complex nature of microsimulation models has led some in the academic community to express outright distrust of what appear to them to be impenetrable "black boxes" (see, e.g., Taussig, 1980). Similarly, academic researchers have been disturbed that such models appear to be overly sensitive to the choices of their developers and generally to lack ready mechanisms with which to judge the reliability and generalizability of their results. In addition, the relatively high cost of using mainframe-oriented models strains research budgets, and the necessity of programmer intervention

OCR for page 273
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations runs counter to prevailing academic work habits and limits the experimentation that a researcher can attempt. The interests of and incentive structures for academic social scientists have also operated to limit their involvement with microsimulation modeling. Academic researchers generally have been uninterested in policy applications of microsimulation models, such as estimating the costs and distributional effects of proposed program changes. Policy analysis results are likely to be of little or no theoretical relevance and unlikely to be accepted for publication in academic journals that tend to emphasize new findings and theoretical advances. For the kinds of basic socioeconomic or policy research studies that more naturally fit their agendas, academic researchers in many social science disciplines have seen microsimulation models as costly and of little relevance. Thus, it is not surprising that many social scientists have remained aloof from microsimulation techniques. However, we believe that this situation will change and, indeed, is already changing, and that microsimulation modeling will prove increasingly useful and cost-effective for a growing number of analytical situations. The cost structure of microsimulation modeling is poised for a fundamental change in the very near future. Improvements in both hardware and software technologies should make it possible to operate very large, complex models in a manner that facilitates direct user interaction and lowers costs of access and use. Improved computing environments should facilitate the ability of users to run alternative models, modify important model components, and validate model outputs—all activities that are critical to researchers' acceptance of microsimulation modeling as an important social science analytical tool. Other factors should also contribute to an improved climate for social science research applications of microsimulation models. Rich longitudinal microdata sets that provide the wealth of variables and repeated observations of individuals that are needed to estimate dynamic behavioral functions are available. These data sets include the Panel Study of Income Dynamics, the National Longitudinal Surveys of Labor Market Experience, SIPP, and others. Some of these surveys provide information for 20 years or more, have been extensively analyzed by many researchers, and are maintained by central facilities. These centers, in turn, provide a broad range of services to the users, such as documentation, training, and bibliographic reference services. Not only do these data sets furnish important inputs for modeling, but the central facilities that support them may serve as prototypes for the support of microsimulation models for academic research purposes. As noted above, some of our recommendations to policy analysis agencies are directed specifically toward facilitating academic involvement with microsimulation models that are used for policy analysis. We suggest ways to attract statisticians and other academics to the issues involved in validating microsimulation model results. For example, data sets, such as the results generated from our TRIM2 validation experiment, can readily be made available to

OCR for page 273
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations researchers for more extensive analysis. Policy analysis agencies can also actively encourage research on improved methods for validating microsimulation model output through such mechanisms as fellowships that allow researchers to work on-site with agencies or their contractors. We also suggest ways for the agencies to obtain more focused research from academic social scientists that would make possible improved modeling capabilities in such areas as incorporating behavioral response. Such involvement on the part of academic researchers is important to maintain and replenish the intellectual capital that underlies the use of microsimulation models for policy analysis purposes. In turn, such involvement may help to stimulate academic interest in microsimulation techniques for more basic social science research. We note that demographers, particularly those working in the sub discipline of family demography, have a long and rich tradition of the development and use of microsimulation models for research purposes. (Anthropologists and geneticists have also been involved in microsimulation-based research studies related to human populations.) The earliest studies in this tradition were initiated in the mid-1960s. By 1972, at a conference sponsored by the Social Science Research Council (Dyke and MacCluer, 1973), researchers reported on the use of dynamic microsimulation techniques to carry out such diverse studies as the survival probabilities of small, closed populations over long time periods under different rules regarding marriage among relatives and clans and the probabilities of pregnancy outcomes (e.g., still versus live births) as a function of consanguinity and genetic inheritance. These models, like cell-based population projection models, typically modeled marriage, fertility, and mortality; however, they applied transition probabilities to individual persons rather than to age-sex groups and hence could accommodate a much greater level of detail: for example, simulating complicated societal marriage rules and other aspects of "marriage markets," such as age preferences for spouses, and often including added functions such as divorce, remarriage, and birth outcomes. One early demographic microsimulation model, POPSIM (Population Simulation Model), was used for policy analysis of alternative family planning methods (Rao et al., 1973; see also Horvitz et al., 1971). In the early 1970s, the National Science Foundation supported the development of the SOCSIM (Social Simulation) model, which simulated demographic processes with the added dimension of household structure (Hammel et al., 1976). One innovative application of SOCSIM, in the field of historical demography, was to model the household composition of preindustrial English villages over a 150-year period under different assumptions about demographic rates and societal preferences for living arrangements (Wachter, with Hammel and Laslett, 1978). The model was invoked to shed light on a troubling discrepancy between theory and evidence, namely, the much higher-than-expected percentages of nuclear families in listings of household membership in preindustrial English settlements. The simulation, which included 15 scenarios, each replicated 64

OCR for page 273
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations times with different random numbers, demonstrated that the empirical data were unlikely to result from demographic constraints. SOCSIM has continued to be used and expanded. One policy-relevant set of simulations looked at the likely kin structures of the U.S. population in the year 2000 under alternative sets of assumptions about fertility rates (Hammel, Wachter, and McDaniel, 1981; see also Reeves, 1987). The study addressed such questions as the proportion of middle-aged people who could be expected to have both older and younger dependents and the proportion of the elderly who could be expected to have kin on whom to call for support, including siblings, children, and grandchildren. (Smith, 1987, reports on the use of a similar model, CANSIM, to simulate kin sets and obtain counts of various types of kin at various stages of the life cycle.) In a paper for a volume on methods of family demography, Wachter (1987:217) comments that "simulation studies function in some respects as the social scientist's substitute for the natural scientist's controlled experiments under laboratory conditions." He summarizes the contributions of microsimulation methods for studies of complex family and household structures (also pointing to their use for simulating fertility processes); notes their advantages and disadvantages vis-a-vis analytic solutions and macrosimulations (what we term cell-based models); and indicates areas for further work. He comments that analytic, macro, and micro approaches are often all pertinent to a research problem in household structure; however, microsimulation has particular advantages when (p. 219) "the theory to be simulated specifies choices for individuals depending on their detailed circumstances…[when] the outputs desired include measurements of random dispersion as well as central tendency…[and when] complex configurations of kin…are of interest." We also note that, in basic research fields outside social science, microsimulation modeling has become quite commonplace. For example, in the 1960s, it was an ordinary part of "atom smashing" research with cyclotrons to use Monte Carlo microsimulation of subatomic particles to assess the impact of phenomena that were not directly observable (for example, pi to mu meson decay between the time the particle left the cyclotron and smashed into the target in scattering experiments). Modern cosmology is a very active area of academic research that, for many types of studies, such as the dynamics of star clusters, relies fundamentally on forms of microsimulation. It is widely accepted that contemporary research in fields as diverse as the archaeology of settlement patterns, the management of inventories for large manufacturing plants, transportion and other queuing problems, and simulations in neural biology of neural networks, to name some examples, may require the development of large-scale microdata-based simulation models. For social science research, we accept the premise of Orcutt's dream, namely, that microsimulation techniques afford the opportunity to address a number of complex analytical problems and thereby advance the state of

OCR for page 273
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations knowledge. As just described, the field of demography provides examples of the utility of microsimulation to draw out the implications of complicated interactions that conventional methods of demographic analysis, such as life tables and their extensions, are unable to handle. Another application of microsimulation modeling that has been closely linked to academic research—namely, the simulation of systems of behavior in urban areas—also illustrates the utility of the technique. Because of complex locational factors interacting with individual behavioral decisions, researchers have found microsimulation modeling to be useful to integrate research about locational choices and consumer demand with housing and transportation policies. See, for example, de Leeuw and Struyk (1975); Ingram, Kain, and Ginn (1972). We believe that there are opportunities to expand the role of microsimulation for studies that combine demographic, sociological, and economic concerns and that may have policy relevance as well. For example, it has been shown that labor force experience on the part of young women increases their likelihood of getting divorced (Rowe, 1990). But researchers are generally interested in learning more than this: What effect does this factor have on the amount of time children can expect to spend growing up in a single-parent household? As another example, if similar longitudinal microdata analyses have examined the determinants of labor force entry and exit, it would be interesting to know how these two processes interact. Microsimulation models have the potential to address these and other research questions involving complex interactions. Bergmann (1990), in describing a highly stylized microsimulation model of job search and hiring behavior on the part of workers and firms, argues strongly for the research utility of microsimulation techniques. (Her model finds, contrary to conventional analysis, that the availability of unemployment insurance does not always raise the unemployment rate.) Bergmann (1990:100-101) comments: Microsimulation models have a number of advantages over conventional methods used by economists. One advantage is the explicitness with which the interactions on the micro level can be depicted. Conventional methods of economic analysis often provide cogent derivations of individuals' contingency plans, but tend to be weak in describing the process that allows all the contingency plans to mesh together…Simulation can be especially helpful in dealing with problems of information and expectations…Simulation models…can be set up to be completely recursive, as real life is…Nonlinearities, switches of regime, and other complexities are easily accommodated. In a similar vein, microsimulation has the potential to address a number of problems that confront the conventional economic theory and analysis of the firm. Anderson's (1989) analysis suggests that individual firms' investment behavior responds nonlinearly and, indeed, discontinuously to tax status. Arthur (1990) argues cogently that increasing returns to scale and chance events are

OCR for page 273
Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling, Volume I - Review and Recommendations important factors in the dynamics of firms, and that these factors have a dominant impact on the kinds of market structures that emerge. Microsimulation models can be developed to take account of all these factors. There is no need to impose simplified patterns of behavior on individual decision units in order to satisfy aggregation conditions. It is natural in a microsimulation framework to represent all the diversity and heterogeneity of the individuals, families, or firms that available microdata can support. There is no need to assume away major aspects of the real world in the theory of the firm, such as increasing returns to scale and "satisfying" behavior (see, e.g., Nelson and Winter, 1982, or Eliasson, 1985, for a Swedish example). Nor is there a need to ignore important determinants of key demographic processes, such as divorce (see Wolfson, 1989a), or to confine analysis to steady-state population structures, as illustrated by the multigeneration population dynamics models developed by Wachter and others (described above). Population heterogeneity, complex nonmaximizing patterns of behavior, increasing returns to scale, and disequilibrium dynamics can all be encompassed within microsimulation modeling. The time is ripe for a new look at the utility of microsimulation methods to advance the state of social science knowledge. This objective is important in and of itself, and it can lead to advances in the state of the art of public policy research and analysis. We are hopeful that the confluence of factors identified above—including reduced costs of using microsimulation models, the availability of richer data sets, and the existence of a large and growing community of researchers who have hands-on experience in working with complex microdata—will lead not only to better policy making but also to the rebirth of a role for microsimulation modeling in social science research and, ultimately, to the full realization of Orcutt's dream.