National Academies Press: OpenBook
« Previous: CHOICE OF COMPONENT MODULES AND ALTERNATIVES FOR STUDY IN TRIM2
Suggested Citation:"OUTPUTS EXAMINED." National Research Council. 1991. Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers. Washington, DC: The National Academies Press. doi: 10.17226/1853.
×
Page 281
Suggested Citation:"OUTPUTS EXAMINED." National Research Council. 1991. Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers. Washington, DC: The National Academies Press. doi: 10.17226/1853.
×
Page 282

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

A VALIDATION EXPERIMENT WITH TRIM2 281 which was run to provide simulations of 1987 law with the 1983 database (from the March 1984 CPS). A fourth factor that was initially considered for study was the algorithm for deciding which eligible households choose to participate in social welfare programs. (See Giannarelli [1989b, 1990a] for a description of the current TRIM2 participation function and the calibration procedure that was used for the 1983 and 1987 baseline files [both adjusted and unadjusted for undercoverage] to align the participation function to match selected administrative control totals. See also Citro and Ross, Chapter 3 in this volume.) The possibility of replacing the current probit equation with a logit equation, least squares, or other alternative was considered but not carried through because of the time demands necessary for reestimating the equation and because the Urban Institute was then embarking on a project to reestimate the parameters of the participation function. Also, since the AFDC module depends on the participation function, reestimating it would have delayed portions of the experiment. Nevertheless, this is a module that clearly needs this type of examination. As Dick Michel (Urban Institute, personal communication, May 26, 1989) has pointed out: The participation adjustments of all microsimulation models are really the core of the model's credibility…. Further, to the extent that participation functions are up-to-date and can account for changes in policy variables (like the benefit level), they will automatically provide us with a best estimate of the potential behavioral responses to a policy change. In addition to the 16 variations simulating 1987 law with the 1983 database (from the March 1984 CPS), we had available (1) the 1983 baseline files, both adjusted and unadjusted for undercoverage, which simulated 1983 law with the March 1984 CPS;2 (2) the 1987 baseline files, both adjusted and unadjusted for undercoverage, which simulated 1987 law with the March 1988 CPS; and (3) tabulations from the IQCS of characteristics of the AFDC recipient population in both 1983 and 1987.3 These are shown as the QC83 and QC87 runs, respectively, in Table 1. (Table 1 provides a description of the 16+6=22 runs that constitute the experiment.) OUTPUTS EXAMINED The choice of which outputs to examine and the weight given to various 2These simulation runs were used as baselines to help estimate changes from 1983 to 1987. Notice that the only baselines carried out were for runs with MONTHS and no aging. Therefore, they are imperfect for estimating change for trials using either old MONTHS or the various aging methods, although it is difficult to conceive of how to run a baseline for the aging methods. 3The IQCS sample is a large monthly sample of AFDC, Medicaid, and food stamp case records that is taken to monitor the administration of welfare programs in each state. For aggregates, such as total caseloads, rather than use this sample information, actual administrative totals were used here. See Giannarelli (1989c, 1989d) for details.

A VALIDATION EXPERIMENT WITH TRIM2 282 TABLE 1 Description of TRIM2 Experimental Runs Run Identification AFDC Law CPS Year Adjustment Months Aging Base83, 83 1983 1983 No Current — Base87, 87 1987 1987 No Current — 1 1987 1983 No Current None 2 1987 1983 No Current Demo 3 1987 1983 No Current Unemp 4 1987 1983 No Current Full 5 1987 1983 No Old None 6 1987 1983 No Old Demo 7 1987 1983 No Old Unemp 8 1987 1983 No Old Full A.Base 83, 83 1983 1983 Yes Current — A.Base 87, 87 1987 1987 Yes Current — 9 1987 1983 Yes Current None 10 1987 1983 Yes Current Demo 11 1987 1983 Yes Current Unemp 12 1987 1983 Yes Current Full 13 1987 1983 Yes Old None 14 1987 1983 Yes Old Demo 15 1987 1983 Yes Old Unemp 16 1987 1983 Yes Old Full IQCS83 IQCS data for 1983 IQCS87 IQCS data for 1987 NOTE: Base83,83 and Base87,87 are the TRIM2 baseline files for 1983 (from the March 1984 CPS) and 1987 (from the March 1988 CPS), respectively, that simulate 1983 and 1987 AFDC program law. A.Base83,83 and A.Base87,87 are the corresponding TRIM2 baseline simulations using CPS files that have been adjusted for the undercoverage. IQCS83 and IQCS87 contain AFDC administrative date for samples of cases from the 1983 and 1987 Integrated Quality Control System, respectively. degrees of correspondence with the comparison values for those outputs (i.e., choice of loss function) will result in different rankings of the alternatives to TRIM2. In terms of policy relevance, the importance of various outputs will vary depending on the programs under consideration; the quality of the input data; and the economic, social, and demographic dynamics of the period in question. For this reason it is not meaningful to declare a specific loss function resulting from the choice and weighting of outputs as a method for scoring microsimulation models over situations. There are, however, three reasons to proceed informally in this direction. First, for a particular situation, an a priori choice of loss function does make comparisons between models more objective. Second, a comparison using several responses has the benefit of reducing attention to individual responses that might be subject to large

Next: Total Variability Due to Use of Alternative Modules »
Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers Get This Book
×
Buy Paperback | $100.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

This volume, second in the series, provides essential background material for policy analysts, researchers, statisticians, and others interested in the application of microsimulation techniques to develop estimates of the costs and population impacts of proposed changes in government policies ranging from welfare to retirement income to health care to taxes.

The material spans data inputs to models, design and computer implementation of models, validation of model outputs, and model documentation.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!