Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
A VALIDATION EXPERIMENT WITH TRIM2 281 which was run to provide simulations of 1987 law with the 1983 database (from the March 1984 CPS). A fourth factor that was initially considered for study was the algorithm for deciding which eligible households choose to participate in social welfare programs. (See Giannarelli [1989b, 1990a] for a description of the current TRIM2 participation function and the calibration procedure that was used for the 1983 and 1987 baseline files [both adjusted and unadjusted for undercoverage] to align the participation function to match selected administrative control totals. See also Citro and Ross, Chapter 3 in this volume.) The possibility of replacing the current probit equation with a logit equation, least squares, or other alternative was considered but not carried through because of the time demands necessary for reestimating the equation and because the Urban Institute was then embarking on a project to reestimate the parameters of the participation function. Also, since the AFDC module depends on the participation function, reestimating it would have delayed portions of the experiment. Nevertheless, this is a module that clearly needs this type of examination. As Dick Michel (Urban Institute, personal communication, May 26, 1989) has pointed out: The participation adjustments of all microsimulation models are really the core of the model's credibilityâ¦. Further, to the extent that participation functions are up-to-date and can account for changes in policy variables (like the benefit level), they will automatically provide us with a best estimate of the potential behavioral responses to a policy change. In addition to the 16 variations simulating 1987 law with the 1983 database (from the March 1984 CPS), we had available (1) the 1983 baseline files, both adjusted and unadjusted for undercoverage, which simulated 1983 law with the March 1984 CPS;2 (2) the 1987 baseline files, both adjusted and unadjusted for undercoverage, which simulated 1987 law with the March 1988 CPS; and (3) tabulations from the IQCS of characteristics of the AFDC recipient population in both 1983 and 1987.3 These are shown as the QC83 and QC87 runs, respectively, in Table 1. (Table 1 provides a description of the 16+6=22 runs that constitute the experiment.) OUTPUTS EXAMINED The choice of which outputs to examine and the weight given to various 2These simulation runs were used as baselines to help estimate changes from 1983 to 1987. Notice that the only baselines carried out were for runs with MONTHS and no aging. Therefore, they are imperfect for estimating change for trials using either old MONTHS or the various aging methods, although it is difficult to conceive of how to run a baseline for the aging methods. 3The IQCS sample is a large monthly sample of AFDC, Medicaid, and food stamp case records that is taken to monitor the administration of welfare programs in each state. For aggregates, such as total caseloads, rather than use this sample information, actual administrative totals were used here. See Giannarelli (1989c, 1989d) for details.
A VALIDATION EXPERIMENT WITH TRIM2 282 TABLE 1 Description of TRIM2 Experimental Runs Run Identification AFDC Law CPS Year Adjustment Months Aging Base83, 83 1983 1983 No Current â Base87, 87 1987 1987 No Current â 1 1987 1983 No Current None 2 1987 1983 No Current Demo 3 1987 1983 No Current Unemp 4 1987 1983 No Current Full 5 1987 1983 No Old None 6 1987 1983 No Old Demo 7 1987 1983 No Old Unemp 8 1987 1983 No Old Full A.Base 83, 83 1983 1983 Yes Current â A.Base 87, 87 1987 1987 Yes Current â 9 1987 1983 Yes Current None 10 1987 1983 Yes Current Demo 11 1987 1983 Yes Current Unemp 12 1987 1983 Yes Current Full 13 1987 1983 Yes Old None 14 1987 1983 Yes Old Demo 15 1987 1983 Yes Old Unemp 16 1987 1983 Yes Old Full IQCS83 IQCS data for 1983 IQCS87 IQCS data for 1987 NOTE: Base83,83 and Base87,87 are the TRIM2 baseline files for 1983 (from the March 1984 CPS) and 1987 (from the March 1988 CPS), respectively, that simulate 1983 and 1987 AFDC program law. A.Base83,83 and A.Base87,87 are the corresponding TRIM2 baseline simulations using CPS files that have been adjusted for the undercoverage. IQCS83 and IQCS87 contain AFDC administrative date for samples of cases from the 1983 and 1987 Integrated Quality Control System, respectively. degrees of correspondence with the comparison values for those outputs (i.e., choice of loss function) will result in different rankings of the alternatives to TRIM2. In terms of policy relevance, the importance of various outputs will vary depending on the programs under consideration; the quality of the input data; and the economic, social, and demographic dynamics of the period in question. For this reason it is not meaningful to declare a specific loss function resulting from the choice and weighting of outputs as a method for scoring microsimulation models over situations. There are, however, three reasons to proceed informally in this direction. First, for a particular situation, an a priori choice of loss function does make comparisons between models more objective. Second, a comparison using several responses has the benefit of reducing attention to individual responses that might be subject to large