Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
A VALIDATION EXPERIMENT WITH TRIM2 279 by the use of an imperfect surrogate for the truth were ignored. Whenever feasible, analysts should search for the sources of major discrepancies between model estimates and the comparison values in both data systems. In particular, discrepancies between model estimates and comparison values that are less than the distance explained by sampling variability should be ignored. CHOICE OF COMPONENT MODULES AND ALTERNATIVES FOR STUDY IN TRIM2 The second major step was to identify several TRIM2 modules currently in use that could be replaced by alternative ones that were, a priori, reasonable substitutes. TRIM2 would then be run with various combinations of alternatives for these modules to determine which choices and combinations of choices resulted in the greatest variability in the model's outputs. In addition, by using the IQCS comparison values, it could also be determined which alternatives were more successful in approximating these quantities. The choice of modules for the experiment was based on considerations of ready availability of alternatives and substantive interest. Identification of alternatives was facilitated when they had either been used previously or been considered as substitutes for the algorithms in use. Some modules of strong interest were excluded on grounds that it would be costly to reprogram them. (Through software redesign, cost should ultimately become a less important consideration in deciding which modules to study as part of a sensitivity analysis.) Our criteria resulted in the choice of the following three modules: 1. Adjustment for CPS undercoverage TRIM2 currently does not attempt any correction for undercoverage of certain population groups in the March CPS. (See Citro, Chapter 1 in this volume, for a discussion of the extent of undercoverage, which is believed to be sizable for welfare-eligible populations.) By modifying the models in Fein (1989), a logistic regression model was derived that specified the rates of undercoverage for households with various characteristics. (For related research, see Fay [1989] and Ericksen and Kadane [1985].) Some of the variables included in this logistic regression model were race of head of household, marital status of head of household, indicator as to whether or not household head is less than 30 years of age, and household location in a central city. (See Giannarelli and Moore [1989] for details on the model's parameters and the remaining variables used.) The effectiveness of this logistic regression model was limited by the variables that existed in the TRIM2 database and by the necessity of giving each person in a given household the same weight rather than weighting individuals to adjust for undercoverage. Nevertheless, the model provided an opportunity to see if weighting for undercoverage would make a difference in TRIM2's outputs. 2. Imputation of monthly employment and income Because the March CPS provides information only on annual employment and earnings, while the
A VALIDATION EXPERIMENT WITH TRIM2 280 AFDC program operates on a monthly accounting basis, TRIM2 includes a module referred to as MONTHS. This module endeavors to capture monthly variations in employment, unemployment, and earnings by using variables from the individual CPS records on work behavior during the previous year and aligning to data from the Bureau of Labor Statistics on aggregate monthly labor force characteristics. A simpler procedure used in an earlier version of TRIM2, referred to here as old MONTHS, simulates a maximum of two spells during the year, one working and one not working. (See Citro and Ross, Chapter 3 in this volume, for a description of the old and new MONTHS modules in TRIM2 and similar modules in MATH [Micro Analysis of Transfers to Households] and HITSM [Household Income and Tax Simulation Model].) Initially there was interest in including a second more sophisticated alternative to MONTHS through the addition of more intrayear variability, but the expected effect of such an alternative was thought to be small because few AFDC cases have earned income (though for those outputs specifically related to earned income the effect could be large). Also, we wanted to limit the number of alternatives for any single factor so that as many factors as possible could be studied within a fixed number of model runs. 3. Aging Although static aging modules are available for TRIM2, the model, as currently applied, does not invoke them. Static aging involves reweighting the records to conform to projected population counts, adjusting labor force data to match projected unemployment rates in a future time period, and adjusting income amounts for projected economic growth and price changes. We specified three alternatives in addition to ânot aging;â see Giannarelli (1989a) for details. The first alternative was to invoke the demographic aging routine to reweight the records in the March 1984 CPS to match target values for population totals by age, sex, and race generated from the March 1988 CPS. The second alternative, unemployment aging coupled with demographic aging (referred to here as unemployment aging), additionally invoked the routine to adjust labor force activity on the demographically aged file to meet targets from the March 1988 CPS for unemployment during the week of the survey and the preceding calendar year. The third alternative, referred to here as full aging, additionally invoked the routine to adjust income amounts for price changes and economic growth between 1983 and 1987. In every case, known control totals were used from the March 1988 CPS to eliminate the source of variability due to erroneous demographic and macroeconomic forecasts. These three modules then constituted the three factors of our factorial experiment. In the abbreviated terminology used henceforth, the first factor was âagingâ at four levels (none, demographic, unemployment, and full), the second was âadjustmentâ at two levels (no or yes), and the third was âmonthsâ at two levels (current or old). Thus, we had 4Ã2Ã2=16 combinations, each of
