National Academies Press: OpenBook
« Previous: Comment
Suggested Citation:"Evaluation of Alternative Months Procedures." National Research Council. 1991. Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers. Washington, DC: The National Academies Press. doi: 10.17226/1853.
×
Page 100
Suggested Citation:"Evaluation of Alternative Months Procedures." National Research Council. 1991. Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers. Washington, DC: The National Academies Press. doi: 10.17226/1853.
×
Page 101
Suggested Citation:"Evaluation of Alternative Months Procedures." National Research Council. 1991. Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers. Washington, DC: The National Academies Press. doi: 10.17226/1853.
×
Page 102

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

ALTERNATIVE MODEL DESIGNS: PROGRAM PARTICIPATION FUNCTIONS AND THE ALLOCATION OF ANNUAL TO 100 MONTHLY VALUES IN TRIM2, MATH, AND HITSM Evaluation of Alternative Months Procedures Confronting the same challenge of converting annual to monthly values for employment and income variables, microsimulation model developers have produced months routines that differ in a number of ways. An important question is the extent to which these differences affect the model outputs. For example, do the more detailed procedures for allocating unearned income in the MATH model produce significantly more accurate simulations of the food stamp program in comparison with the simpler procedures used in TRIM2 and HITSM? Unfortunately, there are no cross-model comparisons evaluating the various months routines. In addition to differences across models, there are within-model differences between current and previous versions of the months routines in TRIM2 and MATH. As noted earlier, research findings supported the existence of substantial intrayear variability in income and employment patterns on the part of individuals and families. Hence, the months routines in these two models were revised substantially at about the same time in the early 1980s. One might assume that the revised routines are clearly better, and hence that there is no point in comparing them with the old routines. However, it is not obvious either that a revised routine will always produce more accurate estimates for all outputs of interest from a model or that the extent of the improvements in the quality of the estimates will be sufficient to justify the costs of the revision. Hence, comparison of current with old routines seems important to carry out, as does evaluation of proposed revisions of routines before they are implemented (perhaps through some type of prototyping). In the case of the TRIM2 and MATH allocation routines, limited evidence is available about the impact of the different versions on the estimates from each model. We consider the MATH case first. The MOINC allocation routine used in MATH prior to 1984 differed significantly from the current ALLOY approach. Instead of creating employment and income values for each month in the year like ALLOY, MOINC created monthly income for a single month using both the last week data from the March CPS and the last year data. MOINC determined the following: • Earnings for the month either (1) as reported annual earnings divided by months worked last year for people both currently employed and employed last year, (2) as imputed monthly earnings based on regression equations for people currently employed but not employed last year, or (3) as zero for people not currently employed regardless of their previous year's employment pattern. • Public assistance for the month using one of the two amounts generated by PBLAST—either the amount applicable to the period of work or that applicable to the period of nonwork, based on employment status in the survey

ALTERNATIVE MODEL DESIGNS: PROGRAM PARTICIPATION FUNCTIONS AND THE ALLOCATION OF ANNUAL TO 101 MONTHLY VALUES IN TRIM2, MATH, AND HITSM week, in each case dividing the PBLAST amount by the number of months in the period. • Unemployment compensation for the month for people unemployed in the survey week based on last year's weeks worked and unemployment insurance benefits (if any) or an imputation of benefits based on user-defined probabilities and average benefits. Persons with no earnings in the last year were not assigned unemployment compensation even if currently unemployed. • Income from all other unearned sources for the month as the annual amounts received divided by 12. Doyle (1984a, 1984b; see also Beebout, 1989) analyzed the ISDP data to provide an empirical foundation for the annual to monthly income translation. With the exception of income from assets, Doyle's findings generally did not support the earlier assumption that unearned income could be appropriately allocated by dividing by 12. The current MATH ALLOY routine was developed to incorporate the findings from the ISDP. The new module resulted in increasing the estimate of households eligible for food stamps by 13 percent and people eligible for food stamps by 16 percent compared with the previous procedure (Lubitz and Doyle, 1986).6 No other evidence is available about the impact of the changed allocation procedures. Turning to the TRIM2 case, prior to 1984 the TRIM2 MONTHS routine simply allocated weeks worked by starting in a random month (in the first week of that month) and continuing until the reported annual weeks of work were exhausted. Periods extending beyond December wrapped around to the first months of the same year. MONTHS then allocated weeks looking for work until they were exhausted. The remaining weeks were weeks out of the labor force. Thus, depending on the starting month, an individual could be simulated to have one or at most two spells of unemployment during the year. Moreover, no account was taken of seasonal variations. The old MONTHS module then allocated employment-related and other income in the same manner as the current MONTHS module. 6However, as noted before, the PBLAST routine has not been modified to compute monthly benefits for the AFDC, SSI, and GA programs using monthly income generated by ALLOY. Instead, PBLAST continues to run first (as it did with MOINC), producing annual benefits for units simulated to participate in cash public assistance programs. The benefit amount is the sum of benefits calculated for a weeks-worked period and a weeks-not-worked period, where the two periods are determined in a manner that takes account of the type of filing unit and the characteristics of the unit head and/or spouse. In making the benefit calculation, PBLAST allocates earned income to the period of work, unemployment insurance to the period not working, and all other unearned income evenly across all 12 months. The ALLOY routine runs next, making adjustments to its procedures for determining monthly employment status to be consistent with PBLAST for simulated public assistance participants. ALLOY also has a procedure to allocate the simulated public assistance benefits by month. The monthly variables calculated by ALLOY are then used by the food stamp module FSTAMP. For some number of food stamp participants, the assumptions regarding their monthly income stream differ in the PBLAST and FSTAMP routines.

ALTERNATIVE MODEL DESIGNS: PROGRAM PARTICIPATION FUNCTIONS AND THE ALLOCATION OF ANNUAL TO 102 MONTHLY VALUES IN TRIM2, MATH, AND HITSM The validation experiment conducted by the Panel to Evaluate Microsimulation Models for Social Welfare Programs with the TRIM2 model examined the effects on estimates of the AFDC program of using the current versus the old MONTHS routine (see Cohen et al. in this volume for details of the experiment).7 Unlike the case in which the current MATH months routine produced many more units eligible for food stamps than the old routine, use of the current TRIM2 MONTHS routine made virtually no difference in the total number of units eligible for AFDC or recipients of AFDC compared with the old routine.8 These results are likely to be due to the fact that the food stamp caseload contains a higher proportion than the AFDC caseload of recipients with employment and earnings. (For example, administrative data show that, in 1987, 21% of food stamp recipients, compared with 8% of AFDC recipients, had earnings; see U.S. House of Representatives [1990:587,1267].) Employed people are less likely overall to be simulated as eligible for program benefits, but more of them will be made eligible—on a part-year basis—by the current months routines that simulate greater intrayear employment and income variability. These suppositions are borne out by the findings from the TRIM2 validation experiment with regard to estimates for the component of the AFDC population with earnings. The current MONTHS routine produced 18 percent more units eligible for AFDC and recipients of AFDC with earnings compared with the old routine. Although a small proportion of the total, the component of the caseload with earnings has often been of interest to policy makers. Given that the choice of routine makes a difference, there is another separate question of which routine is better in the sense of producing more accurate estimates. The panel's experiment with TRIM2 included an external validation component. It used a 1983 database to project the AFDC population in 1987 under 1987 law and hence could invoke administrative data for 1987 as a standard of comparison (see Cohen et al. in this volume). In this one study the two MONTHS routines were both within 2 percent of the target number of AFDC participants in 1987 projected from a 1983 database. However, the current routine overestimated AFDC participants with earnings by 12 percent, while the old routine underestimated them by 5 percent.9 Looking at other characteristics of the AFDC caseload, such as race of head, there was little 7All of the findings reported from the panel's validation experiment with TRIM2 should be treated with great caution as they represent only one study involving one time period and one program comparison. 8 The current routine generated 0.1 percent fewer total estimated AFDC eligible units and 1.0 percent fewer estimated recipient units than the old routine. Both routines were implemented on a 1983 CPS baseline file that was used to simulate 1987 AFDC law. The comparisons cited are for runs 1 and 5 of the experiment, where run 1 is the current TRIM2 model in all particulars, while run 5 varies only in the use of the old MONTHS routine. 9As discussed further in the section on calibration of the participation function, the March CPS has historically produced higher estimates of AFDC and food stamp recipients with earnings compared with administrative data. The current MONTHS routine exacerbate these discrepancies. However, it for a single model.) A useful exercise would be to conduct the same set is not clear whether the problem lies solely with errors in the CPS or also involves errors in the case records.

Next: Need for Further Validation »
Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers Get This Book
×
Buy Paperback | $100.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

This volume, second in the series, provides essential background material for policy analysts, researchers, statisticians, and others interested in the application of microsimulation techniques to develop estimates of the costs and population impacts of proposed changes in government policies ranging from welfare to retirement income to health care to taxes.

The material spans data inputs to models, design and computer implementation of models, validation of model outputs, and model documentation.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!