Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
EVALUATIONS OF MICROSIMULATION MODELS: LITERATURE REVIEW 261 omitted systematic variation. The variation was estimated by using a random effects analysis-of-variance model. It was impossible to validate this alternative model using DYNASIM for so short a time period. However, estimates of the percentage of residual variation attributable to systematic variation found in the PSID were larger than would be expected assuming the autoregressive structure used in DYNASIM. As a result, Holden suggested that the method for estimating hours supplied might need modification. In modeling hourly wage rates, DYNASIM used an approach that was analogous to the model for hours supplied. Through comparisons with PSID data, there appeared to be substantial error in this component of DYNASIM. For example, women who worked in only one of the 6 years had wage rates that, on average, were 71 cents below those predicted by DYNASIM, a relative error of about 40 percent. Holden suggested use of an alternative combined model for both hours and wages. HAYES (1982) Hayes (1982) carried out a very detailed external validation of the Micro Analytic Simulation System (MICROSIM). A 10 percent sample of the 1976 Survey of Income and Education (SIE) was statically aged in reverse to 1970 and then input into MICROSIM. The model was then run for 6 years, which, after aligning to 1975 income and population control totals, generated data for 1976. This 6-year time period was considered long enough to determine whether the model was tracking major demographic and economic trends. The resulting outputs were examined for their performance in macro terms, as well as for their performance distributively, by comparing them to data from the 1976 SIE and the CPS. The retrospective aging was preferable to running MICROSIM with 1976 SIE data for less than 6 years or to revising the MICROSIM code to enable it to read CPS data. The reverse static aging was complicated by several problems. One was the great disparity in unemployment between 1970 and 1975 and the necessity of using current survey week employment status in place of annual averages of unemployment to set adjustment factors. One method used to evaluate the quality of MICROSIM was to determine the degree of scaling required by the model. To do so, over the 6-year duration of the study, macroadjustment factors were set equal to 1 for each year. The model was then rerun with adjustment factors set at values such that the 1975 national estimates fell within 5 percent of targets from published data. Generally, the adjustment factors were fairly large. Except for marriage, labor force participation, and unemployment incidence, the factors reported were all 10 percent or more above or below the corresponding unaligned factors. Hayes offers this technique as one that could be valuable in identifying biases in model predictions. He also discusses a number of reasons why discrepancies
EVALUATIONS OF MICROSIMULATION MODELS: LITERATURE REVIEW 262 between these adjustment factors and unity might not be indicative of model failings, including comparability and definitional problems, sampling error, and income underreporting. However, he believes that the differences between macroadjustments and comparable factors for the unadjusted model, resulting in errors in output frequency distributions, are more selectively affected by model specification error. Hayes also examined the ability of MICROSIM to predict 1979 values by using macroadjustment factors frozen at their 1975 values, which is what one would necessarily be limited to in the usual forecasting situation. This analysis indicated that MICROSIM could not predict such changes as a reduction in unemployment and growth in female labor force participation, which did occur during this period. The same analysis was tried for a 10-year forecasting window as well, which indicated that MICROSIM produced estimates that were relatively stable over time. The direction of trends was typically accurate, but the rate of change was not generally on target. Hayes summarizes by saying that his simulation results were snapshots of MICROSIM's forecasts at three different times in the 16-year simulation period. The results suggested that MICROSIM, once aligned to a recent period, did not have a tendency to explode in a long-run (up to 10-year) simulation but that it had limited ability to track cyclical employment conditions. To compare MICROSIM with respect to its ability to match micro or distributional estimates, the estimates were compared with values obtained from the 1976 SIE and the 1976 and 1980 CPS. This was also done for 1969 estimates to determine the extent of initial disagreement and for 1985 estimates to measure the stability of MICROSIM with respect to distributional properties. Results of this analysis depend on the output of interest. After mentioning a number of important qualifications of his analysis, Hayes summarizes with two conclusions: (1) of MICROSIM's eight behavioral demographic/human capital modules (e.g., births, deaths, marriages, and divorce), the disability and education modules worked well, and (2) of MICROSIM's six behavioral economic modules, the wage rate and labor force participation modules and the incidence portion of the unemployment module performed well on a disaggregated basis. Also, the modules for unemployment, transfer and property income, homeownership, and wage rate needed distributional adjustments; the modules for labor force hours and wealth did not. Hayes recommended investigation of the use of (1) more detailed control totals or (2) more sophisticated modules that attempt to model interactions, including those between demographic and economic modules. He said that his evaluation results showed that MICROSIM required macroadjustment to nearly all operating characteristics. Also, the adjusted model predicted reasonably stable and accurate aggregate behavior, but it required additional adjustment for transfer income components, business cycles, and fertility and disability
