Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
A VALIDATION EXPERIMENT WITH TRIM2 302 addressed. With respect to an external validation, issues that need examination include (1) the number of time periods studied and the degree to which they can be considered replications; (2) the relative size of the policy changes and the coincident social, economic, and demographic changes and their impact on the validation analysis; (3) the responses for which information is needed and their relative importance; and (4) the quality of the comparison values used for the various responses of interest. For sensitivity analyses, issues that need examination include the first three mentioned above for external validation. In addition, information is needed about the modules to be studied and the number of alternatives to be examined for each module. The choices will depend on the individual needs of each analysis, which limits the value of any general advice. However, with experience, greater understanding will be developed about, for example, the typical extent of association between consecutive years with no policy change or with a major policy change. With time, questions about the number of replications and the magnitude of policy changes worth examining, among others, will be easier to answer. SUMMARY The analysis presented here gives mixed signals about the effectiveness of TRIM2. That TRIM2 is very sensitive to the inclusion or exclusion of various factors is apparent. Our main goal was to show how sensitivity and validity studies for microsimulation models might be undertaken. It is quite reasonable to speculate that similar studies of other models would produce comparably mixed resultsânamely, that a model will prove to be most useful for some quantities and not as good for others. This knowledge can be valuable to the analysts using models to inform policy makers and to the people involved in making improvements to the models. We have made a small start toward this end. REFERENCES Doyle, Pat, and Trippe, Carole 1989 Validation of the Food Stamp Microsimulation Model. Final report to the Food and Nutrition Service, U.S. Department of Agriculture. Washington, D.C.: Mathematica Policy Research, Inc. Ericksen, E., and Kadane, J. 1985 Estimating the population in a census year: 1980 and beyond. Journal of the American Statistical Association 80:98â131. Fay, Robert E., III 1989 An analysis of within- household undercover age in the Current Population Survey. Pp. 156â175 in Proceedings of the Bureau of the Census Fifth Annual Research Conference. Washington, D.C.: U.S. Department of Commerce.
A VALIDATION EXPERIMENT WITH TRIM2 303 Fein, David 1989 The Social Sources of Census Omission: Racial and Ethnic Differences in Omission Rates in Recent U.S. Censuses. Dissertation submitted to Sociology Department, Princeton University. Giannarelli, Linda 1989a Aging the 1983 CPS file to 1987. Urban Institute memorandum of 11/21/89 to Connie Citro, Committee on National Statistics, Commission on Behavioral and Social Sciences and Education, National Research Council, Washington, D.C. 1989b Alignment of AFDC Results to Target Data. Urban Institute memorandum of 11/29/89 to Dick Michel. Available from Committee on National Statistics, Commission on Behavioral and Social Sciences and Education, National Research Council, Washington, D.C. 1989c Simulation Results Using Files Adjusted for the Undercount. Urban Institute memorandum of 12/29/89 to Connie Citro, Committee on National Statistics, Commission on Behavioral and Social Sciences and Education, National Research Council, Washington, D.C. 1989d Simulations with Old Months Combined With Aging. Urban Institute memorandum of 11/14/89 to Connie Citro, Committee on National Statistics, Commission on Behavioral and Social Sciences and Education, National Research Council, Washington, D.C. 1990a Alignment Factors for AFDC Runs on Files Adjusted for the Undercount. Urban Institute memorandum of 1/10/90 to Connie Citro, Committee on National Statistics, Commission on Behavioral and Social Sciences and Education, National Research Council, Washington, D.C. 1990b Differences Between 1983 and 1987 Law for AFDC and SSI. Urban Institute memorandum of 1/10/90 to Connie Citro, Committee on National Statistics, Commission on Behavioral and Social Sciences and Education, National Research Council, Washington, D.C. 1990c Looking at the Quality of the 1987 Baseline AFDC Run, and the Quality of the March 1988 CPS. Urban Institute memorandum of 2/28/90 to Connie Citro, Committee on National Statistics, Commission on Behavioral and Social Sciences and Education, National Research Council, Washington, D.C. 1990d Time Spent by Urban Institute Staff on National Academy Study. Urban Institute memorandum of 1/30/90 to Connie Citro, Committee on National Statistics, Commission on Behavioral and Social Sciences and Education, National Research Council, Washington, D.C. Giannarelli, Linda, and Moore, Margaret 1989 Adjustment of CY 1983 and CY 1987 File for the Undercount. Urban Institute memorandum of 11/29/89 to Connie Citro. Committee on National Statistics, Commission on Behavioral and Social Sciences and Education, National Research Council, Washington, D.C. Haveman, Robert H., and Lacker, Jeffrey M. 1984 Discrepancies in Projecting Future Public and Private Pension Benefits: A Comparison and Critique of Two Micro-Data Simulation Models. Institute for Research on Poverty Special Report 36. Madison: University of Wisconsin Press. Kormendi, Roger C., and Meguire, Philip 1988 Dynamic Validation of the Tr im Welfare Simulation Model. Draft report to
A VALIDATION EXPERIMENT WITH TRIM2 304 the Urban Institute. University of Michigan and the Mid America Institute for Public Policy Research. Lehmann, E.L. 1975 Nonparametrics: Statistical Methods Based on Ranks. San Francisco, Calif.: Holden- Day.
