National Academies Press: OpenBook
« Previous: Suggestions
Suggested Citation:"Critique." National Research Council. 1991. Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers. Washington, DC: The National Academies Press. doi: 10.17226/1853.
×
Page 339
Suggested Citation:"Critique." National Research Council. 1991. Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers. Washington, DC: The National Academies Press. doi: 10.17226/1853.
×
Page 340
Suggested Citation:"Critique." National Research Council. 1991. Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers. Washington, DC: The National Academies Press. doi: 10.17226/1853.
×
Page 341
Suggested Citation:"Critique." National Research Council. 1991. Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers. Washington, DC: The National Academies Press. doi: 10.17226/1853.
×
Page 342

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

DOCUMENTATION FOR MICROSIMULATION MODELS: A REVIEW OF TRIM2, MATH, AND HITSM 339 2. The documentation should be carefully reviewed by a copy editor to make the format consistent, to eliminate typographical errors, and to generally improve user friendliness. 3. Better quality control mechanisms should be instituted at the production stage to minimize flaws. 4. Chapter I of the TRIM2 Reference Manual should be rewritten with a nontechnical audience in mind. It should clearly indicate the distinction between databases and simulation modules. 5. A reorganization of the documents should be considered. For example, all database-related documentation such as codebooks, conversion routines, aging, and imputation routines could be collected together in an expanded Codebook. A User's Reference Manual with the “how to” material from the Reference Manual and the purpose statements and input parameters from the Simulation Modules could be developed. Finally, a more technical Programmer's Reference Manual could be developed for the system programmers who will be developing new modules or making major changes to existing ones. MATH Like the TRIM2 documentation, the MATH model's documentation consists of three parts—Technical Description (Doyle et al., 1989), User's Guide (Social & Scientific Systems, Inc., n.d.), and Codebook (Doyle, 1989). The preface to the Technical Description indicates that the particular document that the panel reviewed was an abbreviated version of another document, also entitled MATH Technical Description (Doyle and Bernhardt, 1983). The version that was reviewed focuses on the routines currently used by the U.S. Department of Agriculture's Food and Nutrition Service. The documentation does not provide the reader with any key to the purposes of the three parts or how to use them, so one is left with presumptions about their use. Apparently, the Technical Description is intended for analysts and the User's Guide is intended for programmers. The Codebook provides specifics concerning the underlying data, so it would be useful to analysts and programmers. Critique MATH's Technical Description is organized by an initial introductory chapter intended for a general nontechnical audience, followed by major sections on initial data processing, aging routines, taxes and transfers, and utilities. Descriptions of several routines comprise each of the major sections. The table of contents for the document lists the order of the material in the book using routine numbers (not page numbers!). While the order of the material is, of course, useful, the reader is given no clue as to the purpose or meaning of the routine numbers. For the most part, they are three-digit numbers between

DOCUMENTATION FOR MICROSIMULATION MODELS: A REVIEW OF TRIM2, MATH, AND HITSM 340 021 and 506, but some numbers have letters attached, one is “H-1,” and some routines do not have a number. Figure 3 of the introduction (the document has no list of tables or list of figures, and Figure 3 has no page number) provides some clue about the different routines and how they fit into the model, but there are ma ny routines in the table of contents that are not included in the figure and some routines in the figure that are not included in the table of contents. The Introduction to the Technical Description is a much better nontechnical description of microsimulation and MATH than the similar chapter in TRIM2's Reference Manual. It gives a clear distinction between the underlying database, its preparation, and the simulation routines. It also includes a flowchart that greatly enhances the text. In both the TRIM2 introductory chapter and here, however, it is bothersome to encounter marketing text (touting the power and advantages of the model). Presumably, by the time the reader encounters the technical documentation, he or she is already sold on the product. The rest of the Technical Description consists of technical specifications for the model's various routines. Again, it was difficult to evaluate the accuracy of the specifications. Furthermore, a reader is given almost no help in navigating the document. Consequently, reviewing the document from front to back leads to a sensation of disconnectedness. It is like looking at the individual pieces of a jigsaw puzzle with no guidance as to how the pieces fit together. Some observations on the Technical Description: 1. The format of the individual sections is inconsistent. Often, the individual chapters have introductions or statements of purpose, but sometimes these are in section I of a chapter and sometimes they are in section A, B, or C of particular subsections of chapters. 2. As with TRIM2, production of MATH's Technical Description appears to have lacked quality control. For example, the chapter on the demographic aging routine (#208) has an interesting introduction that explains much of the theory and empirical work on which the routine is based. However, upon first reading the description, a reader discovers that page 2 of the introduction has apparently not been copied, but rather page 1 is followed by page 4. Later, it is discovered that page 2 is copied onto the reverse side of page 26.5. As another example, in the specifications for the routine to convert the March CPS to a MATH database (#021), the text refers to a list in Table 1. There are two lengthy tables in the chapter labeled Table 1a and 1b, and there is another table, much later in the chapter, labeled Table 1. Finally, in the specifications for the alternative demographic aging routine (#202),4 reference is made to a Table 1, but no table exists in this section. 4The text does not explain why a user would be interested in using this alternative given that there is another demographic aging routine (#208).

DOCUMENTATION FOR MICROSIMULATION MODELS: A REVIEW OF TRIM2, MATH, AND HITSM 341 3. The contents are often very poorly written. To give the reader a feel for what it is like to encounter this documentation, Figure 1 is a table from the description of the federal income tax simulation routine (#306). The documentation text refers to Table 6, whereas the table is labeled 6A. There is no Table 6 or 6B, for that matter. The title of the table appears to be missing a word or punctuation mark after the word “DEDUCTIONS.” The title uses the term “SELECTED YEARS PRIOR TO… 1986.” However, no years are given in the table, and the data in the table seem to have no relationship to time. A phrase at the bottom of the table, which is presumably a general note, indicates that the table entries are scaled in thousands. In fact, whereas columns 2 and 4 are scaled in thousands, columns 3 and 5 are scaled in millions, and columns 1, 6, and 7 are not scaled. These errors are typical of the document. 4. The MATH Technical Description is a very lengthy and complex document (over 300 pages). The complexity and length seem unnecessary, however. For example, in the routine to impute a health disability variable (#106), several pages describe imputations that the document indicates are obsolete. The documentation updating procedure should eliminate such descriptions rather than tell the reader that they are obsolete. Furthermore, the imputations are simple recodes of two or three existing variables. It would be easy to use a single page rather than 11 pages to document the imputations. Also, it seems oblique to use an entire page for a “generalized marital status definer” that indicates that the model considers an observation to be married if the underlying data report that the individual is “married, spouse present.” 5. Finally, the degree of referencing to external data sources and documents is highly uneven. When references are given, though, they seem to be more complete than those in the TRIM2 documentation. The description of the simulation of federal income taxes and the simulation of the participation algorithms in AFDC, supplemental security income (SSI), and food stamps have very few references. On the other hand, the benefits and eligibility parameters in the AFDC and SSI simulations have good documentation. The writers of the MATH User's Guide had a very particular meaning in mind for the word “user.” This document seems to be useful only to programmers who are debugging model errors or making changes to the model. It is much too technical for an analyst or a research assistant with minimal programming knowledge who might be charged with applying the model. There is no introductory material to explain how the model processes data, the types of parameters, or the JCL set-ups, for example. Because of its highly technical content, the panel was again forced to focus on format clarity rather than accuracy. The MATH User's Guide has ma ny advantages over the Technical Description in this respect. The format of the individual sections is consistent (at least for the sections identified as Release 89.1). Each section has a brief statement of purpose (including a

DOCUMENTATION FOR MICROSIMULATION MODELS: A REVIEW OF TRIM2, MATH, AND HITSM 342 FIGURE 1 Sample table with numerous errors from the MATH Technical Description.

Next: Suggestions »
Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers Get This Book
×
Buy Paperback | $100.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

This volume, second in the series, provides essential background material for policy analysts, researchers, statisticians, and others interested in the application of microsimulation techniques to develop estimates of the costs and population impacts of proposed changes in government policies ranging from welfare to retirement income to health care to taxes.

The material spans data inputs to models, design and computer implementation of models, validation of model outputs, and model documentation.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!