Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
DOCUMENTATION FOR MICROSIMULATION MODELS: A REVIEW OF TRIM2, MATH, AND HITSM 339 2. The documentation should be carefully reviewed by a copy editor to make the format consistent, to eliminate typographical errors, and to generally improve user friendliness. 3. Better quality control mechanisms should be instituted at the production stage to minimize flaws. 4. Chapter I of the TRIM2 Reference Manual should be rewritten with a nontechnical audience in mind. It should clearly indicate the distinction between databases and simulation modules. 5. A reorganization of the documents should be considered. For example, all database-related documentation such as codebooks, conversion routines, aging, and imputation routines could be collected together in an expanded Codebook. A User's Reference Manual with the âhow toâ material from the Reference Manual and the purpose statements and input parameters from the Simulation Modules could be developed. Finally, a more technical Programmer's Reference Manual could be developed for the system programmers who will be developing new modules or making major changes to existing ones. MATH Like the TRIM2 documentation, the MATH model's documentation consists of three partsâTechnical Description (Doyle et al., 1989), User's Guide (Social & Scientific Systems, Inc., n.d.), and Codebook (Doyle, 1989). The preface to the Technical Description indicates that the particular document that the panel reviewed was an abbreviated version of another document, also entitled MATH Technical Description (Doyle and Bernhardt, 1983). The version that was reviewed focuses on the routines currently used by the U.S. Department of Agriculture's Food and Nutrition Service. The documentation does not provide the reader with any key to the purposes of the three parts or how to use them, so one is left with presumptions about their use. Apparently, the Technical Description is intended for analysts and the User's Guide is intended for programmers. The Codebook provides specifics concerning the underlying data, so it would be useful to analysts and programmers. Critique MATH's Technical Description is organized by an initial introductory chapter intended for a general nontechnical audience, followed by major sections on initial data processing, aging routines, taxes and transfers, and utilities. Descriptions of several routines comprise each of the major sections. The table of contents for the document lists the order of the material in the book using routine numbers (not page numbers!). While the order of the material is, of course, useful, the reader is given no clue as to the purpose or meaning of the routine numbers. For the most part, they are three-digit numbers between
DOCUMENTATION FOR MICROSIMULATION MODELS: A REVIEW OF TRIM2, MATH, AND HITSM 340 021 and 506, but some numbers have letters attached, one is âH-1,â and some routines do not have a number. Figure 3 of the introduction (the document has no list of tables or list of figures, and Figure 3 has no page number) provides some clue about the different routines and how they fit into the model, but there are ma ny routines in the table of contents that are not included in the figure and some routines in the figure that are not included in the table of contents. The Introduction to the Technical Description is a much better nontechnical description of microsimulation and MATH than the similar chapter in TRIM2's Reference Manual. It gives a clear distinction between the underlying database, its preparation, and the simulation routines. It also includes a flowchart that greatly enhances the text. In both the TRIM2 introductory chapter and here, however, it is bothersome to encounter marketing text (touting the power and advantages of the model). Presumably, by the time the reader encounters the technical documentation, he or she is already sold on the product. The rest of the Technical Description consists of technical specifications for the model's various routines. Again, it was difficult to evaluate the accuracy of the specifications. Furthermore, a reader is given almost no help in navigating the document. Consequently, reviewing the document from front to back leads to a sensation of disconnectedness. It is like looking at the individual pieces of a jigsaw puzzle with no guidance as to how the pieces fit together. Some observations on the Technical Description: 1. The format of the individual sections is inconsistent. Often, the individual chapters have introductions or statements of purpose, but sometimes these are in section I of a chapter and sometimes they are in section A, B, or C of particular subsections of chapters. 2. As with TRIM2, production of MATH's Technical Description appears to have lacked quality control. For example, the chapter on the demographic aging routine (#208) has an interesting introduction that explains much of the theory and empirical work on which the routine is based. However, upon first reading the description, a reader discovers that page 2 of the introduction has apparently not been copied, but rather page 1 is followed by page 4. Later, it is discovered that page 2 is copied onto the reverse side of page 26.5. As another example, in the specifications for the routine to convert the March CPS to a MATH database (#021), the text refers to a list in Table 1. There are two lengthy tables in the chapter labeled Table 1a and 1b, and there is another table, much later in the chapter, labeled Table 1. Finally, in the specifications for the alternative demographic aging routine (#202),4 reference is made to a Table 1, but no table exists in this section. 4The text does not explain why a user would be interested in using this alternative given that there is another demographic aging routine (#208).
DOCUMENTATION FOR MICROSIMULATION MODELS: A REVIEW OF TRIM2, MATH, AND HITSM 341 3. The contents are often very poorly written. To give the reader a feel for what it is like to encounter this documentation, Figure 1 is a table from the description of the federal income tax simulation routine (#306). The documentation text refers to Table 6, whereas the table is labeled 6A. There is no Table 6 or 6B, for that matter. The title of the table appears to be missing a word or punctuation mark after the word âDEDUCTIONS.â The title uses the term âSELECTED YEARS PRIOR TOâ¦ 1986.â However, no years are given in the table, and the data in the table seem to have no relationship to time. A phrase at the bottom of the table, which is presumably a general note, indicates that the table entries are scaled in thousands. In fact, whereas columns 2 and 4 are scaled in thousands, columns 3 and 5 are scaled in millions, and columns 1, 6, and 7 are not scaled. These errors are typical of the document. 4. The MATH Technical Description is a very lengthy and complex document (over 300 pages). The complexity and length seem unnecessary, however. For example, in the routine to impute a health disability variable (#106), several pages describe imputations that the document indicates are obsolete. The documentation updating procedure should eliminate such descriptions rather than tell the reader that they are obsolete. Furthermore, the imputations are simple recodes of two or three existing variables. It would be easy to use a single page rather than 11 pages to document the imputations. Also, it seems oblique to use an entire page for a âgeneralized marital status definerâ that indicates that the model considers an observation to be married if the underlying data report that the individual is âmarried, spouse present.â 5. Finally, the degree of referencing to external data sources and documents is highly uneven. When references are given, though, they seem to be more complete than those in the TRIM2 documentation. The description of the simulation of federal income taxes and the simulation of the participation algorithms in AFDC, supplemental security income (SSI), and food stamps have very few references. On the other hand, the benefits and eligibility parameters in the AFDC and SSI simulations have good documentation. The writers of the MATH User's Guide had a very particular meaning in mind for the word âuser.â This document seems to be useful only to programmers who are debugging model errors or making changes to the model. It is much too technical for an analyst or a research assistant with minimal programming knowledge who might be charged with applying the model. There is no introductory material to explain how the model processes data, the types of parameters, or the JCL set-ups, for example. Because of its highly technical content, the panel was again forced to focus on format clarity rather than accuracy. The MATH User's Guide has ma ny advantages over the Technical Description in this respect. The format of the individual sections is consistent (at least for the sections identified as Release 89.1). Each section has a brief statement of purpose (including a
DOCUMENTATION FOR MICROSIMULATION MODELS: A REVIEW OF TRIM2, MATH, AND HITSM 342 FIGURE 1 Sample table with numerous errors from the MATH Technical Description.