National Academies Press: OpenBook

Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers (1991)

Chapter: Incorporation of Sensitivity Analyses - Measurement of Total Uncertainty

Suggested Citation:"Incorporation of Sensitivity Analyses - Measurement of Total Uncertainty." National Research Council. 1991. Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers. Washington, DC: The National Academies Press. doi: 10.17226/1853.
Page 250

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

VARIANCE ESTIMATION OF MICROSIMULATION MODELS THROUGH SAMPLE REUSE 250 Incorporation of Sensitivity Analyses—Measureme nt of Total Uncertainty While the measurement of the variability of microsimulation model output resulting from the sampling variability in the primary input data base is relatively straightforward, the above discussion makes clear that assessment of the variability resulting from secondary sources, such as the use of regression equations, imputations, and statistical matches can be more problematic. This is due to both theoretical and practical considerations. An example of a theoretical issue is that the sample space of alternative model specifications for, say, a behavioral equation may be very dependent on an analyst's opinion (see below). Practically, even if the various secondary data sets used to determine various control totals, regression coefficients, etc., are fairly small and accessible, estimation of the covariance of various estimated parameters may be difficult, causing the parametric bootstrap procedure discussed above to be difficult to apply. Furthermore, since the secondary data sets may be unavailable to the analyst running the microsimulation model, if estimates of these variances and covariances are not available in the literature, their measurement may not be possible. In addition, if the input is the result of a complicated computation, access to the data set and the computational algorithm is crucial for use of the bootstrap. However, even if the various secondary data sets and procedures are available, creating bootstrap pseudosamples for each of these data sets, some of which are extremely large and have complicated sample designs, would at times involve a massive effort. As a currently more feasible alternative to assessment of overall uncertainty, it may be possible to use sensitivity analysis in conjunction with bootstrap resampling to develop estimates of total model uncertainty. Roughly speaking, one could use the bootstrap to measure variance and a sensitivity analysis to weakly measure bias. To do this, one would bootstrap the primary database as described above. However, instead of developing a distribution of values for other inputs (such as child care expense regression equations), one could, subjectively, select, say, two of what appear to be the most important inputs and, again subjectively, develop, say, two variants of one input and three variants of the other (such as three child care expense equations). One would then have six (2×3) different parameter sets to apply to, say, four pseudosamples of the primary database for a total of 24 model runs. In this way, uncertainty due to model mispecification could be better understood. However, the resulting range would not have an associated coverage probability. While this failure to produce a range with known coverage is troublesome, there are instances when this is the best that one can do. For many of the modules used within microsimulation models, the specific choice of which module to use is somewhat arbitrary, either because the underlying theory is uninvestigated or unclear. There is therefore a contribution to the uncertainty of the output from microsimulation models resulting from the choice of which

Next: Number of Replications Needed »
Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers Get This Book
Buy Paperback | $100.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

This volume, second in the series, provides essential background material for policy analysts, researchers, statisticians, and others interested in the application of microsimulation techniques to develop estimates of the costs and population impacts of proposed changes in government policies ranging from welfare to retirement income to health care to taxes.

The material spans data inputs to models, design and computer implementation of models, validation of model outputs, and model documentation.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook,'s online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!