National Academies Press: OpenBook
« Previous: VALIDATING POPULATION PROJECTIONS
Suggested Citation:"Sensitivity Analysis." National Research Council. 1991. Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers. Washington, DC: The National Academies Press. doi: 10.17226/1853.
×
Page 310
Suggested Citation:"Sensitivity Analysis." National Research Council. 1991. Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers. Washington, DC: The National Academies Press. doi: 10.17226/1853.
×
Page 311
Suggested Citation:"Sensitivity Analysis." National Research Council. 1991. Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers. Washington, DC: The National Academies Press. doi: 10.17226/1853.
×
Page 312
Suggested Citation:"Sensitivity Analysis." National Research Council. 1991. Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers. Washington, DC: The National Academies Press. doi: 10.17226/1853.
×
Page 313

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

EVALUATING THE ACCURACY OF U.S. POPULATION PROJECTION MODELS 310 migration. They might also incorporate evaluations of the internal logic of the projection model itself. External validations are concerned with the extent of agreement between projected population figures and subsequent realized outcomes. As already noted, conditional population forecasts must be correct unless some computational error has been made. But these projections are usually treated by the user community as unconditional forecasts, so it is reasonable to ask how close the projections come to what actually occurred. Estimation of variance is related to a discussion of confidence intervals and is reserved for the following section. Sensitivity Analysis At one level of understanding, population projections are necessarily valid. For example, projections by the Census Bureau or the SSA cannot be wrong, assuming no computational errors have been made, because agency analysts simply lay out the necessary outcomes of a series of hypothetical inputs. Nevertheless, because inputs to a population projection model are relatively simple, it is generally of interest to carry out sensitivity analyses on each component, given reasonable bounds on each. Different aspects of a projection are affected to a lesser or greater degree by each of the components. For example, if the potential demand for day care is of interest, fertility inputs will be of primary importance. On the other hand, if future outlays for old-age security benefits are at issue, mortality inputs will prove to be much more crucial, at least for the next 65 years. When population projections are made, alternative assumptions are typically offered in the form of high, middle, and low assumptions for fertility, mortality, and immigration. It is possible to observe the variability of a certain population output to each of these inputs. By taking each of the three high assumptions and each of the three low assumptions as equally likely, it becomes apparent which component matters most to the final outcome.4 In 1989 the Census Bureau published 27 projections of the U.S. population, one for each combination of the three inputs (fertility, mortality, and immigration) at three levels (high, middle, and low).5 Table 1 shows the projected population in the year 2010 that results from varying only one component at a time, while holding the other two constant at their middle levels. The first panel gives the population under age 5, the relevant population for most day care centers. Variation is clearly driven by the fertility component. In the second panel, projections are presented for the population age 75 and over. Sometimes 4One could always set the high and low inputs for a particular component at such extreme values that the outcome is necessarily driven by that component. Valid sensitivity tests depend on the reasonableness of the input components. 5Three additional projections are made, under the assumption of no immigration.

EVALUATING THE ACCURACY OF U.S. POPULATION PROJECTION MODELS 311 designated the old-old, this is the population group of interest in determining the demand for nursing care. Because the entire population over age 75 in the year 2010 has already been born, alternative assumptions of fertility are irrelevant here.6 Migration matters to some extent, but alternative assumptions of mortality are the predominant driving force in determining the size of this population. The final panel of Table 1 presents the total population figures. No single component is as clearly dominant as in the projections of the very young and the very old. Alternative assumptions regarding fertility are more important than those regarding mortality or immigration, but the difference is not large.7 TABLE 1 Projected U.S. Population Size in 2010 Under Varying Assumptions of Fertility, Mortality, and Immigration Population and Component that Varies Level of Varying Component Percentage Change From Middle High Middle Low High Low Population under age 5 (in thousands) Fertility 20,314 16,899 14,340 20.2 −15.1 Life expectancy at birth 16,956 16,899 16,850 0.3 −0.3 Immigration 17,590 16,899 16,392 4.1 −3.0 Population age 75 and over (in thousands) Fertility 18,328 18,323 18,321 0.0 −0.0 Life expectancy at birth 20,339 18,323 16,918 11.0 −7.7 Immigration 18,378 18,323 18,204 0.3 −0.6 Total population (in thousands) Fertility 292,775 282,575 274,603 3.6 −2.8 Life expectancy at birth 287,271 282,575 278,654 1.7 −1.4 Immigration 290,671 282,575 275,856 2.9 −2.4 NOTE: For each component that varies, the other two components are held at the middle level. SOURCE: Spencer (1989 Projections 5, 11, 13, 14, 15, 17, 23). It has become common practice in presenting population projections to focus on three series, representing a “best guess” scenario and a high and a low alternative. This is the approach taken by the Census Bureau, the SSA, and the United Nations (1989). The high and low alternatives are taken to be outer bounds within which the final outcome is expected to fall. The Census Bureau's projections, for example, combine high (low) fertility, low (high) mortality, and high (low) net immigration to maximize (minimize) the expected population size. The SSA is less interested in the total population size than it is in the 6Slight variations exist in the projections because the handling of census undercount and overcount is slightly different in each. 7Alho and Spencer (1985) indicate that the high and low fertility assumptions are considerably more narrow than those regarding mortality and migration. If so, the fertility component is in fact more important for population size than Table 1 indicates.

EVALUATING THE ACCURACY OF U.S. POPULATION PROJECTION MODELS 312 financial well-being of the Old-Age Survivors and Disability Insurance (OASDI) program. As a result, the outer bounds presented are those that minimize (or maximize) the ratio of the old-age population to the working-age population. These figures are derived by combining high (low) fertility, high (low) mortality, and high (low) immigration. These projections are referred to as optimistic and pessimistic, respectively. FIGURE 1 U.S. population projections by the Bureau of the Census and the Social Security Administration. NOTE: The three lines for each agency represent high, middle, and low scenarios; see text for discussion. SOURCES: Data for the Census Bureau from Spencer (1989); data for SSA from Wade (1988). Figures 1 and 2 demonstrate these points by examining the projection fans of the population through 2080. Projections of the total population are shown in Figure 1. The fans for the Census Bureau's projections are considerably wider (i.e., the difference between the low series and the high series of projections is far greater) than those for the SSA.8 The SSA is no more certain than the Census Bureau about future trends, but it is interested in placing bounds on different aspects of growth. The eventual size of the population is relatively unimportant for the financial well-being of the OASDI program. Figure 2 shows projections of the old-age dependency ratio through 2080. In this case the SSA fans are considerably wider. 8The “middle” assumption for the SSA is somewhat higher than that for the Census Bureau for a number of reasons. First, the area covered by the SSA includes Puerto Rico, other islands, and citizens overseas, whereas the Census Bureau excludes these areas. Second, the SSA base population is adjusted for net census undercount. Also, the underlying assumptions of the two models differ.

EVALUATING THE ACCURACY OF U.S. POPULATION PROJECTION MODELS 313 FIGURE 2 Old-age dependency ratios projected by the Bureau of the Census and the Social Security Administration. NOTES: The dependency ratio is the population aged 65 and older divided by the population aged 20–64. The three lines for each agency represent high, middle, and low scenarios; see text for discussion. SOURCES: Data for the Census Bureau from Spencer (1989); data for the Social Security Administration from Wade (1988). Model validation typically includes consideration of the appropriateness of the model. In the present case there is little reason to doubt the cohort component model. If fertility, mortality, and immigration are correctly predicted, there will be no error in the resulting population projection; persons can enter and leave the population in only three ways—being born, dying, or migrating. This is not the case, for example, with projections of future GNP, which ma y depend on many unknown factors. But alternatives to the cohort component model do exist— some more complicated, requiring detailed economic inputs, others quite simplistic. More complicated models, including various economic-demographic models, explicitly generate values for fertility, mortality, and migration on the basis of projected economic circumstances. In contrast, these inputs are chosen on a more subjective basis in the official SSA and Census Bureau projections. The appropriateness of these more elaborate models is largely an empirical question; do they in fact produce more accurate predictions than those based on subjective assessments? At the other end of the spectrum, simple extrapolation models may produce reasonable results, even though they do not take into account the basic demographic events responsible for population change.

Next: External Validation »
Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers Get This Book
×
Buy Paperback | $100.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

This volume, second in the series, provides essential background material for policy analysts, researchers, statisticians, and others interested in the application of microsimulation techniques to develop estimates of the costs and population impacts of proposed changes in government policies ranging from welfare to retirement income to health care to taxes.

The material spans data inputs to models, design and computer implementation of models, validation of model outputs, and model documentation.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!