example, U.S. EPA 200424). These pitfalls also apply to the MMOD programs and include (1) not allowing for optimal weighting of the probabilities and consequential errors; (2) not permitting a reliable comparison of alternative decisions; (3) failing to communicate the range of control options that would be comparable with different assessments of the true state of nature; and (4) precluding the opportunity for identifying research initiatives. Examples of these pitfalls are characteristic of the CARA/COLA programs where there is a significant lack of uncertainty analysis associated with those programs (see Chapter 9).
In general, NASA MMOD programs have embraced some of the principles identified in Box 5.1. However, as both the agency and the MMOD programs mature, it becomes increasingly important to better characterize risk and uncertainty in all aspects of the MMOD problems being addressed. Other issues to be addressed include the sources of the uncertainties, how to reduce uncertainties, identifying those that cannot be significantly reduced or removed over the near term, and estimating the time and effort required to reduce significantly extant uncertainties. Of course, natural variability in the MMOD environment will prevent uncertainties from being removed entirely, no matter how sophisticated and detailed testing programs and modeling efforts become.
It is equally important that uncertainty information continue to be communicated to decision makers or program leaders because they are the ones who will determine how to handle this information within the NASA framework of mission planning and operations. While the calculation and communication of uncertainty information to decision makers, including those who plan space missions, has improved at NASA, it is also apparent that a fully integrated cataloging and assessment of MMOD-related uncertainties does not routinely occur in mission-planning and decision-making activities as noted above, this type of information is typically conveyed to management when the uncertainties are either small enough to be ignored, or large enough to be obvious so that either more data or some sort of corrective action is required. Since many of these decisions appear to be made at the program level, effective communication of uncertainty information both to the public and to the proper management levels is an issue of considerable importance that needs constant reevaluation and oversight.
Recommendation: NASA’s meteoroid and orbital debris programs should increase their efforts to reduce the uncertainty and variability in models through acquisition of measurements (and where necessary, to do testing and analysis) for continually improving assessment of risk and characterization of uncertainty. Together with its MMOD efforts, NASA should continue to advance the agency’s efforts to present information on uncertainty in risk analyses. Special attention should be given to maximizing public understanding of uncertainty analysis through peer-reviewed papers and other publications.
24 Environmental Protection Agency, An Examination of EPA Risk Assessment Principles and Practices, EPA/100/B-04/001, Washington, D.C., March 2004.