noted that Congress understands risk, although not in a rigorous way, and often has to make decisions with little information (which, he observed, is one type of risk management). Congress is looking at whether the President’s new plan is safe enough. However, the perception of risk and safety can vary over time, with perception immediately after an accident quite different from at other times. The questions Congress must address are wide ranging. One is the cost estimate. Is it right? Too low? Another is performance risk—what has to work right for the program to succeed? Is a miracle involved? What if performance falls short of the goal? Is the initiative resilient? What is political risk? It can mean risk to a member of Congress, but more broadly, it is the vulnerability of a mission to the external political environment, such as a new administration, a new economic climate, or a new geopolitical environment. The lack of clear information on risk will make Congress less likely to commit to a new vision.
Obermann also noted that members of Congress perceive risk based on their past experience with NASA, and previous problems with the agency’s cost estimates make them more likely to view costs as high risk. Members also want milestones or tracking points to make sure a program proceeds on target. Obermann mentioned that on a number of occasions Congress has questioned the rigor and realism of some of NASA's planning efforts.
On the subject of probabilistic risk, Obermann asked how confident one could be that a risk is low probability. He pointed out that the risk of losing a shuttle was originally set at about 1 in 100,000. After the Challenger accident, it was set at 1 in 100. He asked whether that risk is one of losing just a single space shuttle or if it is the risk of losing the entire human spaceflight program. The risk and reward calculation depends on what is at stake, and different members of Congress have different opinions. It is a matter not only of how confident we can be that a risk is low probability but also of whether we can achieve consensus on what the cost or consequence is.
Obermann said that risk planning tools (such as were used for ASTRA) are useful for internal planning but will not convince Congress that risk has been eliminated. More useful is a clear delineation of high risks and what the NASA strategy is to alleviate them.
In response to questions, Obermann said that while members of Congress understand statistics they also know that statistics can be used in different ways and are skeptical of a number in isolation. They also tend to cite the statistics that buttress their arguments. Once burned by a statistic, they are very wary of similar statistics. Asked about the extent to which his comments pertained also to agencies like the Government Accounting Office, the Congressional Research Service, and the Congressional Budget Office, Obermann pointed out that support agencies respond to questions from members and the analysts from these agencies present the desired information paired with the assumptions they use in their analyses. He also noted that with the demise of the Congressional Office of Technology Assessment, there is no entity to provide real risk analysis for members of Congress.
In response to a comment about the small statistical sampling that often constrains the modeling of low-probability events, Obermann noted that now and then members are interested in risks from near-Earth asteroids, but it becomes hard to understand the meaning of such probabilities. If the risk models were better for such low-probability events, perhaps there would be more interest in the issue.