and zero benefits for the Reference Case and the High Oil and Gas Prices scenario. Because the relative importance of the different scenarios is a consequence of policy judgments, the difference in benefits of a particular technology under different scenarios can be crucial information for decision makers.
Both the lighting and the sequestration panels observed an inconsistency between stated program goals, which were predicated on certain funding levels, and the actual funding, which was inadequate. In both cases, however, the potential benefits associated with full funding were large. In allocating resources, it is essential that decision makers know the benefits expected to be produced by a given funding pattern.
DOE often sets stretch goals for its programs. Some panels found that the probability of meeting those goals at current funding levels is low. Nevertheless, the estimated benefits of high-risk, high-payoff programs might exceed their projected cost by a significant amount. For example, the vehicle fuel cell panel believed there was little probability that DOE’s stretch goals would be achieved but concluded that the expected economic benefit would still be well in excess of program costs. This result is consistent with the rationale for public funding of high-risk, high-payoff research. In addition, the methodology helps to pinpoint the key risks to which benefits are sensitive, serving as a basis for monitoring the progress of such programs in a useful way.
Even when the stretch goals are unlikely to be met, DOE’s programs might achieve lower levels of performance at which benefits still exceed their costs. For example, the stationary fuel cell subpanel estimated that achieving a $600 per kilowatt cost goal (instead of $400 per kilowatt) would still have positive benefit. Similarly, the lighting panel estimated that the expected benefits of achieving a 100-lumens-per-watt (lpw) solid state device would be similar to those of achieving the stretch goal of 150 lpw and that the benefit of achieving either would substantially exceed the cost of a program aimed at the stretch goal.
In developing its recommended methodology and process, the committee has been sensitive to the demands it places on DOE’s resources. Although the committee believes that a reasonable balance between rigor and practicality has been struck, applying the methodology will require the use of scarce DOE financial, program management, and analytic resources.
Accordingly, the committee recommends that DOE explicitly recognize in its resource allocation processes the need for resources to support use of the methodology. The committee believes that such resources are not likely to be large in relation to the size of programs being evaluated and that the value of prospective benefits analysis justifies making them available.
The goal of Phase One of the prospective benefits project has been to develop a methodology that can be applied at the program level. The committee believes that the experience amassed during Phase One is sufficient to accomplish this goal. The committee expects that experience gained during Phase Two will make the methodology more robust. Further, in concentrating on developmental issues, a number of other important issues were intentionally deferred to Phase Two.
Now that the proposed methodology is in hand, the committee recommends that the following steps be taken in Phase Two to integrate benefits estimation into the program budget process and to maximize the value of such estimation to decision makers:
Review the proposed template for a two-page summary panel report with its intended users to ensure that their decision-making needs are met.
Prepare guidance for the program reviews to be conducted in Phase Two.
Make an initial analysis of the application of the benefits methodology to portfolio design.
Improve the process for estimating national security and environmental benefits.
Refine and amplify the quality control process.
Recommend how to integrate the benefits estimation process into the program-budget cycle.
Select anywhere from two to six programs to which the methodology will be applied in Phase Two.