definition of various kinds of benefits would ensure the comparability of benefits estimates across programs.

DOE calculates benefits assuming that the performance and cost goals of a technology are fully attained. The panels generally found that DOE had set stretch goals that by their nature would be hard to meet. Knowing the benefits of achieving these stretch goals is important, of course. However, since their achievement is uncertain and, in most cases, not very likely, the expected value of the benefit is a more suitable parameter for comparing program benefits. The difference can be substantial: The sequestration panel, for example, calculated an expected benefit of $35 billion, while it estimated that full attainment of DOE’s stated goals, on schedule, would produce a $120 billion benefit.

Comparability of benefits across programs requires using comparable and reasonable assumptions. The vehicle fuel cell subpanel found significant incompatibilities in the assumptions DOE used to calculate benefits. For example, DOE assumed that fuel cell vehicles would, by 2040, have 3.4 times the fuel economy of conventional gasoline vehicles of the same vintage, while gasoline hybrid vehicles would have only 2.0 times the fuel economy of conventional vehicles by 2040. In the view of the fuel cell panel, the assumed gasoline hybrid fuel economy was too conservative and resulted in DOE overestimating the benefits of the vehicle fuel cell program.3

Insights Useful for Decision Making

Benefits can vary significantly from one future scenario to the next. For example, the sequestration panel estimated large benefits for the Carbon Constrained scenario and zero benefits for the Reference Case and the High Oil and Gas Prices scenario. Because the relative importance of the different scenarios is a consequence of policy judgment, the differences in benefits of a particular technology under different scenarios can be crucial information for decision makers.

Both the lighting and the sequestration panels observed an inconsistency between stated program goals, which were predicated on certain funding levels, and the actual funding, which was inadequate. In both cases, however, the potential benefits associated with full funding were large. In allocating resources, it is essential that decision makers know the benefits expected to be produced by a given funding pattern.

As noted above, DOE often sets stretch goals for its programs. Some of the panels/subpanels found that the probability of meeting those goals at current funding levels is low. Nevertheless, the estimated benefits of high-risk, high-payoff programs might exceed their projected cost by a significant amount. For example, the vehicle fuel cell subpanel assigned a very low probability for the achievement of DOE’s stretch goals but concluded that the expected economic benefit was still well in excess of program costs. A conclusion such as this is consistent with the rationale for public funding of high-risk, high-payoff research. In addition, the methodology for prospective evaluation helps to pinpoint the key risks to which benefits are sensitive, serving as a basis for monitoring the progress of such programs in a useful way.

Even when the stretch goals are unlikely to be met, DOE’s programs might achieve lower levels of performance at which benefits would still exceed their costs. For example, the stationary fuel cell subpanel estimated that achieving a $600/kW cost goal (instead of $400/kW) would still have a positive benefit. Similarly, the lighting panel estimated that the expected benefits of achieving a 100 lpw solid state device would be similar to those of achieving the stretch goal of 150 lpw and that the benefits of achieving either would substantially exceed the cost of a program aimed at the stretch goal.

The committee stresses that the above examples are only that—illustrations of the kinds of insight that careful analysis of program benefits can provide.

BALANCE BETWEEN ANALYTIC RIGOR AND PRACTICAL UTILITY

The methodology and process for prospective benefits calculations recommended by the committee are intended to balance the need for analytic rigor and the need for practical utility. Rigor is required to assure that the benefits information offered to decision makers is credible and consistent across DOE’s programs. At the same time, the committee is aware that benefits estimation can become excessively complex, bringing resistance to the application of the methodology.

To characterize how the committee has balanced these conflicting demands, it could be useful to examine how the six main issues raised by the panels have been resolved. (These issues were presented in Chapter 2, “From Retrospective to Prospective Evaluation.”)

  • Calculating benefits. The recommended methodology calls for a simplified spreadsheet model for calculating benefits instead of relying on NEMS. This approach allows a panel to explore alternative outcomes quickly and efficiently, conforms to the committee’s definition of benefits, and helps make the analytic assumptions more transparent. An initial NEMS run calibrates the spreadsheet model to ensure consistency with the key energy prices and quantities used in benefits calculation. Preplanning of the panel’s activities is suggested to establish agreement on, and the transparency of, the assumptions that go into the NEMS runs.

3  

It is crucial to note, however, that the panel calculated large potential benefits of hydrogen vehicle fuel cells even adjusting for these assumptions.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement