grams are complemented by policy measures such as tax incentives to encourage early adoption of advanced technologies by consumers, efficiency standards for household appliances, and production tax credits for certain renewable energy sources (EIA AEO, 2006).

The programs evaluated by the NRC were limited to R&D within FE and to the portion of EERE’s R&D devoted to energy conservation.4 Research within FE has traditionally been divided between the Office of Coal and Power Systems (CPS) and the Office of Oil and Natural Gas. CPS administers a suite of clean coal R&D, which has the goal of ensuring the generation of clean, reliable, and affordable electricity from coal. The Office of Oil and Natural Gas supports research and policy options to ensure clean, reliable, and affordable supplies of oil and natural gas for American consumers.5 The energy conservation portion of EERE’s R&D is related to technologies for the efficient end-use of fuels and electricity in vehicles, in industrial processes, and within building envelopes.

Two general activities have been the focus of this Phase Two study: refining the methodology developed in Phase One and applying it to additional R&D projects. The committee improved the methodology for estimating environmental benefits (from, for example, reduced emissions) and estimating national security benefits (from, for example, reduced oil imports). In parallel, the committee selected six R&D activities to be the subject of case studies, which were carried out by separate expert panels appointed by the NRC. The activities selected for review included three within EERE—the Chemicals subprogram of the Industrial Technologies Program, the Distributed Energy Resources Program, and the activities related to light-duty hybrid electric vehicles within the Vehicles Technologies Program—and three within FE—the Integrated Gasification Combined Cycle subprogram, the Carbon Sequestration Program, and the Exploration and Production activities of the Natural Gas Technologies Program. This Phase Two study shows that the basic analytical structure, using decision trees, works well and can be implemented with the appropriate panels.



The primary effects of DOE’s programs are seen to be these: (1) they reduce technical risk, (2) they reduce market risk, and (3) they accelerate the introduction of the technology into the marketplace. The methodology developed by the Phase One committee used expert panels to review the DOE R&D program and estimate the expected economic, environmental, and energy security benefits of the program in three different global economic scenarios, with the results summarized in a matrix such as that shown in Figure S-1 (see Appendix F for generalized definitions of economic, environmental, and energy security benefits). The expert panel evaluation process is facilitated by a decision analysis consultant, and the panels construct simple decision trees to describe the main technical and market uncertainties associated with the program and the impact of DOE support on the probability of various technical and market outcomes. The decision trees used by all the panels assessed changes in technical and market risks. The acceleration effect was considered separately by each panel. In some cases, acceleration increases the likelihood that a project will attain the program goals of completion by a critical date, which is then accounted for in the assessment of technical risk. In other cases, the panels accounted for acceleration in their benefit calculations, which assume that if the technology is ultimately developed in the absence of the government program the net benefits accrue only for a limited time.

The calculations based on the decision trees allow the benefits of each R&D project to be estimated for combinations of outcomes—technical and market—in each of these global economic scenarios. These scenario- and outcome-specific results would typically be estimated using a simple spreadsheet model in conjunction with more sophisticated models such as the Energy Information Administration’s (EIA’s) National Energy Modeling System (NEMS).6 For example, NEMS might be used to estimate prices and demand for various energy sources in a particular global economic scenario. A spreadsheet model might be used to estimate the demand for the particular technology of interest given different levels of effectiveness and/or costs.

The overall benefit of the DOE R&D program is given as the difference between the expected benefits with DOE support and the expected benefits without DOE support, where the expected benefits are given as a probability-weighted average of the benefits in particular technical and market outcomes. To ensure consistency across the expert panels, the process calls for the use of common scenarios and assumptions across evaluations and an oversight committee that provides guidance to the panels. This kind of panel-based probabilistic assessment of R&D programs is common in many industries, in particular, the pharmaceutical industry (Sharpe and Keelin, 1998).


The remainder of EERE’s R&D was devoted to energy supply from renewable resources. Beginning with FY06, funding for all FE and EERE R&D programs was consolidated into one appropriations account subject to the jurisdiction of the House Appropriations Subcommittee on Energy and Water Development and Related Agencies (CRS, 2005).


Available at <>.


NEMS is a computer-based, energy-economy system for modeling U.S. energy markets that projects the production, imports, conversion, consumption, and prices of energy, subject to assumptions about macroeconomic and financial factors, world energy markets, resource availability and costs, behavioral and technological choice criteria, cost and performance characteristics of energy technologies, and demographics.

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement