power trains for light-duty vehicles. The program has R&D activities in combustion, lightweight materials for vehicle components and structures, electrical and power electronic systems, emission control technologies, and fuels. The panel will be composed of about 8 members who will have experience with hybrid vehicle technologies, combustion engine technology, emission controls, the automotive market and sector, economics, and vehicle engineering and marketing. Panel members may have expertise in more than one area.

The panel will apply the committee’s methodology by first assessing the probability of success of meeting the DOE’s time frame and targets for technology development, consider alternative paths of development with and without DOE funding, review DOE estimates of the economic, environmental, and national security benefits of its program on light-duty vehicle technology, and estimate the benefits under the alternative future states of the world (scenarios) that have been specified by the committee, namely, a base case EIA scenario, a high oil/gas price scenario, and a carbon-constrained scenario. Consideration of technical and market risk will be a key element of this assessment.

It is expected that the panel will hold two 2-day meetings and will write a short 10-page report on its assessment of the benefits of this program and fill in the benefits matrix and explanation of it as outlined in the committee’s report, Prospective Evaluation of Applied Energy Research and Development at DOE (Phase One): A First Look Forward.


The benefits methodology presented in this report is intended to provide consistent information that is useful to decision makers who are considering what programs should be continued, expanded, scaled back, or eliminated. The committee’s charge did not extend to reaching conclusions or making recommendations about such decisions for the six programs that it reviewed for this Phase Two project. However, the committee has developed some insights that may assist decision makers with interpreting and applying the results of the analysis. These insights are recorded in the next section, “Commonly Encountered Issues.”

Although the committee has developed a process whereby all the panels evaluating DOE programs will use a common methodology and has suggested a procedure whereby all evaluations will be performed on a consistent basis, it is important that the decision makers understand the assumptions that the panelists had to use in assessing the programs. This understanding is critical to giving them the confidence to use the results as they make decisions about DOE programs and as they recommend funding.

Commonly Encountered Issues

The committee’s review of the six panel reports of Phase Two, along with the experience from the two earlier benefits evaluations, noted that several issues of interpretation came up frequently. Decision makers may thus encounter these same issues as they apply the results of the methodology presented here.

  1. A clear understanding of the reason for government-funded research is fundamental to the entire evaluation. For this reason, the program summaries produced in Phase Two require an explicit statement of the reason for government action. The government must demonstrate on the one hand the importance of a technology and the compelling need for it, and, on the other hand, that it will not be achieved in a timely fashion without the government’s participation. Readers are encouraged to consider this statement of justification carefully to verify that it accurately expresses the basic reason for the program.

  2. Several of the expert panels concluded that the probability of meeting the technical goals of a DOE program on the stated timetable was very small. While it is important for a decision maker to know this, it is also frequently true that a program can produce significant benefits at lower levels of technical achievement or over longer times. The committee recommends that users investigate these trade-offs carefully.

  3. The budgets of some programs may be inconsistent with the program goals. In some cases, this inconsistency is the reason that the program’s stated goals are not likely to be met, as noted above. But in other cases the issue is that budgets have been declining for several years; the combined heat and power (CHP) program and the ITP–Chemicals subprogram in Phase Two study exhibited this characteristic. In the latter case, the benefits of the subprogram may be over-ambitious because the analysis assumes that the goals will be accomplished when the reality is that declining budgets, relative to those planned, make that unlikely. In these cases, decision makers should recognize that realizing the estimated benefit may require increasing (or at least stabilizing) the budget. At some point, declining budgets probably reach a threshold below which the program is not viable.1

  4. The benefits of some programs are strongly influenced by nonresearch policy decisions. For example, the benefits of carbon capture and sequestration depend on the size and timing of the carbon tax (or equivalent policy intervention in the market). In this case, a delay in the imposition of the tax, by delaying an incentive to the private sector, would accelerate technology development attributable to government funding, although not on a one-for-one basis. Imposition of appliance efficiency standards might have a similar effect.


At some point, however, a small program to maintain a federal presence might be justifiable. For example, the Panel on DOE’s Distributed Energy Resources Program believes that the federal program is helpful in encouraging states to fund similar programs.

The National Academies of Sciences, Engineering, and Medicine
500 Fifth St. N.W. | Washington, D.C. 20001

Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement