National Academies Press: OpenBook

Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two) (2007)

Chapter: 2 Results of Applying the Methodology

« Previous: 1 Introduction
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×

2
Results of Applying the Methodology

CASE STUDY SELECTION AND EXECUTION

The statement of task for the study called for case studies of the prospective benefits of approximately six DOE programs on energy efficiency and fossil energy R&D. Early in the study, during the open session of its July 14, 2005, meeting, the committee on prospective benefits held discussions with federal stakeholders on its pending selection of case studies. Based on that input, the committee finalized its choices and on July 22, 2005, sent letters to DOE—one each to the Office of Fossil Energy (FE) and the Office of Energy Efficiency and Renewable Energy (EERE)—informing them of the six selected programs (see Appendix D), which were as follows:

  • FE programs and activities:

    • Integrated Gasification Combined Cycle (IGCC);

    • Carbon Sequestration; and

    • Natural gas technologies (exploration and production).

  • EERE programs and activities:

    • Distributed Energy Resources (DER) program (enduse system integration and interface);

    • Vehicle technologies program (hybrid and electric propulsion, advanced combustion R&D and materials technology—excluding projects related to heavy-duty vehicles); and

    • Industrial Technologies Program–Chemicals.

In a few cases, the scope of the evaluation was limited to a subset of activities within a program. This is indicated in parentheses, where applicable.

The NRC formed six panels to conduct the case studies. The chairs of five of the six panels were members of the committee on prospective benefits. (A list of panel activities is included as Appendix E.) Each panel held two meetings between September and December 2005. A description of the evaluation process and methodology (see Appendix F) was distributed to panelists before the first panel meetings. After discussing this methodology early in the first meeting, the panel heard presentations delivered by DOE staff on their programs’ FY05 budgets, objectives, R&D portfolios, and estimated future benefits; the latter had been calculated by DOE assuming program technical goals would be met. The panels used this and other information available to them, along with their own knowledge of the subjects, to construct a decision tree showing the timing and likelihood of the technical success and market acceptance of the technology or technologies supported by the program. The panels next calculated the benefits—economic, environmental, and security—associated with each level of technical and market success in three global economic scenarios. The overall benefit of the DOE R&D program is given as the difference between the expected benefits with DOE support and the expected benefits without DOE support. To ensure consistency, the panels employed the process recommending common scenarios and assumptions across evaluations, and they were aided by the committee, which served in an oversight capacity. A decision analysis consultant assisted with the preparation of decision trees and benefits calculations. The panels wrote short reports summarizing their findings and calculations and submitted them to the committee.

STATEMENT OF TASK

The statements of task for the six panels were nearly identical. Here is the statement of task for the Panel on Light-Duty Vehicle Hybrid Technology:


The Panel on Light-Duty Vehicle Hybrid Technology will apply the methodology developed by the Committee on Prospective Benefits of Energy Efficiency and Fossil Energy R&D Programs to assess the potential benefits of DOE’s R&D activities that are focused on hybrid electric vehicle technologies using more efficient internal combustion engine

Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×

power trains for light-duty vehicles. The program has R&D activities in combustion, lightweight materials for vehicle components and structures, electrical and power electronic systems, emission control technologies, and fuels. The panel will be composed of about 8 members who will have experience with hybrid vehicle technologies, combustion engine technology, emission controls, the automotive market and sector, economics, and vehicle engineering and marketing. Panel members may have expertise in more than one area.


The panel will apply the committee’s methodology by first assessing the probability of success of meeting the DOE’s time frame and targets for technology development, consider alternative paths of development with and without DOE funding, review DOE estimates of the economic, environmental, and national security benefits of its program on light-duty vehicle technology, and estimate the benefits under the alternative future states of the world (scenarios) that have been specified by the committee, namely, a base case EIA scenario, a high oil/gas price scenario, and a carbon-constrained scenario. Consideration of technical and market risk will be a key element of this assessment.


It is expected that the panel will hold two 2-day meetings and will write a short 10-page report on its assessment of the benefits of this program and fill in the benefits matrix and explanation of it as outlined in the committee’s report, Prospective Evaluation of Applied Energy Research and Development at DOE (Phase One): A First Look Forward.

ADVICE TO USERS

The benefits methodology presented in this report is intended to provide consistent information that is useful to decision makers who are considering what programs should be continued, expanded, scaled back, or eliminated. The committee’s charge did not extend to reaching conclusions or making recommendations about such decisions for the six programs that it reviewed for this Phase Two project. However, the committee has developed some insights that may assist decision makers with interpreting and applying the results of the analysis. These insights are recorded in the next section, “Commonly Encountered Issues.”

Although the committee has developed a process whereby all the panels evaluating DOE programs will use a common methodology and has suggested a procedure whereby all evaluations will be performed on a consistent basis, it is important that the decision makers understand the assumptions that the panelists had to use in assessing the programs. This understanding is critical to giving them the confidence to use the results as they make decisions about DOE programs and as they recommend funding.

Commonly Encountered Issues

The committee’s review of the six panel reports of Phase Two, along with the experience from the two earlier benefits evaluations, noted that several issues of interpretation came up frequently. Decision makers may thus encounter these same issues as they apply the results of the methodology presented here.

  1. A clear understanding of the reason for government-funded research is fundamental to the entire evaluation. For this reason, the program summaries produced in Phase Two require an explicit statement of the reason for government action. The government must demonstrate on the one hand the importance of a technology and the compelling need for it, and, on the other hand, that it will not be achieved in a timely fashion without the government’s participation. Readers are encouraged to consider this statement of justification carefully to verify that it accurately expresses the basic reason for the program.

  2. Several of the expert panels concluded that the probability of meeting the technical goals of a DOE program on the stated timetable was very small. While it is important for a decision maker to know this, it is also frequently true that a program can produce significant benefits at lower levels of technical achievement or over longer times. The committee recommends that users investigate these trade-offs carefully.

  3. The budgets of some programs may be inconsistent with the program goals. In some cases, this inconsistency is the reason that the program’s stated goals are not likely to be met, as noted above. But in other cases the issue is that budgets have been declining for several years; the combined heat and power (CHP) program and the ITP–Chemicals subprogram in Phase Two study exhibited this characteristic. In the latter case, the benefits of the subprogram may be over-ambitious because the analysis assumes that the goals will be accomplished when the reality is that declining budgets, relative to those planned, make that unlikely. In these cases, decision makers should recognize that realizing the estimated benefit may require increasing (or at least stabilizing) the budget. At some point, declining budgets probably reach a threshold below which the program is not viable.1

  4. The benefits of some programs are strongly influenced by nonresearch policy decisions. For example, the benefits of carbon capture and sequestration depend on the size and timing of the carbon tax (or equivalent policy intervention in the market). In this case, a delay in the imposition of the tax, by delaying an incentive to the private sector, would accelerate technology development attributable to government funding, although not on a one-for-one basis. Imposition of appliance efficiency standards might have a similar effect.

1

At some point, however, a small program to maintain a federal presence might be justifiable. For example, the Panel on DOE’s Distributed Energy Resources Program believes that the federal program is helpful in encouraging states to fund similar programs.

Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×

While not directly evaluated in this study, the social welfare impact of such policies could be considerable.

  1. Programs having very large benefits might alter the supply and demand characteristics of the entire energy system. For example, a successful deployment of IGCC technology appears to be accompanied by a large role for DER in order to meet peak electricity demand. This, of course, would affect the benefits calculation for DER. NEMS is particularly useful in understanding these changes in the overall energy system.2

Using the Benefits Analysis for Decision Making

The expected net benefits calculated according to the methodology provide a useful basis for comparing programs. However, other elements of the analysis will contribute to the decision-making process. During its discussion of the results of the Phase Two work, the committee identified three such considerations that it believes deserve comment:

  • The decision maker will often want to know how best to spend incremental resources (or how to minimize the impact on benefits of a reduction in resources). It is not the case, however, that incremental resources should be spent on programs with the highest benefit-cost ratio. What counts most in making this decision is finding the greatest incremental benefit produced by the additional resources. The methodology does not calculate marginal benefits and so is not directly useful in making these incremental decisions. However, the committee suggests that the decision tree analysis would be of considerable use in identifying actions most likely to increase the expected benefit. A recommended method for calculating marginal benefits appears in Chapter 5.

  • The methodology presents benefits for each of three scenarios that describe future states of the world, but it does not attempt to combine the three sets of benefits into a single set. As a result, the decision maker must weigh the alternative scenarios in arriving at judgments about the benefits of the overall research portfolio. Ideally, a well-designed portfolio would contain a balance of projects that would produce acceptable results across the range of outcomes for different scenarios.

  • The methodology estimates public benefits in three areas—economic, environmental, and energy security. While these three types of benefits reflect DOE’s strategic goals (DOE, 2005a), the committee recognizes that other kinds of benefits may be important in evaluating some projects. For example, markets will demand that automobile manufacturers produce cars that not only meet the fuel economy of importance to DOE but have several additional attributes as well. Another example might be employment impacts. Because including such benefits could affect the comparability among programs, the committee recommends that they be considered apart from the results matrix developed by the methodology or that, if included, they should be part of the text of a panel report. The panel should also discuss noteworthy assumptions it has made in calculating benefits, particularly if the results are sensitive to that assumption or the assumptions itself is novel or interesting.

RESULTS OF CASE STUDIES

The six expert panels that evaluated DOE applied energy programs estimated prospective benefits—economic, environmental, and security—using quantitative outcome indicators. In some cases, security and environmental benefits had to be characterized qualitatively. These estimates were facilitated by an assessment of the key technical and market risk faced by the technology being developed. Each set of results is reported in a two-page format accompanied by a decision tree. The latter illustrates how the expert panel assessed the possible future outcomes of the R&D program in terms of the timing, probability, and levels of technical performance and market acceptance of the technology. The goals and history of the program are summarized, and the expert panels have offered further observations about the program. The expert panels have contemplated outcomes that might occur with and without federal funding and taken the difference between the outcomes as the net expected value of the program.

The findings of the six panels are presented in summary fashion in Figures 2-1 through 2-6:

  • Figure 2-1, integrated gasification combined cycle technology, pages 17-19,

  • Figure 2-2, carbon sequestration, pages 20-22,

  • Figure 2-3, natural gas exploration and production, pages 23-28,

  • Figure 2-4, distributed energy resources, pages 29-31,

  • Figure 2-5, light-duty hybrid vehicles, pages 32-36, and

  • Figure 2-6, Industrial Technologies Program––Chemicals, pages 37-39.

The complete panel reports from which this information was derived are found in Appendixes H through M.

2

NEMS determines capacity additions in its electricity market module as follows: “Capacity expansion is determined by the least cost mix of all costs, including capital, O&M, and fuel.” And further: “Fossil-fired steam and nuclear plant retirements are calculated endogenously within the model. Plants are retired if the market price of electricity is not sufficient to support continued operation.” (EIA, 2003, pp.43-46)

Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×

FIGURE 2-1 Findings for DOE’s gasification technologies R&D.

Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×

FIGURE 2-2 Findings for DOE’s carbon sequestration program.

Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×

FIGURE 2-3 Findings for DOE ’s natural gas exploration and production program.

Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×

FIGURE 2-4 Findings for DOE’s distributed energy resources program.

Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×

FIGURE 2-5 Findings for DOE’s light-duty vehicle hybrid technology R&D.

Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×

FIGURE 2-6 Findings for DOE’s chemical industrial technologies program.

Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 14
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 15
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 16
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 17
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 18
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 19
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 20
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 21
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 22
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 23
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 24
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 25
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 26
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 27
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 28
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 29
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 30
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 31
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 32
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 33
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 34
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 35
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 36
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 37
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 38
Suggested Citation:"2 Results of Applying the Methodology." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 39
Next: 3 Methodology for Prospective Evaluation of Deparment of Energy Programs »
Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two) Get This Book
×
Buy Paperback | $69.00 Buy Ebook | $54.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Since its inception in 1977 from an amalgam of federal authorities, the U.S. Department of Energy (DOE) has administered numerous programs aimed at developing applied energy technologies. In recent years, federal oversight of public expenditures has emphasized the integration of performance and budgeting. Notably, the Government Performance and Results Act (GPRA) was passed in 1993 in response to questions about the value and effectiveness of federal programs. GPRA and other mandates have led agencies to develop indicators of program performance and program outcomes. The development of indicators has been watched with keen interest by Congress, which has requested of the National Research Council (NRC) a series of reports using quantitative indicators to evaluate the effectiveness of applied energy research and development (R&D).

The first such report took a retrospective view of the first 3 years of DOE R&D programs on fossil energy and energy efficiency. The report found that DOE-sponsored research had netted large commercial successes, such as advanced refrigerator compressors, electronic lighting ballasts, and emission control technology for flue gas desulfurization. However, some programs were judged to be costly failures in which large R&D expenditures did not result in a commercial energy technology. A follow-up NRC committee was assigned the task of adapting the methodology to the assessment of the future payoff of continuing programs.

Evaluating the outcome of R&D expenditures requires an analysis of program costs and benefits. Doing so is not a trivial matter. First, the analysis of costs and benefits must reflect the full range of public benefits that are envisioned, accounting for environmental and energy security impacts as well as economic effects. Second, the analysis must consider how likely the research is to succeed and how valuable the research will be if successful. Finally, the analysis must consider what might happen if the government did not support the project: Would some non-DOE entity undertake it or an equivalent activity that would produce some or all of the benefits of government involvement?

This second report continues to investigate the development and use of R&D outcome indicators and applies the benefits evaluation methodology to six DOE R&D activities. It provides further definition for the development of indicators for environmental and security benefits and refines the evaluation process based on its experience with the six DOE R&D case studies.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!