Cover Image

Not for Sale



View/Hide Left Panel

THE NATIONAL ACADEMIES

Advisers to the Nation on Science, Engineering, and Medicine

Board on Energy and Environmental Systems

500 Fifth Street, NW
Washington, DC 20001
Phone: 202 334 3344
Fax: 202 334 2019

December 14, 2005

Mr. David Garman

Undersecretary for Energy, Science, and Environment

U.S. Department of Energy

1000 Independence Avenue, S.W.
Washington, DC 20585

Dear Mr. Garman:

The National Research Council (NRC) has established the Committee on Prospective Benefits of DOE’s Energy Efficiency and Fossil Energy R&D Programs, Phase Two, and the committee has begun work. The committee’s purpose is to continue to develop methodology for estimating the economic, environmental, and energy security benefits associated with DOE’s Energy Efficiency and Fossil Energy R&D Programs and to apply its proposed methodology to several DOE programs. The committee’s statement of task is provided in Attachment A and its members are listed in Attachment B.

To obtain feedback on its proposed methodology and its then-pending selection of DOE programs for further case study, the committee held a workshop on July 14, 2005, in Washington, D.C., attended by stakeholders. In this letter,1 the committee discusses the principal comments made during the workshop, the case studies it intends to perform in phase two, and the changes to the process and methodology that have occurred since phase one.

1  

This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the NRC’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this report: William Agnew, NAE, General Motors (retired); David Bodde, Clemson University; Charles Lave, University of California, Irvine; John J. Wise, NAE, Mobil Research and Development Corporation (retired); and James Wolf, independent consultant.

Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations, nor did they see the final draft of the report before its release. The review of this report was overseen by John Ahearne, NAE, Sigma Xi. Appointed by the NRC, he was responsible for making sure that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1
Letter Report for the Committee on Prospective Benefits of DOE’s Energy Efficiency and Fossil Energy R&D Programs THE NATIONAL ACADEMIES Advisers to the Nation on Science, Engineering, and Medicine Board on Energy and Environmental Systems 500 Fifth Street, NW Washington, DC 20001 Phone: 202 334 3344 Fax: 202 334 2019 December 14, 2005 Mr. David Garman Undersecretary for Energy, Science, and Environment U.S. Department of Energy 1000 Independence Avenue, S.W. Washington, DC 20585 Dear Mr. Garman: The National Research Council (NRC) has established the Committee on Prospective Benefits of DOE’s Energy Efficiency and Fossil Energy R&D Programs, Phase Two, and the committee has begun work. The committee’s purpose is to continue to develop methodology for estimating the economic, environmental, and energy security benefits associated with DOE’s Energy Efficiency and Fossil Energy R&D Programs and to apply its proposed methodology to several DOE programs. The committee’s statement of task is provided in Attachment A and its members are listed in Attachment B. To obtain feedback on its proposed methodology and its then-pending selection of DOE programs for further case study, the committee held a workshop on July 14, 2005, in Washington, D.C., attended by stakeholders. In this letter,1 the committee discusses the principal comments made during the workshop, the case studies it intends to perform in phase two, and the changes to the process and methodology that have occurred since phase one. 1   This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the NRC’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this report: William Agnew, NAE, General Motors (retired); David Bodde, Clemson University; Charles Lave, University of California, Irvine; John J. Wise, NAE, Mobil Research and Development Corporation (retired); and James Wolf, independent consultant. Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations, nor did they see the final draft of the report before its release. The review of this report was overseen by John Ahearne, NAE, Sigma Xi. Appointed by the NRC, he was responsible for making sure that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution.

OCR for page 1
Letter Report for the Committee on Prospective Benefits of DOE’s Energy Efficiency and Fossil Energy R&D Programs EVALUATING THE FEDERAL INVESTMENT IN APPLIED ENERGY R&D From the time the Department of Energy was formed in 1977, successive administrations in Washington, D.C., have looked to technological innovation as a critical tool for ensuring that the nation has a reliable supply of affordable, clean energy. Recognizing the importance of technological innovation, DOE, the Office of Management and Budget (OMB), and congressional committees have given increasing attention to understanding the effectiveness of federal funding for applied energy research and development (R&D).2 Evaluating government investment in applied energy R&D programs requires assessing their costs and benefits. Doing so is not a trivial matter. First, the analysis of costs and benefits must reflect the full range of public benefits—environmental and energy security impacts as well as economic effects. Second, the analysis must consider how likely the research is to succeed and how valuable the research will be if it is successful. Finally, the analysis must consider what might happen if the government did not support the project: Would some private entity undertake it or an equivalent activity that would produce some or all of the benefits of government involvement? Congress provided funds for “a continuing annual review by the [National] Academy [of Sciences] of programs … to measure the relative benefits expected to be achieved and to inform decision making on what programs should be continued, expanded, scaled-back, or eliminated.”3 The NRC has completed two studies to date. The first study committee, whose report was published in 2001,4 conducted a retrospective examination of the first 22 years of DOE-funded R&D on energy conservation.5 A second NRC committee adapted the methodology developed by its predecessor committee for use in prospectively assessing the benefits of the portfolio of ongoing R&D directed at energy conservation. Its report,6 published in April 2005, culminated phase one of the prospective study. The methodology suggested by the phase one committee uses expert panels to review the DOE R&D program and estimate the expected economic, environmental, and energy security benefits of the program in three different global economic scenarios, with the results summarized in the matrix shown in Attachment C. The expert panel evaluation process is facilitated by a decision analysis consultant, and the panels construct simple decision trees to describe the main technical and market uncertainties associated with the program and the impact of DOE support on the probability of various technical and market outcomes. The 2   An applied energy R&D program addresses a specific technology with defined performance and cost targets and milestones, whereas a research program has as its objective increased understanding and knowledge. 3   House Report 107-564, p. 125. July 11, 2002. U.S. Government Printing Office: Washington D.C. 4   National Research Council. 2001. Energy Research at DOE: Was It Worth It? Washington D.C.: National Academy Press. This report was requested by Congress in the conference report of the Consolidated Appropriations Act for fiscal year (FY) 2000 (House Report 106-479, p. 493. November 18, 1999. U.S. Government Printing Office: Washington, D.C.). 5   These programs include only those that were at the time under the jurisdiction of the U.S. House Appropriations Subcommittee on the Interior and Related Agencies. 6   National Research Council. 2005. Prospective Evaluation of Energy Research and Development at DOE (Phase One): A First Look Forward. Washington, D.C.: The National Academies Press. Interested readers are referred to this report for a complete description of the methodology for prospective evaluation of R&D benefits, subject to the modifications discussed herein.

OCR for page 1
Letter Report for the Committee on Prospective Benefits of DOE’s Energy Efficiency and Fossil Energy R&D Programs benefits of each R&D project are then estimated in each of these technical and market scenarios; the phase one report emphasized the potential need to use simple spreadsheet models in conjunction with more sophisticated models (such as the Energy Information Administration’s National Energy Modeling System;7 NEMS) to estimate these benefits. The overall benefit of the DOE R&D program is given as the difference between the expected benefits with DOE support and the expected benefits without DOE support. To ensure consistency across the panels, the process calls for the use of common scenarios and assumptions across evaluations and an oversight committee that provides guidance to the panels reviewing individual activities. Phase two of the NRC’s prospective study calls for testing, refining, and extending the proposed methodology. The committee intends to apply the phase one methodology for prospective evaluation of six applied energy R&D activities residing within DOE’s Office of Energy Efficiency and Renewable Energy (EE) and Office of Fossil Energy (FE). In addition, the committee will continue to revise the methodology as further experience with the panels warrants. The goal of this evaluation process is to enhance the value of DOE’s R&D programs by helping to establish a basis for increasing the funding of socially valuable programs and for transferring resources from programs that are less socially valuable, as well as justifying total funding. PRINCIPAL COMMENTS FROM THE WORKSHOP At the July 14, 2005, workshop, the committee heard presentations from representatives of OMB, Congress, the Office of Science and Technology Policy (OSTP), and DOE. It reviews each of their comments in turn. OMB. OMB representatives at the July 2005 workshop were quite supportive of the committee’s proposed approach, both the analytic methodology and the proposed process. Specifically, OMB supported the use of simple spreadsheet models in conjunction with NEMS, noting that the approach has the “potential to improve resolution, transparency, [and] ease of sensitivity analysis.”8 It also endorsed the committee’s decision tree framework as an “appropriate way to model risk for technical outcome, [and] market acceptance.” On the process side, OMB supported the use of balanced external review panels and stressed the importance of the oversight committee in ensuring the consistency of assumptions (about macroeconomic factors, next-best technologies, program funding, and so on) across panels. OMB’s representatives indicated that OMB was quite comfortable with the level of complexity of the proposed analysis and emphasized the need to summarize results in a single page for high-level policy analysis, as proposed in the committee’s matrix (see Attachment C). Although pointing out that the quantitative benefit estimates provided were 7   NEMS is a computer-based, energy-economy system for modeling U.S. energy markets that projects the production, imports, conversion, consumption, and prices of energy, subject to assumptions about macroeconomic and financial factors, world energy markets, resource availability and costs, behavioral and technological choice criteria, cost and performance characteristics of energy technologies, and demographics. 8   Leo Sommaripa, “Prospective Benefits Estimation for DOE’s Applied R&D—NRC Phase II; OMB Perspective and Interest,” presentation to the Committee on Prospective Benefits of DOE’s Energy Efficiency and Fossil Energy R&D Programs, Phase Two, July 14, 2005. OMB was also represented at the workshop by Rob Sandoli, program examiner, Energy Branch.

OCR for page 1
Letter Report for the Committee on Prospective Benefits of DOE’s Energy Efficiency and Fossil Energy R&D Programs certainly helpful, OMB staff noted that the qualitative issues identified by the review panels in their reports were also very helpful in OMB’s reviews of DOE programs. While the NRC’s current process for evaluating DOE R&D programs focuses on measuring net benefits for the U.S. economy, OMB indicated that it would be helpful if the NRC’s benefits evaluations also distinguished between producer and consumer surpluses so that beneficiaries of DOE’s R&D programs could be more readily identified. OMB’s representatives also made some suggestions regarding programs to review in phase two that are discussed below. Congress. A congressional view presented at the July 2005 workshop was also quite positive about the NRC’s proposed approach to prospective evaluation of DOE R&D programs.9 It echoed many of OMB’s comments, citing the benefits of the independent external reviews, the more transparent modeling, and the accessible short summaries. It noted that different users in Congress may have different preferences for the quantitative and/or qualitative information provided by panel reports and emphasized the need for both kinds of information. Also expressed was the desire that the panels’ analyses more explicitly identify likely beneficiaries of DOE applied energy R&D programs, as well as a reservation regarding whether this kind of analysis was appropriate for the NRC panels. OSTP. OSTP’s representative at the July 2005 workshop gave an overview of the OSTP mission and its role in setting energy R&D policy and talked about the value of a rigorous approach to estimating benefits, such as that proposed by the NRC phase one study, and the potential use of such an approach in portfolio allocation and program management. DOE. DOE was represented primarily by two staff members—one from the Office of Fossil Energy and one from the DOE Office of Energy Efficiency and Renewable Energy and the Office of Fossil Energy.10 A number of other DOE representatives and contractors attended the workshop and participated in the discussions throughout the day. DOE’s FE representative emphasized two issues. First, he considered the use of expert panels, questioning whether a single panel can effectively evaluate the broad range of technologies involved in a major system, such as a zero-emission coal plant or the hydrogen fuel program. The second issue was the use of NEMS. The phase one committee report criticized NEMS as being opaque and cumbersome to run and noted that DOE analyses frequently considered consumer savings while neglecting impacts on producers. As indicated above, the phase one committee report proposed the use of simple models in conjunction with NEMS to estimate net benefits in a given scenario. Expressing concerns that these simple models “may take too many shortcuts,” he invited Kevin Forbes of Catholic University to describe an approach for calculating net benefits using multiple NEMS runs. DOE’s FE representative concluded by calling for more interaction between the DOE and NRC panels during the evaluation process and for better documentation of the evaluation panels’ discussions and the logic underlying their risk assessments. 9   Kevin Carroll, House Committee on Science, Subcommittee on Energy, July 14, 2005. 10   Sam Baldwin, DOE Office of Energy Efficiency and Renewable Energy, and Jay Braitsch, DOE Office of Fossil Energy, “EERE-FE Observations on the NRC Report: Prospective Evaluation of Applied Energy Research and Development at DOE (Phase One): A First Look Forward,” presentation to the Committee on Prospective Benefits of DOE’s Energy Efficiency and Fossil Energy R&D Programs, Phase Two, July 14, 2005.

OCR for page 1
Letter Report for the Committee on Prospective Benefits of DOE’s Energy Efficiency and Fossil Energy R&D Programs DOE’s EE representative offered a number of observations on the phase one report that were further documented in an accompanying memorandum.11 He echoed his colleague’s concerns about the use of a single panel of experts for each program and about the use of simplified models in conjunction with NEMS; he also called for more interaction between DOE and the NRC panels during the evaluation process and better documentation. He expressed concerns about the consistency of the process, the lack of clear metrics, the use of single-point estimates, and decision trees not fully capturing the “flexibilities of actual management practices.”12 He went on to describe the activities of a risk team within DOE whose goal is to develop “scaleable risk analysis methods” that can be used by project/program managers, portfolio managers, and political leaders. He described a prototype Monte Carlo simulation-based tool for analyzing wind turbine systems but noted that “many challenges remain to develop/implement these tools.” The committee was pleased to hear about DOE’s efforts to improve its ability to calculate the benefits of R&D, both through properly calculating net benefits in NEMS and through developing sophisticated risk analysis models that can be used for program management and evaluation. Whenever available, the results of these analyses should inform panel evaluations of program benefits. The committee agrees that the individual program evaluations would benefit from improved interactions between DOE and the NRC panels and also agrees that DOE and DOE laboratories can contribute meaningfully to the ongoing development of the proposed methodology. Indeed, the modifications to the methodology (described in the final section of this letter) are focused primarily on improving these interactions. The committee also agrees that review panels should discuss the logic underlying their risk assessments. Although it is sympathetic to DOE’s concerns about the use of expert panels and overreliance on simple models, the committee remains optimistic that the proposed process can lead to evaluations that are useful to decision makers. The committee emphasizes that the proposed process is quite similar to processes used routinely to evaluate applied R&D projects in industry. For example, Sharpe and Keelin13 describe a process used for evaluating R&D projects at SmithKline Beecham that uses simple decision tree models for projects and uses independent review panels to review these assessments. Like applied energy R&D projects, modern pharmaceutical R&D projects are also quite complex and require consideration of both scientific and market issues. Sharpe and Keelin’s discussion of the SmithKline Beecham experience emphasizes how the independence and consistency of the evaluation process led to improved communication and credibility: “by tackling the soft issues—such as information quality and trust—SB improved its ability to address the hard 11   “EERE and FE Observations on the NRC Report: Prospective Evaluation of Applied Energy Research and Development at DOE (Phase One): A First Look Forward,” background paper delivered to the Committee on Prospective Benefits of DOE’s Energy Efficiency and Fossil Energy R&D Programs, Phase Two, July 14, 2005. 12   Sam Baldwin, DOE Office of Energy Efficiency and Renewable Energy, and Jay Braitsch, DOE Office of Fossil Energy, “EERE-FE Observations on the NRC Report: Prospective Evaluation of Applied Energy Research and Development at DOE (Phase One): A First Look Forward,” presentation to the Committee on Prospective Benefits of DOE’s Energy Efficiency and Fossil Energy R&D Programs, Phase Two, July 14, 2005. 13   P. Sharpe and T. Keelin. 1998. “How SmithKline Beecham Makes Better Resource-Allocation Decisions.” Harvard Business Review. Cambridge, Mass.: Harvard Business School Publishing. March-April.

OCR for page 1
Letter Report for the Committee on Prospective Benefits of DOE’s Energy Efficiency and Fossil Energy R&D Programs ones: how much and where to invest.”14 In that case, senior management ultimately concluded that increased R&D funding would be a worthwhile investment. The committee believes that with the cooperation and support of DOE, the proposed process developed in phase one can be similarly successful and can improve communication with stakeholder groups and the credibility of program evaluations. CASE STUDIES SELECTED FOR PHASE TWO For phase two, the committee has selected six DOE applied energy R&D activities to be the subject of a prospective assessment of benefits. The selected activities are from the FE and EE programs and are as follows (where applicable the specific subprogram that will be the focus of the assessment is noted in parentheses): FE Integrated gasification combined cycle, Sequestration, and Natural gas technologies (exploration and production). EE Distributed energy program (end-use system integration and interface), Vehicle technologies program (hybrid and electric propulsion, advanced combustion R&D, and materials technology—excluding projects related to heavy duty vehicles), and Industrial technologies program (chemicals). Prior to its selection of these six activities for case studies, the committee held discussions with major stakeholders, including congressional committee staff, DOE, and OMB. The discussions with congressional appropriations staff occurred before the July 14, 2005, workshop. At these meetings, congressional and agency staff recommended that since the funds for the prospective benefits studies (phases one, two, and three; phase three will apply the benefits methodology to a new set of case studies) had been appropriated by the Appropriations Subcommittee on the Interior and Related Agencies, the case studies in phase two should be drawn from the energy conservation programs within FE and EE, even though all funds for DOE now fall under the jurisdiction of the newly reorganized Appropriations Subcommittee on Energy and Water Development. At the July 14 workshop, OMB15 and DOE16 offered suggestions for the case studies. Letters (see Attachment D) were sent by the 14   P. Sharpe and T. Keelin. 1998. “How SmithKline Beecham Makes Better Resource-Allocation Decisions.” Harvard Business Review. Cambridge, Mass.: Harvard Business School Publishing. March-April. Page 45. 15   Leo Sommaripa, “Prospective Benefits Estimation for DOE’s Applied R&D—NRC Phase II; OMB Perspective and Interest,” presentation to the Committee on Prospective Benefits of DOE’s Energy Efficiency and Fossil Energy R&D Programs, Phase Two, July 14, 2005. 16   Sam Baldwin, DOE Office of Energy Efficiency and Renewable Energy, and Jay Braitsch, DOE Office of Fossil Energy, “EERE-FE Observations on the NRC Report: Prospective Evaluation of Applied Energy Research and Development at DOE (Phase One): A First Look Forward,” presentation to the Committee on Prospective Benefits of DOE’s Energy Efficiency and Fossil Energy R&D Programs, Phase Two, July 14, 2005.

OCR for page 1
Letter Report for the Committee on Prospective Benefits of DOE’s Energy Efficiency and Fossil Energy R&D Programs committee to DOE on July 22, 2005, indicating which activities had been selected for case studies and identifying information the review panels would need for their deliberations. There was final agreement on the case studies after an August 4, 2005, meeting with the undersecretary for energy, science, and environment and DOE staff. The sequestration R&D program was selected as a case study for phase one and again for phase two. Although the phase one study was not intended to produce accurate quantitative results, the committee thought it would be useful to test in phase two the currently proposed methodology and compare the results with those from phase one. Environmental and energy security benefits are being further defined during phase two. Evaluation of energy security benefits will be relevant for the chemicals subprogram of the industrial technologies program (ITP), distributed energy resources R&D, hybrid vehicle technology R&D, and natural gas technologies R&D. Evaluation of environmental benefits will be relevant for integrated gasification combined cycle R&D and sequestration R&D. The chemicals subprogram of ITP and the hybrid vehicle technologies program include many separate program elements and provide the opportunity to aggregate several activities. Thus the committee should have an opportunity to comment on how to aggregate programs, the usefulness of spreadsheet models, and methods to account for competing and complementary benefits. MODIFICATIONS TO METHODOLOGY The methodology proposed by the phase one committee was developed after its review of the results of three pilot case studies. In phase two, the committee will test the methodology in the new case studies and work to make it extensible to consideration of environmental and energy security benefits. The feedback on methodology received in the July 2005 workshop and other venues has raised many issues that the committee and the review panels will have to bear in mind in conducting the case studies and in further developing the methodology. However, this feedback has not led the committee to propose fundamental changes to the methodology before applying it to these new case studies. The primary change to the proposed process and plan is the decision to work to improve communications and interactions with DOE, to the extent permissible under the Federal Advisory Committee Act, Section 15, and the rules of the National Research Council. Some examples of opportunities for improved communication and interaction are as follows: Selection of case studies. Prior to selecting the case studies, the committee obtained input from DOE, which was also asked to suggest experts for the various panel chairs. Per the recommendations in the phase one report (pp. 33-34),17 the panel chairs are meeting in person or via telephone with DOE program managers to discuss the methodology and the information being requested. DOE was asked to make presentations at the first panel meeting following the template18 that was developed by the NRC committee in order to ensure that 17   National Research Council. 2005. Prospective Evaluation of Energy Research and Development at DOE (Phase One): A First Look Forward. Washington, D.C.: The National Academies Press. 18   The template is given in Appendix K of National Research Council. 2005. Prospective Evaluation of Energy Research and Development at DOE (Phase One): A First Look Forward. Washington, D.C.: The National Academies Press.

OCR for page 1
Letter Report for the Committee on Prospective Benefits of DOE’s Energy Efficiency and Fossil Energy R&D Programs the information is provided to all panels in a consistent and complete manner. At the end of the first panel meeting, a decision tree will be constructed that will be sent to DOE to obtain its probability estimates and any suggested modifications of the decision tree. The second panel meeting will include open sessions with DOE participants at which DOE’s probability estimates and any suggested modifications will be discussed. The primary change in the process is that DOE has the opportunity to comment on and share its views on each panel’s proposed process, before the end of the panel study. Development and refinement of methodology. The further development of the evaluation methodology will also involve DOE as well as experts within DOE’s laboratories. The July 14, 2005, workshop provided a forum for offering feedback on the phase one methodology and for supplying initial input regarding energy security and environmental benefits. At the September 13, 2005, meeting, FE and EE described their activities related to estimating environmental and energy security benefits. As refinement of the methodology continues, there will be informal conversations with the committee chair, some committee members, and DOE and other stakeholders about the proposed process and enhancements or modifications to it. It is in the interest of the NRC, DOE, and other participants, sponsors, and stakeholders to develop a methodology for evaluating prospective benefits of DOE R&D that is both rigorous and transparent. The committee looks forward to its work with DOE in the months ahead and welcomes your feedback on its proposed processes. Sincerely, Maxine Savitz, Chair Committee on Prospective Benefits of DOE’s Energy Efficiency and Fossil Energy R&D Programs, Phase Two

OCR for page 1
Letter Report for the Committee on Prospective Benefits of DOE’s Energy Efficiency and Fossil Energy R&D Programs Attachment A Statement of Task PROSPECTIVE BENEFITS OF DOE’S ENERGY EFFICIENCY AND FOSSIL ENERGY R&D PROGRAMS—PHASE 2 Project Scope: The Phase 2 activity follows the completion of Phase 1, which resulted in the issuance of two reports on methodology for estimating prospective benefits and evaluating energy R&D programs at DOE. These reports [Energy Research at DOE: Was It Worth It?, and Prospective Evaluation of Applied Energy Research and Development at DOE: A First Look Forward] are posted in the project record with project identification number BEES-J-03-01-A in the Current Projects System. At least three issues will require attention as part of the Phase 2 Task. These issues include: (a) further improving the estimation of the value of environmental benefits (e.g., reduced emissions), (b) further improving the estimation of the value of security benefits (e.g., reducing oil imports or ensuring more reliable electricity supplies), and (c) determining how to estimate the overall benefits of the options under a variety of scenarios. The first two issues involve the public good rather than direct economic benefits. The committee will build on the foundation of work from Phase 1 and the body of literature that exists to determine appropriate values for these factors. The committee might commission white papers defining the state of knowledge and suggesting how the methodology could incorporate these estimates. For (c), options evaluation, the committee will consider the extent to which an analytical foundation is appropriate, building on the Phase 1 work and incorporating the full range of benefits for representative scenarios. In addition, the committee will consider mechanisms for quantifying knowledge benefits and include them as appropriate in the overall evaluation. The committee will also provide a peer review of how DOE is evaluating prospective benefits of various Energy Efficiency (EE) and Fossil Energy (FE) programs/projects. As in Phase 1, several panels will be separately appointed to assist the committee in Phase 2. A workshop will be held early in Phase 2 to discuss the Phase 1 reports and methodology, following which the committee will write a letter report that will set the stage for the work to be accomplished in Phase 2. A final report will be issued at the conclusion of Phase 2, about the end of April 2006. The panels will write panel reports documenting the results of the analyses of the prospective benefits of the various programs/projects in EE and FE chosen by the committee to evaluate. These panel reports may be issued separately or incorporated into the Phase 2 final report. The project is sponsored by the U.S. Department of Energy. The approximate starting date for this project is March 15, 2005. Project Duration: 14 months

OCR for page 1
Letter Report for the Committee on Prospective Benefits of DOE’s Energy Efficiency and Fossil Energy R&D Programs Attachment B Committee Roster COMMITTEE ON PROSPECTIVE BENEFITS OF DOE’S ENERGY EFFICIENCY AND FOSSIL ENERGY R&D PROGRAMS, PHASE TWO NAE Maxine L. Savitz (Chair) General Manager, Technology Partnerships Honeywell, Inc. (retired) Linda Cohen Professor, Department of Economics University of California, Irvine James Corman President, Energy Alternatives Studies, Inc. Paul DeCotis Director, Energy Analysis New York State Energy Research and Development Authority (NYSERDA) Ramon Espino Professor, Department of Chemical Engineering University of Virginia Robert W. Fri Visiting Scholar Resources for the Future W. Michael Hanemann Professor, Department of Agricultural and Resource Economics University of California, Berkeley NAE Wesley Harris Head, Department of Aeronautics and Astronautics Massachusetts Institute of Technology Martha A. Krebs Director, Energy R&D Division California Energy Commission IOM Lester B. Lave Professor, Tepper School of Business Carnegie Mellon University Richard G. Newell Council of Economic Advisors Jack Siegel President, Technology & Markets Group Energy Resources International, Inc. James E. Smith Professor, Fuqua School of Business Duke University Terry Surles Director Pacific International Center for High Technology Research James L. Sweeney Professor, Management Science and Engineering Stanford University Michael Telson Director of National Laboratory Affairs University of California

OCR for page 1
Letter Report for the Committee on Prospective Benefits of DOE’s Energy Efficiency and Fossil Energy R&D Programs Attachment C Committee’s Template for Presenting Panel Results PANEL NAME: Program Name: Program Goals: Year Goals Expected to be Achieved: Program Costs: Funding to Date: $ Current Funding: $ Proposed Year Funding: $ Expected Cost to Completion: $ Industry and Foreign Government Funding: $ Key Complementary/Interdependent DOE Programs: All benefits are cumulative through 2050 and are reported in 20XX year dollars.

OCR for page 1
Letter Report for the Committee on Prospective Benefits of DOE’s Energy Efficiency and Fossil Energy R&D Programs Comments and Observations: One to two paragraphs e.g., provide a summary of the panel’s completed assessment and estimate of expected benefits of the DOE program. Technical Risks: 5 to10 lines e.g., describe the program’s risks in sufficient detail and clarity, noting program interdependencies, technical and infrastructure innovations and breakthroughs needed, and competitive alternatives, and so on. Market Risk: 5 to 10 lines e.g., note factors that might affect market acceptance, including customer preferences, pricing, competitive domestic and foreign activities; next-best technologies issues, regulatory concerns, and so on. Benefits: 5 to 10 lines e.g., discuss specifically the estimation of benefits, uses and interpretations, caveats, outstanding issues, and so on. Program Observations: 5 to 10 lines e.g., notable accomplishments/gaps, opportunities, spin-offs, and so on.

OCR for page 1
Letter Report for the Committee on Prospective Benefits of DOE’s Energy Efficiency and Fossil Energy R&D Programs Attachment D Letters Sent to DOE by the Committee Board on Energy and Environmental Systems 500 Fifth Street, NW Washington, DC 20001 Phone: 202 334 3344 Fax: 202 334 2019 July 22, 2005 Allan Hoffman EE-3B Forrestal Building U.S. Department of Energy 1000 Independence Avenue Washington D.C. 20585 Dear Dr. Hoffman: At last week’s meeting, there was discussion as to which DOE activities would be the subject of benefits assessments during phase two. Accordingly, the committee has selected three EE activities for benefits assessment. The selected activities are included in the following list, with the area that will be the focus of the assessment noted in parentheses: Distributed energy program (end-use system integration and interface); Vehicle technologies program (hybrid and electric propulsion—excluding projects related to heavy vehicles; advanced combustion R&D—limited to the combustion and emission control R&D activity, only; and materials technology—excluding projects related to heavy vehicles and excluding the high temperature materials laboratory activity); and Industrial technologies program (chemicals). The committee requests that, for the above activities, DOE provide the necessary program description and model runs (using NEMS, for example) as set forth in Figure K-1 of Prospective Evaluation of Applied Energy Research and Development at DOE (Phase One): A First Look Forward. In addition, a brief history of the activity is requested. It is suggested that EE designate a point of contact for each activity listed above to facilitate requests for information. Please contact Martin Offutt of the NRC at 202-334-2904 or moffutt@nas.edu with the names of these contacts. In addition, it is requested that a meeting take place in the near future between the program managers from DOE and the NRC panel chairs to discuss the information request. Thank you for your assistance with this request. Sincerely, Maxine Savitz Chair, Committee on Prospective Benefits of DOE’s Energy Efficiency and Fossil Energy R&D Programs (Phase Two) Enclosure: Appendix K, Prospective Evaluation of Applied Energy Research and Development at DOE (Phase One): A First Look Forward cc: David Garman, Undersecretary for Energy, Science, and Environment Rob Sandoli, Office of Management and Budget Terry Tyborowski, Committee on Appropriations, U.S. House of Representatives

OCR for page 1
Letter Report for the Committee on Prospective Benefits of DOE’s Energy Efficiency and Fossil Energy R&D Programs Board on Energy and Environmental Systems 500 Fifth Street, NW Washington, DC 20001 Phone: 202 334 3344 Fax: 202 334 2019 July 22, 2005 Jay Braitsch FE-24 Forrestal Building U.S. Department of Energy 1000 Independence Avenue Washington D.C. 20585 Dear Mr. Braitsch: At last week’s meeting, there was discussion as to which DOE activities would be the subject of benefits assessments during phase two. Accordingly, the committee has selected three FE activities for benefits assessment. The selected activities are as follows, and where applicable the specific subprogram that will be the focus of the assessment has been noted in parentheses: Integrated Gasification Combined Cycle; Sequestration; and Natural gas technologies (exploration and production). The committee requests that, for the above activities, DOE provide the necessary program description and model runs (using NEMS, for example) as set forth in Figure K-1 of Prospective Evaluation of Applied Energy Research and Development at DOE (Phase One): A First Look Forward. In addition, a brief history of the activity is requested. It is suggested that FE designate a point of contact for each activity listed above to facilitate requests for information. Please contact Martin Offutt of the NRC at 202-334-2904 or moffutt@nas.edu with the names of these contacts. In addition, it is requested that a meeting take place in the near future between the program managers from DOE and the NRC panel chairs to discuss the information request. Thank you for your assistance with this request. Sincerely, Maxine Savitz Chair, Committee on Prospective Benefits of DOE’s Energy Efficiency and Fossil Energy R&D Programs (Phase Two) Enclosure: Appendix K, Prospective Evaluation of Applied Energy Research and Development at DOE (Phase One): A First Look Forward cc: David Garman, Undersecretary for Energy, Science, and Environment Leo Sommaripa, Office of Management and Budget Terry Tyborowski, Committee on Appropriations, U.S. House of Representatives