• Transparency. Although the projects being evaluated are complex, in order for the results to be understood and used, decision makers must be able to understand the critical assumptions underlying the analysis, including the assumptions about the likelihood of various levels of success and the benefits in each scenario.

• Consistency. Although each project has its own technical goals and potentially different markets, the committee requires that the same summarizing matrix be used for each project. The expected economic, environmental, and security benefits in the matrix must be defined in the same way, and each program must be considered under the same three scenarios. Using a consistent set of definitions and assumptions makes it easier to study a portfolio of programs,¹⁷ as the investment in learning the terminology associated with the evaluation of one program pays dividends when studying subsequent programs. It also facilitates comparisons between and aggregation across programs, as the expected benefits of different programs can be compared and totaled.

The committee expects to release its Phase 1 final report on methodology for estimating R&D benefits in February 2005. (In Phase 2, the committee will apply the methodology developed to a larger set of DOE programs and resolve any remaining methodological issues.) The Phase 1 final report will contain the following:

• A description of the prospective benefits matrix, which the committee expects will be similar to the outline provided in this letter report;

• Detailed guidance on the methodology for completing the matrix;

• An outline of the process for applying the methodology, including the use of expert panels;

• A summary of the reports prepared by the expert panels for the three test programs—advanced solid state lighting, carbon sequestration, and fuel cells—that were selected for evaluation; and

• Recommendations for key issues to be resolved in the Phase 2 study, as well as suggestions to DOE on ways to improve its prospective benefits evaluations.

In its work to date, the committee has benefited greatly from the input and support of the DOE staff. It looks forward to their continued assistance in its future endeavors.

Robert W. Fri (Chair)

Senior Fellow Emeritus

Resources for the Future

Linda Cohen

Professor, Department of Economics

University of California, Irvine

James Corman

President

Energy Alternatives Studies, Inc.

Paul A. DeCotis

Director of Energy Analysis

New York State Energy Research and Development Authority (NYSERDA)

Wesley Harris, NAE¹⁸

Head, Department of Aeronautics and Astronautics

Massachusetts Institute of Technology

Martha A. Krebs

Consultant (Science Strategies)

George W. Norton

Professor, Agricultural and Applied Economics

Virginia Polytechnic Institute and State University

Rosalie Ruegg

President

Technology Impact Assessment (TIA) Consulting, Inc.

Maxine L. Savitz, NAE

General Manager, Honeywell, Inc. (retired)

Jack Siegel

President, Technology and Markets Group

Energy Resources International, Inc.

James E. Smith

Professor, Fuqua School of Business

Duke University

Terry Surles

Vice President

Electricity Innovation Institute

James L. Sweeney

Professor of Management Science and Engineering

Stanford University

John J. Wise, NAE

Vice President (retired)

Mobil Research and Development Corporation