were provided with the proposed benefits framework but were given considerable flexibility in applying the methodology for completing the matrix. It was the judgment of the committee that this flexible approach would stimulate critical thinking about the methodological needs and practical problems of providing the relevant information. The committee believes that the panels met this objective in exemplary fashion.
Receive a report from each panel and use it as a basis for recommendations.
Evaluate the experience reported by the panels and modify the methodology accordingly.
Present conclusions in the Phase One report, which will also contain recommendations for specific steps to be taken in Phase Two of the ongoing effort.
The committee prepared a letter report, dated November 15, 2004, responding to a request from the staff of the House Appropriations Subcommittee on the Interior and Related Agencies to “… provide an overview of the study committee’s methodological approach; how that approach differs from the retrospective study delineated in the Academies’ report (also initiated by the Subcommittee through the appropriations process) Energy Research at DOE, Was It Worth It? (NRC, 2001); a synopsis of the current committee’s activities; and what is expected to be contained in the committee’s final report.” The letter report is included as Appendix D.
It is important to emphasize that Phase One was focused tightly on the basic problem of adapting the retrospective methodology to a prospective context. As a result, some important issues had to be deferred to Phase Two. For example, the issue of how to characterize national security benefits has not been explored. Security threats to the energy system have multiplied in number and kind since the retrospective study, but the committee intentionally deferred this and other issues to Phase Two of the project. In addition, the committee cautions against using the three expert panel case studies to assess the benefits of the three programs because the panels’ findings on the programs were developed for the express purpose of helping the committee to refine its methodology.
The balance of this report is organized as follows: Chapter 2 describes the evolution of the conceptual framework from retrospective to prospective evaluation and recalls the chief methodological and application problems encountered by the expert panels. Chapter 3 presents the committee’s recommended methodology. Chapter 4 presents the committee’s recommendations for applying the methodology in a consistent but practical way. Chapter 5 summarizes the work of the panels. Chapter 6 presents the committee’s overall conclusions about the Phase One project and its recommendations for Phase Two.