Introduction

The U.S. Department of Energy's (DOE's) Office of Environmental Management (EM) was established in 1989 to address the health, safety, and environmental challenges associated with cleanup of the nation's nuclear weapons complex. Within EM, the Office of Science and Technology (OST) was created to promote the development of new and improved technologies to lower cleanup costs and risks (to workers, the public, and the environment) and to improve cleanup capabilities. OST supports the entire range of technology development activities—beginning with basic research through the new EM Science Program (NRC, 1997) and extending through development, demonstration, and (with the assistance of industrial partners) deployment into the cleanup program.

In fiscal year 1997 the annual budget for EM is about $6 billion, of which about $300 million is devoted to technology-development activities within OST. The importance of technology development to EM's mission has been recognized in Accelerating Cleanup: Focus on 2006 (DOE, 1997a), a draft planning document for cleanup of the weapons complex. This document discusses the importance of technology development to reduce the "mortgage" at the complex—the long-term costs of maintaining contaminated buildings, equipment, and sites until they can be remediated.

OST sponsors close to 300 research and development (R&D) projects at universities, national laboratories, and private-sector companies on topics ranging from remote detection of contaminants in the subsurface using geophysical techniques to development of melters for waste vitrification. OST uses various types of reviews (e.g., programmatic reviews, technical assessment reviews, and peer reviews)1 in its technology-selection process (see Appendix C for a description of the different types of reviews used within OST). These reviews are used to assess the merit of individual projects as well as the merit of entire technology development thrusts within the office.

Several recent NRC reports evaluated DOE-OST's technology-selection process and recommended that OST should develop and apply an independent, external review process to all of its technology-development programs. The report Improving the Environment recommended that "technology selection should incorporate a knowledgeable independent review group that has no vested interests in the outcome and that includes people from outside the Department who work in the commercial use of technologies" (NRC, 1995c, p. 104). The NRC's Committee on Environmental Management Technologies (CEMT) also evaluated this technology-selection process in its 1994 and 1995 annual reports (NRC, 1995b, 1996). In particular, these reports recommended that OST should develop and apply a standardized, rigorous, and independent external peer review process to all of its technology-development programs. These findings were echoed in a subsequent General Accounting Office (GAO) report, which concluded that "although the lead sites used significantly different systems to select projects, none of them used disinterested reviewers to determine the technical merit of the proposed work" (GAO, 1996, p. 7).

1  

These peer reviews are termed "technical peer reviews" by OST.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 5
--> Introduction The U.S. Department of Energy's (DOE's) Office of Environmental Management (EM) was established in 1989 to address the health, safety, and environmental challenges associated with cleanup of the nation's nuclear weapons complex. Within EM, the Office of Science and Technology (OST) was created to promote the development of new and improved technologies to lower cleanup costs and risks (to workers, the public, and the environment) and to improve cleanup capabilities. OST supports the entire range of technology development activities—beginning with basic research through the new EM Science Program (NRC, 1997) and extending through development, demonstration, and (with the assistance of industrial partners) deployment into the cleanup program. In fiscal year 1997 the annual budget for EM is about $6 billion, of which about $300 million is devoted to technology-development activities within OST. The importance of technology development to EM's mission has been recognized in Accelerating Cleanup: Focus on 2006 (DOE, 1997a), a draft planning document for cleanup of the weapons complex. This document discusses the importance of technology development to reduce the "mortgage" at the complex—the long-term costs of maintaining contaminated buildings, equipment, and sites until they can be remediated. OST sponsors close to 300 research and development (R&D) projects at universities, national laboratories, and private-sector companies on topics ranging from remote detection of contaminants in the subsurface using geophysical techniques to development of melters for waste vitrification. OST uses various types of reviews (e.g., programmatic reviews, technical assessment reviews, and peer reviews)1 in its technology-selection process (see Appendix C for a description of the different types of reviews used within OST). These reviews are used to assess the merit of individual projects as well as the merit of entire technology development thrusts within the office. Several recent NRC reports evaluated DOE-OST's technology-selection process and recommended that OST should develop and apply an independent, external review process to all of its technology-development programs. The report Improving the Environment recommended that "technology selection should incorporate a knowledgeable independent review group that has no vested interests in the outcome and that includes people from outside the Department who work in the commercial use of technologies" (NRC, 1995c, p. 104). The NRC's Committee on Environmental Management Technologies (CEMT) also evaluated this technology-selection process in its 1994 and 1995 annual reports (NRC, 1995b, 1996). In particular, these reports recommended that OST should develop and apply a standardized, rigorous, and independent external peer review process to all of its technology-development programs. These findings were echoed in a subsequent General Accounting Office (GAO) report, which concluded that "although the lead sites used significantly different systems to select projects, none of them used disinterested reviewers to determine the technical merit of the proposed work" (GAO, 1996, p. 7). 1   These peer reviews are termed "technical peer reviews" by OST.

OCR for page 5
--> In response to these NRC and GAO reports, the OST recently has instituted a peer review program that uses the American Society of Mechanical Engineers (ASME), with administrative and technical support provided by the Institute for Regulatory Science (RSI), to conduct peer reviews of technologies (or groups of technologies) at various stages of development. According to the OST, the objective of this new program is to serve as a "management tool for assuring that the technology is of high quality and effective, that critical needs have not been overlooked, and that the technology has the best chance possible for implementation."2 OST asked the NRC to convene an expert committee to evaluate the effectiveness of its new peer review program and to make specific recommendations to improve the program, if appropriate. In particular, the committee was asked to review the following: internal procedures used by OST to identify the need for timely peer review of projects and programs; structures, protocols, and procedures for obtaining peer reviews of OST projects and programs, including who decides what will be peer reviewed, what criteria for peer review are used, and when in the R&D process peer review is requested; and feedback of peer review results into program management and development decisions. In performing this assessment, the committee was asked to compare OST's practices to generally accepted norms for scientific and technical peer review, including practices for selection of peer reviewers and screening for bias and conflict of interest. The responsibility for establishing and executing an effective peer review program lies entirely with EM and specifically with OST. OST has chosen to establish a relationship with two independent organizations, the ASME and RSI, to carry out some aspects of its peer review program. When the committee first began this review and in subsequent meetings, officials from ASME and RSI expressed their concerns about the scope of the study and, in particular, whether the committee would be evaluating the performance of either of these organizations. In undertaking this review, the committee has focused its attention exclusively on the design and effectiveness of the peer review program; the committee has not addressed the performance of ASME or RSI, nor has it evaluated or endorsed the organizational arrangements among ASME, RSI, and OST. The committee's findings and recommendations are directed at OST which, as noted above, has ultimate responsibility for the peer review program. Another NRC Committee, the Committee on Prioritization and Decision Making in the Department of Energy-Office of Science and Technology, is currently conducting a parallel review of the decision-making process in OST's technology-development program. One aspect of its work will be to examine the role and importance of peer reviews in OST's decision-making process. Our committee therefore has focused its work on OST's peer review program itself (including the peer review results used as an input to decision making), but has not addressed OST's decision-making process explicitly. This study is being carried out over a 15-month period (from January 1997 through March 1998), during which time the committee will produce both an interim (this report) and a final report. The committee has been briefed on the newly instituted peer review program by DOE staff at three committee meetings, and members and staff of the committee observed peer 2   Presentation to committee by Anibal Taboas, DOE, February 24, 1997, Washington, D.C.

OCR for page 5
--> reviews conducted on the In Sire Redox Manipulation project in Richland, Washington, the Decontamination and Decommissioning (D&D) Large Scale Demonstration projects and three D&D Technology Development projects in Morgantown, West Virginia, the MAG*SEP technique3 in Atlanta, Georgia, the Small In-Tank Processing Modules and Small Modular In-Can Vitrification projects in Columbia Maryland, and several High Temperature Melter and Characterization projects in Idaho Falls, Idaho. The committee also has reviewed all peer review reports that have been produced under the new program from its initiation in October 1996 through June 1997. In addition, during its meetings, the committee was briefed on current peer review practices at the Environmental Protection Agency (EPA), EPA's Science Advisory Board, the U.S. Nuclear Regulatory Commission (USNRC), the DOE-Office of Basic Energy Sciences (BES), Information Dynamics, Inc. (which conducts reviews for NASA's Office of Life and Microgravity Sciences and Applications), and the American Institute of Biological Sciences (AIBS). This is the first of two reports to be prepared by this committee on OST's new peer review program. OST requested this interim report to provide a preliminary assessment of its new peer review program. In particular, OST asked the committee to consider whether it is moving in the fight direction toward the implementation of a credible, effective, and defensible peer review program. In this report, the committee describes the essential components of a credible peer review program and provides a preliminary assessment of OST's new peer review program and the status of its implementation. Recognizing that this is a new program in its early stages of implementation, the committee has focused on broad issues and has tried to offer constructive recommendations to assist OST in successfully implementing this program. In the final report, the committee will provide a more detailed assessment of OST's peer review program after its first complete annual cycle. This final assessment will develop a general framework for evaluating the level of development (or "maturity level") of a peer review program, focusing on specific components of the peer review process, and will include more detailed recommendations on how to improve OST's peer review program (including issues raised in this report), if appropriate. 3   The MAG*SEP technique is a means of recovering selected radionuclides and heavy metals from water and other liquids through sorption onto specially coated particles.

OCR for page 5
This page in the original is blank.