National Academies Press: OpenBook

Peer Review in the Department of Energy-Office of Science and Technology: Interim Report (1997)

Chapter: Appendix C: Description of OST's Review Program

« Previous: Appendix B: Description of OST's Technology Investment Decision Model
Suggested Citation:"Appendix C: Description of OST's Review Program." National Research Council. 1997. Peer Review in the Department of Energy-Office of Science and Technology: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/5939.
×

Appendix C
Description of OST's Review Program1

The DOE-Office of Science and Technology (OST) has developed an extensive review program with the goal of improving the design, management, and implementation of its programs and technical projects (see Figure C.1). OST seeks to integrate the results of its many reviews to aid managers in decision making. Results may feed into many decisions and activities, such as improving programs and management, formulating budgets, setting priorities, determining technology maturity and availability, and accelerating or decelerating programs. Reviews take place throughout the various levels of the EM organization and can be assigned to three different categories, described below:

  • Programmatic reviews are designed to assess the appropriateness and effectiveness of the structure, goals, management, and budget of a particular organization or program. This type of review may be conducted internally (by OST staff) or externally (by standing bodies and groups with EM or OST oversight responsibilities). Reviewers must possess a high level of familiarity with the mission and activities of the organization or program being evaluated.
  • Technical assessment reviews are designed to evaluate one or more projects with respect to organizational needs, objectives, responsibilities, or budget, and are used to provide the program or project manager with technical advice and direction. Evaluation criteria include effectiveness in contributing to meeting needs, cost effectiveness, and public and regulatory acceptability. Technical assessment reviews are conducted internally. Reviewers include technology developers and users, who possess program and project knowledge expertise, and knowledgeable stakeholders, for example, regulators.
  • Technical peer reviews, also called merit reviews, are used to evaluate a project on its scientific or engineering basis, the competence of the researchers, the soundness of the research plan, and the likelihood of success. Reviewers may consider budgetary aspects of a project only from the standpoint of whether the proposed budget is adequate to complete the work. Technical peer reviews are conducted externally and independently, such that the investigators whose work is being reviewed play no part in the selection or organization of the review panel. From the technical peer review, OST seeks to gain independent, unbiased, technical input or justification for funding project development. This type of review fits most closely the generally accepted definition of "peer review."

These three types of reviews are performed across the organizational levels of the OST, which comprise department, program, and project levels. Reviews conducted at each of these levels are discussed below.

1  

The material in this appendix is based on OST's description of its program (DOE, 1996), not on the committee's evaluation.

Suggested Citation:"Appendix C: Description of OST's Review Program." National Research Council. 1997. Peer Review in the Department of Energy-Office of Science and Technology: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/5939.
×

Figure C.1

Diagram of OST's review program showing the different types of reviews, the  offices to which reviews are submitted, and the level of the organization at  which reviews occur. ASME = American Society of Mechanical Engineers;  CEMT = Committee on Environmental Management Technologies; EM-1 = Office of the  Assistant Secretary for Environmental Management; EM = Office of Environmental  Management; EMAB = Environmental Management Advisory Board; ER = Office of  Energy Research; FA/CC = Focus area/crosscutting area; FETC = Federal Energy  Technology Center; NRC = National Research Council; OST = Office of Science and  Technology; SSAB = Site-Specific Advisory Board; STCG = Site Technology  Coordinating Group. Modified from DOE (1996).

Suggested Citation:"Appendix C: Description of OST's Review Program." National Research Council. 1997. Peer Review in the Department of Energy-Office of Science and Technology: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/5939.
×

Department-Level Reviews

Department-level reviews assess the effectiveness of EM's technology-development program, and all are classified as programmatic reviews. These reviews are submitted to the Assistant Secretary for EM. OST considers department-level reviews to be external in that they are initiated and conducted independently of OST and the focus areas. The following groups conduct department-level reviews:

  • U.S. General Accounting Office (Gao). The GAO provides congressionally mandated programmatic oversight to the Department of Energy, and the observations and conclusions from GAO reports are made available to and used by OST.
  • National Research Council (NRC)'S Committee On Environmental Management Technologies (CEMT). Until it was discontinued in September 1997, the CEMT2 reviewed the broad issues of technology development, implementation, and evaluation within EM. It also evaluated specific technologies that EM considers most important toward reaching its goals. The CEMT submitted two annual reports to the Assistant Secretary for EM (NRC 1995b, 1996). The responsibilities of the CEMT have been assumed by six committees, including the present committee.
  • Environmental Management Advisory Board (EMAB). The EMAB provides advice and recommendations to the Assistant Secretary for EM on a broad range of issues relevant to EM. The EMAB is supported by EM and is chartered under the Federal Advisory Committee Act (FACA) and is composed of representatives from tribal, state, and local governments; other federal agencies; environmental and citizen groups; labor organizations; science organizations; and academia. Of the five subcommittees under the EMAB, the Technology Development and Transfer Committee is most directly related to OST.
  • Environmental Management Site Specific Advisory Board (EM SSAB). The EM SSAB provides a means for community members to contribute to site-specific policy and technical decisions (e.g., future land use, integrated risk management, resource allocation, and EM priority setting). The board is supported by EM, chartered under FACA and includes all board members from each local EM board throughout the DOE complex. The local boards have unique mission statements, operating procedures, and meeting schedules. Local board members are appointed by the Assistant Secretary for EM and include community members, members from local and tribal governments, and ex officio representatives from DOE, EPA, and state governments.

Program-Level Reviews

OST uses program-level reviews to chart progress being made and to assess the suitability of OST and focus area objectives, policies, and plans. These reviews are initiated and conducted within OST and the focus areas, and use both internal and external experts. Program-level reviews include the following:

  • OST Board Of Directors Reviews. The OST Board of Directors provides guidance and management for EM's science and technology programs, reviews entire program areas, and weighs high-level policy issues. The board consists of the Deputy Assistant Secretary for OST

2  

Formerly the parent body of this committee.

Suggested Citation:"Appendix C: Description of OST's Review Program." National Research Council. 1997. Peer Review in the Department of Energy-Office of Science and Technology: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/5939.
×
  • and the field office managers at the Savannah River, Idaho, and Hanford operations offices, and at the Federal (formerly Morgantown) Energy Technology Center (FETC). The OST Board of Directors reviews are classified as programmatic reviews.
  • OST Business Reviews. OST business reviews are conducted on a monthly basis and evaluate business management, program design, and execution of technology development activities, as presented by FA/CC program managers. The Deputy Assistant Secretary for OST performs these programmatic reviews.
  • Focus Area Mid-Year Reviews. These reviews, conducted for each focus area, assess program management, direction, technical emphasis, and overall soundness of the respective focus area's technology-development program. Both internal and external experts participate as reviewers, which include representatives of private industry, EM user groups, the scientific and academic communities, other federal agencies, and, in some instances, members of the Community Leaders Network (OST's primary stakeholder organization). Mid-year reviews are held in the second or third quarter of the fiscal year, and are classified as programmatic reviews.
  • OST Year-End Review. This internal review is an evaluation of individual projects in the context of overall program direction and available resources and is intended to guide EM-50 headquarters staff as they finalize programs for the budget year. In assessing projects, the staff review associated technical peer review reports and consider progress made. The OST year-end review is classified as a technical assessment review.
  • Focus Area And Crosscutting Program Reviews. These programmatic reviews assist managers in balancing the respective program's portfolio of technologies to fit the needs of users and stakeholders. Program reviews are conducted by the FA/CC program managers and are attended by technology users, stakeholders, and external industry and academic experts. The results of program reviews feed into EM's strategic planning.

Project-Level Reviews

Project-level reviews may be internal or external, and are used to assess proposed or ongoing projects for scientific merit, potential for meeting a site need, potential for risk reduction and safety improvements, cost-effectiveness, regulatory and public acceptability, and commercial viability. Project-level reviews include the following:

  • American Society of Mechanical Engineers (ASME) Peer Reviews. The ASME operates under a grant from DOE to conduct technical peer reviews of proposed or ongoing focus area projects. Reviewers are subject matter experts independent of DOE. Review panels are convened to review specific technologies or groups of technologies and are disbanded after issuing their reports. The ASME peer review process is described in further detail in Appendix A.
  • Site Technology Coordinating Group (STCG) Reviews. STCG reviews are internal evaluations used to identify and prioritize site technology requirements. STCGs are located at each DOE Operations Office, and are, in general, headed by the Technical Program Officer and composed of site technology users, technology developers, site contractors, and stakeholders. The groups review focus area plans in order to ensure that technology-development decisions address site cleanup needs. STCGs also serve as liaisons to regulators and stakeholders (including local SSABs), and ensure that these perspectives are incorporated into site technology decisions. The work of the STCGs can be categorized as technical assessment reviews.
Suggested Citation:"Appendix C: Description of OST's Review Program." National Research Council. 1997. Peer Review in the Department of Energy-Office of Science and Technology: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/5939.
×
  • Procurement Reviews. Procurement reviews are evaluations of proposals received in response to a solicitation for environmental restoration research and development projects. These solicitations are used when DOE is able to define, but unable to solve, a specific problem. Procurement reviews are conducted by FETC and are considered technical assessment reviews.
  • Environmental Management Science Program (EMSP) Reviews. The EMSP is an EM-sponsored research program designed to bridge the gap between fundamental research and needs-driven technology development for environmental restoration. Two types of reviews are conducted as part of the proposal evaluation process for the EMSP. The Office of Energy Research (ER) conducts a peer review to evaluate the merit of proposals received and forwards selected proposals to a panel of EM managers. The EM managers then conduct a technical assessment review to evaluate the proposals' relevance to EM remediation needs.
Suggested Citation:"Appendix C: Description of OST's Review Program." National Research Council. 1997. Peer Review in the Department of Energy-Office of Science and Technology: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/5939.
×
This page in the original is blank.
Suggested Citation:"Appendix C: Description of OST's Review Program." National Research Council. 1997. Peer Review in the Department of Energy-Office of Science and Technology: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/5939.
×
Page 47
Suggested Citation:"Appendix C: Description of OST's Review Program." National Research Council. 1997. Peer Review in the Department of Energy-Office of Science and Technology: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/5939.
×
Page 48
Suggested Citation:"Appendix C: Description of OST's Review Program." National Research Council. 1997. Peer Review in the Department of Energy-Office of Science and Technology: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/5939.
×
Page 49
Suggested Citation:"Appendix C: Description of OST's Review Program." National Research Council. 1997. Peer Review in the Department of Energy-Office of Science and Technology: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/5939.
×
Page 50
Suggested Citation:"Appendix C: Description of OST's Review Program." National Research Council. 1997. Peer Review in the Department of Energy-Office of Science and Technology: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/5939.
×
Page 51
Suggested Citation:"Appendix C: Description of OST's Review Program." National Research Council. 1997. Peer Review in the Department of Energy-Office of Science and Technology: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/5939.
×
Page 52
Next: Appendix D: Biographical Sketches »
Peer Review in the Department of Energy-Office of Science and Technology: Interim Report Get This Book
×
Buy Paperback | $29.00 Buy Ebook | $23.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The Office of Science and Technology (OST) of the U.S. Department of Energy's (DOE's) Office of Environmental Management (EM) recently has instituted a peer review program that uses the American Society of Mechanical Engineers (ASME), with administrative and technical support provided by the Institute for Regulatory Science (RSI), to conduct peer reviews of technologies (or groups of technologies) at various stages of development.

OST asked the NRC to convene an expert committee to evaluate the effectiveness of its new peer review program and to make specific recommendations to improve the program, if appropriate. This is the first of two reports to be prepared by this committee on OST's new peer review program. OST requested this interim report to provide a preliminary assessment of OST's new peer review program. In the final report, the committee will provide a more detailed assessment of OST's peer review program after its first complete annual cycle.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!