Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 52
Appendix B List of Recommendations from Peer Review in Environmental Technology Development Programs: The Department of Energy 's Office of Science and Technology COMMITTEE ON THE DEPARTMENT OF ENERGY-OFFICE OF SCIENCE AND TECHNOLOGY'S PEER REVIEW PROGRAM C. HERB WARD, Chair, Rice University, Houston, Texas BARRY BOZEMAN, Georgia Institute of Technology, Atlanta RADFORD BYERLY, JR., University Corporation for Atmospheric Research (retired), Boulder, Colorado LINDA A. CAPUANO, AlliedSignal Aerospace, San Jose, California RICHARD A. CONWAY, Union Carbide Corporation (retired), South Charleston, West Virginia THOMAS A. COTTON, JK Research Associates, Vienna, Virginia FRANK P. CRIMI, Lockheed Martin Advanced Environmental Systems Company (retired), Saratoga, California JOHN C. FOUNTAIN, State University of New York, Buffalo DAVID T. KINGSBURY, Chiron Corporation, Emeryville, California GARETH THOMAS, University of California, Berkeley RECOMMENDATIONS Linkage of Peer Reviews to Management Decisions As part of the documentation provided to peer review program management during the process of selecting projects for review, OST program managers should be required to clearly identify the upcoming decision or milestone for which the results of the peer review will be used. 52

OCR for page 52
APPENDIX B 53 Selection of Projects for Review To aid in the selection of projects to review, OST should adopt two additional criteria to choose from those projects that satisfy one of the three existing selection criteria: (1) technologies that are being considered for deployment, and (2) technologies for which a request for further funding has been received or is anticipated. Although the two additional selection criteria would assist OST in identifying those projects for which peer review is of highest priority, application of these criteria would still leave a large number of projects that are not peer reviewed. To address this issue, the committee recommends that OST should expand its practice of evaluating a number of related technologies in a single peer review, whenever possible. Selection of Reviewers OST should establish a more systematic approach to accessing reviewer information from other databases (e.g., chemical engineers, geologists, physicists, materials scientists, biologists) or from other professional societies, as needed, to ensure the appropriate range of expertise for all review panels. The reviewer selection process should in general avoid DOE staff as peer reviewers and should ensure that the DOE-affiliated persons are never more than a small fraction of a panel's membership. Documentation for Peer Reviews A detailed proposal or statement of work should be required for all peer reviews. It should describe the specific activities that will be carried out if the project is funded. Addressing the Backlog of Peer Reviews OST should consider adopting a triage approach that would allow far greater numbers of technologies to be peer reviewed. This approach would involve a formal prescreening of projects by peer reviewers based exclusively on the written documentation on the project in effect, a "mail review" of projects, followed by a formal meeting of the panel to discuss and rank them. The approach would help OST to fulfill its policy that "all projects are to be peer

OCR for page 52
54 TECHNOLOGIES FOR ENVIRONMENTAL AL4NAGEMENT reviewed" in the short term (as of May 1, 1998, OST had reviewed only 43 of 226 projects that were receiving funding from the program). Evaluation and Improvement Mechanisms OST management should develop an effective evaluation and improvement process for the peer review program that includes regular benchmarking against other peer review programs and the collection of activity and performance metrics. OST's Organizational Structure and Leadership OST leadership should develop an explicit strategy to accomplish a change in its organizational culture by distributing (1) educational materials that summarize the basic principles and benefits of peer review as a tool for decision making, (2) case histories illustrating how peer review input served to improve specific projects, and (3) summaries of key performance metrics that demonstrate how peer reviews are used to meet the overall objectives of OST's program. Potential Applications of Peer Reviews in OST OST should carefully evaluate the objectives and roles of all its existing reviews and then determine the most effective use of peer reviews (of various types) in meeting its overall objectives.