Peer Review in Environmental Technology Development Programs (1998) / Chapter Skim
Currently Skimming:
7 Improving the Effectiveness of OST's Peer Review Program
7 Improving the Effectiveness of OST's Peer Review Program
Pages 85-93
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 85... ...
The first step for OST leadership is to ensure that peer reviews are effectively linked to OST decision making. In order to continue to develop and achieve a more effective peer review program, OST leadership also will have to commit to a process of continuous assessment and improvement involving cycles of planning, execution, and evaluation.
|
From page 86... ...
Because for FY98, the FA/CC program managers selected the projects to be reviewed, the times for review, and the technology-specific review criteria, the committee expects that the results of the peer review should fit more logically with the decision-making process. The ASME Peer Review Committee also has identified the lack of a clear relationship between the peer review results and OST decision making as a recurring problem with the peer review program.
|
From page 87... ...
Therefore, the committee recommends that as part of the documentation provided to peer review program management during the process of selecting projects for renew, OST program managers be required to clearly identify the upcoming decision or milestone for which the results of the peer review will be used. This Formation also should be provided to peer reviewers as part of the documentation that they receive in preparation for the review EVALUATION AND IMPROVEMENT MECHANISMS Benchmarking One approach for guiding the development of an internal evaluation procedure for the peer review program would be for OST peer review program managers to proactively seek out and learn from other organizations that have more mature peer review processes.
|
From page 88... ...
The benefits of improving the peer review process through benchmarking must be weighed against the possible negative effects of constantly changing procedures, however. Benchmarking efforts should be targeted to specific weaknesses in the peer review process and should be initiated at a logical time in OST's annual peer review cycle when specific areas of improvement have been identified (e.g., after the release of this report, or after the annual ASME Peer Review Committee meeting)
|
From page 89... ...
. Although OST has not yet established metrics or a benchmarking process for its peer review program, it has begun to develop performance metrics for its technology programs as part of its annual performance planning.
|
From page 90... ...
These activity and performance metrics are provided as examplesultimately OST management will have to establish its own set of metrics based on the success criteria it sets for the technology development program. The committee recommends that OST management develop an effective evaluation and improvement process for the peer review program that
|
From page 91... ...
The committee recommends that OST carefully evaluate the objectives and roles of all of its existing reviews, and then determine the most effective use of peer reviews (of various types) in meeting its overall objectives.
|
From page 92... ...
but composed of world-class engineers and scientists (another example of such a standing group, the NRC Board on Assessment of NIST has been described in Box 3.5~. OST'S "ORGANIZATIONAL" CULTURE AND LEADERSHIP One feature characteristic of organizations that effectively use peer review as a tool for management of their research and development portfolios is a peer review process that is ingrained in their organizational cultures; in other words, for these organizations, peer review is "standard operating procedure" for providing input to their decision-making process.
|
From page 93... ...
educational materials that summarize the basic principles and benefits of peer review as a tool to decisionmaking, (2) case histories illustrating how peer review input served to improve specific projects, and (3)
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.