National Academies Press: OpenBook
« Previous: Introduction
Suggested Citation:"Definition of Peer Review." National Research Council. 1997. Peer Review in the Department of Energy-Office of Science and Technology: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/5939.
×

Definition of Peer Review

Although one can argue legitimately that "peer review"4 is the name given to any judgment of technical5 merit by other experts working in or close to the field in question, the scientific and engineering communities generally use the term "peer review" in a narrower sense. In this report, we adopt these communities' sense of peer review, as articulated in the definition developed by the USNRC:

A peer review is a documented, critical review performed by peers [defined in the USNRC report as "a person having technical expertise in the subject matter to be reviewed (or a subset of the subject matter to be reviewed) to a degree at least equivalent to that needed for the original work"] who are independent of the work being reviewed. The peer's independence from the work being reviewed means that the peer, a) was not involved as a participant, supervisor, technical reviewer, or advisor in the work being reviewed, and b) to the extent practical, has sufficient freedom from funding considerations to assure the work is impartially reviewed.

A peer review is an in-depth critique of assumptions, calculations, extrapolations, alternate interpretations, methodology, and acceptance criteria employed, and of conclusions drawn in the original work. Peer reviews confirm the adequacy of the work. In contrast to peer review, the term "technical review" . . . refers to a review to verify compliance to predetermined requirements; industry standards; or common scientific, engineering, and industry practice. (USNRC, 1988, p. 2)

In this definition, the term peer review has the following characteristics:

  • expert (including national/international perspectives on the issue),
  • independent,
  • external, and
  • technical.

Most importantly, peer reviews must be carded out by independent reviewers who are experts in the technical issues relevant to the projects under review. Such reviewers must be highly qualified6 and independent in order to evaluate credibly the scientific and engineering merit of

4  

The choice of the term "peer review" versus "merit review" is somewhat subjective. Because "merit review" is often used to describe evaluations that include programmatic/nontechnical aspects of projects (Royal Society, 1995), the committee has chosen to use the term "peer review" in this report.

5  

in this report, the committee uses the term "technical" to mean "relating to special and/or practical knowledge of an engineering or scientific nature."

6  

Determined by reputation and standing in the field (for example, publications and status in professional societies) and relevance to the project being reviewed.

Suggested Citation:"Definition of Peer Review." National Research Council. 1997. Peer Review in the Department of Energy-Office of Science and Technology: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/5939.
×

the project with respect to current technologies, both domestic and international. In the report Allocating Federal Funds for Science and Technology (NRC, 1995a, p. 69), peers are defined as "established working scientists or engineers from diverse research institutions who are deeply knowledgeable about the field of study and who provide disinterested technical judgments as to the competence of the researchers, the scientific significance of the proposed work, the soundness of the research plan, and the likelihood of success." Note that such reviewers need not be expert in or familiar with the agency program or relevant contextual factors. Those are the proper province of agency management.

The USNRC's definition of "peer review" provides some guidance on the issue of potential conflict of interest by explicitly excluding potential reviewers who have been involved with the specific project being reviewed or who have financial interests in the outcome of the reviews. As Chubin and Hackett (1990) have pointed out, however, it is often difficult to identify true "experts" on a subject or technology who do not have some biases that could be perceived as a potential conflict of interest (e.g., competing interests, personal relationships). The committee will provide more specific recommendations on the issue of conflict of interest in its final report.

In the past, OST has used the term "peer review" generally to refer to internal reviews of OST projects by qualified EM technical staff who were not involved directly in the specific project under review. This use of terminology has caused confusion and misunderstanding within both OST and external review groups (e.g., GAO, NRC) who have continued to criticize OST for a lack of a credible peer review program. Although internal reviews of this type are necessary and should continue, they should not be confused with peer review because the term "internal peer review" is not consistent with the usual meaning of peer review (Bozeman, 1993). The committee believes that at least part of the criticism leveled at the OST project review process has resulted from inconsistent and inaccurate descriptions of the processes involved (e.g., internal peer review, technical peer review). To avoid misunderstanding, OST should restrict the term "peer review" to only those technical reviews conducted by independent, external experts. OST should adopt alternative terms, such as "technical review," for its internal reviews of scientific merit and pertinency. Careful attention to nomenclature will eliminate much of the confusion about the nature of OST's review process and could increase appreciation and respect for OST's new peer review program.

Suggested Citation:"Definition of Peer Review." National Research Council. 1997. Peer Review in the Department of Energy-Office of Science and Technology: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/5939.
×
Page 9
Suggested Citation:"Definition of Peer Review." National Research Council. 1997. Peer Review in the Department of Energy-Office of Science and Technology: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/5939.
×
Page 10
Next: Benefits of Peer Review »
Peer Review in the Department of Energy-Office of Science and Technology: Interim Report Get This Book
×
Buy Paperback | $29.00 Buy Ebook | $23.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The Office of Science and Technology (OST) of the U.S. Department of Energy's (DOE's) Office of Environmental Management (EM) recently has instituted a peer review program that uses the American Society of Mechanical Engineers (ASME), with administrative and technical support provided by the Institute for Regulatory Science (RSI), to conduct peer reviews of technologies (or groups of technologies) at various stages of development.

OST asked the NRC to convene an expert committee to evaluate the effectiveness of its new peer review program and to make specific recommendations to improve the program, if appropriate. This is the first of two reports to be prepared by this committee on OST's new peer review program. OST requested this interim report to provide a preliminary assessment of OST's new peer review program. In the final report, the committee will provide a more detailed assessment of OST's peer review program after its first complete annual cycle.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!