among the logic, social dynamics, procedures, and validity of various forms of peer review. For example, peer review of a prospective (planning) document is typically more tentative than review of a final work product. Similarly, peer reviews of broad programs are different from reviews of individual reports.

LIMITATIONS OF PEER REVIEW

Peer review is not quality assurance or quality control per se. It is essentially advisory, not controlling. Although it can be an important guide and aid to those responsible for ensuring quality, the essence of peer review is to criticize constructively, not to decide. Peer review relies on impartial, independent experts who might have expertise only on some portion of the scope of the work and typically have many other demands competing for their time. The experts performing reviews cannot be expected to be aware of all scientific areas, practical considerations, or constraints of the subject of review, nor should they be held responsible for the ultimate decisions beyond their own review comments. They cannot be held responsible for matters beyond their expertise or, ultimately, for the quality of a work product they did not produce. Those decisions should reside with the individuals and organizations responsible for the outcome. A decision-maker needs to know the views of qualified peers, but such peers often cannot be expected to integrate factors outside the document presented for review, such as the relevance, need, and priority of a new research activity or the role of research findings in a context that necessarily includes statutory requirements, economics, and many other considerations.

The value of the peer-review process in assessing and improving a scientific or technical work product depends on a strong commitment to conduct and apply the results of peer review appropriately in judging or improving the technical merit of the product. The benefits of peer review are diminished if the integrity of the peer-review process is compromised or if the criticisms and suggestions received from independent peer reviewers are to some degree ignored or taken lightly by decision-makers who may be more interested in meeting a deadline or producing a desired answer than in judging or enhancing technical merit.

Peer review is an expensive and personnel-intensive process. It re-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement