REVIEW PROCEDURES FOR WATER RESOURCES PROJECT PLANNING

Panel on Peer Review

Committee to Assess the U.S. Army Corps of Engineers Methods of Analysis and Peer Review for Water Resources Project Planning

Water Science and Technology Board

Ocean Studies Board

Division on Earth and Life Studies

NATIONAL RESEARCH COUNCIL OF THE NATIONAL ACADEMIES

THE NATIONAL ACADEMIES PRESS
Washington, D.C. www.nap.edu



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page R1
Review Procedures for Water Resources Project Planning REVIEW PROCEDURES FOR WATER RESOURCES PROJECT PLANNING Panel on Peer Review Committee to Assess the U.S. Army Corps of Engineers Methods of Analysis and Peer Review for Water Resources Project Planning Water Science and Technology Board Ocean Studies Board Division on Earth and Life Studies NATIONAL RESEARCH COUNCIL OF THE NATIONAL ACADEMIES THE NATIONAL ACADEMIES PRESS Washington, D.C. www.nap.edu

OCR for page R1
Review Procedures for Water Resources Project Planning THE NATIONAL ACADEMIES PRESS 500 Fifth Street, N.W., Washington, DC 20001 NOTICE: The project that is the subject of this report was approved by the governing board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance. Support for this project was provided by the U.S. Army Corps of Engineers under contract no. DACW72-01-C-0001. International Standard Book Number 0-309-08508-X Copies available from The National Academies Press 500 5th St., NW Lockbox 285 Washington, DC 20055 1–(800)–624–6242 or (202) 334–3313 Internet http://www.nap.edu Copyright 2002 by the National Academy of Sciences. All rights reserved. Printed in the United States of America.

OCR for page R1
Review Procedures for Water Resources Project Planning THE NATIONAL ACADEMIES Advisers to the Nation on Science, Engineering, and Medicine The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Bruce M.Alberts is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Wm.A.Wulf is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V.Fineberg is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Bruce M.Alberts and Dr. Wm.A.Wulf are chair and vice chair, respectively, of the National Research Council. www.national-academies.org

OCR for page R1
Review Procedures for Water Resources Project Planning This page in the original is blank.

OCR for page R1
Review Procedures for Water Resources Project Planning COMMITTEE TO ASSESS THE U.S. ARMY CORPS OF ENGINEERS METHODS OF ANALYSIS AND PEER REVIEW FOR WATER RESOURCES PROJECT PLANNING: PANEL ON PEER REVIEW* JAMES K.MITCHELL, Chair, Virginia Polytechnic Institute and State University, Blacksburg MELBOURNE BRISCOE, Office of Naval Research, Arlington, Virginia STEPHEN J.BURGES, University of Washington, Seattle LINDA CAPUANO, Honeywell, Inc., San Jose, California DENISE FORT, University of New Mexico, Albuquerque PORTER HOAGLAND, Woods Hole Oceanographic Institution, Woods Hole, Massachusetts DAVID H.MOREAU, University of North Carolina, Chapel Hill CRAIG PHILIP, Ingram Barge Company, Nashville, Tennessee JOHN T.RHETT, Consultant, Arlington, Virginia RICHARD E.SPARKS, Illinois Water Resources Center, Urbana BORY STEINBERG, Steinberg and Associates, McLean, Virginia National Research Council Staff JEFFRY W.JACOBS, Study Director ELLEN A.DE GUZMAN, Research Associate JON Q.SANDERS, Project Assistant *   The Panel on Peer Review was one of four panels, operating under the auspices of a coordinating committee, that was convened by the National Academies’ Water Science and Technology Board (lead) and Ocean Studies Board to carry out studies mandated in the Water Resources Development Act of 2000. The panel’s charge is described in chapter 1. The “parent bodies” are listed in Appendix E.

OCR for page R1
Review Procedures for Water Resources Project Planning This page in the original is blank.

OCR for page R1
Review Procedures for Water Resources Project Planning Foreword In the early 1800s the U.S. Congress first asked the U.S. Army Corps of Engineers (which was created in 1775) to improve navigation on our waterways. From that beginning, the Corps began a program of public works that has reshaped virtually all of the nation’s river basins and coastal areas. Today we share in the benefits of those works: a reliable water transportation network, harbors that help link our economy to global markets, previously flood-prone land that is productive for urban and agricultural uses, hydroelectric power, and widely-used recreational facilities. Now, at the beginning of the twenty-first century, the Corps program is under intense scrutiny. Traditional constituencies press the Corps to complete projects that have been planned for many years and campaign for new projects to serve traditional flood control and navigation purposes. At the same time, environmental and taxpayer groups express concerns about these projects in Congress and in the courts. Some of these groups have exposed technical errors in analyses that have been used to justify projects. For these critics, the Corps’ water project development program must be reformed and the budget reduced or redirected. Some of these same groups are pressing the administration, the Congress, and the agency itself toward a new Corps mission, broadly described as environmental restoration. However, the concept of restoration awaits more precise definition, and the science of ecosystem restoration is in its infancy. But it is clear that restoration is a call for water resources management that accommodates and benefits from, as opposed to controls, annual and multi-year variability in the patterns and timing of river flows and the extremes of flood and drought. Meanwhile, the Corps is affected by a general trend in all federal agencies of smaller budgets and staffs. As demands for reform mount,

OCR for page R1
Review Procedures for Water Resources Project Planning the Corps’ current staffing and organization may need to be reconfigured to provide improved and more credible planning reports. As a result of this national debate over the Corps’ programs and the quality of its planning studies, the U.S. Congress in Section 216 of the 2000 Water Resources Development Act, requested The National Academies to conduct a study of the procedures for reviewing the Corps’ planning studies. In addition, the Congress requested a review of the Corps’ “methods of analysis” used in its water resources planning. In response to this request, the Water Science and Technology Board of The National Academies’ National Research Council, in collaboration with the NRC’s Ocean Studies Board, appointed four study panels—(1) Peer Review, (2) Planning Methods, (3) River Basin and Coastal Systems Planning, and (4) Resource Stewardship and Adaptive Management—and a coordinating committee to follow these panels’ progress and to write a synthesis report. Our study panels and coordinating committee held several meetings over the course of the study period beginning in 2001. We spoke with dozens of Corps of Engineers personnel, visited several Corps projects, and heard from different groups with interests in Corps projects. We came away with an appreciation for the dedication of Corps personnel and the complications and challenges they face in trying to being responsive to local project sponsors and the nation’s taxpayers. This is not the first study of the Corps by the Academies. However, these past studies were often focused on specific projects or on particular planning aspects. The reports in this series address the agency’s programs in a wider context. Because we appreciate the importance of the U.S. Congress and the sitting administration in directing Corps’ programs, many of our recommendations are directed to them. The Corps has a long history of serving the nation and is one of our oldest and most-recognized federal agencies. But it is today at an important crossroads. The nation, through the administration and the Congress, must help the agency chart its way for the next century. Leonard Shabman Chair, Coordinating Committee

OCR for page R1
Review Procedures for Water Resources Project Planning Preface The U.S. Army Corps of Engineers, through its civil works program, can take pride in its contributions throughout our nation’s history to the development of waterways infrastructure, navigation, flood damage reduction, water resources development and protection, and environmental restoration. Many projects that have been pioneering in their concept and bold in their execution were made possible by the creativity and dedication of outstanding scientists, engineers, and builders. The Corps has always had review processes for evaluation of its planning studies and projects, with the focus often being largely on the technical aspects. In recent years, however, increased consideration of such factors as environmental impacts, economic evaluations, political pressures, and new paradigms about flood control and management has engendered increased criticism and concern that some of the Corps’ studies may have led to conclusions, recommendations, and project decisions that are not adequately supported by the assumptions and analyses that were used. Our panel was charged to review “peer review procedures” and to assess both “an independent review process” and “existing technical review procedures.” As these terms imply different views regarding “independent peer review,” our panel chose to not use the term “peer review,” instead simply referring to both independent and internal procedures as “reviews.” The focus of our panel’s report is on review of Corps of Engineers studies, with careful attention given to the need for independent, external reviews by panels of well-qualified and impartial experts for large, complex, and sensitive projects. Our panel’s principal conclusions relate to the increasing need for independence of the reviewers and the review process from the organization undertaking and responsible for a planning study or project (in this case, the Corps of Engineers) as project complexity, cost, and controversy increase. A

OCR for page R1
Review Procedures for Water Resources Project Planning fully independent review can only be accomplished by reviewers who are free of conflict of interest and who are appointed by a group external to the Corps. Our recommendations call for the establishment of an Administrative Group for Project Review (AGPR) to administer the review process—to be housed either in the Office of the Assistant Secretary of the Army for Civil Works or in the Office of the Chief of Engineers—and for a Review Advisory Board (RAB) to provide oversight of the AGPR activities. To provide background and to set the stage for our work, we were briefed at our first meeting by Lt. Gen. Robert B.Flowers, Chief of Engineers; Dr. James Johnson, Chief of Planning; and Mr. Richard Worthington of Corps Headquarters in Washington. We extend our thanks also to Dr. Ronald Kostoff of the Office of Naval Research in Arlington, Virginia, who provided a briefing on review concepts and approaches at the panel’s first meeting. We also owe thanks to Dr. Jack Fritz of the National Academies and Mr. Tim Searchinger of Environmental Defense, both of whom discussed review procedures with the panel at its second meeting. The panel members provided diverse expertise and a wealth of experience in the many disciplines and topics relevant to this study—peer review, water resources engineering and planning, environmental and water law, river navigation and transport, ports, and Corps of Engineers history and operations. Each member brought a creative and fresh perspective to the study, and participated in the crafting of the several conclusions and recommendations and in the drafting of the report. We were also fortunate to have Dr. Leonard Shabman, Chair of the Coordinating Committee, participate in two of our panel’s three meetings. Len’s knowledge of the Corps of Engineers and its civil works program made his input especially valuable. The panel was supported and guided in its work by the outstanding staff of the Water Science and Technology Board. WSTB director Stephen Parker got us on our way by setting the stage for the study. Dr. Jeffrey Jacobs, the study director, carried the bulk of the burden. His knowledge of the Corps and of river management issues, his ability to understand and synthesize information, his creative and effective writing, his initiative and responsiveness, and his enthusiasm made him a pleasure to work with. This study and report could never been completed without his tireless effort. We also acknowledge with appreciation the logistical support of Mr. Jon Sanders and Ms. Ellen de Guzman, and editorial guidance from Ms. Rhonda Bitterli. The report was reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise in accordance with the procedures approved by the NRC’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript

OCR for page R1
Review Procedures for Water Resources Project Planning remain confidential to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this report: Mr. Richard Conway (retired), Union Carbide, Charleston, West Virginia; Mr. Robert Crangle, Rose and Crangle, Ltd., Lincoln, Kansas; Mr. Steve Dola, Office of the Assistant Secretary of the Army (retired), Arlington, Virginia; Dr. William Graf, University of South Carolina, Columbia; Dr. Ronald Kostoff, Office of Naval Research, Arlington, Virginia; Mr. Thomas Maddock, Boyle Engineering, Newport Beach, California; Ms. Deborah Moore, independent consultant, Berkeley, California; Dr. Herb Ward, Rice University, Houston, Texas; Mr. Garrett Westerhoff, Malcolm Pimie, White Plains, New York. Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations, nor did they see the final draft of the report before its release. The review of this report was overseen by Dr. Richard Goody, Harvard University (emeritus). Appointed by the National Research Council, he was responsible for making certain that an independent examination of the report was carefully earned out in accordance with the institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution. We appreciate the opportunity to address an issue of importance to the future success of the Corps of Engineers mission in meeting the nation’s needs for navigation, flood damage reduction, river and wetlands environmental protection and restoration, and water resources development. JAMES K.MITCHELL Chair

OCR for page R1
Review Procedures for Water Resources Project Planning This page in the original is blank.

OCR for page R1
Review Procedures for Water Resources Project Planning Contents     Executive Summary   1 1   REPORT PURPOSE AND SCOPE   10     Peers and Peer Review,   12     The Need for and Value of Review,   14     Selecting Reviewers for Corps Planning Studies,   15 2   HISTORY OF REVIEW OF CORPS OF ENGINEERS WATER RESOURCES PLANNING STUDIES   19     The Board of Engineers for Rivers and Harbors,   20     Water Resources Development Acts,   23     Changes to the Review Process,   26     Current Review Procedures,   29     Commentary,   30 3   CORPS OF ENGINEERS WATER RESOURCES PROJECT PLANNING PROCEDURES   32     Types of Studies,   32     Commentary: Inserting Review Into the Planning Process,   37 4   INDEPENDENT REVIEW PRINCIPLES AND CONSIDERATIONS   39     Components of Ideal Independent Review,   39     Establishing a Review Process,   41     Selecting the Type and Level of Review,   44     Commentary,   47 5   ALTERNATIVE APPROACHES TO REVIEW   48     Criteria for Evaluating Alternatives,   50     Variations on Independence,   52

OCR for page R1
Review Procedures for Water Resources Project Planning     Conclusion,   56 6   AN ADMINISTRATIVE GROUP FOR PROJECT REVIEW   57     Institutional Home,   58     AGPR Review Options,   59     Ensuring Effective and Credible Review,   63 7   RECOMMENDATIONS   70     References   73     Acronyms   75     Appendixes         A Water Resources Development Act 2000. Section 216   79     B Section 3 of the 1902 River and Harbors Act   82     C Izzo Memorandum Regarding Organization of the Office of the Assistant Secretary of the Army (Civil Works)   84     D Review Processes in Other Federal Agencies   85     E Rosters of:         (1) Committee to Assess the U.S. Army Corps of Engineers Methods of Analysis and Peer Review for Water Resources Project Planning: Coordinating Committee   88     (2) Water Science and Technology Board   89     (3) Ocean Studies Board   90     F Biographical Information of Panel Members and Staff   92