The nation’s water resources infrastructure features thousands of dams, an extensive levee system, and many harbors that can accommodate large ocean-going vessels. The U.S. Army Corps of Engineers planned, designed, and constructed much of this infrastructure, and now maintains it. Corps of Engineers feasibility studies for such projects are today generally more comprehensive and more complex than in the past, and they are often conducted in a politically charged environment. These studies may be subjected to the careful scrutiny of many interest groups, with some of these groups retaining highly qualified analysts to review the Corps’ fundamental assumptions and analytical methods. The complexity of some Corps planning studies and the challenges to some of these studies—especially the Corps’ Upper Mississippi River-Illinois Waterway draft feasibility study in the late 1990s—by different interest groups led Congress to request the National Academies to provide advice on implementing improved review procedures for Corps water resources planning studies.
Corps of Engineers water resources projects have long been subjected to some degree of review. The Corps’ former Board of Engineers for Rivers and Harbors, which was composed of senior Corps officials, reviewed Corps planning studies from the Board’s inception in 1902 to its termination in 1992. Whatever merits the Board brought to the review process, the Board was not independent of the Corps. There is a strong and direct correlation between the independence of reviewers—in terms of both knowledge and association with a project and organizational affiliation—and the credibility, both real and perceived, of review.
Whatever type of review process is implemented within the Corps, the role of review panels should be to identify, evaluate, explain, and comment on key assumptions that underlie technical, economic, and en-
vironmental analysis. Review panels should highlight areas of disagreement and controversies to be resolved by the Administration and Congress. A review panel should be given the freedom to comment on those topics it deems relevant to decision makers, leaving it to the recipient of the review to decide whether those issues constitute “technical” issues or “policy” issues. Review panels should also be able to evaluate whether interpretations of analysis and conclusions based on analysis are reasonable. But review panels should not be tasked to provide a final “thumbs up/thumbs down” judgment on whether a particular alternative from a planning study should be implemented, as the Corps of Engineers is ultimately responsible for this final decision.
EXECUTING REVIEW WITHIN THE CORPS
The Corps’ more complex water resources project planning studies (this report adopts a broad definition of “planning studies” that includes “reoperations” and retrofit-type studies for existing projects, as well as feasibility studies for new water resources projects) should be subjected to external, independent review. One or more panels of impartial, highly qualified experts should conduct this external review. External review panels should not include Corps of Engineers staff members and should not be selected by the Corps. External reviews should be overseen by an organization independent of the Corps, which will provide the highest degree of credibility of review. Examples of organizations that might lead these independent reviews include professional science societies and engineering societies and the National Academy of Public Administration. Responsibility for independent review could also be delegated to an independent federal oversight group (the Department of Energy’s Defense Nuclear Facility Safety Board is one model).
Not all Corps of Engineers water resources project planning studies will require external, independent review, but the Corps should institute external review for studies that are expensive, that will affect a large area, that are highly controversial, or that involve high levels of risk. As a rule, the more independent this external review process is from the Corps, the greater the credibility—both real and perceived—the review will have. Internal reviews should be conducted for Corps planning studies that are less complex and less costly and that involve lower levels of risk. Internal reviews should be conducted by a panel that usually includes a balance in the number of Corps of Engineers staff and external experts. The Corps should select the panelists for internal reviews.
An Administrative Group for Project Review
Corps of Engineers water resources project planning studies span a spectrum from small, low-impact projects to large, complex planning studies that consider a range of potentially large economic and environmental impacts. This diversity of planning studies calls for a review process that employs various levels of independence of review, depending on the project. Effective execution of this responsibility requires the establishment of a small, full-time, permanent body of professional staff. Congress should thus direct the Secretary of the Army to establish a small group—which we term the Administrative Group for Project Review (AGPR)—to administer the Corps’ review processes. The Administrative Group for Project Review itself should not conduct reviews; rather, it should decide which Corps planning studies will require a review, and whether a review will be conducted externally or internally or with the current review process.
There are two practical options for the institutional home of the Administrative Group for Project Review: (1) the Office of the Assistant Secretary of the Army for Civil Works (ASA(CW)) and (2) the Office of the Chief of Engineers of the Corps. A review of the relations between these two offices and their respective histories shows that the balance of responsibilities for review between the Assistant Secretary of the Army for Civil Works’ office and the Chief of Engineers has occasionally shifted since the ASA(CW) office was created in 1970. The nature of how Corps water resources project planning studies have been reviewed has also evolved. This panel noted that there are advantages and disadvantages of placing the AGPR in either body, and concluded that neither location is clearly preferential to the other.
The decision regarding the type of review (external, internal, or current procedures) to be performed should be made by the AGPR. But this decision should be open to review upon petition by interested parties, and a mechanism for the appeal of the decision should be established. If the AGPR is located in the Office of the Chief of Engineers, parties should be permitted to file an appeal to the ASA(CW). If the AGPR is located in the Office of ASA(CW), appeals should be submitted to an executive-level body such as the Council on Environmental Quality or the Office of Management and Budget. The entire appeal process should extend no longer than 60 days. Furthermore, the Administration (perhaps the director of the Office of Management and Budget) or Congress (via congressional resolution or other legislative action, but not simply committee
language) should be able to request a review of a particular Corps planning study.
The AGPR should assist reviewers and panels in understanding the assumptions and methods in the study at hand. To facilitate the tasks of review panels, the AGPR should compile a document for each review panel that clearly summarizes and explains the contents, assumptions, models, and methods contained within a Corps planning study. The AGPR and the Corps’ District Engineer(s) and other Corps analysts should be available to a panel during its review to answer questions about the study’s evolution and contents, and to help review panels understand the implications of the panel’s recommendations. The AGPR also should facilitate communication between review panels and appropriate other federal agencies, interest groups, and the public.
The Administrative Group for Project Review should produce a document that clearly explains the Corps’ review procedures. The Corps’ review procedures will evolve and mature, and they should be revised as the scientific, economic, social, and organizational context of Corps planning changes. This set of written review procedures should be viewed as flexible and amenable to changes and the AGPR should periodically update and revise this document.
In the case of external review panels, the appropriate independent organization should publish and disseminate reports. The AGPR should organize, publish, and disseminate reports authored by internal review panels.
A Review Advisory Board
The Administrative Group for Project Review would benefit by periodic, independent review of its mandate, structure, and decision-making processes. Periodic review and advice from an independent, interdisciplinary group of experts—a Review Advisory Board (RAB)—should be part of the implementation of the Corps’ review procedures. Congress should establish this Review Advisory Board to provide periodic independent advice to the Corps regarding review procedures for its water resources project planning studies.
The Review Advisory Board would not perform study reviews nor would it select reviewers. The Review Advisory Board should assess review processes to help ensure consistency, thoroughness, and timeliness of reviews, and it would suggest changes for improving the review process. In doing so, it would use background material provided by the Administrative Group for Project Review, make necessary site visits, and incorporate information from public comments. The Review Advisory
Board should consider both past studies and prospective studies and projects. The Review Advisory Board should periodically review the proposed scope and task statements of selected proposed reviews. To ensure the clarity and comprehensiveness of the planning study summary documents produced by the Administrative Group for Project Review, the Review Advisory Board should periodically review samples of these summary documents. The Review Advisory Board should report periodically to the office that houses the Administrative Group for Project Review.
The proposal for a Review Advisory Board clearly has merit. However, a board constituted with a mission focused exclusively on review procedures might be too narrowly focused to attract the interest of a broad range of well-qualified water resources planning experts. To ensure that the Corps’ review procedures are reviewed by well-qualified analysts, the functions of a Review Advisory Board may have to be part of the mandate of a body charged with more comprehensive review of the Corps’ planning procedures.
Independence of Review and Reviewers
The highest degree of credibility of external reviews will be achieved if the responsibility for coordinating the external review process is granted to an organization independent of the Corps. Such an independent organization must be in charge of selecting reviewers, all of whom should be independent of the Corps and free of conflicts of interest. Examples of organizations that could oversee independent reviews include professional science and engineering societies and the National Academy of Public Administration. There are other options as well, such as the Office of the Chief of Engineers, the Office of the Assistant Secretary of the Army for Civil Works (ASA(CW)), and the Office of Management and Budget. These options, however, do not offer the degree of independence that will provide highly credible review of controversial planning studies.
An important issue related to independence relates to use of the term “peer review.” During the course of this study, it became apparent that this term means different things to different people. Moreover, our panel was charged to evaluate and comment upon both an “independent review process” and the existing technical review process of Corps planning studies. To avoid possible confusion, the term “peer review” is not used in this report. Rather, the report focuses on review of Corps of Engineers
planning studies, with careful attention given to the need for independent, external reviews for large, complex, and sensitive projects.
Conceptual and Process Issues
An important step in ensuring effective use of the results of review is to clarify at the review’s outset the review panel’s roles and how results from the panel’s report are to be used. Recommendations of review panels cannot be binding. A review panel is to provide a credible assessment of the planning study, which should serve as an evaluation aid to the Chief of Engineers, who is ultimately responsible for the final decision. A review panel should also be able to evaluate whether interpretations of analysis and conclusions based on analysis are reasonable. A review panel should not, however, present a final judgment regarding whether a project alternative or a particular operations plan should be implemented.
Results of the review should be presented to the Chief of Engineers before a final decision on a planning study has been made. A review panel’s report should be a public document. Results of reviews should appear in Corps water resources project planning studies that are submitted to Congress. An instance in which a Corps District Engineer, and not the Chief of Engineers, may be the primary client of a review is when review is initiated in the early stages of a planning study (discussed below). To help ensure effective use of a review’s results, the review’s primary client—usually the Corps’ Chief of Engineers—should respond in writing to each key point contained in a review. The Chief should either agree with the point and explain how it will be incorporated in the planning study, or rebut the comment and explain why the Corps is choosing to reject it.
Timing, continuity, and costs of review are important considerations for the Corps. It is not always clear when reviews should be conducted, or whether they should be conducted periodically or continuously. Concerns over the added costs of review are important given that local water resources project sponsors are required to contribute some portion of project costs. Project cosponsors often question the value of review, especially in instances where the local sponsor has a clear idea of the type and scale of project desired.
Corps of Engineers planning studies are conducted in two phases, a reconnaissance phase and a more detailed feasibility phase (Figure ES-1). The reconnaissance phase, lasting no more than one year, is used to determine whether there is a federal interest in a given water resources
problem or opportunity. In the event it is determined that a federal interest exists—and the majority of reconnaissance studies conclude that no federal interest exists—the Corps conducts a feasibility study.
In the idealized Corps of Engineers water resources project planning study, the feasibility phase lasts roughly two years. The point at which a review should be initiated will not always be clear, and it will vary depending upon a study’s complexity and duration. It is important that review be initiated early enough in the Corps’ study process so that the review’s results can be applied to the feasibility study. Conducting review early within a planning study will also lend credibility to the process, as reviews conducted in a study’s latter stages are more likely to be viewed as pro forma exercises.
With highly controversial studies, reviews should generally be initiated early in the feasibility phase, and there may be instances in which review could be initiated during the reconnaissance phase. Results of reviews initiated during a planning study’s early stages should be submitted to the Corps’ District Office. When reviews are initiated early in a planning study, the Corps’ District Engineer should prepare a written response explaining how the Corps will incorporate the review’s results into its planning study.
Reviews conducted at various stages of Corps planning studies also may have value. For example, in highly controversial studies, a review panel might conduct an initial review early in the feasibility phase, then meet later during the feasibility phase to conduct a more comprehensive review. Such multiple-stage reviews may be less practical for more focused and highly technical planning studies, but they should be used in more controversial and complex planning studies. These reviews may be especially useful for the Corps’ most challenging planning studies, some of which may require over 10 years to complete. At the same time, it is important that panelists focus on their review of the planning study, and not become defenders of their recommendations. To guard against this—especially in lengthy planning studies—different review panels may need to be appointed at different stages of the study.
The cost of review is an important consideration, especially since the 1986 Water Resources Development Act (WRDA 1986) mandates stringent cost-sharing responsibilities for local sponsors. Since passage of WRDA 1986, local project sponsors have complained that the Corps study process takes too long and costs too much. There may be instances in which review could prove redundant and should be kept to a minimum. However, there are also instances in which Corps water resources project planning studies—especially controversial ones—could be more scientifically sound and could require less time and resources if they are given the benefit of review early in the process. There are examples of
Corps feasibility studies that were not reviewed and that extended over several years, requiring considerable resources, but that were not completed in a timely manner because of uncertainties, controversies, and criticisms. Review should not be viewed as a burden, but rather as an essential part of Corps project planning that provides quality control. In fact, there will be instances in which review will help reduce planning costs and time. Effective review will ideally result in water resources planning processes that are transparent and accountable.
The current planning context features a huge backlog of Corps of Engineers projects; the construction of projects based upon recently completed planning studies will not begin for years. Extending a planning study by a few months for careful review thus seems to be of little consequence and is likely to represent time well spent. In terms of costs, especially in the case of large, expensive projects, adding careful review represents a small fraction of total costs and will generally represent a wise expenditure of resources.
To help implement this report’s recommendations, Congress should provide the resources necessary to help the Secretary of the Army reformulate and strengthen the Corps’ review procedures for its water resources project planning studies.