Alternative Approaches to Review
Corps of Engineers water resources project planning studies could be reviewed and evaluated within many different organizational structures and by many different criteria, some of which are used by other government agencies. In considering how the Corps’ review procedures might be improved, it is instructive to examine review procedures of other organizations, including other federal agencies (Appendix D lists a sample of review procedures employed in other federal agencies). Some review procedures in other federal agencies were formulated in response to specific legislative directives. Others were established pursuant to broader administrative and legislative mandates, such as the Government Performance and Results Act (GPRA) of 1995. Each of these review frameworks was adapted to the unique policy and organizational setting in which the referenced activities take place. It is important to note that there is no comprehensive structure for review that covers all federal programs, nor is there any single standard approach to review employed across federal agencies. One reason for this lack of a review standard is differences in agency structures, missions, and histories. For example, the nature of the Corps’ activities and of its planning studies and projects usually varies greatly from that of most other federal agencies. In other agencies, clearly defined and specific mission-related research programs and projects often are the subjects of review. For example, the U.S. Department of Energy, the U.S. Environmental Protection Agency, and the U.S. National Science Foundation employ review processes for scientific research programs or funding proposals that are typically more specific
and more focused than the Corps’ civil works program for water development.
Beyond U.S. agencies, many international organizations have developed review procedures. The World Bank, for example, commonly enlists “independent panels of experts” to review its project appraisal documents and to monitor project implementation. The World Bank also established an inspection panel that processes claims filed by people affected by projects, and it also independently reviews the extent to which World Bank policies and guidelines have been followed. Another example is the organization Transparency International, which has established review procedures across a variety of sectors, including public procurement, financial accounting, and voting procedures (http://www.transparency.org; last accessed July 9, 2002).
The World Commission on Dams (1998–2000), an independent body that reviewed the worldwide impacts of dams, developed guidelines for best practices for independent review. The following excerpt from its report (WCD, 2000) illustrates the Commission’s perspective on review and independent review panels (IRPs):
IRPs further provide a quality control function to assure the developer, regulator, financing agency and affected groups that the necessary standards are being met and that laws or guidelines are complied with.… They usually perform functions in the social and environmental domain similar to independent engineering inspection for technical issues…. The IRP is independent of all parties and its terms of reference should allow the panel to look into any issues deemed important without the need to justify such examination.
Corps of Engineers planning studies have unique features that merit consideration in the establishment of appropriate review procedures. Corps flood damage reduction, navigation enhancement, and other civil works projects often involve major economic and environmental consequences, and they may also entail significant social and cultural considerations. The potential for widespread controversy over the conclusions and recommendations of Corps planning studies is thus high in comparison to the potential for controversy in agencies where the focus tends to be primarily on research and development programs. This is not to say that useful review paradigms in other organizations cannot be used to inform the Corps’ review procedures, and the Corps should draw upon lessons from successful review programs in other organizations and federal agencies. Nonetheless, the Corps is a unique agency, and implementation of review procedures must be made with consideration of the Corps’ unique roles and projects.
This chapter discusses relevant criteria for evaluating alternative approaches to review. Several options for constructing a process for the Corps are then examined in light of these criteria. Findings from this examination form the basis for the recommendations presented in Chapter 6.
CRITERIA FOR EVALUATING ALTERNATIVES
Several different review processes could be formulated for review of Corps water resources project planning studies. Several different criteria must be considered in weighing these alternate approaches to review. The degree of independence from influence by the Corps of Engineers is a preeminent criterion, as it is strongly related to a review’s credibility. The process by which reviewers are nominated and selected is also important, as it will affect a review’s independence and credibility. The affiliation(s) of the group or individuals selecting the reviewers is a key issue, as are the affiliations and backgrounds of the reviewers themselves. Issues related to conflicts of interest and biases may arise in connection with review processes within federal agencies such as the Corps of Engineers, and care must be taken to minimize these concerns. There is also the challenge of selecting review panels that are viewed as credible and balanced, but that also have adequate knowledge of the Corps’ often highly complex planning guidance and analytical methods.
Independence of review begins with the nomination and selection of reviewers. Credibility of this process does not necessarily require that the selection process be totally divorced from the Corps for all decisions. For example, the Corps should be allowed to nominate panelists for an independent review panel—but it should not select them. In fact, some degree of participation by the Corps in the review will generally help increase the review’s usefulness, even in fully independent reviews. The Corps should help inform the review panel of a planning study’s key assumptions and methods, and it should discuss with the panel ways in which the panel’s findings might be most useful. The fact remains, however, that in large, controversial projects, a review’s credibility will be a function of the distance between the reviewer selection process and the Corps. To reiterate, the two most important considerations in establishing a review panel’s independence are (1) who selects the reviewers, and (2) who the reviewers are.
If the purpose is to improve the quality of Corps water resources project planning studies, the results of review will be more useful to the Corps before it prepares a final recommendation on a planning study. Comments from reviewers can be addressed before a final project rec-
ommendation is made public. If a review is primarily intended to provide to Congress and the public a fully independent judgment about a project proposed by the Corps, the review could be deferred until after the Corps’ recommendation.
In addition to informing Congress and the public, the review should also assist the Corps in the process of reaching its final recommendation. Results of review should thus be directed to the Corps—usually to the Chief of Engineers—before the Chief develops a final decision. This recommendation does not imply that results of a review should be provided confidentially to the Chief of Engineers. In fact, review panel reports should be made public and should be incorporated in the record of the project that is sent to the Office of Management and Budget (OMB) and Congress. The Chief of Engineers should also address each key finding or recommendation in the report, either by agreeing and stating what steps will be taken in response, or by disagreeing and rebutting the comments.
Reviews should not duplicate other review processes required by law or included within normal executive functions of the government. The Fish and Wildlife Coordination Act of 1934, the National Environmental Policy Act, and other statutes require external review of Corps projects by select parties. Comments from other agencies may identify the need for review to provide advice on particular issues, but the review should not duplicate studies of other agencies. In addition, OMB routinely reviews Corps planning studies for consistency with Administration goals and priorities before those studies are included in the budget proposed by the Administration.
The group that selects reviewers for Corps projects should be knowledgeable of the Corps’ mission, its statutory authorities and related administrative regulations, and other planning and evaluation procedures. The Corps operates within authorities and directives given to it by Congress, and it has a set of guidelines and regulations that provide a decision-making framework. That framework leaves considerable discretion to Corps staff as they execute the various steps in the planning process. Review panels should thus include, or have available as a resource, experts familiar with the guidance and regulations under which the Corps operates. To ensure that review panels have this knowledge, the group that selects reviews either should be familiar with the community of external water resources experts who have knowledge of the Corps’ decision-making and planning framework, or should be able to draw on individuals who can provide the needed expertise.
Reviewers may find themselves in disagreement with the results of Corps planning studies. These results may have been driven by specific
regulations or guidelines, or they may have resulted from staff exercising discretion within the regulations and guidelines. Reviewers should aim to draw distinctions between criticisms of the regulations and guidelines and criticisms of how well the Corps conformed to planning guidance.
Finally, any arrangement for implementing a review process should consider the implications for staff and supporting resources. Some reviews may entail a greater number of reviewers than others, some reviews might be conducted by videoconferencing or by mail, and some meetings might employ a professional facilitator. Some arrangements might entail a large staff with the full complement of skills necessary to review projects in detail, while others may be more selective in the aspects of decisions that will be reviewed.
As it moves to implement a more thorough and credible review process to meet contemporary and future water resources management challenges, the Corps should consider a wide variety of criteria and options. In creating an institutional mechanism to help facilitate a revised review process, the Corps should ensure that the following functions, responsibilities, and capabilities are established for the review process: recruit and maintain quality review panels, gather information from stakeholders, prepare high-quality draft feasibility studies in a timely fashion, arrange for external and internal reviews, receive the Corps’ responses to review recommendations, and follow up with inquiries regarding Corps actions based on review recommendations, where appropriate. Full coverage of these items will require a significant and sustained level of resources.
VARIATIONS ON INDEPENDENCE
The Corps has several institutional options for helping ensure independence of the processes for nominating and selecting reviewers. Examples of institutions that could conceivably assume some reviewer selection duties and that are completely external to the Corps include, but are not limited to, the National Academies, the National Academy of Public Administration, professional science and engineering societies, and independent federal oversight groups. Options more closely related to the Corps are the Office of Management and Budget (OMB), the Office of the Assistant Secretary of the Army for Civil Works (ASA(CW)), and the Office of the Chief of Engineers.
Options External to Corps
National Academies and National Academy of Public Administration
The National Research Council (the National Academies’ research arm) is frequently enlisted to conduct independent reviews on a range of controversial federal projects. The Upper Mississippi River-Illinois Waterway draft feasibility study and the Comprehensive Everglades Restoration Plan are two examples of studies for which NRC (through its Water Science and Technology Board) has reviewed water project plans.
Placing a permanent, independent review panel for the Corps within the National Academies and its National Research Council is not feasible, however, because such action would delegate a function of government to the National Academies, which would be inconsistent with the Academies’ mission and mandate. However, various NRC boards convene “standing committees.” These committees have rotating memberships and provide programmatic advice to a sponsoring agency. They operate under guidelines that apply to all NRC committees, convening meetings and providing advice through reports to agencies that fund their activities.
The National Academy of Public Administration (NAPA) is an independent organization dedicated to improving the performance of governance systems. It provides advice to public institutions, nonprofit organizations, and private companies that share in the implementation of public policy. The National Academy for Public Administration, an independent, nonprofit organization chartered by Congress, responds to specific requests from public agencies. This is a legitimate option in the independent review for the Corps, as the National Academy of Public Administration specializes in administrative and governance issues central to the execution of the Corps’ planning guidance, and it has also reviewed policies and administrative arrangements for natural resources management. The NAPA also assembles standing panels on specific topics.
Technical Societies and Professional Associations
Other alternative groups for overseeing independent reviews of Corps planning studies include professional engineering and scientific societies and associations. These groups, which include bodies such as the American Association for the Advancement of Science and the
American Society of Civil Engineers, meet a key criterion of being independent from the Corps of Engineers. Such professional groups, however, are unlikely to be able to effectively identify and assemble the breadth of expertise (e.g., engineering, economic, ecological, policy) required for comprehensive review of the Corps’ most complex planning studies.
In addition to these bodies, another option would be to establish an independent federal oversight group, an example of which is the Department of Energy’s Defense Nuclear Facility Safety Board, which provides advice on public health and safety issues at nuclear facilities (see http://www.dnfsb.gov; last accessed July 8, 2002).
Options Closer to the Corps
Several options for selecting reviewers and managing the review process are administratively more closely related to the Corps. These are the Office of Management and Budget (OMB), the Office of the Assistant Secretary of the Army for Civil Works (ASA(CW)), and the Office of the Chief of Engineers. Although these options are less independent than those described above, they better satisfy other criteria compared with other options within the executive branch of government. They likely represent a greater level of knowledge of Corps authorities and procedures and are likely to have greater continuity over time.
Office of Management and Budget
The OMB is currently part of the process for reviewing Corps projects, but the reviews usually do not take place until after projects have been finalized and reviewed by the Chief of Engineers. The OMB reviews a project for consistency with planning guidelines, for quality of the project as indicated by benefit-cost analysis and other criteria, and for its impact on the budget. External review could be organized and administered by OMB, but that review could duplicate its existing review, and it could inject OMB deeply into Corps planning. If OMB is involved in review prior to final review by the Chief of Engineers, it is not clear that it could serve as an impartial reviewer after the Chief of Engineers approves a planning study.
Assistant Secretary of the Army for Civil Works
The Office of the Assistant Secretary of the Army for Civil Works (ASA(CW)) represents an alternative organization for coordinating review. An advantage of this option is that, while moving the process away from the Corps’ direct management, reviews could be kept within the administrative agency that has authority over the Corps. Such an arrangement would make it possible to more easily incorporate review comments before a project is recommended to the OMB. A disadvantage of placing the review function in the Assistant Secretary’s Office is that the Office of the ASA(CW) may be too closely related to the Corps to provide truly independent review of Corps planning studies.
Another review option could be an intra-Corps review process where reviewers would be Corps personnel and the process would be administered within the Corps. This process would be similar to that used by the former Board of Engineers for Rivers and Harbors, although the staff requirements would not be as great those of the former Board. The primary advantage is that the review process would be easily managed and easily integrated into the Corps’ planning process. It would draw upon experts most knowledgeable of Corps projects and procedures (it could also consult with independent experts as it deemed appropriate).
There are at least two important shortcomings to such a structure: it would not be independent of the Corps, and it would fail to take advantage of expertise outside the Corps. Critics may argue that independence from the Corps is essential—and for controversial projects, independence is essential to establishing and maintaining credibility—but establishing an “intra-Corps” review process is an option for the Corps.
A variation on this option would be to have the Corps appoint a panel of reviewers from outside the Corps. The Corps would provide staff support, but the panel would operate independently of the Corps. This process would give the Corps the opportunity to identify experts from a variety of disciplines relevant to the mission of the Corps and its project purposes. It would be seen as being more independent of the Corps than the first option, but would still be affiliated with the Corps. Because the panel would be selected by and possibly paid by the Corps, it would not be a truly independent review body. For controversial projects that receive favorable comments by a review panel, project opponents would likely accuse the Corps of creating an improperly biased panel.
No option under the authority of a single organization can satisfy equally all the criteria that might be considered important in creating an effective review process. Options that represent a high level of independence from the Corps may not provide sufficient knowledge of Corps planning and decision-making processes and procedures, they may not provide sufficient continuity to the process, or they may not be willing or able to commit sufficient resources to support review activities of the Corps. Options closer to the Corps may not be sufficiently independent of the Corps to provide needed levels of independence and hence credibility.
An option that would best satisfy the multiple evaluation criteria is one administered by a small professional staff with expertise in Corps planning processes and procedures, but using external reviewers when appropriate. Such a review staff should hence be established and should examine all planning studies and projects as they are initiated to determine the appropriate level (current procedures, internal, or external) of review, administer the process by which reviewers are nominated and selected, and provide staff support for external reviews and dissemination of findings.
We term this group the Administrative Group for Project Review. Its structure, duties, and operation are discussed more fully in Chapter 6.