Analytical Methods and Approaches for Water Resources Project Planning (2004)

The U.S. Army Corps of Engineers has long constructed civil works projects to help manage the nation’s inland and coastal water systems. The Corps’ earliest water-related missions were to manage flood risks and enhance navigation. The U.S. Congress added several missions, including beach and hurricane protection, hydropower production, water supply, as well as ecosystem restoration (the latter responsibility was added in the 1990s), to the Corps work program over the years. In promoting these objectives, the agency constructed dams, levees, and other civil works to help reduce high river flows and flood damages, and to support inland and harbor navigation. Although the Corps has never been without its critics, through the 1960s the agency could generally rely upon widespread congressional and public support for its civil works projects. An area in which the Corps was not traditionally criticized was the quality of the agency’s planning and evaluation methods and the technical soundness of its planning studies.

But the planning context has changed greatly since the 1960s, with criticisms of the Corps’ basic objectives and of its planning methods becoming increasingly sharp. The agency has made concerted efforts to adapt to these criticisms, and the typical Corps of Engineers planning study today incorporates vast amounts of economics and engineering data, exhaustive environmental analyses, and is several hundreds pages long. These efforts toward increasing sophistication and thoroughness, however, have not silenced the Corps’ critics. In fact, the Corps’ efforts to improve and refine its planning and analytical techniques seem to have only invited additional scrutiny and criticism. To compound matters, the criticisms come at a time when the national interest in water development is in a state of flux. Moreover, engineering capabilities in the private sector have greatly increased, and the Corps no longer enjoys a once near-monopoly in civil engineering expertise.

This report was part of a larger study that was conducted in response to a request from the U.S. Congress in the Water Resources Develop-

ment Act of 2000 for the National Academy of Sciences to review the Corps’ peer review methods and analytical approaches (this report’s Foreword and Chapter 1 discuss these studies). This panel reviewed the Corps’ analytical procedures and planning methods, largely in the context of the federal Economic and Environmental Principles and Guidelines for Water and Related Land Resources Implementation Studies, also known as the Principles and Guidelines or “P and G”(P&G), as well as the Corps’ Planning Guidance Notebook (PGN). This panel’s statement of task was:

This report provides recommendations for improving the processes and methods outlined in the federal P&G and the Corps PGN. An observation noted during the course of this study is that substantial improvements to Corps planning methods will only be realized if they are linked to changes in broader, federal-level organizational and policy structures and processes that frame and guide those methods. Before presenting findings and recommendations regarding Corps planning and

evaluation methods, it is thus appropriate to review federal-level governance issues relevant to these methods.

The contemporary setting of U.S. water management features numerous well-informed and active stakeholders with water management objectives that are often mutually exclusive. In this setting, clear direction from the Administration and the Congress—which have ultimate responsibility for federal water project management decisions—regarding the nation’s de facto body of water policy, is essential. This clarification is especially important given that the Corps’ future work program is likely to center upon more efficient management of existing infrastructure—which will entail more trade-off decisions and less new project construction.

This is not to say that the Corps lacks federal-level guidance. On the contrary, the Corps is governed by 219 public laws (as of December 2000) that date back to the late nineteenth century. Executive orders, congressional committee language, and Administration guidance also guide the Corps. But internal inconsistencies within this body of guidance often pose problems for the Corps, as this body was not designed according to a master plan, but rather accreted over time with little regard given to whether new laws and directives were fully consistent with existing ones. For example, the Corps must abide by environmental legislation such as the National Environmental Policy Act, the Clean Water Act, and the Endangered Species Act. At the same time, however, the Corps is also mandated to provide services such as navigation channels. This results in conflicts when, for example, an endangered species’ habitat requirements (e.g., a flow regime that includes high flows, low flows, and occasional overbank floods) clash with an authorization(s) for a navigation channel of a given, reliable depth. Inadequate guidance on how to resolve these types of conflicts places the Corps in the position of setting policy, often resulting in confused policy direction, inefficient operations, and criticism of the agency from all corners.

1. To provide clearer direction to the Corps, the Administration and the Congress, in cooperation with the states, should reconcile inconsistencies within the existing, de facto, body of national water policy.

Resolving inconsistencies within the body of national water policy, and resolving inconsistencies among principles and methods employed by various federal water resources agencies, will require sustained attention, with controversial, high profile cases likely to require careful individual consideration. Given the large number of federal agencies (more than a dozen) with responsibilities for managing water and related resources, meaningful progress on this front will require a body specifically devoted to this task.

2. A body should be specifically charged to coordinate water resources policies and activities among the Administration, the Congress, the States, and federal agencies with water resources management responsibilities.

The Corps of Engineers’ key planning guidance documents are the federal Principles and Guidelines (1983) and the Corps’ Planning Guidance Notebook (2000), which provides guidance consistent with the P&G. The P&G document was approved in 1983 and is derived in part from the 1972 federal Principles and Standards. The Principles and Guidelines document, and its Principles and Standards predecessor, represented many relevant and useful economic and planning concepts. Nonetheless, the P&G is based in part upon water management and development paradigms of the 1970s (in some cases earlier). The P&G has not been revised for over twenty years, a period of substantial changes in planning approaches, improvements in analytical methods, and shifts in social preferences and scientific paradigms. The P&G document defines the current federal water and related land resources objective: “to contribute to national economic development, consistent with protecting the nation’s environment, pursuant to national environmental statutes, applicable executive orders, and other Federal planning requirements” (WRC, 1983). This “National Economic Development” objective is clear, and it

likely reflects the contemporary views of many congressional representatives and many of their constituents. Nonetheless, its viability has diminished in today’s context of more vigorous stakeholder participation and a broader set of demands from the nation’s river and coastal systems.

The P&G prescribes a set of detailed methods designed to quantify costs and benefits associated with a proposed water project. The P&G relies heavily upon predictive models and monetization techniques as the basis for water resources investment decisions. The theories and methods within the P&G, however, do not adequately reflect contemporary planning approaches or settings. The limits of models to provide precise predictions are today better understood and appreciated. Similarly, the limits of monetization techniques—which have realized some notable recent analytical advances—to accurately quantify all the important variables related to a water management decision are also better understood. Stakeholder groups, especially project co-sponsors, today demand a strong voice in important water management decisions. Adaptive management principles are being implemented in some places to help managers adjust to unknown and unforeseen anthropogenic impacts on ecosystem processes. The Corps recognizes these issues, but the P&G does not incorporate these types of considerations into the planning calculus.

The Corps naturally looks to credible sources to guide its planning decisions. This is especially the case given that the agency’s planning methods and decisions are carefully scrutinized, and that internally inconsistent federal legislation and other policy directives generally do not provide clear project management objectives. Corps planners use the P&G as an analytical touchstone to navigate through this context. But a quest for greater credibility through the development and application of increasingly detailed analytical methods has resulted in those methods becoming a product in themselves, rather than serving as a means to reach sound water management decisions.

3. The Principles and Guidelines should be revised to better reflect contemporary management paradigms, analytical methods, legislative directives, and social, economic, and political realities. The new planning guidance should apply to water resources implementation studies and similar evaluations carried out by all federal agencies. A revised version of the P&G document should be periodically and formally reviewed and updated.

No significant action has yet taken place within the Administration in response to this recommendation that has been voiced multiple times by previous groups.

4. Therefore, even if the Administration should choose not to revise the P&G, the Corps should draft a revision to its Planning Guidance Notebook that is consistent with this report’s recommendations and propose this revision to the Administration.

The new planning guidance should recognize uncertainties in planning and the limits of predictive models. It should reflect the reality that planning decisions are not purely analytical, that significant uncertainties often exist, and that they are influenced by factors such as stakeholder preferences. The reconciliation of some existing discrepancies among legislation and other guidance would provide clearer guidance for the Corps. This would allow for new planning guidance to be less prescriptive than the P&G, which would better accommodate stakeholder input and would encourage the creativity of Corps analysts. The new planning guidance should be periodically reviewed and updated.

A revised P&G document will not be a panacea for the Corps, as a revised document will eliminate neither differences between competing interests and values nor eliminate the analytical complexities of planning and managing large water projects. But because the Corps is obliged to follow the P&G, because the agency takes the document seriously, and because the P&G does not adequately reflect contemporary realities and best practices, its revision should prove useful to the Corps.

Pursuant to federal legislation and administration directives, a proposed water project is viable only if its projected benefits exceed projected costs as documented in a Corps planning study. Although this rule imposes discipline on the planning process, it often pressures analysts to make questionable assumptions or to configure a study such that it produces a given benefit-cost ratio. Benefit-cost analysis is an important component of sound decision making. But benefit-cost analysis may not adequately consider uncertainties and relevant public policy considerations such as stakeholder opinions, non-market values, and equity.

5. Benefit-cost analysis should not be used as the lone decision criterion in judging whether a proposed planning or management alternative in a Corps planning study should be approved. Benefit-cost analysis is a useful decision making guide, and it is important that

Corps planning studies retain the discipline imposed by comparing project benefits and costs. But it should be used only as one criterion in reaching final judgment on a proper alternative; criteria such as stakeholder opinions, political preferences, equity, and non-market values such as biodiversity are important factors in water resources investments and policy decisions that are not captured in benefit-cost analysis. Rather than being the product of lengthy analysis (as currently embodied within the P&G), benefit-cost analyses in future Corps studies should aim to be more concisely presented, with clear explanations of assumptions and models employed.

The Corps of Engineers does not systematically review outcomes of its water projects after construction or review how closely planning study projections matched actual results. A lack of retrospective, or “ex post,” reviews represents missed opportunities to better understand how demands upon water projects have changed over time, strengths and weaknesses of planning methods, and how project operations have (or have not) changed to meet changing conditions. The Corps has authorities that allow for post-construction evaluations to be conducted and for operations to be adjusted accordingly, but Congress generally has not provided resources to the Corps for comprehensive post-construction studies. Evaluations of water project outcomes, however, are essential to sound water resources management (the report from the 216 study Coordinating Committee provides more detailed advice on the role of study authorities in improving water planning decisions).

6. Periodic reviews of completed projects should be a routine part of Corps water project planning and management. Congress should provide resources to conduct these “ex post” evaluations.

The Corps of Engineers conducts its planning studies in two phases, a reconnaissance phase and a feasibility phase. The Corps’ internal

guidelines limit those reconnaissance studies to one year and to $100,000. A reconnaissance study is conducted to determine if a national interest exists in addressing a given water resources problem or opportunity. It also provides an opportunity to consider a broad range of planning alternatives. The given time and resource constraints may be reasonable for smaller, less expensive studies. Larger, more sophisticated studies, however, may require additional time and resources. Inadequate study reconnaissance in important projects may preclude the consideration of viable alternatives, which may create blind spots and contribute to conflicts during the feasibility study.

7. Resources and time allocated for Corps reconnaissance studies should be commensurate with the scale and complexity of the water resources issue at hand.

The Corps has been a leader in developing and implementing stakeholder participation procedures. Since the late 1980s, however, the Corps has been less systematic in incorporating stakeholder participation into planning and management decisions. Although stakeholder participation is required in all project planning, its design and conduct takes place at the Corps district (local) level, with only general standards grounded in legal requirements like those associated with the National Environmental Policy Act. This has resulted in inconsistency in stakeholder participation practices across projects and Corps offices.

8. The Corps should conduct a comprehensive review of district-level experiences with stakeholder participation procedures and activities. The Corps should also develop training and reference materials on stakeholder participation standards.

Corps of Engineers planning studies, and attending appendixes and other documents, are often hundreds of pages in length. This quantity of data often makes it difficult to identify and comprehend all important assumptions, alternatives, models employed, data sets, and other factors.

9. A summary document that identifies key environmental and social issues, primary assumptions, alternatives considered and evaluated, objectives sought, benefits and costs (monetized and non-

monetized), trade-offs and stakeholder perspectives and differences, presented with a consistent format across studies, should be a standard in Corps planning studies.

Three aspects of the Corps’ engineering analysis and methods bear close attention in the years ahead. The importance of these relates to changing paradigms of U.S. water resources management and to changing needs of Corps projects and activities. These are (1) systems engineering aspects of water resource planning, (2) impacts of risk and uncertainty on planning, and (3) integrating engineering methods of analysis with ecosystem restoration planning. Systems engineering focuses on interactions among project components, which may significantly amplify both benefits and costs. The analysis of risk and uncertainty in project planning illuminates the possible impacts of deviations from “best estimates” in projections of benefits and costs. The development of engineering methods of analysis for ecosystem restoration provides a way to integrate ecological components within more traditional Corps planning analysis and approaches.

10. The Corps should strengthen its programs in the areas of systems engineering aspects of water resources, risk and uncertainty analysis, and the integration of engineering and ecosystem analyses. Part of this strengthening should include the development of updated design manuals that better reflect contemporary methods and theories. These manuals should be used as general guidance rather than as “cookbooks” that specify a series of steps that must be strictly adhered to.

Water management theories, practices, and methods are constantly being tested and advanced in many different disciplines, in a variety of institutions, and in nations outside the United States. The resources required to employ experts that can stay abreast with these advances in all relevant disciplines and specialties, however, transcends the budgets of most federal agencies. Nonetheless, it is important that the Corps be familiar with current thinking and practices. An alternative to employing a

suite of experts in every Corps District office is to periodically enlist independent experts to review and comment upon programs and on planning studies. Given the level of sophistication in Corps planning studies today, the participation of independent experts is often useful in ensuring that methods employed are consistent with current and credible thinking and practice. In today’s planning environment, independent, expert advice is also essential for credibility. Other longer-term, potentially useful means for infusing ideas into the Corps are through the existing “visiting scholars” programs at the Corps Institute for Water Resources and the revived “Associates Program” within the Corps.

11. Independent experts from outside the Corps of Engineers should be routinely enlisted to provide advice in Corps programs and planning studies.