National Academies Press: OpenBook
« Previous: 3 ENVIRONMENTAL ISSUES
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 114
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 115
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 116
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 117
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 118
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 119
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 120
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 121
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 122
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 123
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 124
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 125
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 126
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 127
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 128
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 129
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 130
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 131
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 132
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 133
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 134
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 135
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 136
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 137
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 138
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 139
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 140
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 141
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 142
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 143
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 144
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 145
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 146
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 147
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 148
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 149
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 150
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 151
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 152
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 153
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 154
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 155
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 156
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 157
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 158
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 159
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 160
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 161
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 162
Suggested Citation:"4 RISK METHODOLOGY." National Research Council. 1995. Flood Risk Management and the American River Basin: An Evaluation. Washington, DC: The National Academies Press. doi: 10.17226/4969.
×
Page 163

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

4 Risk Methodology The U.S. Army Corps of Engineers (USAGE) has adopted new risk and uncertainty analysis procedures for project evaluation that explicitly include un- certainties in the hydrology, hydraulics, and economics of a planning study (USAGE, EC 1105-2-205, 1994) (hereafter referred to as EC 1105-2-2051. This procedure represents an extension of the traditional paradigm for flood control project planning and community flood protection evaluation. USACE observed that the new risk and uncertainty methodology is similar to present practice but differs in that uncertainty is explicitly quantified and integrated into the analysis (USAGE, EC 1 105-2-205, 1994). The 1994 Alternatives Report (USAGE, Sacramento District, 1994a) indi- cated that USAGE's analysis now considers "varying degrees of uncertainty in the causes of flooding, such as inflow to Folsom Reservoir, regulated outflow- frequency relationships for Folsom Dam, river stages, and levee stability." The methodology computes the risk of flooding due to combinations of hydrologic events, hydrologic parameter uncertainty, uncertainty in stage-discharge rela- tions, and levee performance. This change in methodology is important to the American River Watershed Investigation (ARWI) because the ongoing evaluation of flood control alterna- tives for the basin by the Sacramento District is one of the first applications of the approach, and almost certainly the most complex application yet attempted by USACE. The risk and uncertainty methodology specifically addresses many assumptions in the 1991 ARWI that were subject to controversy, and which the committee was charged to review. Whether the controversy will be resolved remains to be seen. 114

RISK METHODOLOGY 115 In particular, assumptions about levee freeboard for American River basin levees are replaced by a distribution on the stage at which levees fail. Likewise, hydrologic uncertainty that was described by an expected probability adjustment, and assumptions about delays between the beginning of the flood and increased releases, are now described by explicit probability distributions. Issues that were in contention have not disappeared; what some viewed as conservative values have been replaced by probability distributions, which may also be contested. For decades, civil engineers have realized that it is not practical to protect communities in the floodplain from all conceivable floods (Foster, 1924; Riggs, 1966~. Such protection measures would be prohibitively expensive, even if they were practicable. Communities and individuals who choose to locate in flood- prone areas will generally be exposed to some risk of flooding. However, it is often economically advantageous to provide protection from flood events that have a 1 in 50, 1 in 100, or a 1 in 500 chance of occurring in any year, depending on the value of the property at risk, the chance of loss of life, and the costs of flood risk reduction opportunities. Derivation of probability distributions to describe the possible magnitude of flood flows has been a practice in civil engi- neering since the early part of the century. They provide a description of hydro- logic risk. When a particular flood flow with a 1 in T chance of being exceeded in any year serves as a design flood for a project, USACE has said that the project provides a T-year "level of protection." The new USACE risk and uncertainty methodology explicitly introduces into the planning process consideration of hydrologic, hydraulic, and economic uncertainty. Before, the USACE planning procedure selected a level of protec- tion corresponding to perhaps the 1 in 250 chance event (often called the 250- year flood), and then determined the corresponding design flood flow. Use of an expected probability correction did incorporate hydrologic uncertainty into flood risk estimates (Beard, 1960, 1978; Stedinger, 1983a). Alternative hydraulic flood control structures including levees, flood storage capacity in dams, and channel improvements, in addition to flood-proofing efforts, were selected to control a flood of that magnitude. In the evaluation of flood control projects, there are a number of uncertain- ties that make it difficult to determine whether a specified flood can be passed safely. For example, flood control dams might have surcharge capacity that was not included in the flood routing calculations. Levees are a more common con- cern. Levee failure depends on factors such as the structural integrity of the levee embankment, possible scour and undercutting, variation in the state of levee repair, and other factors in addition to high water levels. Hydraulic predictions of the flood stage associated with different flow rates may also be in error. Levee failure stage predictions and stage-discharge relationships are affected by survey- ing inaccuracies in the measurements of channel geometry and riverbed eleva- tions, errors in estimation of flow resistance, simplifications in hydraulic routing

6 FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN calculations, waves and wave effects, and possible settling of levees that affect crest elevation. Risk-based analysis of hydrologic and hydraulic engineering problems has been and is an active area of research (Davis et al., 1972; Tung and Mays, 1981; Ha~mes and Stakhiv, 1986, 1989, 1990; Duckstein et al., 1987; USACE, 1992a,b; Haimes et al., 1993; Taylor et al., 1993~. In most risk analysis applications, the risks of concern arise from the distributions of annual flood peaks, rainfall depths, and other hydrologic variables (Mays and Tung, 1992~. In a few cases, project performance is described probabilistically (Duckstein and Bernier, 1986; Chow et al., 1988, section 13.4; Mays and Tung, 1992~. Uncertainty in structure perfor- mance was important in several studies addressing dam rehabilitation and dam safety (McCann et al., 1985; Goicoechea et al., 1987; Von Thun, 1987; Stedinger et al., 1989; Bowles, 1990; see also NRC, 19851. There are relatively few applications where risk analyses have considered the natural variability in hydrologic and hydraulic variables as well as the uncer- tainty in the parameters of fitted flood-flow frequency curves and calculated stage-discharge relationships, and in economic quantities; these analyses might best be described as risk and uncertainty analyses to make the distinction clear. The Bayesian~ framework that is appropriate for hydrologic uncertainty has been employed in proposals to include hydrologic parameter uncertainty in planning studies (Benjamin and Cornell, 1970; Duckstein et al., 1975; Vicens et al., 1975; Wood, 1978; Stedinger, 1983a). The USACE use of expected probability adjust- ments is one way to include parameter uncertainty in flood control project evalu- ation. RISK AND UNCERTAINTY: A PRIMER Uncertainties, Safety Factors, and the Meaning of Level of Protection USACE traditionally has included safety factors in its design of facilities and the specification of operating policies to address important hydraulic uncertain- ties in flood control planning calculations. Surcharge storage in reservoirs might be one safety factor. For levees, engineers have required that the design flood The statistical literature includes several methods for dealing with parameter estimation, statisti- cal inference, and decisionmaking. Bayesian statistical methods treat unknown statistical parameters (the population mean, population variance, or a probability or quartile) as random variables whose probability distributions reflect the degree to which the value of a parameter can be resolved from available sample information as well as prior beliefs and other sources of information. With the traditional statistical procedures employing standard confidence interval estimators and classical hypothesis tests, such parameters are treated as if they have fixed (but unknown) values, and prob- ability distributions describing the sample-to-sample variability of sample statistics and parameter estimators are the focus of the analyses. The topic is addressed in more detail in the text.

RISK METHODOLOGY 117 pass through the levee system with some specified freeboard. Such a safety factor enables the engineer and the planning agency to be confident that in an actual flood event approximating the design storm, there will be sufficient chan- nel capacity to pass that flood flow without the levees failing from overtopping or excessive stages. In planning studies, encroachment within levee freeboard might be treated as sufficient to cause levee failure, even though in an actual flood failure might not necessarily occur at that stage. From an economic perspective, one can ask how much freeboard is justified economically to increase project reliability (Davis, 1991 J. The practice of including freeboard in design suggests that at the design flood associated with a target probability, called the "level of protection," there will often be some residual safety factor before actual flooding would occur. If there is, then the true chance of levee failure resulting in major flooding is less than the specified target probability. The question arises as to what was meant by the traditional "level of protection." Should it have been viewed as (1) an esti- mate of the chance of flooding due to levee overtopping or breaching, or was it simply (2) the exceedance probability of the design flood that a reservoir and levee system was designed to pass with some safety margin? Generally, evacuation plans would begin before a levee breached or was overtopped. Thus the "level of protection" could be viewed as the probability that the design event would be exceeded and thus that emergency measures would be required, even though widespread flooding might not occur. What seems clear is that there is confusion on this issue. Although calcu- lated levels of protection might appear to address (2) above, their use to estimate expected damages suggests that they are often used as an answer to (1~. This has led to the conventional wisdom that USACE projects provide more protection than acknowledged because safety factors built into levee design and reservoir operating policies appear to add an additional increment of safety. If this conven- tional wisdom is true, then by lowering the apparent benefit-cost ratios this prac- tice may have worked against some proposals to provide needed flood protection. For example, if levees can almost always pass flood flows that encroach within the specified design freeboard, they actually provide protection from larger floods than has been assumed in many analyses. However, the inclusion of safety factors in reservoir-levee system design to compensate for hydraulic uncertainty may not be sufficient to actually decrease the risk of levee system failure or levee overtopping. If levee settlement in one location ensures that a levee system failure will occur before the design flood event is reached, excess channel capacity or extra freeboard at other locations will not improve system reliability. In a levee system, failure occurs at the weakest point. However, if in a flood event a reservoir operator can vary releases in response to actual developments within the channel-levee system, it is possible that variation in reservoir operations taking advantage of excess surcharge stor

8 FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN age could avoid levee system failures due to other deficiencies within some range of hydrologic loading. Planners and engineers also realize that the condition of levees and some equipment degrades with time. Safety factors are a reasonable way for designers of flood control works to ensure that over time a system can continue to pass the design flood without levee overtopping or breaching. However, it may not be immediately clear how safety factors included in different components of a reser- voir-channel-levee system interact to affect overall system reliability. Definitions for Risk and Uncertainty USACE will be wrestling for some time with the implementation of its new risk and uncertainty methodology. Of concern will be both a consistent scientific methodology, and a vocabulary and style for the presentation of the results to technical audiences and the public. The choice of words is very important be- cause they help us distinguish one concept or idea from another. In this regard, the terms "risk" and "uncertainty" can cause problems because different authors have ascribed to them significantly different ideas. Risk has been used to convey each of the following meanings (USAGE, 1992a, pp. 10-1 1~: 1. The idea of hazard, when something is described as being "at risk." 2. The expected losses or risk related to a venture. 3. The probability of some outcome, such as the risk that a levee will be overtopped. All three definitions attribute to risk a probabilistic character related to the possibility of an adverse and unwanted event in a particular system. Risk may be due solely to physical phenomenon or to the interaction between man-made systems and natural events. The tea uncertainty has been given a broad and sometimes conflicting range of meanings. There is a literature wherein the term uncertainty is used to describe events for which objective probabilities are not available (USAGE, 1992a). On the other hand, it could simply to be used to describe situations that are not certain; USACE (1992a) stated that "uncertainty means simply the lack of certainty. It is the reality of inadequate information. When information is imprecise or absent, that is uncertainty." The USAGE's guidelines provide the following operational definitions of risk and uncertainty (USAGE, 1992a, p. 123: Risk: The potential for realization of unwanted, adverse consequences; estima tion of risk is usually based on the expected result of the conditional probability of the occurrence of event multiplied by the consequence of the event, given that it has occurred.

RISK METHODOLOGY Uncertainty: Uncertain situations are those in which the probability of potential outcomes and their results cannot be described by objectively known probabili- ty distributions, or the outcomes themselves, or the results of those outcomes are indeterminate. Those guidelines indicate that actual uncertain planning situations are lo- cated on a continuum between situations of known risk (where the probability distributions of interest are well specified) and situations characterized by uncer- tainty (where those distributions are hardly specified at all; USACE, 1992a'. 119 A Distinction Between Risk and Uncertainty Although the cited distinctions between risk and uncertainty are some times useful, they are not the distinctions that are needed for our discussion of the USACE methodology for risk and uncertainty analysis. Of particular concern here with regard to the USACE risk and uncertainty methodology are: · models of natural and operational variability and randomness, including probability distributions describing flood flows, event-to-event variability in stage-discharge relationships and reservoir operations, and variability in flood damages due to factors not captured by flood stage, and . uncertainty representing limited understanding of system processes and the lack of accuracy with which the parameters in models describing natural processes can be specified, including the parameters of a probability distribution, the cross-sections used to derive a stage-discharge curve, and the value and the count of the number of dwellings in a protected portion of the floodplain. In some cases one may be uncertain as to which of several competing models to employ, such as alternative probability distributions. Uncertainty refers to our lack of understanding of characteristics of nature that we conceptualize as being fixed at any given time. Ideally, the values of various model parameters could be determined. However, due to data limitations there are generally residual errors in our understanding of those characteristics of nature that cannot be eliminated with reasonable levels of effort. The first situation is referred to here as natural variability or randomness in the indicated process. The second situation is referred to as "specification error," or simply uncertainty. This use of uncertainty to describe lack of knowledge is not strictly consistent with the operational definition for the term suggested in USACE (1992a), although it may be consistent with the way the term is used. This use is consistent with the definitions adopted by other groups (IS O TAG 4, 1993; Taylor and Kuyatt, 1993; NRC, 1994~.

120 FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN Sources of Uncertainty Recently, in a report on risk assessment of hazardous air pollutants, the National Research Council (NRC, 1994) recommended making a clear distinc- tion between parameter uncertainty, which is associated with the parameters of a particular model, and uncertainty as to the appropriate model, or model uncer- tainty. The report noted that parameter uncertainty often is described by continu- ous parameter ranges (NRC, 1994) that result in corresponding uncertainty inter- vals associated with predictions; however the choice among competing health risk models generally corresponds to distinct and mutually exclusive choices. The authors observed that "indiscriminately" combining the two types of uncer- tainty in health risk assessment could result in the calculation of average health risks and uncertainty ranges that are inconsistent with any of the alternative models. The report recommended that parameter uncertainty be evaluated sepa- rately for each competing model. Hydrologists face similar issues when choos- ing between alternative flood flow probability distributions or between methods for calculation of stage-discharge relations. Hydrologists often consider what can be classified as a third type of uncer- tainty, which arises due to model imprecision, or model prediction error. Thus, even with the best parameters, operational hydrologic models may fail to pre- cisely predict flood stages at some locations in a system; such model prediction errors are another source of uncertainty in the analysis of flood projects. The error here is not due to natural variability, which might be best described explic- itly, or to a failure to have the best set of model parameters, which is described by model parameter uncertainty, but is instead due to lack of model accuracy and thus is a source of uncertainty associated with model predictions. Such predic- tion errors can be thought of as a type of model uncertainty, because if one had a more accurate model, such errors might be eliminated. However, better models in most cases would have greater data requirements, requiring a finer spatial description of channel cross-sections and roughness coefficients with fewer lumped representations of watershed and channel characteristics. In fact, most operational models deliberately employ simplifications and lumped representa- tions of natural processes to restrict the parameter space to a manageable dimen- sion so that available data are sufficient for model calibration. Thus uncertainty due to model prediction error often reflects both data/parameter limitations and model uncertainty. In this discussion, model prediction error is included with other parameter and model uncertainties. A FRAMEWORK FOR RISK AND UNCERTAINTY ANALYSES A framework is needed to understand the structure of risk and uncertainty analysis efforts for flood protection project evaluation and to understand the relative roles of the natural variability of flood volumes, reservoir operations,

RISK METHODOLOGY / Q _ / In '~ ~ ~ 3 / i a ~\ Reservoir Outflow 121 - S(O) :> Failure /PL(S) D(S) us no a River Stage Possible values Possible reservoir Outflow peak 0 Probability levee Stage alone used to of annual flood operation determines determines breaches is estimate damages D. flows Q . outflow peak O for downstream stage S determined by Dotted line is without any inflow peak Q. at points of interest. stage S. levee overtopping or breach. FIGURE 4.1 Deterministic and stochastic processes contributing to flood risk. Perfor- mance of a flood control system involving both reservoirs and downstream levees can depend on deterministic and stochastic components. Possible values of the inflow peak for any year are described by a tree with branches, as are reservoir operations during that event, because both are described as random processes. The transformation of the out- flow peals O to downstream stage 5 is described by a deterministic relationship, though there is uncertainty associated with parameters of that relationship. Likewise, damages are described as a deterministic function of river stage for the levee breach/overtopping case, and the case without levee failure or over topping. Levee failure is probabilistic and occurs with a probability pit which depends on the stage S. hydraulic system performance, stage-discharge errors, and uncertainty in hydro- logic, hydraulic, and economic parameters. Figure 4.1 provides a conceptual model for describing hydrologic risk, variation in reservoir operations, use of a river stage-reservoir outflow relationship, levee reliability, and finally estimates of the economic damages that would result should a levee fail. Several of these relationships are stochastic, while others are described by deterministic relation- ships. The committee developed the event tree in Figure 4.1 to describe how the volume distribution of the largest flood volume in a year is transformed first into a river stage distribution and eventually into a damage distribution. This event tree can be used to evaluate the probability that flood protection works are over- whelmed and flooding occurs at some damage site, called the annual failure probability (AFP). Likewise, it can serve as the basis for calculating the expected annual damages (EAD), which would be the foundation of the economic evalua- tion of project performance. In Figure 4.1, a process is modeled either as being deterministic or as having some random component reflecting natural process variability. To understand the impact of specification errors or uncertainty in parameters of the selected discharge-frequency model or in economic parameters, it is useful to note that for each step in Figure 4.1 there is a set of parameters that define the relationship or model employed at that step. For example, in the first step the flood flow frequency relation requires specification of the parameters of that distribution;

22 FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN these are often taken to be the mean, variance, and skewness coefficient of the logarithms of the flows. Likewise, the variability in reservoir operation at the second step will be described by some selected probability distribution, which will also have parameters. In this presentation, uncertainty analysis focuses on the parameters of the selected models; those models are assumed to accurately reflect the probability distributions of processes that are variable (such as the largest inflow in a year and the actual timing of reservoir releases in a future flood event) and of deterministic processes such as the stage associated with different channel flow, if only the correct or best values of the models' parameters could be determined. When the problem is structured as it is in Figure 4.1, one can identify the parameters of each of the models that determine the numbers that enter into calculation of risk and expected damages. One might then ask, how well or how precisely are those model parameters defined? Or, how uncertain are values of the project performance criteria AFP and EAD owing to the uncertainty or speci- fication errors in various parameters? There are several sources of variability in the economic damages that will be experienced in any year. Extreme variability results from the magnitude of the floods that may occur. Less important but still significant variability is intro- duced by flood hydrograph timing and shape, variations in reservoir operations, possible levee failure stage, and differences in the actual damages that would occur to a structure depending on the duration of flooding, wave attack, and differences in warning times; the effects of these factors are not captured by the specification of stage alone. Planners understand that this variability exists and so base their plans on AFP and EAD, which reflect the decision to average over the probability distributions describing annual maximum flood volumes and other variable processes. In structuring the problem, as has been done in Figure 4.1, engineers can also clarify how the various processes are thought to work. For example, the stage- discharge relation can be conceived of as being time-invariant or deterministic, so that a specific stage always corresponds to the same discharge. Then the relevant uncertainty would pertain to the precise functional relation between discharge and stage. Alternatively, there are certain stream reaches where the stage-discharge relation varies significantly over time because of channel changes, sediment movement, or the stages of tributaries or other streams with which the river of interest merges. Such stage-discharge relations hence might best be described by some random process. While in this second case the stage-dis- charge relation might best be described as a source of variation, there would still be uncertainty as to the best values of the parameters that describe that process. Economic damages depend on several factors, and some are deterministic while others are random. In particular, actual flood damages vary depending on flood duration, the presence of ice and sediment, wave action, and warning time. Flood damage uncertainties related to the number, types, and value of structures

RISK METHODOLOGY 123 in flood-prone areas would not change much from year to year, unless a major flood occurs. The source and nature of variability and uncertainty in levee performance present similar issues. USACE needs a clear framework for its risk and uncertainty calculations to be able to articulate and explain its treatment of such issues. Even so, it will not always be clear what should be described as variability and what to represent as uncertainty. Including Uncertainty in the Analysis Planners should know by how much the estimates of AFP and EAD might be in error. For example, a flood-frequency curve is based on a limited flood record. By how much might the parameters of the discharge frequency function be in error, and how big a change in AFP and EAD would result? Likewise, in deter- mining the stage-discharge relationship, a limited amount of effort goes into the surveying and the description of river cross-sections, geometry, and roughness coefficients: the hydraulic model has a limited amount of detail. What errors might this introduce into the evaluation of AFP and EAD? Similarly, limited effort is devoted to determining the value of property at risk in flood-prone areas. Additional effort could refine the data base describing the property at risk. Given a statistical description of the likely specification errors in economic and struc- tural survey data, a planner could quantify the magnitude of the corresponding errors in AFP and EAD. These questions can be addressed by sensitivity analysis procedures. The document Guidelines for Risk and Uncertainly Analysis in Water Resources Plan- ning (USAGE, 1992a), developed by the USACE Institute for Water Resources, defined sensitivity analysis as the technique of varying assumptions to examine the effects of alternative as- sumptions on plan formulation, evaluation and selection. This can include variation of model parameters as variation of benefit, cost and safety parame- ters. One of the important uses of sensitivity analysis is to investigate how different values of certain critical assumptions and parameters could result in changing the choice of the selected project and report recommendations. Sensi- tivity analysis is the systematic evaluation of the impacts on project formulation and justification resulting from changes in key assumptions underlying the anal- ys~s. Sensitivity analysis can be used to bracket forecasts, parameters, benefit and cost estimates, and other factors for which a range of values can be expected to occur. Generally, each model or process parameter is varied, one at a time, and the result observed (USAGE, 1992a'. However, there are often so many parameters in the models employed to evaluate flood protection projects that it would be difficult to integrate such one-at-a-time evaluations, or to decide how they should be incorporated into decisions (Moser, 1994~.

24 FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN Describing Uncertainty Useful descriptions of uncertainty can be developed by describing the speci- fication errors or uncertainty in various economic parameters by probability dis- tributions. This must be done with care so that the resultant distributions truly reflect the probabilities planners should ascribe to the various parameters given the sample information at their disposal, general information they have about the processes of interest, and what is reasonable for the location in question. Then, using those probability distributions over the uncertain parameters, a statistical description of the uncertainty in AFP, EAD, and other performance criteria can be computed. For the purpose of developing a more mathematically precise notation for describing uncertainty, denote a possible set of model parameters for the event tree in Figure 4.1 by co. If the event tree in Figure 4.1 is evaluated with param- eters ce, let the resulting values of AFP and EAD be denoted AFP(co) and EAD(co). One can then ask what statistics should be calculated for the purposes of planning and project evaluation. A reasonable and simple procedure would be for planners and engineers to select their best estimate of ce, denoted here as Obese, and employ the value of AFP and EAD calculated with that best estimate: AFP(cubes~) and EAD(Cobes~) This is what is done in many studies. It is generally satisfactory when model parameter uncertainty is small. Alternatively, if a probabilistic description has been developed to describe the likelihood of different values of is, a different method could be employed. Just as EAD(co) is obtained by averaging over the probability distribution for annual floods, one could average the values EAD(co) over the probability distri- bution for co. The resulting descriptions of average flood risk and average eco- nomic losses are the average annual failure probability (denoted Avg~AFP]), and the average expected annual damages (denoted AvgtEAD]), where Avg{AFP] = Lover `,, {AFP(~) } Avg[EAD] = Eover i,, ~ EAD(co) ~ and where E denotes expectation over the indicated variable. The choice between AFP(cebes~) and Avg~AFP] and between EAD(cobes~) and AvgtEAD] reflects a philosophical choice in planning. The choice should also reflect how well plan- ners believe the available distribution for ~ has been specified. Even if Obese is simply the average value of ce, because of the nonlinear relationship between a probability distribution's parameters and exceedance probabilities, there will gen- erally be a difference between the two descriptions of AFP and EAD. Whether one uses average values of AFP and EAD or uses values of AFP

RISK METHODOLOGY 125 and EAD computed using CObes~,, those single values should be augmented with a description of their uncertainty, which results from specification errors in the model parameters co. One of the major contributions of risk and uncertainty analyses is the quantification of specification errors and other uncertainties, the evaluation of the resultant uncertainty in predictions and estimates of benefits and costs, and quantification of the value of additional information (Morgan and Henrion, 1990; NRC, 1994, pp. 160-61; 184-85~. A description of the uncertainty in AFP and EAD can be computed by a Monte Carlo sampling procedure using the distribution of ce to determine the distributions of AFP(co) and EAD(co), or just the standard deviations. The distri- butions of AFP and EAD can be used to determine the distribution of the benefit- cost ratio (BCR), the probability that the national development objective is less than zero for a particular project alternative, or the probability that BCR is less than one. (Such calculations and ideas were illustrated in USACE, 1992b and NRC, 1994,p.180.) ESTIMATION OF FLOOD DAMAGE INCORPORATING HYDROLOGIC UNCERTAINTY Congress asked the committee to consider the issue of expected probability. An expected probability adjustment to flood frequency curves is a method that has been employed by USACE to incorporate hydrologic uncertainty into flood risk assessments (Beard, 196O, 1978~. With the new risk-based planning method- ology being employed by USACE that correction is no longer made explicitly. However, by including hydrologic uncertainty in its Monte Carlo evaluation of expected flood damages, USACE has implicitly introduced hydrologic parameter uncertainty (frequency-curve parameter-specification error) into the flood risk and expected damage calculations. Adding discharge-quartile uncertainty into the Monte Carlo evaluation of flood damages corresponds to what has been called an "expected damages" approach (Arnell, 1989), as opposed to the "ex- pected probability" correction that Beard (1960, 1978, 1990) advocated. In the framework described above, the choice would be between the use of planning criteria such as AFP(cobes~) and EAD(cobes~), which use "best" available estimates of the parameters, and AvgLAFP1 and AvgtEAD], which incorporate parameter uncertainty into the estimates of those planning criteria. Reasonable arguments suggest that use of Avg[AFP1 and AvgtEAD] should be entirely ap- propriate. However, this is true only if uncertainty in the various parameters is described well. The analysis in this section shows that use of classical statistical ideas to incorporate uncertainty into the evaluation of AvgLAFP] results in a biased exceedance probability estimator and biased estimators of flood damages. Thus the use of the classical approach is not recommended.

126 FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN An Example to Consider the Estimation of Flood Damages Consider the statistical performance of flood damage estimators for a rela- tively simple situation to illustrate the consequences of including hydrologic and other specification errors and uncertainties in the evaluation of flood damages. In particular, consider flood peaks Q whose logarithms X = logy) are normally distributed with mean ~ and standard deviation 6. This corresponds to a log normal distribution for flood peaks Q. which is a special case of the Water Resources Council log-Pearson type 3 distribution, with a log-space skewness of zero (IACWD, 1982~. Let M be the sample mean and S the sample standard deviation of the avail- able systematic record of the logarithms ~Xi) of gauged flood flows Hi). Let q be a discharge of interest, possibly the discharge necessary to inundate significant buildings or overtop an existing levee designed to protect property. Then, fol- lowing the Water Resources Council's (WRC's) Bulletin-17B, an estimator of the true but unknown cumulative (non-exceedance) probability p~q) associated with q is (IACWD, 1982) p(q) = ~(logfq)-M)/S) where ~ is the cumulative distribution function of the standard normal distribu tion. If the flood damages corresponding to levee failure or overtopping are zero for flood peaks less than the fixed flow q, and have constant value D for flood peaks greater than or equal to q, then the true but unknown expected damages ED are ED = E{Damages} =DP[Q > q] = D[1 - 4) {log(q) - ~)/6}] where ~ and ~ are the true but unknown mean and standard derivations of flood flow logarithms Xi. For a model with fixed damages when a levee fails or is overtopped, the conventional estimator of flood damages is Estimator-of-ED = D [ l - p (q)] = D [ l - ~ ~ (l og fq) - M)/S } ~ This is a very simple description of the nature of flood damages, and is used in the investigation below. In a river system like the American, the flood stage in protected areas after a levee fails may approach that in the river, and the water level beyond the levees could continue to rise if the river continues to rise. The simpler model of damages employed above illustrates the significant relationship between the estimated probability that various stages are exceeded and the esti- mated damages that would be computed.

RISK METHODOLOGY 127 The analysis above was based on the assumption that q is fixed and indepen- dent of the sample statistics M and 5. Beard (1990) was critical of this assump- tion. Indeed, the value of q may be affected by floods that have occurred: when siting a building, the owner might have knowledge of the flood peaks that had occurred before that time. In the case of the American River, the historical levee system and Folsom Dam have provided protection from natural flows for the last three decades, so there should be little relationship between the events in the flood record and recent construction. Of course, older construction could not have anticipated the magnitude of subsequent flood peaks. Thus it is reasonable to ignore the possible interaction between the magnitude of events in the flood record and the location and value of property in the floodplain. Beard proposed another model for flood damages that would place the prop- erty at risk at a stage corresponding to a flow M + tS for some fixed scalar t (Beard, 1990J. Thus the location of valuable property would be determined completely by the sample mean M and sample standard deviation 5 of the loga- rithms of the flood record that would be available when a study was performed. This is clearly an impossibility for older property and represents for newer prop- erty unusual social responsiveness to revealed flood hazard. In general, it is not a credible basis for a flood damage model. Analysis Without Hydrologic Uncertainty From both risk and economic loss points of view, the accuracy of any estima- tor of the cumulative probability p~qJ associated with a fixed critical flow q are of great importance to the accuracy of the calculation of expected damages. For the simple damage model discussed above, the expected damages are simply Dt1 - p~q)~. Unfortunately, many estimators are biased, which means that their ex- pected values (or average values over many samples) do not equal the target value. The difference between the expected value and the target value is called the "bias" of the estimator. A Monte Carlo experiment was conducted to evaluate the expected value of t1 - pi for flood records of length n = 10, 25, 50, 100, and 200, with q ranging from the 10 percent to the 0.1 percent chance exceedance events, denoted q0 ~ and q0 00~, respectively. The results in Table 4.1 are based on 1000 generated samples yielding different values of M and S. In this instance, the results do not depend on the assumed parameters for the normal distribution describing the logarithms of the flood flows. One can see that there is relatively little bias in the estimators of the exceedance probabilities of q0 ~ up to q0 0~ when n 2 50. For q0 002 and q0 00~ the bias is more severe, particularly for small n. Analysis Incorporating Hydrologic Uncertainty The new USACE risk analysis procedure proposes to include hydrologic

28 FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN TABLE 4.1 Average Value of Estimated Exceedance Probability t1 - pi for Specified Critical Flow q Using Sample Mean and Variance from Normal Samples of Size n Without Adjustment for Hydrologic Uncertainty Critical Flow n go. 1 qO.04 qO.02 qO.O 1 qO.002 qO.OO 1 10 0.11 0.048 0.028 0.016 0.0055 0.0035 25 0.10 0.044 0.023 0.013 0.0034 0.0020 50 0.10 0.042 0.022 0.011 0.0027 0.0015 100 0.10 0.041 0.021 0.011 0.0024 0.0012 200 0.10 0.041 0.021 0.010 0.0022 0.0011 uncertainty associated with the sample estimates M and S into calculation of expected annual benefits to reflect these model-parameter specification errors. If this is done correctly without approximations, it is equivalent to the expected probability adjustment that was previously employed. (Arnell (1989) analyzed both cases.) With the expected probability adjustment, the probability distribu- tion describing the distribution of floods X is a Student t distribution with location M, scale parameter S(1 + 1/n)05, and degrees of freedom n-1. This expected probability model has been employed because the ratio (X-M) / t(1 + 1/n)05 S1 has a Student t distribution when X, M, and S are all considered to be random (Moran, 1957; Beard, 1960; Stedinger, 1983a). That analysis is also the basis of the expected probability adjustment for the log-Pearson type 3 distribution de- scribed in Bulletin 17B (IACWD, 1982, Appendix 14, p. 14~. Practically, the expected-probability adjustment yields an estimated flood frequency distribution defined by the quartile estimators Xp = M + (1 + 1/n)05 tpn_~S where tp n ~ is the pth quartile of the Student t distribution with n-1 degrees of freedom. For clarity, this estimate is called the "expected-probability quartile estimator." There is a temptation to assume that because the expected-probability quan- tile estimator Xp provides a good estimate of the design flood that is exceeded with the target probability, then the corresponding frequency curve would be an appropriate relationship for estimating the probability that various fixed flow values are exceeded. However, because of the nonlinear transformations in- volved, the inverse of the expected-probability quartile estimator is not particu- larly good for estimating exceedance probabilities of fixed flood flow values.

RISK METHODOLOGY 129 In particular, to use the expected-probability quartile estimator Xp to evalu- ate the risk that a specified flow q is exceeded, one solves log~q)=M+ tS(1 + 1/n)05 for the corresponding t value t = Logy - M1 1 IS ( 1 + 1/n)° s] and then looks up the corresponding cumulative probability in the tables of the Student t distribution. That calculation yields the probability estimator p #(q) = Ft Logy - M1 1 ES ( 1 + 1/n)° 5 1 ~ where F is the Student t probability distribution function with (n - 1) degrees of freedom. For clarity, this estimate is called the "expected-probability probability estimator." The Monte Carlo experiment described in Table 4.1 was repeated to evaluate the expected value of the expected-probability probability estimator t1 - p#(q)] for the same cases. Again, the value of q is fixed, as are the parameters of the normal distribution describing the logarithms of the flood flows. The results are reported in Table 4.2. For every value of the sample mean and variance, the expected probability adjustment increases the estimated exceedance probability associated with the critical flow q. The estimators in Table 4.1 that ignored hydrologic uncertainty had some upward bias. The expected probability adjustment males that bias worse in every case considered. Still, there is relatively little bias in the estima- tors of the exceedance probabilities of q0 ~ up to qO 0~ when n 2 100. For the larger thresholds q0 002 and qO OOl, the bias is severe, particularly for n < 100. TABLE 4.2 Average Value of Estimated Exceedance Probability t1 - p ~q)] for Specified Critical Flow q Using Sample Mean and Variance from Normal Samples of Size n with Expected Probability Adjustment Reflecting Hydrologic Uncertainty Critical Flow n qO.] qo.O4 qO.02 qO.Ol qO.002 qO.OO] 10 0.13 0.069 0.046 0.031 0.0148 0.0111 25 0.11 0.053 0.031 0.018 0.0063 0.0041 50 0.11 0.047 0.025 0.014 0.0040 0.0023 100 0.10 0.043 0.023 0.012 0.0029 0.0016 200 0.10 0.042 0.021 0.011 0.0025 0.0013

130 FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN The natural (or conventional) estimator 1 - p (q) of the exceedance probabil- ity of a levee or other flood control structures is upwardly biased. An expected probability adjustment makes that bias larger and thus does not seem to be advis- able if Avg[AFP] is to be used as a decision criterion. Others have debated the issue of bias in calculations of expected flood dam- ages (Hardison and Jennings, 1972, 1973; Thomas, 1976; Doran and Irish, 19804. Gould (1973) and Stedinger (1983a) noted that the expected probability method was likely to increase the bias in expected flood damage. Arnell (1989) provided a clear analysis of expected annual flood damage estimates obtained with (1) the conventional estimator without a correction for hydrologic uncertainty, (2) the expected probability estimator described above, and (3) an expected damage method. The expected damage method computes for every probability level p the expected damages associated with floods with cumulative probability p given the uncertainty in the discharge associated with p; it then integrates those "expected damages" over p. This is what the new USACE risk-based planning method does using Monte Carlo simulation (D. Ford, consultant, personal communication, January 19, 19941. Arnell (1989) provided results such as those in Tables 4.1 and 4.2 in addition to considering two different nonlinear damage functions that begin at thresholds with exceedance probabilities of 20, 4, and 1 percent. The results obtained with those damage functions are like the results above, except that the biases are generally larger because more of the damages occur with flood flows substan- tially greater than the threshold flows. He concluded that, all methods overestimate expected annual damages, particularly when damage commences in infrequent events, but the conventional method is least biased.... Although the degree of difference varies with damage function, the results clear ly show that the use of either expected probabilities or the "expected damage" method would produce very biased estimates of expected annual damages. Explanation of Bias It is useful to understand why the expected probability adjustment, which has a legitimate theoretical motivation, results in such a biased estimator of the prob- ability that existing structures at fixed locations would be flooded. Consider its origin (Moran, 1957; Beard, 1960~. Let X = logy) be the normally distributed logarithm of a possible future flood flow Q. where M and S are the sample estimators of the mean and standard deviation of X based on possible historical samples of size n. Q. X, M, and S are all considered to be random variables. Then for random X and M, the difference (X- M) is normally distributed with mean zero and variance Var FIX - M1 = o2( 1 + fin)

RISK METHODOLOGY As a result, the random variable defined as T =(l+l/n)-°s(X- M)15 has a Student t distribution with n - 1 degrees of freedom. 131 Let tp no be the quartile of the Student t distribution with cumulative prob- ability p. Then a possible future flood X will exceed the random quantity Xp = M + tp n_,S(1 + 1/n)0 5 on average (over the distributions of X, M, and S) with probability (1 - p). This means that if many projects are built nationwide, using design flows Xp based on calculated sample means M and standard deviations S. the nationwide annual failure rate will be equal to the target probability (1 - p). Here Xp can be viewed as an estimator of ~ + zp6. Why not use Xp = M + zpS instead? Stedinger (1983a) showed that on average the probability that a future (and therefore random) X exceeds the random quantity Xp is greater than (1 - p), sometimes substantially in small samples. To understand these issues, observe that Xp is a nearly unbiased estimator of the quartile of interest ~ + zp6, and X does exceed ,u + zp c, with probability (1 - p). When (1 - p) is small, say 1 percent, consider what happens when the A estimator Xp happens to be too large: then X exceeds this random quartile esti- mator with a probability less than 1 percent, perhaps 0.7 percent. On the other hand, consider what happens when Xp is too small: then X exceeds this random quartile estimator with a probability that can be much greater than 1 percent, perhaps 1.5 or 2 percent. This asymmetry results from the curvature of the cumulative probability function of Xfor x values corresponding to p near 1. The result is that the random quartile estimator Xp is exceeded too frequently on average (over the M and S distributions). The expected probability correction yields an estimator Xp of p for which the exceedance probability averaged over the distributions of M and S is indeed 1 - p. This, however, is different from the situation where one needs to evaluate the expected damages for existing structures at fixed locations or river stages. Then to evaluate the probability of flooding for any predetermined and fixed stage x, an estimate of (x-,u)/o is required. The conventional estimator (x-M)/S is itself nearly unbiased, but again because of the curvature of the cumulative probability function, values of (x-M)/S that are too small assign relatively more exceedance probability to the fixed flow x than values of (x-M)/S, which are too big and correspond to relatively smaller exceedance probabilities. Table 4.1 shows that use of

32 FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN p=~{(x - M)IS} yields exceedance probabilities for the fixed flow x that on average are too large. Table 4.2 shows that use of the expected probability estimators results in an even greater bias. This is due to use of the Student t distribution and the scale factor (1 + 1/n)05. To eliminate these biases, one might propose a downward adjustment of the probability curve (an expected-damage bias-adjustment correction for damage sites at fixed locations). The committee does not propose such an adjustment. The conventional estimator without adjustment for hydrologic uncertainty can be thought of as a compromise. For designing a nationwide system of small structures, such as highway culverts, to meet mandated failure-rate criterion, the expected-probability adjust- ment can be appropriate. However, when evaluating the expected damages asso- ciated with existing structures and population centers at fixed locations and stages, the conventional estimator without a correction for expected probabilities or hydrologic uncertainty is the better choice. The Bayesian Viewpoint The problem of hydrologic uncertainty, to the extent it is due to records of limited length, represents a classic statistical sampling problem. The statistical literature contains two methods for representing such uncertainty. Confidence intervals are a means of expressing uncertainty in terms of intervals that in repeated sampling will bracket the true value of a parameter with a specified frequency, called the confidence of the interval. They are the most commonly used method of describing sampling uncertainty. The second method employs Bayesian statistics (sometimes called inverse probability). Bayesian procedures describe the possible values of a parameter by a probability distribution that represents an engineer's or statistician's degree of belief as to the likelihood that the parameter has different values. Such distribu- tions generally depend upon the available data as well as the engineer's or statistician's prior beliefs and other sources of information such as regional hy- drologic experience (Benjamin and Cornell, 1970~. Many examples of Bayesian procedures in flood frequency analysis are available (e.g., Davis et al., 1972; Vicens et al., 1975; Wood, 1978; Stedinger, 1983a; Bernier, 1987J. There are also empirical Bayesian methods that explicitly use regional information to elimi- nate use of subjective distributions to describe prior beliefs and regional experi- ence (Kuczera, 1982~. To illustrate the importance of nonsample information, consider a hydrolo- gist's estimate of a probability distribution (which in a Bayesian analysis is called the posterior probability distribution) for the probability that a levee is over- topped based on (1) 20 years of data without any levee overtopping events and (2)

RISK METHODOLOGY 133 a reasonable understanding of the hydrology of the basin. While the sample data may have convinced the hydrologist that a levee overtopping event is unlikely, there would also surely be some risk of such an event. Thus a reasonable estimate of the probability of levee overtopping would not be zero but should reflect a broader sense of what is likely and physically reasonable. There are important conceptual and practical differences between the- classi- cal statistical approach and a Bayesian analysis. In particular, a proper Bayesian analysis employs an informative prior distribution for the unknown parameters (such as the probability of levee overtopping in the example above). That infor- mation may result in the posterior distribution having a smaller or a larger mean and variance than the sample moments M and S2. As a result, a Bayesian analysis will provide, on average across basins, an unbiased estimator of flood damages (Stedinger, 1983a); thus the introduction of a legitimate prior distribution is very important. However, this is different from an estimator being unbiased at each site. In cases where the available data overwhelm a proper informative prior distribution, parameter uncertainty is likely to be relatively unimportant, and it should not matter whether classical or Bayesian methods are employed. It is interesting that the distribution obtained for X with an expected probabil- ity adjustment is equivalent to a Bayesian posterior distribution using a so-called noninformative prior (Stedinger, 1983a). So why is there a problem with an expected probability adjustment if it is equivalent to a Bayesian estimator? The problem is that the expected probability estimator results from use of a noninformative Bayesian prior, which always inflates the mean and variance of the Bayesian posterior flood distribution. Thus it assigns an infinite prior mean and prior variance to flood flows. As a result, it is not surprising that the expected probability adjustment always yields an upward biased estimator of flood risk and damages. Recommendation To avoid the problem of bias in estimating expected annual damages, it seems most appropriate that the economic assessment and descriptions of the probability of flooding be based on best estimates of the parameters of models, AFP(cobes~) and EAD(cebes~. These two statistics still involve calculation of the expectation over significant processes contributing to flood risk and variability in system operation, perhaps as illustrated in Fissure 4.1. The alternative would be for USACE to adopt a correct Bayesian analysis of flood risk uncertainty with a proper informative prior based on regional hydrologic information as well. Given the lack of precedence, and the need for uniform and accepted procedures for the selection of prior distributions to describe uncertainty in the parameters of impor- tant models, use of proper Bayesian procedures does not appear feasible at this time. The use of AFP(cl)bes~) and EAD(cobes~) as the primary criteria to summarize

134 FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN the most likely performance of a project has other advantages as well. They will not be dependent on the descriptions of uncertainty employed to describe hydrau- lic and economic models. Thus they will be conceptually easier to understand. Moreover, the descriptions of uncertainties used to describe many of these pro- cesses will be fairly subjective and not particularly well determined. While it is often a challenge to determine the best estimators of each of the processes in Figure 4.1, it is much more difficult to describe well the uncertainty in those parameters. Use of AFP(cobes~) and EAD(mbeS~) will separate the description of the likely operation of the system from problems related to the description of uncertainties. These best estimates, AFP(cobes~) and EADcobes~) should still be supplemented by descriptions of their accuracy. In particular, possible values of AFP and EAD could be generated by Monte Carlo simulation procedures to illustrate the uncer- tainty in these performance criteria. Similarly, the uncertainty in AFP and EAD could be described by their standard error or particular quartile ranges, while the impact of that uncertainty could be illustrated by the probability that the national economic development objective is negative. Those descriptions of uncertainty in AFP and EAD and its impact would depend on the selected representations of uncertainty in hydrologic, hydraulic, and economic parameters, and would allow agency planners and the public to assess the accuracy of AFP(cobes~) and EAD(cobes~. NRC (1994, pp. 184-85) made similar recommendations for health risk analyses and also recommended that risk assessors carefully explain qualita- tively the basis for such numbers so as to minimize public misunderstanding (NRC, 1994,p.13~. This proposal appears to be consistent with the requirement in EC 1105-2- 205 (1994, p. 4) that "the estimate of NED benefits will be reported both as a single expected value and on a probabilistic basis (value of the benefit and its associated probability), for each planning alternative." Table B-3 in EC 1105-2- 205 (USAGE, 1994J illustrated the presentation of economic and risk criteria for different project alternatives (corresponding to levee heights) and included with- project residual damages (equivalent to AvgtEAD]) and the median probability of exceedance (equivalent to AFP(ct)bes~. The table also reports the simulated stage-exceedance probability (equivalent to AvgLAFP]~. AFP(cebest) and EAD(cobes~) include expectations over the distributions of annual maximum flood events and perhaps also reservoir operation and levee failure stage. They would also be based on the expected value of damages for each stage in a flood damage zone. EC 1105-2-205 (in its Appendix A) described the derivation of such an expected value and the associated uncertainty. METRICS FOR PROJECT PERFORMANCE EVALUATION Fundamental questions in flood protection project evaluation are,

RISK METHODOLOGY · What is the probability that target areas will be flooded? · Do economic parameters justify proposed projects? and perhaps, . 135 How reliable is the economic analysis of alternative projects? Thus a thorough risk-based flood-protection analysis should calculate the follow- ing values for each project alternative: The best estimate of the annual probability of flooding for target loca- tions, called the annual failure probability (AFP). 2. The expected economic benefits and costs for each project, based on the expected annual damages (EAD). 3. Measures of the uncertainty, lack of accuracy, or likely specification er rors associated with (1) and hopefully also (25. 4. Measures of the reliability of system performance that contribute to an engineer's and the public's understanding of system dynamics and how indi- vidual components of the system are likely to perform. While project selection is for the most part determined by an economic evaluation of the alternatives, the best estimate of the risk of flooding at target locations is perhaps of most interest to the public and many public officials. They are interested in risk of flooding both without any project and with various alternative projects. For this reason, this performance index has been listed first. The justification for most projects is ultimately economic, though environ- mental, social, and equity impacts should not be neglected. Thus the second performance criterion is the economic efficiency or national economic develop- ment (NED) objective, which depends on the expected annual damages associ- ated with different alternatives. Because of the importance of the economic evaluation of a project, and the inherent uncertainty in the performance of flood protection projects, it is impor- tant to evaluate the uncertainty in the economic performance of alternative projects. This point was made clearly in USACE (1992a); USACE (1992b) provided an extended example of the calculation of the uncertainty in estimates of the national economic development objective and the benefit-cost ratio (BCR). Economic uncertainty was illustrated by the probability that the BCR is actually less than one, the mean and coefficient of variation of the BCR, the distribution of net benefits, and the distribution of the BCR (USAGE, 1992b). All of these are reasonable approaches for illustrating the impact of uncertainties that affect the economic performance of a project. The uncertainty in the economic performance of a project depends largely on the uncertainty associated with flood flow frequency distributions, hydraulic re- lationships, specification of when levee failure occurs, and the economic value of

136 FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN property and the damages it might sustain. EC 1105-2-205 illustrated the impact of structural and content values in the evaluation of the uncertainty of economic measures of project performance. Following Davis (1991) in the application of the USACE risk and uncertainty methodology, the Sacramento District also pro- posed to calculate various reliability indices to help explain project performance. Clearly, the value of such indices is less important than either the overall risk of flooding faced by residents of the floodplain or the relative economic attractive- ness of alternative projects. Moreover, the attractiveness of alternative plans should be judged primarily on risk of flooding and economic efficiency rather than on whether some internal measure of system reliability meets an arbitrary standard. These issues are discussed in greater detail later in this chapter. USACE RISK-BASED PROCEDURES The sections above discuss a general formulation and structure of risk and uncertainty analyses for flood protection project evaluation, and metrics for project evaluation. The committee's task included looking carefully at the risk and uncertainty methodology used by the Sacramento District. The sections below first review the general approach USACE has adopted and then focus on the implementation of that philosophy in the American River basin. The New Methodology USACE (1994) has observed that "risk and uncertainty are intrinsic in water resources planning and design" (EC 1105-2-205, 4(a)~. In the past, USACE first developed its best estimate of the most likely values of "key variables" for the evaluation and design of flood damage reduction projects. Then sensitivity analy- sis was used as the primary tool to investigate the importance of uncertainty in planning parameters. However, this approach fails to integrate sources of uncer- tainty, their interaction, and their relative likelihoods (Moser, 1994~. The new USACE risk-based procedures quantify the risks and uncertainties in various parameters and components of the planning process and the design of facilities. USACE has described the new risk-based analysis framework as "an approach to evaluation and decision making that explicitly, and to the extent practical, ana- lytically incorporates considerations of risk and uncertainty." USACE has decided that the traditional "level of protection" will no longer be used in describing project performance, and levee freeboard will be replaced with a probabilistic description of levee performance (EC 1105-2-205~. Previ- ously, levee height determination was caught between the designer's view that it should ensure that a project can reliably pass the design flood, and the economist's view that freeboard should be economically justified. The new risk-based analy- sis will allow USACE analysis to address the economic and reliability tradeoffs associated with levee freeboard (Davis, 1991~. EC 1 105-2-205 indicated that,

RISK METHODOLOGY 137 "Risk-based analysis enables risk issues and uncertainty in critical data and infor- mation to be explicitly included in project formulation and evaluation" (EC 1105- 2-205, p. A-11. The new risk-based decisionmaking procedures combine traditional hydro- logic risk with tainty), · uncertainty in parameters describing hydrologic risk (hydrologic uncer · variability and uncertainty in stage-discharge (hydraulic) relationships, · levee performance variability, · variability and uncertainty in stage-damage (economic) relationships, and · variability in other operating assumptions. Some of these uncertainties arise because of limited hydrologic records, while others reflect limited data and errors in measurements of channel geometry, roughness and slope, or the range and character of economic activities. USACE (EC 1 105-2-205, 1994, p. A-3) observed that "The proposed strategy is similar to present practice but differs in that uncertainty is explicitly quantified and inte- grated into the analysis." In addition, USACE risk-based analyses will initially concentrate on includ- ing uncertainty in the following key variables (Moser, 19941: Economic Hydrologic Hydraulic Structure, first-floor Discharge associated Conveyance elevation with exceedance roughness Structural values frequency Cross-section Content values geometry Figure 4.2 (from Davis, 1991; also EC 1105-2-205, p. A-2) illustrates pos- sible uncertainties in discharge-frequency, stage-discharge and damage-stage re- lationships. Table 4.3 summarizes the sources of risk, variability, and uncertainty that can be considered in a risk and uncertainty-based planning methodology. Reasons given to support this evolution in the USACE planning methodol- ogy include 1. The inherent uncertainty in planning. 2. That hydrologic and hydraulic engineering have advanced sufficiently to reduce the need to design for unquantified uncertainties. As a result the degree of certainty in performance (engineering reliability) can be quantified and advanta- geously used in project selection. 3. The broader range of risk, costs, reliability, and associated trade-offs that it will allow the planning process to address (Moser, 1994, pp. 1 to 21.

138 C, cat C' C, - CD o, cat CO FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN Flood Risk / ' ! Exceedance Probability Uncertainty in Stage Uncertainty in Discharge - ~D 03 ro c' Q / / hi/ / Exceedance Probability Uncertainty in Damage | _ / / Discharge (Q) FIGURE 4.2 Uncertainty in discharge, stage, and damage. Stage (S) USACE indicated that its risk-based planning procedures should allow ex- amination of the total effect of risks and uncertainties on design values and economic viability. Thus better decisions can be made on the trade-offs between risks and costs. Increasingly, USACE is confronted with severe budget con- straints, new customer cost-sharing requirements, and concern among its custom- ers and the public with project performance and reliability. System performance and planning uncertainties now need to be addressed more explicitly as part of the assessment of water resource investments. The new risk-based procedures are

RISK METHODOLOGY TABLE 4.3 Sources of Risk, Variability, and Uncertainty That Can Be Considered in a Risk and Uncertainty-Based Planning Methodology 139 1. Hydrologic risk: Discharge Q associated with exceedance probability p. 2. Hydrologic uncertainty: Variability in estimators of moments of the Q-distribution and the accuracy of derived frequency curves. 3. Flood stage: stage S corresponding to discharge Q is not perfectly determined owing to variability and imperfect knowledge of channel geometry, roughness, flow regime, bed form, flow debris, and inexact analytical techniques. 4. Levee performance: Stage L at which levee fails is uncertain owing to lack of understanding of internal structure, and possibility of surface erosion, piping problems, underseepage, slides within the levee embankment, and foundation soil weaknesses. 5. Flood damage uncertainty: Damage D on floodplain associated with river stage S is uncertain owing to mix of structures, elevations, and structural and content damage potential, which determines damage distribution about stage-damage curve. EC 1105-2-205 (p. A-3) indicated that damage uncertainty can describe uncertainty in the extent of the physical damage or in the "cost data." presented with the expectation that more explicit consideration of risk and uncer- tainty should improve USACE investment decisions and the planning process (EC 1105-2-205, 4(d)~. Explicitly introducing hydraulic and levee-performance variability into the analysis should improve estimates of the true overall risk of levee system failure, as well as identification of the critical processes most likely to result in failure. For example, one can ask how the risk of flooding would change as a result of rigorous inspection and some structural improvements in levees (without actually attempting to raise their crest), as opposed to developing increased flood control storage in reservoirs, which would address the issue of flood control more explic- itly. On a philosophical basis the USACE proposal is the logical and appropriate next step in the evolution from the previous flood-damage-reduction evaluation methodology. USACE is to be commended on beginning the development of this planning capability. However, the new procedures will not necessarily be easy to implement. Safety factors are often an easy way to avoid messy issues. When they are replaced by probability distributions that are actually used to describe project performance, much more attention needs to be paid to some critical uncer- tainties. Models will be needed to represent when levees actually fail as a func- tion of stage, levee characteristics, and levee length. The need for these functions is new, and they may be relatively important. Likewise, variability in hydraulic calculations defining stage-discharge relationships will also be needed; such cal- culations are an extension of traditional hydraulic sensitivity analyses. USACE recognizes these problems and has embarked on a vigorous research effort (EC 1 105-2-205, Appendix C). The ability of the new planning procedure to better represent flood risk,

40 FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN system design, and operating trade-offs depends on how well many modeling issues are handled in each application of the procedures. Providing general guidance and training for district engineers on procedures and data bases would strengthen the engineering judgments and modeling assumptions they make. In- stituting report review at the district and national level should also improve consistency and accuracy. Using USACE research organizations as centers of excellence would help to develop and disseminate the needed expertise and expe- rience. The committee realizes that the increased complexity of the new risk and uncertainty-based analyses may lead to less reliable estimates of flood risk such as Avg~AFP], at least until the method is better understood by those using it. For example, it will be hard to capture the effectiveness of flood-fighting efforts, and the feedbacks between system weaknesses and variations in reservoir operations to avoid failure. Nevertheless, ultimately the new methodology can provide a more accurate estimate of the true residual flood risk associated with a project and the uncertainty in average estimates of performance criteria due to unavoid- able model specification errors. Engineers also should be careful not to mix the new estimates of the chance of flooding, such as Avg~AFP], with the old "level of protection." Such confusion is a major problem with the 1994 Alternatives Report (USAGE, Sacramento District, 1994a). The two approaches have differ- ent sets of assumptions. Description of Risk and Uncertainty in the New Methodology This section provides a brief description of the treatment of risk and uncer- tainty in the new USACE risk-based planning methodology. The structure of this section follows Table 4.3. Hydrologic Risk More effort and work have gone into capturing and describing hydrologic risk than have gone into the other issues. As a result, it is much easier to criticize the procedures in light of the many flood frequency procedures that have been proposed. USACE employs WRC Bulletin 17B procedures (IACWD, 1982), which are recommended federal guidelines, and uses a log-Pearson type 3 (LP3J distribution to describe the frequency relationship. Issues associated with that procedure and recently developed alternatives are discussed elsewhere (Thomas, 1985; Potter, 1987; Cunnane, 1988; Potter and Lettenmaier, 1990; Stedinger et al., 19931. Hydrologic Uncertainty Hydrologic uncertainty is simpler to deal with than other sources of uncer

RISK METHODOLOGY 141 tainty, when the analysis is based on a stationary gauged record. For the most part, hydrologic uncertainty in estimators of the parameters is determined by the limited length of the flood series used. In that sense, the uncertainty is objective and is described by standard statistical sampling theory (IACWD, 1982; Stedinger, 1983b; Chow et al., 1988; Chowdhury and Stedinger, 1991~. When a flood record must be corrected for development, storage, or channel changes, then the length of record is still likely to be the primary determinant of hydrologic uncertainty, though subjective assessments of the quality of any ad- justments to measured flows are also important. Possible nonstationarity due to subtle shifts in climate and storm paths is difficult to detect and document, but is sometimes a concern. If regional relationships are used to develop flood curves, then the corresponding estimates of prediction error should be employed (Tasker and Stedinger, 19899. The proposed analysis for gauged sites bases its description of hydrologic uncertainty upon the confidence interval calculation procedure in Bulletin 17B (IACWD, 1982), which contains procedures that federal agencies agreed to em- ploy in the mid- 1970s. The Bulletin 17B procedure for calculating confidence intervals employs the assumption that the coefficient of skewness of the loga- rithms of the floods is correctly specified, independent of the data (Stedinger, 1983b). In fact, the actual coefficient of skewness employed is generally a weighted average of the at-site sample skewness and a regional or generalized skewness estimator (IACWD, 19823. Clearly, the weighted skewness estimators incorporate estimation error because of sampling error in the at-site skewness estimators and also the regional skewness estimators (McCuen, 1979; Tasker and Stedinger, 1986~. As a result the calculated intervals with the Bulletin 17B procedure are too small. Formulas that incorporate variability in weighted skew- ness estimators are available (Chowdhury and Stedinger, 1991; Stedinger et al., 1993~. As noted above, the problem of hydrologic uncertainty, to the extent it is due to records of limited length, represents a classic statistical sampling problem. The two approaches in the statistical literature for representing such uncertainty are traditional confidence intervals, which are interval estimators that contain an unknown but fixed parameter with a specified frequency, and Bayesian inference, which describes the uncertainty in unknown parameters by a probability distribu- tion. The USACE risk and uncertainty methodology employs a Monte Carlo procedure to generate alternative values of flood quartiles reflecting hydrologic uncertainty. This is used to estimate the probability of levee failure and expected annual damages. Given this approach, it would appear that USACE would need to adopt a Bayesian framework to be conceptually consistent. In a Bayesian framework hydrologic variability and uncertainty are integrated to obtain the posterior distribution for flood flows (Zellner, 1971~. Descriptions of the uncer- tainty in flood quartiles and flood-distribution parameters by probability distri- butions are inconsistent with the theory supporting traditional confidence inter

142 FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN vats because that theory depends on the parameters being fixed; only in repeated sampling does the concept of "confidence" associated with a confidence interval have meaning (Stedinger, 1983a). However, the USACE procedure employs the confidence interval procedures from Bulletin 17B to generate alternative flood quartiles associated with each generated exceedance probability p and then uses these values to compute a probability of flooding. Flood Stage Uncertainty At some gauged sites the uncertainty in stage estimates for a given flow is related to the accuracy of the stage-discharge rating curve and its stability. Agen- cies such as the U.S. Geological Survey that are responsible for the estimation and updating of the rating curve should be able to provide information about its accuracy and stability. In most cases, hydraulic models will be required to compute water surface elevation profiles based on surveys of channel capacity and in some cases perhaps also on the operation of storage facilities. At some locations the analysis may be plagued by complex hydraulics and junctions with other rivers and streams or with hydraulic control structures and weirs that divert flood flows. In such instances determination of the stage associated with a given flow may be difficult. Development of descriptions of flood stage uncertainty at ungauged sites can be viewed as an application of sensitivity analysis. The difference is that more care will need to go into the specification of the uncertainty distribution for different parameters. First-order uncertainty analysis techniques can be used in situations with relatively small errors to derive the resulting distribution of errors in stage (Benjamin and Cornell, 1970; Burges, 1979~. Mays and Tung (1992, section 5.3 J illustrated the application of this method to Manning's formula for open channel flow. Kuczera (1988) discussed the accuracy of more complex rainfall-runoff calculations. USACE (EC 1105-2-205, p. A-16) has provided guidelines for the estimation of stage-discharge relations. Levee Performance Variability The reliability of levees will be an important component of risk-based plan- ning studies. USACE has outlined procedures for developing these distributions based on the opinions of experts and its review of available data (Memorandum for Major Subordinate Commands and District Commands, Policy Guidance Letter No. 26, Benefit Determination Involving Existing Levees, 23 Dec. 1991~. Flood Damage Uncertainty Flood damage estimates for residential and commercial areas are based on (1) the number of different types of buildings, (2) structural value by building

RISK METHODOLOGY 143 type and usage, (3) contents of building by type and usage, (4) first floor eleva- tion, (5) damage percentage as a function of flood depth, and (6) flood depths at damage locations as a function of river stage. The errors in such calculations can be estimated by considering the errors likely in each component of the analysis (EC 1105-2-205, pp. A-l9 to A-39). The analysis is more difficult for the calcu- lation of with-project stage-damage functions because one should anticipate the response of floodplain occupants to new construction and other projects that are intended to reduce flood risk. Using the USACE Risk-Based Analysis Framework for the American River Information about how the risk-based procedures are being applied to the planning activities in the American River basin were provided to the committee largely in a presentation on August 11, 1994, by USACE and in letters from consultant Dr. David Ford (D. Ford, consultant, personal communications, Au- gust 23 and September 1 and 19, 19941. The risk and uncertainty analysis is summarized in Figure 4.3. The risk and uncertainty procedures for the American River basin focus on the national economic development (NED) objective, risk of flooding, and sys- tem reliability. Expected annual damages (EAD) and expected annual failure probability (AFP' are computed by repeatedly sampling from the discharge- frequency function and the levee stage-stability probability function, as well as the error distributions for frequency, stage-discharge, and stage-damage. USACE often refers to the annual failure probability as the "annual exceedance probabil- ity." Because failure can occur with floods of different magnitudes depending on reservoir system operation and levee performance, the word "exceedance" may have lost its original meaning, which referred to a unique flood with a specified exceedance probability. If occurrences of failure events are independent from year to year, the risk of occurrence of at least one failure in a T-year period can be computed from the annual failure probability as Risk(T) = 1 - ( 1-AFP)T American River Risk-based Simulation Analysis The analysis procedure for the evaluation of risk and expected economic losses is based on sampling the relevant performance and uncertainty distribu- tions, as illustrated roughly in Figure 4.2. The steps described by Ford (personal communication, September 19, 1994), after some reorganization, are repeated below. The algorithm simultaneously incorporates hydrologic risk and variabil- ity in flood operations, and hydrologic and hydraulic uncertainty. Averaging the

44 FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN R&U ANALYSIS ~ RIVER srAGE RATING <) REGULATED OUTFLOW <' UNREGULATED INFLOW UJ In R&V ANALYSIS <) 0 Trt~ Out. ~\k , V ~ ~,- _~.~ ~s"~.~.~. 1,~! 1 , DISCHARGE OUT \loo 200 400 \ FFtECUEN~ / \ NDISCHARGE IN \ l \] W)~- -i,, 641 ·', ~-~1! WN , or o ~ `~; |_ ~ ~ _ ~ _ _ , _, ~ ,- ~ Boas ~ _ Cach -credt'f"b ,, - ~7al NN Solve ~1110~: SACRA~E NTO SYSTEM STACK FREOllENCY Observed _1_ Computed N I N.S. _ ~ ~ ·~ETR.OU~ ~ ~ ~ ~ ~"N8tD~NS . t00 to. 4 UOU~ t ~ · Coo 'ma ~_~;,, /~ OVT`OW HYDROC;RAPttS 400 >: A, 100 FIGURE 4.3 American River project risk-based reliability hydrology and hydraulics. SOURCE: USACE, Sacramento District, August 11, 1994.

RISK METHODOLOGY 145 generated values yields AvgLAFP] and Avg tEAD]. Unfortunately, economic uncertainty is ignored in the American River study (Ford, personal communica- tion, September 1, 1994~. Discharge-Frequency Relationship (Hydrologic Variability, The first step of the simulation is to sample the "median" discharge-fre- quency function (the fitted flood-frequency curve) to obtain a nominal flood flow value Q. Bulletin 17B (IACWD, 1982) describes the procedures employed to obtain this distribution. This step corresponds to use of the frequency distribu- tion in Figure 4.3 and in Figure 4.2, which is represented by a solid line. Hydrologic Uncertainty The second step incorporates hydrologic uncertainty into the analysis. Using the confidence interval procedure in Appendix 9 of Bulletin 17B (IACWD, 1982), the flood flow selected in step 2, Q. is modified to reflect possible errors in the "median" flood frequency curve. The resulting flood flow reflecting possible error in the estimation of Q is denoted Q. This step corresponds to use of the error distribution about the frequency distribution in the upper right-hand corner in Figure 4.2 and the upper-right-hand corner in Figure 4.3. In Figure 4.2 that error distribution is represented by a bell- shaped curve. Reservoir Operation In the third step, the unregulated Folsom inflow must be transformed (by storage routing using the reservoir' s operating rules) into a peak outflow rate for evaluation of downstream damages in the American River corridor. The Sacra- mento District considers variability in this transformation due to variations in initial storage, possible delays in making releases, use of a one- or two-wave model of the inflow hydrograph, outlet works operation, and spillway operation efficiency. The worst-case, most likely, and best-case values for operational perfor- mance and decisions were analyzed to determine for various inflow levels a possible distribution of peak outflow rates. A triangular distribution for outflow was determined by assigning probabilities of 5 and 95 percent to the best- and worst-case outflows, respectively, and computing the non-exceedance probabil- ity of the most-likely outcome so as to yield a legitimate distribution function (Ford, personal communication, September 19, 19941. That distribution allows the assignment of a peak outflow reflecting possible variation in reservoir opera- tion o ~ Qp] to each computed error-affected inflow with error Qp from step 2. This is illustrated in the middle graph at the top of Figure 4.3.

46 FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN The Sacramento District will eventually need to justify the selected ranges for each of the factors considered in this step. It would also be useful for the district to provide an analysis illustrating which factors were most important in determining the variance of the outflow distribution so that attention can be focused on the key factors. In particular, if initial delays in releases are particu- larly important, then planners and engineers should investigate general policies, operating procedures, and warning and alert systems that might be able to reduce such delays. Stage-discharge In the fourth step, stages S ~ o ] at various locations in the American River corridor can be estimated given the Folsom outflow peak 0 . This is illustrated in the lower-left-hand corner of Figure 4.2 and the upper-left-hand corner of Figure 4.3. Hydraulic Uncertainty Hydraulic uncertainty is quantified in the fifth step. The calculated stage S ~ o] is an imperfect estimator, so an error-affected stage is generated, which is denoted S ~ O]. This corresponds to the error distribution shown in the lower left-hand corner of Figure 4.2 and the upper left-hand corner of Figure 4.3. Levee Performance Whether or not a levee fails is determined by the height of the water in the channel, though other factors such as the duration of flooding could be important. For the sixth step USACE (Engineering Technical Letter 1110-2-328 22 March 1993) prescribes describing levee reliability for existing levees by two points: Probable Failure Point, PFP, and Probable Nonfailure Point, PNP (see Figure 4.4~. As shown in Figure 4.4, there is a 15 percent probability that the levee would fail at the PNP, and the probability increases from 15 percent at the PNP stage linearly with stage until it reaches 85 percent at the PFP stage. At the PFP stage the failure probability increases discontinuously to unity. Geotechnical engineering evaluations are the basis of these two stages. The selected stages replace the use of a single stage to define when a levee would fail (with some residual freeboard included as a safety factor). However, USACE has indicated that for new levees, PNP and PFP both equal the stage of the levee crest, perhaps with an allowance for settling. If the levee does indeed fail, then the error-affected river flow with error ~ ~ O should be used to determine a new error-affected stage S ~ 0 ~ for damage sites of interest.

RISK METHODOLOGY Stage Probable Failure Point (PFP) , ;.:~ 147 Probable Nonfailure Point (PNP) 0.00 0.15 0.85 1.00 Probability of failure i! water surface reaches stage shown FIGURE 4.4 Failure relationship of PNP and PEP. SOURCE: USACE, letter 1110-2- 328, March, 1993. Stage-Damage Relationship In a system without levees, the error-affected stage with error S ~ o] from step 5 is used to determine the economic damages. This is the situation consid- ered in the example in EC 1105-2-205 (p. A-21. In a system with levees, the situation is more complex because one must consider if the levee fails, and the flooding that would result if it does. In step 7, if a levee is overtopped or fails for other reasons, the error-affected stage, given levee failure stage S [o] deter- mines the damages. Stage-Damage Uncertainty EC 1105-2-205 (pp. A-l9 through A-40), and USACE (1992b, pp. FC-23 through FC-33) described the origin of errors in the estimation of expected dam- ages. Many of these are related to the limited resources available to determine the number, types, and value of structures in areas likely to be flooded. Other uncertainties relate to factors besides stage that determine the damages from flooding. These include flood duration, the presence of ice or wave action, and warning time, as well as fundamental problems in determining the costs of dam- age to property. These are represented in the lower-right-hand corner of Figure 4.2. If the only statistic of interest were the expected annual damages (EAD), there would be no need to consider uncertainty in damages; it would suffice to employ with each error-affected stage S ~ 0 ~ the average estimated damages that would result were the river to reach that stage. However, as illustrated by USACE (1992b), uncertainty in the EAD and benefit-cost ratio resulting from the uncer- tainty in damage estimates and the discharge-frequency relation can be substan- tial. It was the committee's understanding that uncertainty in estimated damages would not be part of the risk-based analysis performed in the American River

48 FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN study (Ford, personal communication, September 1, 1994~. That is unfortunate because as a result it will not be possible to determine how uncertainty in key economic parameters might affect the ranking of flood control alternatives for the American River basin, and the overall viability of the more attractive projects. Organization of the American River Study Analysis There are a number of ways to organize the risk-analysis computations. The simplest procedure would be to randomly generate values of all of the variables and count the number of times the levee fails to determine the levee failure probability (which determines the AFP), and to average the resulting damages to determine the expected annual damages (EAD). Ford (personal communication, September 1, 1994) describes this procedure for calculating failure probabilities. However, the Monte Carlo simulation described above can be simplified by introducing an analytical evaluation of levee failure, as suggested by Figure 4.1. Given a calculated e~Tor-affected stage S ~ O ], one can determine the probability of levee failure for that stage, denoted PL Suppose the resulting damages if the levee failed would be DfO,, and zero otherwise. Then the average damages associated with the generated stage S ~ o] are just p~D(O) + (1 - pt)O = p~D(O) By averaging across the Monte Carlo replicates the expected damages pads O ~ associated with each e~or-affected stage S ~ O i, one obtains the expected annual damages (EAD). A similar simplification can be employed when calculating the expected annual levee failure probability. For each randomly generated stage S ~ 0 i, one obtains the corresponding probability of levee failure ply S ~ Old; the average of these probabilities is the expected annual failure probability (AFP). The commit- tee understands that the Sacramento District employed the second of these ideas (Ford, personal communications, September 1 and 19, 19941. The two shortcuts eliminate from the Monte Carlo analysis variability due to the random generation of different levee failure stages. Other significant simplifications are possible. As EC 1105-2-205 (p. A-3) pointed out, the problem as a whole is quite complex. As a result, analytical, or analytical-numerical evaluation of the entire problem may not be as attractive as Monte Carlo sampling schemes. However, analytical approximations could be introduced in some places to make the computations simpler and more accurate. For example, if one chooses to introduce discharge-frequency uncertainty, that uncertainty might be combined with the original "median" frequency curve to get an error-affected discharge-frequency distribution curve. That error-affected in- flow curve could then be convolved numerically with the distribution for the

RISK METHODOLOGY 149 inflow-outflow transformation to obtain an error-affected outflow-frequency curve to eliminate that simulation step. USACE USE OF RELIABILITY IN PROJECT PLANNING Developers of the new USACE risk-based planning methodology have pro- posed use of a system reliability index as a key description of a system's ability to meet particular performance levels (Davis, 1991~. Examples can be found in the 1994 Alternatives Report (USAGE, Sacramento District, 1994a, Plate 12 and p. 57, Table III-1 and p. 9~. In a case study illustrating the new methodology, Dotson et al. (1994) observed: The risk-based approach has many similarities with the present practice in that the basic data are the same. Best estimates are made of discharge/frequency curves, water surface profiles, and stage/damage relationships. The difference between the current practice and the r~sk-based approach is that uncertainty in technical data is quantified and explicitly included in evaluating project perfor- mance and benefits. Using the risk-based approach, performance can be stated in terms of reliability of achieving stated goals. Also, adjustments or additions of features to accommodate uncertainty, such as adding freeboard or levee/ flood walls, are not necessary. [Italics added.] USACE Guidelines for Use of a Reliability Index The committee struggled to understand how this reliability index would be used and what information it conveyed. Before those issues are examined, the use of reliability in project evaluation needs to be considered. The USACE guidelines for the new risk and uncertainty analysis procedures (EC 1105-2-205) state, The r~sk-based analysis will quantify the reliability and performance of levee heights considered by explicitly incorporating, the uncertainties associated with key variables. This reliability and performance will be reported as the protec- tion for a target percent chance exceedance flood with a specified reliability. For example, the proposed levee project is expected to contain the one-half percent (0.5 percent) chance exceedance flood, should it occur, with a ninety percent (90 percent) reliability. This performance may also be described in terms of the percent chance of containing a specific historic flood should it occur. [Italics added.] With these changes the directive indicates that "the concept of level of pro- tection is no longer useful and will not be used in describing project perfor- mance." To illustrate the concept, EC 1105-2-205 presents several examples in its Appendix B. Table B-2 from that appendix is reproduced here as Table 4.4. (An earlier version of that example appeared in Davis (19919.) For different levee

150 Cal Cal o = sin Cal V) ._ Cal o ;^ ._ ._ Cal ._ au m ¢ A> ~ Ct Cal 5: V ~ ~ X C) ~ c, o ~ z cio~ to of 5 - 5: C) X ~ ~ ~ O a) cat tV ~ Z ° o c40 Cat ~ ~ ~ O O : - C.) ~ G ~ C) Ct Ct .L ~ ~' - ~ Z ~ A_ ~ IS ;> ._ ~ TIC - ~ ~- Go ~ ~ ~ ~ Do rid ~ 0 Go ~ Cal ~ . . . . . . . . . . . . . . . . O O O O - ~ ~ EN ~ At - ~t ~t ~ ~- (= - - CM At ~ ~ of ~ ~ ~ O - ON ~ ~ C~ ~ ~ ~ ~ 00 ~. . . . . . . . . . . ~ ~ . . . ~ o o - ~ ~ ~ ~ ~ o oo ~ ~ ~ ~ M t ') 0 v~ u~ ~ ~ oo 0 ~ ~ - ~ crs ~ ~ ~ cn ~ ~ ~ ~ ~ ~ . . ~ ~ ~ . ~ ~ . . . . . . . o o o c~ ~ o o v) o N ~ ~ oo ~ ~ ~ ~ ~ ~ ~ o - o ~ o ~ ~ ~ ~ . . . . . . . . . . . . . . . . . . . o ~ u~ - ~ ~ c~ Mo o oo ~ ~ ~ oo ~ ~ c~ ~ ~ 00 O0 00 ~ ON C~ ~ ~ C~ ax ON ~ ~ ~ oo ~ ~ ~ ~ ~ oo ~ ~ ~ ~ ~ ~ c~ o ~ ~ ~ ~ ~ - ~ ~ go oo 13%~ =~ oN aN ctN ~3N - ~ ~ ~ 00 C~ ~ ~ C~ ~N ~N ~ GN C~ ~ _ cr~ . . . ~ v~ ~D ~ r~ oo ~ ~ 0 - - c~ ~ ~ v~ ~ ~ ~ oo ~ O - c~ m]

151 a) ._ . _ of ._ IS C) At: ._ 3 ._ o C) - . _ t ~ ;^ ~ ~._ ._ ._ _ _ ._ ._ . _ =, o o o 50 ~ Q ~ 4_ ~ Cal Cal 50 L,, ~C) O O ~_` . . _ ~ Cal 11 11 ~ED Ct Ct V: Cal AD ~ ._ ._ U' ~ =5 ~ ~I_ ;> ~ ~sit Ct Ct ;^ O O O lo) Ct Ct ._ ._ ~C) ~Is: Ct , ~ ~ 3 ~ .> o .> ~ V? ~ IS . . . ;> - a~ . 3 ._ o D . _ au o 5: ~ ._ ._ ._ ._ AD IS S AD ._ ._ C) "S _` ON - - O ~ ~ . _ ~ 3 o 5) o . _ 5 0 .' ~ _ D =1 - ~ , ._ ._ ~ 3 3 V: . Hi ~ ~- = . - ~ ID ct Hi Hi -= ~ o ~ - c Hi Hi 3 _ C,) ~ ,~ O O - ~ Cal O ~ ._ _ c5 11 ._ ~ O ~ ~4 ._ ~ Cal O Al ~ C<' Cal ~ so: ;^ O ' ._ so ~ ._ V: ~ . _ ~ a) _ ~ ._ ~ 3 ~ Ct Ct "D ,= O ~ PA ~ .> ED V O in Cal - - 11 ED Ct Cal ._ ;> a Ct ._ _ ~ ~ - ._ O V ~ ~ 50 ._ > - An: ._ 3 ._ Ct o - ._ 3 A: > C) Cal ;` - ._ Ct o 1 1 C) IS so C) c) o to 11 ~4 Ct U: ._ Cal o to ._ Ct AS So . _ ~ 4~,, o cat ; - ~.. .= .> O -

52 FLOOD RISK MANAGEMENT AND THE AMERICaN RIVER BASIN stages, and different possible flood events defined by the chance of flooding, ignoring hydrologic uncertainty, the table presents the reliability of the levee (the probability it will not be overtopped). The calculated reliability includes hydro- logic uncertainty, uncertainty in calculated stages given river discharge, and vari- ability in levee failure stage. EC 1105-2-205 observes that: Table 2 in Appendix B shows that for a levee height of 25 feet there is a 14 percent chance of containing the 0.2 percent chance flood, a 40 percent chance of containing the 0.4 percent chance flood, a 85 percent chance of containing the 1 percent chance flood, a 98.5 percent chance of containing the 2 percent chance flood, and a 99 percent chance of containing the 4 percent chance flood. In the case of the American River, variability in reservoir operation would also be added. Ford (personal communication, September 19, 1994~; indicated that Reliability, as used in the American River study, describes the frequency with which a proposed plan performs as intended, given the occurrence of a speci- fied event. It is computed via sampling also, with sampling of the frequency function limited to discharge of a specified exceedance probability. For exam- ple, the reliability of an alternative at the 1%-chance event is predicted by repeated sampling of the discharge and stage for a 1%-chance exceedance event (accounting for the error in predicting both the discharge and the stage). Figure 4.5 illustrates the presentation of results for different project alterna- tives using the proposed reliability index. Numbers such as those in Table 4.4 appear in Figure 4.5 in graphical form so they can be better understood. An example is provided by Plate 12 in the 1994 Alternatives Report (Sacramento District, 1994~. In an early presentation of these concepts, Davis (1991) said One could (and likely would) prepare reliability tabulations of the likelihood of exceedance by various flood events to enable characterizing performance by assignment of a level-of-protection, with stated reliability. The approach suggested explicitly acknowledges that there is not a specific, unequivocal, performance level. Levels-of-protection would have to be couched in a reliability context such as "this levee project has a 95 percent chance of protecting against the 100-year exceedance interval flood, should it occur." Acceptable reliability criteria would have to be adopted. Dotson et al. (1994) illustrated how the methodology can "quantify the reli- ability and performance" of a project, which can be expressed in statements such as "there is about a 95 percent chance of containing the 1-percent (100-year) flood, should it occur."

RISK METHODOLOGY 100 _ - ct i_ so _' .5, 50 o - ._ ED Ct ._ - o 153 1 1 1 1 -~ 50 100 200 300 500 year year year year year Return Period of Flood Peak FIGURE 4.5 Illustration of the trade-off between the return period of a flood peak and the reliability of the reservoir-levee system with possible flood flows associated with that return period. Application of Reliability Indices in the ARWI While the committee does not disagree with the analysis in Table 4.4 or Plate 12 of the 1994 Alternatives Report, it cannot see clearly what the public or most engineers would do with such information. There are several concerns: 1. It is not at all clear how one should conceptualize the 1 percent chance event given that it is not converted into a single flow estimate. Instead it is used to generate a set of flows reflecting the hydrologic uncertainty in the computed discharge-frequency relationship. This makes it very hard to anchor the analysis mentally or to know for certain to what it is applied. In the definition of reliabil- ity for the American River study taken from Ford (personal communication, September 19, 1994, quoted above), what is the particular "specified event" to which the chance of failure in Figure 4.5 or in Plate 12 of the 1994 Alternative Report refers? Use of critical historic flood events with known flood flow peaks would help resolve this conceptual vagueness. 2. The Sacramento District needs to clarify its reasons for wanting to calcu- late this reliability index shown in Table 4.4. If the overall probability of levee

154 FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN failure, which describes the residual risk of flooding, is already known for levees of different heights, what does this other reliability calculation add? 3. The analysis does not indicate how much of the reliability (or likelihood of failure) is due to hydrologic uncertainty, how much to stage-discharge uncer- tainty, and how much to variability in levee failure stage. It is not clear what this reliability calculation reflects. The term reliability gave the committee the sense that it was a measure of how certain the Sacramento District is that the levee system would perform as intended. It was suggested that by using the new "reliability" index the Sacra- mento District is trying to tell the public that there is some uncertainty about how particular aspects of the project will perform. Ford (personal communication, September 19, 1994) wrote, "The term reliability, as used in the American River study, describes the likelihood that a proposed plan will perform as intended, given the occurrence of a specified event." Because the committee could find no explicit statement by the Sacramento District of what was intended, it had diffi- culty interpreting such statements. How should one define the intent of an exist- ing system? Because the numbers in Table 4.4, Plate 12, and Figure 4.5 also include the large uncertainty related to converting a "median" exceedance probability for a flood into the correct discharge, the committee found it very difficult to develop a useful interpretation of these numbers. It would be even more difficult for the public to interpret them. Proponents of careful risk communication warn of the pitfalls related to public misinterpretation of descriptions of risk (Plough and Krimsky, 1987; Slovic, 1987; NRC, 1989, 1994~. If each column in Table 4.4 corresponded to a discharge peak of a particular magnitude, corresponding to a historic flood or a selected design hydrograph, then one could interpret the calculated reliabilities as describing the consequences of stage-discharge estimation errors and levee performance uncertainty. For example, one could compute the reliability of the levee-reservoir system for a flood flow with a peak of 300,000 cfs or 500,000 cfs into Folsom Reservoir due to uncertainties in reservoir operation, stage-discharge relationships, and levee performance. One could also provide the estimated probabilities that these par- ticular peak flows are exceeded. For levee systems, a similar calculation would result by having the columns in Table 4.4 represent particular discharges (and perhaps the estimated probability each would be exceeded). 4. Calculations of project reliability may involve some difficulties that are not apparent. In reservoir-levee projects the characteristics of a critical event may depend on the capacity of the reservoirts) considered for different alterna- tives. For a levee-only system, it is the peak inflow that matters most. As one adds more storage, flood inflow volume becomes increasingly important. Thus one wants to select for the columns of a table such as Table 4.4, and for the graph

RISK METHODOLOGY 155 in Figure 4.5, events that for all alternatives are equally critical. It may be important that the public and the engineers who are reviewing project proposals understand how this is done. But this issue is completely hidden in a table such as Table 4.4, in which the character of the actual hydrologic event corresponding to each probability is obscure. 5. The explanation that is reproduced at the beginning of this section from EC 1105-2-205 also gives the sense that from Table 4.4 and Figure 4.5 one can determine the risk of flooding should a particular project be adopted. The re- sidual risk of flooding is certainly a primary concern. Ford (personal communi- cation, September 19, 1994) wrote, The importance of reliability is, to some extent, a function of the consequences of exceedance. If the consequences are great, then high reliability is necessary. For example, if overtopping a levee would inundate a high-density residential development to a depth of 25 feet without warning, high reliability is required. This discussion of the importance of reliability ignores the risk associated with the target flow. The reliability of the system as calculated by USACE is only part of the residual risk. It is the overall risk of flooding that is key, not the reliability of the system for particular events. That overall residual risk of flood- ing, described by AFP or Avg[AFP], and the expected annual damages (EAD) are certainly the two most important system performance criteria. A significant problem with the presentation of system reliability in Table 4.4 and Figure 4.5 is that reliability appears to address the residual risk of flooding, while it actually hides the true answer in a matrix of less meaningful numbers. To compute the actual risk of flooding (as described by AvgLAFP]), one would need to compute the average across all failure probabilities of the reliability of the system. There are insufficient numbers in the table to do this computation, and interested individuals should not have to do it themselves. Engineers and plan- ners should perform these calculations and provide the results. Table 4.4 and Figure 4.5 appear to reflect a desire to hold on to the old idea of "level of protection," expressed by the hydrologic return period T or exceedance probability for a design flood, while moving to a new risk analysis methodology that includes the idea of uncertainty and variability in other pro- cesses. Davis (1991) noted that traditionally projects were defined by the target "level of protection." The problem with the presentation of system reliability versus a target failure probability is that it fails to integrate those two sources of risk. 6. In the American River study, reliability is also used to demonstrate that the reliability of the levee network across the American-Sacramento River sys- tem is not impaired by a project. This is a legitimate concern and one that a risk analysis methodology should be able to address. The Sacramento District has demonstrated how its reliability index can be calculated at different points in the

56 FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN system for different probability levels to demonstrate that "reliability" is not Impaired. The committee wondered if this is the most effective definition of reliability for that purpose. If reliability is used to ensure that for every flood level de- scribed by a cumulative probability p, throughout the river system the probability of flooding is not increased by a project, it is not necessary to include hydrologic discharge-frequency uncertainty. It is much simpler to specify a range of Folsom inflow hydrographs and evaluate the reliability for each. The "reliability of the system for a given inflow" is both simpler and more meaningful than the "reli- ability of the system for a given exceedance probability including our inability to determine the flow actually associated with that exceedance probability." In this regard, requiring reliability to remain the same for every hydrograph is a more demanding requirement than requiring that it not decrease for every median exceedance probability after averaging over hydrologic uncertainty in the frequency curve. The first approach requires that reliability not decrease at every flood flow; the second requires that reliability not decrease for averages over flow ranges. Keying on clearly specified flood hydrographs with their associated peak and volume seems to meet the requirement of ensuring that reliability not de- crease more rigorously than the approach the Sacramento District has adopted. It would also be clearer and easier to understand. Moreover, it is also easier to compute and relate to levee and channel system performance because those un- certainties will not be swamped by the potentially much larger hydrologic uncer- tainty. Overall, the committee applauds the USACE decision to adopt a risk-based planning methodology that better incorporates uncertainties in key variables. However, the committee does not believe that the definition for system reliability that was proposed in USACE guidelines and adopted by the Sacramento District is particularly effective at addressing the relevant issues. In many cases, it seemed unnecessary or misleading. Annual failure probability (AFP, Avg~AFP], or both) is likely to be the most straightforward and easily understood measure of residual flood risk. It could be supplemented by the vulnerability criteria dis- cussed in the risk communication section of Chapter 6. THE 1994 ALTERNATIVES REPORT The committee reviewed the 1994 Alternatives Report (USAGE, Sacramento District, 1994a) and found the document to be particularly confusing. The report provided a summary of its evaluation of different projects consisting of alterna- tive modifications of the system. Unfortunately, essential details of the analysis were omitted, so the committee could not determine what was actually done from reading the report. In particular, the committee could not determine the extent to which some criticisms of the 1991 analyses had been addressed.

RISK METHODOLOGY 157 The report also failed to associate with the estimated net benefits any mea- sure of overall uncertainty due to economic uncertainty, or hydrologic and hy- draulic uncertainties, as recommended by EC 1105-2-205. These uncertainties could be important given the modest benefit-cost ratios calculated for the alterna- tives considered. A very serious concern is how the report addressed issues of risk terminol- ogy and its reporting of flood risk. USACE now has two significantly different ways to calculate flood risk. On pages 8 and 9 of the 1994 Alternatives Report, they were both called "level of protection." No distinction was made between estimates of flood risk calculated with the traditional level of protection method- ology and those calculated with the new risk and uncertainty methodology. Throughout the report a host of different terms and phrases were used inter- changeably to describe these ideas. A layperson would have great difficulty sorting out the following jumble of terms: T-year level of protection, exceedance interval (p. 8), return period (p. 8), recurrence frequency (p. 9), control for Tyears (pp. 18 to 23), T-year flood (p. 9), T-year flood protection (p. 6), T-year protec- tion (pp. 27, 29), T-year return frequency (p. 34), expected exceedance (pp. 37, 39), expected level of protection (p. 57), annual recurrence (Plate 5), and flood event return period (Plate 12J. The report should use a few terms whose defini- tions are both clear and consistent with commonly accepted interpretations. The most common terms in the report are T-year level of protection, T-year protection, control for T years, and T-year flood protection. The use of the term "level of protection" to describe flood risk is inconsistent with the new USACE guidelines for risk and uncertainty analyses (EC 1105-2-205' and confuses the traditional and the new approaches to calculating flood risk. This terminology supports the erroneous idea that one and only one T-year flood occurs every T years. Actual statements in the report reinforce the error. On page 9, flood risk was described as a flood once in 78 years or 103 years, while the executive summary indicated that "levees could fail about once in every 78 years" and "the level of protection (or likelihood that levees would not fail) would be increased to about once in 100 years." These are exactly the analogies that should be avoided. With the new risk and uncertainty methodology, estimates of flood risk are no longer tied to a single T-year design flood, but can depend on different combi- nations of flood flows, operating decisions, and levee performance. Instead of stating that a project has a 200-year "level of protection" or protection for the 200-year flood, the Sacramento District should instead indicate that the annual risk of flooding is 0.5 percent per year, or the annual risk of flooding is 1 in 200 (see Stedinger et al., 1993, p. 18.3~. It is also informative to convert such annual risks into the risk of flooding over 25 to 50 year periods, reflecting the likely length of a mortgage or the anticipated economic life of structures and dwellings. When producing the 1994 Alternatives Report, the Sacramento District was under a great deal of pressure to revise its analysis of flood control alternatives to provide protection for people and property along the lower American River. Its

158 FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN difficulties were increased by the need to use the new risk and uncertainty meth- odology being developed within the Sacramento District for the planning of flood protection projects. Inadequacies in the 1994 Alternatives Report reflect those pressures and constraints. The committee hopes that subsequent documents will more clearly describe how the analyses were conducted and will more clearly explain the basis for the risk and uncertainty analyses. THE PROMISE OF ECOLOGICAL RISK ASSESSMENT USACE has made a commendable effort to apply recently developed risk and uncertainty analysis to the engineering problems faced in minimizing the damage from floods. The question then arises: Should not the relevant ecologi- cal risk and uncertainty that may be the consequence of each of the proposed actions also be subjected to risk analysis? Applying ecological risk assessment to the major areas of uncertainty would be a daunting task. The following discus- sion highlights some of the advantages and disadvantages of such an approach. Development of the Paradigm Formal assessment of risk in ecological science and management is a rela- tively new development. Until very recently, EPA had not developed any guide- lines for risk assessment (Suter, 1993~. Thus far, the principal application of risk assessment to ecological problems has been in the context of considering impacts of hazardous chemicals in the environment, evaluating the risk of extinction of rare or endangered species, or providing management advice for commercial fisheries. Conceptually, there seems to be no reason that the process could not be applied to assessment of potentially adverse effects of water projects such as the ones considered here in the American River. However, the extension to such an analysis is controversial (Lackey, 1994) and probably will not be generally ac- cepted in the scientific community at this time. Ecological risk assessment has evolved slowly over the past two decades, but has received impetus from the National Research Council (NRC) paradigm for human health risk assessment: Risk Assessment in the Federal Government: Man- aging the Process (NRC, 1983~. EPA has recently released a Framework for Ecological Risk Assessment (EPA, 1992), along with a series of case studies (EPA, 1993, 1994~. These publications do not present final policy and proce- dures but are designed to stimulate discussion and development of a process that will be in flux for some time. EPA is developing formal guidelines for conduct- ing ecological risk assessments, which are expected to be released in late 1995 or early 1996. NRC has been in the forefront of such development, with reports on risk communication (NRC, 1989) and on issues in risk assessment, including a sig

RISK METHODOLOGY 159 nificant discussion of ecological risk (NRC, 1993~. Currently an NRC committee is conducting workshops designed to build consensus on the philosophy and methods for ecological risk analysis. In addition, many academic and industrial scientists are developing and evaluating the process (Suter, 19934. The debate about the extension of risk analysis to ecological problems has focused on several contentious points: · The process is based on a human health paradigm; extension to ecological effects, particularly at the ecosystem level, is highly problematic. There is insuf- ficient understanding of ecosystem processes to predict outcomes with any cer- tainty. · Risk assessment has the potential to produce a sort of an ecological triage, whereby particular processes and species thought to be important might receive attention at the expense of some potentially serious problems. · Risk analysis may lead to a consideration of alternatives that is too nar- row, particularly if the focus is on the risk of a particular action versus that of no action. The analysis must consider the full range of alternatives, and benefits as well as risks of all the alternatives. · The process can be tilted in favor of a particular action, given that uncer- tainty is great and the desired level of risk defined; the analysis may simply proceed until the desired endpoint is reached. In spite of these serious concerns, ecological risk analysis has had some success, leading to models that may provide templates for further development. A recent NRC report (NRC, 1993), in a section titled "A Paradigm for Ecological Risk Assessment," recognized significant problems in extension of the health risk approach from NRC (1983~. Nonetheless, that committee concluded that inte- grating ecological risk into the original framework is possible and that such an approach is preferable to developing a completely new framework. Key scien- tific issues limiting the application of ecological risk assessment include the following: · Extrapolation across scales of space, time, and ecological organization. Estimating ecosystem-level response on the basis of laboratory or small-plot experiments is a particular concern. · Quantification of uncertainty, including measurement uncertainty, natural variability in ecological systems, and inadequacy of models. . Validation of predictive tools. Substantial improvements are needed in the models fundamental to effective risk assessment. · Valuation of outcomes. Analysis of both costs and benefits is essential, but generally accepted principles for valuation of ecosystems are not available.

60 FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN Ecological Risk Assessment and the American River This committee believes that the Sacramento District has done a reasonably effective job of framing alternatives in the American River planning activities, particularly in the 1994 Alternatives Report. The recent organization of the Lower American River Task Force under the sponsorship of SAFCA has substan- tially improved communication among the various stakeholders in the basin. Hence there is the potential for appropriate use of ecological risk analysis. None- theless, there is little likelihood that such an analysis would be accepted by the scientific and lay community at this stage in the development of flood control proposals for the American River. One of the most contentious environmental issues faced by the committee is the assessment of the potential effects in the canyons of the North and Middle Forks above a proposed detention dam at Auburn. Great uncertainty surrounds estimates of the probability of mass soil failure and mortality of vegetation fol- lowing inundation. A case study example is available of risk analysis applied to a similar situation, modeling future losses of bottomland forest wetlands in Loui- siana in the face of increased flooding (EPA, 19931. However, this analysis was based on a substantial body of research in that region and on the application of a simulation model adapted for the specific area. No such base of knowledge is available for the American River canyon. Scientific understanding that would allow accurate modeling of the processes involved in hillslope failure and mortal- ity of vegetation is simply not available at this time, and most likely will not be available for years. Significant opportunities were missed when research failed to take advantage of the presence of the cofferdam upstream of the Auburn dam site, though the detention dam concept was not developed until after the dam breached in 1986. One field of resource management has a relatively long history of recogniz- ing uncertainty and may have lessons to provide as ecological risk assessment develops. Managers of marine and anadromous fisheries have long faced uncer- tainty. Stimulus for the development of more robust approaches to prediction in the face of incomplete knowledge has often come from the collapse of large fisheries (Ludwig et al., 1993~. The model of adaptive management advocated by Holling (1978) and Walters (1986) recognizes that uncertainty is a pervasive element of most resource management scenarios. The committee strongly rec- ommends that the water resource issues in the American River be managed in this adaptive context. Some important characteristics of this approach include the following: · recognizing and communicating uncertainty, · treating management as an experiment, and · providing sufficient monitoring to allow managers to learn from the expe- rience gained from observing system behavior.

RISK METHODOLOGY 161 The current direction in models being developed for management advice emphasizes Bayesian analysis (Walters, 1986; Hilborn and Walters, 1992) and statistical decision theory (Frederick and Peterman, 1995~. Another trend in more traditional statistical analysis of natural resource issues has been a focus on statistical power analysis, particularly in the analysis of downward trends in resource abundance (Peterman, 1990~. The approach has promoted more explicit consideration of where the burden of proof properly lies. Incorporation of these concepts into ecological risk analysis should improve future decisions in a wide array of resource conflicts. CONCLUSION From its review of the material provided describing the new USACE risk and uncertainty analysis guidelines, and the 1994 Alternatives Report, the committee reached the following conclusions. · Improvements in Planning Methodology. The USACE risk and uncer- tainty methodology is an innovative and timely development. The explicit recog- nition of modeling uncertainty should result in a better understanding of the accuracy of flood risk and damage reduction estimates. The committee applauds the USACE efforts to develop a better flood protection planning methodology incorporating both risk and uncertainty in hydrologic, hydraulic, and economic parameters and processes. However, USACE and the Sacramento District need to more carefully develop and articulate the structure of their risk and uncertainty methodology, employing an effective vocabulary for distinguishing among risk, variability, uncertainty, and system reliability for use with technical and public audiences. USACE leadership is encouraged to convene an intra-agency work- shop, including outside experts, to review the risk and uncertainty procedures, with special attention to the committee's concerns, and to recommend specific changes to the guidelines as necessary. · Impact of Uncertainty on Performance Criteria. The proposed USACE risk and uncertainty methodology, which directly includes hydrologic uncertain- ties (and potentially other sources of uncertainty) in the calculation of average flood risk and the average annual flood damages that might be averted by a project, inflates those estimates. This upward bias is a concern if the methodol- ogy is adopted nationwide because it could distort the economic evaluation of projects. The committee did not have the resources to determine the actual distortion for the American River study. · Descriptions of Project Performance. To avoid the problem of bias described in the recommendation above, and to simplify the analysis so that it can be more easily understood and is less dependent on hidden assumptions, the committee recommends that the primary descriptions of the expected annual flood damages and of the probability of flooding be based on best estimates of the

162 FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN parameters of models defining the deterministic and significant random pro- cesses contributing to flood risk and flood damage. · Descriptions of Project Performance Uncertainty. Best estimates of expected annual flood damages and the risk of flooding should be supplemented by descriptions of their uncertainty due to hydrologic, hydraulic, and economic uncertainties. Uncertainty can be described by a standard error or the distribu- tion of the likely values of the quantity of concern. The impact of uncertainty can be illustrated by computing the probability that the national economic devel- opment objective is negative, or various quartiles of its distribution. The ap- proach should be consistent with the requirement in USACE guidelines for risk and uncertainty analyses (EC 1105-2-205' that the estimate of NED benefits be reported both as a single expected value and on a probabilistic basis (value of the benefit and its associated probability) for each planning alternative. It is the committee's understanding that the American River study will not address eco . . . noetic uncertainties. · Measures of System Performance Reliability. Estimates of expected annual flood damages and economic benefits associated with different projects, and the probability of flooding at different locations, are likely to be the primary criteria describing flood risk and economic impacts. It will often be useful to calculate other indices of system performance and the reliability of different components of the river channel and levee system. The committee questions in general the value of the system reliability index proposed by USACE documents and employed by the Sacramento District in the American River study. It seems to be an awkward combination of traditional and new concepts. In the case of the American River study, a reliability index did have an important role in demonstrating that different projects do not increase the risk of flooding in any reach of the American-Sacramento River system. Still, it is not clear that the adopted definition is the most effective or easily understood. How- ever, the Sacramento District's use of reliability does not affect the validity or accuracy of the study results and the calculations upon which they are based. · Risk Analysis in USACE Alternatives Report. The committee re- viewed the risk and uncertainty analysis in the 1994 Alternatives Report. The report failed to associate with the estimated net benefits any measure of overall uncertainty due to economic, hydrologic, and hydraulic uncertainties. The com- mittee found the explanation and presentation of the results particularly confus- ing. No distinction was made between estimates of flood risk calculated with the traditional level-of-protection methodology and those calculated with the new risk and uncertainty methodology. Both were called "level of protection" and described by a variety of terms, which further contributed to the confusion. The most common terms in the report are control for T years, T-year level of protection, and T-year flood protection. The use of the term level of protection to describe flood risk is inconsistent with the new USACE guidelines for risk and

RISK METHODOLOGY 163 uncertainty analyses (EC 1105-2-205) and confuses the traditional and the new approaches to calculating flood risk. This terminology and phases appearing in the report fosters the erroneous idea that one and only one T-year flood occurs every T years. Moreover, with the new USACE risk and uncertainty methodology that was employed, failure is no longer related to a single T-year design flood being exceeded, but can depend on different combinations of flood flows, operating decisions, and levee perfor- mance. Instead of stating that a project has a 200-year level of protection, or protection for the 200-year flood, studies should indicate that the risk of flood- ing is 0.5 percent per year, or equivalently that the chance of flooding is 1 in 200 each year. · The Promise of Ecological Risk Assessment. At this time the committee does not believe that the process of ecological risk analysis is sufficiently evolved, nor that there is sufficient knowledge of the ecological system, for this new tool to be applied usefully to problems of flood control in the American River basin. However, ecological risk assessment does provide a new approach that emphasizes the importance of uncertainty in the analysis of the conse- quences of various alternatives. The process will help select questions for inves- tigation and will be increasingly important in broadening the scope of future planning. USACE should follow this rapidly evolving approach and adopt it as soon as it shows promise of improving the decisionmaking process.

Next: 5 FLOOD RISK MANAGEMENT BEHIND LEVEES »
Flood Risk Management and the American River Basin: An Evaluation Get This Book
×
Buy Paperback | $44.00 Buy Ebook | $35.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

This book reviews the U.S. Army Corps of Engineers' (USACE) investigations of flood control options for the American River basin and evaluates flood control feasibility studies for the watershed, with attention to the contingency assumptions, hydrologic methods, and other analyses supporting the flood control options.

This book provides detailed comments on many technical issues, including a careful review of the 1991 National Research Council report American River Watershed Investigation, and looks beyond the Sacramento case to broader questions about the nation's approach to flood risk management. It discusses how to utilize information available about flood hazard reduction alternatives for the American River basin, the potential benefits provided by various alternatives, the impacts of alternatives on environmental resources and ecosystems, and the trade-offs inherent in any choice among alternatives which does not lie in the realm of scientists and engineers, but in the arena of public decisionmaking.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!