Society has been attempting to deal with flooding for several millennia. Initially, those at risk of flooding moved out of the way of rising waters. If that became impractical, leaders would turn to methods that would control floods by keeping them away from the population through use of embankments, channels, and elevating structures. This emphasis on flood control began in the third millennia B.C. and continued through to the middle of the 20th century when it was recognized that structures alone could not overcome the flood challenges (Sayers et al., in press). In the mid-20th century, other tools such as floodproofing, building codes, land-use management, and early warning systems joined flood control structures in flood damage reduction efforts. However, as population and development grew, flood losses continued to rise and the need for the establishment of priorities for the use of scarce resources became apparent (Figure 3-1).
Following in the path of several centuries of experience in the insurance and financial industries, flood management professionals across the globe began to examine risk management methodologies that could be used to assess the flood hazards that had to be faced and the consequences that would result from flooding, and the relative significance of the risks that were identified. By the end of the 20th century, the concept of flood risk management had been widely accepted in Europe and was beginning to take hold in the United States. This evolution of approaches to dealing with floods is described in Figure 3-2.
Flood risk is a function of the characteristics of the hazard, the exposure to the hazard, the vulnerability1 of that which is exposed, and the consequences that could occur should the hazard reach the exposed, and vulnerable elements of the potentially flooded area. Probabilities are a part of all elements of risk—the probability of the event taking place, the probability that the exposed area will be flooded, the probability that the flood mitigation systems in place will successfully reduce vulnerability, the probability that the people and property in the target area will remain in the area and be subject to flooding, and the probability that they have insurance, etc. Flood risk management represents comprehensive efforts, to continuously carry out analyses, assessments, and related mitigation implementation activities to reduce flood risk (FLOODsite, 2004; Sayers et al., 2012).
Movement to flood risk management requires that those responsible enter into an iterative process that seeks to identify the hazards faced by the community and assess the exposure, vulnerability, and potential consequences of these hazards should a flood event occur. Provided with this information, the community then develops and
1 Vulnerability is “the potential for harm to the community and relates to physical assets (building design and strength), social capital (community structure, trust, and family networks), and political access (ability to get government help and affect policies and decisions). Vulnerability also refers to how sensitive a population may be to a hazard or to disruptions caused by the hazard” (NRC, 2012b).
FIGURE 3-1 Estimated Total Flood Damages in the United States, 1934-2000, billions of dollars. Data adjusted to inflation by using the Construction Cost Index (CCI) from the McGraw-Hill Construction Engineering News-Record. To adjust each year’s number, the 2011 CCI was divided by the CCI of the year in question and then multiplied by the raw damage amount for that year.
SOURCE: NOAA (2012).
FIGURE 3-2 The evolution of flood management practice through history. It can be anticipated that in the years ahead, advances in technology will provide capabilities that will permit more effective and efficient capabilities to identify and deal with risk. At the same time, increased communications capabilities will better prepare the population at large to understand and participate in the development and use of risk strategies.
SOURCE: Sayers et al. (in press).
implements a strategy for coping with the risk. The strategy and implementation remain under continuous review, and when circumstances dictate, adjustments are made to the strategy. Risk communication is a continuous effort throughout the process (Figure 3-3). The effectiveness of the flood risk management process is linked to sound data and modern computational tools.
A risk analysis is a detailed examination:
performed to understand the nature of adverse consequences from a particular event to human life, property, or the environmental; an analytical process that provides information about or quantifies probabilities and consequences of an unwanted event. Oftentimes broad definitions of risk analysis include examination of risk communication, risk perception, and risk management alternatives. (NRC, 2010)
A subset of a risk analysis, a flood risk analysis is an analytical process that provides information about or quantifies probabilities and consequences of a flood event. A flood risk analysis is a subset of the broader risk-based approach to flood risk management described throughout this report (Figure 3-3).
The use of risk-based methods to evaluate the performance of civil infrastructure systems such as nuclear power plants and dams (URS/JBA, 2008; ASME/ANS, 2009) and to serve as the basis for building codes (DOE, 2002), evaluation of terror threats (NRC, 2010) is now well established. Similarly, in the context of the insurance industry, risk analysis methods are used routinely to conduct portfolio assessments (AIR, 2012; RMS, 2012). The advancement of risk analysis methods to evaluate major infrastructure systems and extensive portfolios parallels the improvement in technology; software, hardware, and key datasets (e.g., Lidar, building inventories). As a consequence, software tools and hardware to support risk-based analyses have advanced to the point that evaluation of large portfolios has been performed (URS/JBA, 2008; IPET, 2009; AIR, 2012; RMS, 2012).
FIGURE 3-3 The flood risk management cycle illustrates the multistep flood risk management process.
SOURCE: Sayers et al. (in press), NRC (2012a), Moser et al. (n.d).
The use of risk analysis methods to evaluate flood risks and specifically to consider the performance of levees has been ongoing and improving for over a decade (USACE, 1996; URS/JBA, 2008; IPET, 2009). In the 1990s, USACE developed a probabilistic approach and software tools to evaluate flood damages (USACE, 1996). This methodology was used as part of planning studies to evaluate flood loss reduction benefits. The National Research Council (NRC) reviewed the USACE flood loss reduction methodology and reported on its findings in 2000 (NRC, 2000). The NRC found the USACE techniques a significant step forward and commended the agency for embracing risk analysis techniques. The NRC also noted that the USACE approach was technically sound, but aspects of the methodology as it related to the evaluation of uncertainties required improvement.
Following Hurricane Katrina, USACE established the Interagency Performance Evaluation Task Force (IPET) to determine the causes of levee and floodwall failures in New Orleans. Early work by the IPET, which involved over 300 professionals from government agencies, academe, and the private sector, used risk-based techniques in its analysis and the IPET recommended that USACE begin a move to risk-based approaches, to include geotechnical analyses in its work. Shortly thereafter, USACE announced that it would become a risk-based organization and that efforts would be made to accelerate the use of risk-based techniques not only in studies of future work but in analysis of existing structures (USACE, 2008).
Title 44, Section 65.10, of the Code of Federal Regulations (44 CFR §65.10) refers to the “risk study” that is performed when a community is seeking recognition of a levee system. Although the regulation does not specifically spell out what is meant by a risk study, in practice, it is a flood hazard analysis. The flood hazard analysis currently used within the NFIP to assess flood risk establishes a legislated level of protection (the one percent annual chance flood), using standard hydrologic methods that compute the expected elevation of that flood at a given location and assumes that any flood protection structures at that location designed to pass the one percent annual chance flood will in fact do so. To account for uncertainty, structures are required to be built to the one percent annual chance flood elevation plus a specified number of feet of freeboard.
To elaborate, under current NFIP policies a partial risk-based analysis is used with respect to performance of levee systems, where many parts of the analysis (e.g., the geotechnical component) are deterministic in fashion. A levee system is only recognized for its benefits if it is “accredited,” meaning that the levee system has been determined to meet minimum design, operation, and maintenance standards that are consistent with a level of protection associated with the ability to pass the one percent annual chance flood, as specified in section 65.10. In other words, the levee system is only considered if it performs perfectly up to standard; if not, it is ignored. Thus, if a levee system is accredited, its potential failure is included in the flood hazard analysis and it is simply considered as part of the river hydraulics. A levee system that does not meet accreditation standards is not considered in the analysis used to quantify flood risk, even though it provides some (potentially considerable) protection against flooding.
The current approach to flood risk analysis does not address certain components that are critical to a modern flood risk analysis. These include the uncertainties in the hydrology, the probabilities that a protection structure might fail at less than the design elevation, the consequences that will result from the actual flooding, and the probabilities of the success of actions such as, for example, evacuation of the elderly and disabled. Capturing these factors in the NFIP’s approach to flood risk analysis is the source for the committee’s advice that the program move to a modern risk-based analysis.
To elaborate on these issues, communities construct levee systems to protect up to the one percent annual chance flood, enabling new development in areas with significant, but un-quantified exposure to catastrophic flood risk. Thus, protection against the one percent annual chance flood event is the de facto standard for most levees in the United States, with limited regard to residual risk—that is, the consequence of capacity exceedence.2 Furthermore, levee systems that only marginally meet certification standards are vulnerable to decertification. If not properly maintained, the performance of levee systems degrades over time due to erosion, rodent damage, and subsidence. Further, the frequency and magnitude of flood hazards can increase over time due to natural and anthropogenic causes.
2 The State of California recently legislated that levees protecting urban areas should provide protection against the 0.5 percent annual chance flood (200-year) (California Government Code § 65007(l)).
Under current NFIP policies, lands behind accredited levee systems are not designated as being in a Special Flood Hazard Area (SFHA) and are not required to manage flood risk, even though the risk can be significant. Floods greater than the one percent annual chance flood can and do occur. Accredited levees can fail and have failed during smaller floods, causing catastrophic damage to structures and endangering residents. For example, in February 1986 the Linda-Olivehurst levee, a certified levee located on the Lower Yuba River in California, failed (Figure 3-4). Economic and life-safety risks associated with accredited levee systems are not accounted for in the current NFIP policy, which generally means that flood risks have not been effectively evaluated and, as a result, not effectively managed.
FIGURE 3-4 Images of the certified Linda-Olivehurst levee failure in 1986. The failure occurred after the flood had crested 8.6 feet below the levee crest. (Top) The breach was 170 feet wide and (bottom) entire neighborhoods were flooded in the surrounding community. The economic loss was $1.5 billion and lives were lost.
SOURCE: Rogers (2012).
The summation of these factors is that, under current FEMA policies and flood hazard analyses, the flood risk for many lands is not described completely, for a variety of reasons. This is not unknown to FEMA or relevant stakeholders including policy makers. In its 2006 report, the Interagency Levee Policy Review Committee recommended that within 5 years FEMA should, in the evaluation of levees, begin to use the risk-based hydrologic techniques being employed by USACE (ILPRC, 2006). In 2007, USACE, in coordination with FEMA, established the National Flood Risk Management Program, emphasizing the intention of the USACE to move to risk-based approaches to flood management. In 2009, with the creation of the Risk MAP program, FEMA identified risk-based analysis as a critical element in efforts to accurately identify the risk to those who occupy flood-prone areas.3 As indicated in Chapter 2, the Biggert-Waters Act placed emphasis on risk analysis in NFIP activities and directed the formation of the USACE-FEMA Flood Protection Structure Accreditation Task Force to better coordinate the data development of the two agencies in levee accreditation.4
In this report, a modern flood risk analysis is defined as a risk analysis using the best available science and analytical methods to evaluate flood risk. This is more precisely defined as an analysis where the likelihood of adverse consequence (loss, personal injury) is quantitatively evaluated, taking into account
1. the likelihood of flood events occurring (the randomness of future events);
2. the likelihood that the capacity of levee and other flood protection systems may be overwhelmed by the flood (be overtopped) or fail (the randomness of structure performance for given levels of loading), leading to flooding of protected areas; and,
3. given flooding has occurred, an assessment of the consequences that occur.
Each element of the analysis is subject to uncertainty; neither the chance that different-size flood events can occur nor their magnitude can be estimated exactly—they are uncertain. These uncertainties and the related implications to the estimate of risk are quantitatively evaluated in a modern risk-based analysis.
There are a number of clear and specific attributes of a flood risk analysis that are not inherent to the definition, but at the same time, define what it means to perform such an analysis. For instance, the analysis considers the occurrence and consequences for a full range5 of flood events that can occur, impact a community, and affect the assessment of insurance rates; it models and analyzes the hydrologic system, including levees and flood protection systems as systems, physically and logically integrated features that impact the outcome of future events; and it quantitatively evaluates the sources of uncertainty that affect the results and the estimation of future losses.
The ILPRC (2006) and many others (Appendix F) recommend that the NFIP should move to implement a more complete approach to the risk analysis that is the cornerstone of its mapping and insurance program. The committee concurs with these recommendations. A modern risk analysis would take advantage of new computational and mapping capacity to produce state-of-the art risk estimates for all areas that are vulnerable to flooding hazards. The analysis would provide a foundation for the NFIP to directly evaluate flood hazards and the performance of levee and other flood control systems and assess the consequences of flooding using sound engineering assessment and scientific and probabilistic methods. Specifically, the modern risk analysis would
3 Although Risk MAP is focused on new and better analysis to support existing criteria and methodologies of the NFIP, this focus does not prevent the program from considering risk in the broader sense. The committee considers this program an important step in FEMA’s attempt to move toward risk-based analysis.
4 Section 100221, Interagency Coordination Study.
5 The term “full range” is used in general terms to reflect that the likelihood of small as well as large flood events are considered in the analysis, not just a single event. The range that is considered should be broad enough to include the frequency and consequences of events that are meaningful to floodplain management and determination of insurance rates.
• provide direct engineering- and reliability-based assessment of the performance of flood protection systems in the evaluation of flood hazards in protected and unprotected areas;6
• account for the performance of risk reduction that is attributable to nonstructural measures;
• provide a means to establish an insurance program that is based on a continuum of flood risks as opposed to the current approach that is partially deterministic;
• offer a clear and informed basis for risk communication;
• provide a reliability-based means for the assessment of levees and flood protection systems and their integrity;
• improve the long-term stability of flood hazard and risk estimates that are a basis for insurance pricing, presuming that uncertainty and changing conditions are accounted for properly throughout the analyses;
• require that levees and appurtenant structures and subsystems be evaluated and modeled as part of flood protection “systems” whose reliability has a significant impact on the flood risks that communities face;
• afford the opportunity for communities and others to understand their flood risks at the community level and to assess the economic benefit of alternative floodplain management options; and
• provide risk information at a community as well as an individual property scale, which would offer the opportunity for communities to more clearly understand the value of community participation in the NFIP.
Such an analysis would provide an in-depth technical evaluation of flood hazards and directly account for the performance of levee flood protection systems, whether accredited within the NFIP or located within an NFIP community (i.e., levee systems that protect against the one percent annual chance flood or not). It would account for all elements of the flood protection system (levees, gates, other structures, etc.) that significantly affect flood risk, as long as they meet minimum design, operation, and maintenance standards. Hence credit could be given when establishing flood insurance rates for flood protection systems that do not protect to the one percent annual chance standard (or any new standard), but at the same time provide some level of protection. Risk analysis results would also differentiate between protection provided by levees that protect to the one percent annual chance flood and higher, for example, 0.2 percent levees. It would be able to recognize the difference in risk between low areas behind levees and higher ground.
In addition to the foregoing, a risk-based analysis offers other advantages. A modern risk-based analysis will account for the uncertainties that affect the quantitative representation of the flood hazard and the estimated performance of flood protection systems. The current analysis for flood risk estimation neglects significant sources of uncertainty in the hydrologic and hydraulic modeling and it does not account for any aspect concerning the performance of flood protection systems other than the binary aspect (meets or fails to meet criteria). And it does not account for uncertainties in the consequences of flooding. The proposed modern flood risk analysis directly accounts for these uncertainties, building on techniques that have been developed and applied in the evaluation of risks associated with other natural phenomena (e.g., earthquakes) and that have been used in the flood and levee areas (USACE, 2006; URS/JBA, 2008; IPET, 2009).
In a broader context, utilizing a modern risk analysis will provide a sound foundation for transitioning to risk-informed floodplain management. For example, it will provide flood hazard and risk information (at the individual property and at the community level) to communities in areas protected by levees and will identify their risk. The present system has implied to communities that live behind accredited levees that they are “safe.” This implication and the lack of specific risk information can be addressed with risk analysis products that are designed, with the support of risk communication experts, to better inform communities and decision makers about flood risks. Also, a modern risk analysis would provide guidance to the communities for development of new strategies that would encourage wise floodplain management.
In addition to advancing the technical basis for flood risk analysis, the approach described here is, in many ways consistent with the methods being developed and implemented by USACE (Box 3-1). However,
6 The terms “reliability” and “performance” are used differently across the flood risk community. For the purposes of this report, the geotechnical behavior of levees is referred to as performance. Reliability is used to refer to the likelihood under a given condition that the levee or levee system will perform in a way for which it was designed (IPET, 2009).
USACE Risk-Related Guidance
USACE began to develop an approach to using modern risk concepts in the evaluation of flood management projects in the early 1990s. In 1996, the agency published its first formal guidance document on risk, Risk-Based Analysis for Flood Damage Reduction Studies (USACE, 1996). This manual describes how the risk concepts would be used in USACE flood damage reduction studies in the conduct of hydrologic, hydraulic, geotechnical, and other analyses. The manual became the standard for conduct of hydrologic and hydraulic analyses for planning studies but was not adopted by the geotechnical and other communities within USACE. In 2000, USACE issued Design and Construction of Levees to promulgate the basic principles that should be used in design and construction of levees (USACE, 2000). That manual noted the use of a risk-based approach and hydrologic and hydraulic design, but indicated that geotechnical would continue to use a deterministic methodology.
In 2006, in the wake of Hurricane Katrina, USACE issued Risk Analysis for Flood Damage Reduction Studies, to update the 1996 manual on risk-based analysis, noting that
The ultimate goal is a comprehensive approach in which the values of all key variables, parameters, and components of flood damage reduction studies are subject to probabilistic analysis. Not all variables are critical to project justification in every instance. In progressing toward the ultimate goal, the risk analysis and study effort should concentrate on the uncertainties of the variables having a significant impact on study conclusions. (USACE, 2006).
Structural and geotechnical analyses were to be included in overall risk-based analyses.
In 2006, in the wake of levee failures during Hurricane Katrina, USACE directed the conduct of an evaluation of the performance of the New Orleans hurricane protection system. The results of this evaluation, which was carried out by the Interagency Performance Evaluation Task Force, highlighted the need for the use of modern risk analyses in the design and construction of levees and related flood damage reduction structures. This evaluation led to the establishment of the USACE Levee Safety Program with the mission “to assess the integrity and viability of levees and recommend courses of action to make sure that levee systems do not present unacceptable risks to the public, property, and environment.” In response to congressional and administration direction, it also initiated a levee inventory project that has been transformed into the National Levee Database (NLD) (see Chapter 6).
Faced with the need to evaluate many levees that were part of both the USACE and NFIP programs, in 2010, USACE issued the USACE Process for the National Flood Insurance Program (NFIP) Levee System Evaluation. This document provided guidance to field activities on the evaluation of NFIP levees to determine if USACE would certify that the levees met the standards of 44 CFR §65.10. The document indicated that it was “USACE policy to apply a probability and uncertainty analysis framework to NFIP levee system evaluations.” It noted however that, “Probability of exceedance and uncertainty-based methodologies are under development and emerging for structural and geotechnical engineering elements but are not yet sufficiently mature for direct application in NFIP levee system” (USACE, 2010).
In 2008, USACE established a Risk Management Center to develop policies, methods, tools and systems to enhance its risk management activities overall, including as this applied to levees and dams. As part of its tool and policy development activity, the Risk Management Center is developing processes by which levee safety can be analyzed in the field and levee safety classifications determined for use in levee management.
USACE is currently using risk analysis in its programs and continues to develop, along multiple paths, its approach to use of these concepts in levee planning, design, and evaluation. No single approach has been promulgated, but all are founded on the use of risk-based methodologies. Although USACE’s goal is to ultimately develop an integrated approach to risk analysis across its flood risk reduction portfolio, the agency has not yet issued fully coordinated policy guidance and technical instructions that would define the approach and how it will be implemented.
some differences may exist in the USACE approach and the analysis proposed here (e.g., in the evaluation of uncertainties). Yet, by adopting a risk-based approach, the NFIP and USACE can develop a common technical basis for evaluating flood protection systems and defining their future working relationship as it relates to levees in the NFIP.
Thus, the proposal to adopt a modern risk analysis is considerably more than implementing a different technical approach to the determination of insurance rates, but rather a program-wide adoption of and engagement in risk-informed management principles and practices. As such, the implementation of risk analysis methods will be an integrated program activity that extends beyond the NFIP actuaries and risk analysts.
• Flood Hazard Analysis. The hazard analysis estimates the frequency of occurrence and the magnitude of flooding (flood elevations, flow velocities, forces) that may impact a region. A key component of the hazard analysis is an assessment of the sources of epistemic uncertainty and the effect that these uncertainties have on the estimate of the frequency of occurrence and the magnitude of future flood events.
• Levee and Other Hydraulic Structure and Component Fragility Analysis. The purpose of the fragility analysis for structures and components that make up a flood protection system is to estimate their performance as a function of the flood levels (elevations, forces, etc.) that may occur. The elements of the flood protection system may include structures (levees, floodwalls, gates, etc.), mechanical or electrical components (pumps, emergency power systems, etc.) and operators (personnel responsible for gate closures, pump system operation, etc.). The hazard level that causes failure of a structure or component is not exactly known; therefore, the conditional prob-
FIGURE 3-5 Elements of the modern flood risk analysis.
SOURCE: Modified from IPET (2009).
ability of failure or damage (the fragility) is uncertain. A fragility curve shows the variation of the conditional probability of structure or component failure, given a level of flooding. At low levels of flooding, the conditional probability is zero. As flood levels increase, the conditional probability of failure increases, until a point is reached where failure is certain. There are a number of uncertainties in estimating the fragility of a structure or component, thus making the assessment of conditional probability of failure uncertain.
• Systems Analysis. The potential for flooding in a region, and in particular, protected areas, depends on the performance (response) of the natural and man-made system during a flood event. This includes all features that affect flooding along the river system, for example, the hydrologic impact of upstream structures and nonstructural measures. A system model defines the relationship between the hazard and the combination of events (levee performance including failure, nonfailure, and overtopping) that can lead to different flooding outcomes. A system model describes the relationships between different elements of the flood protection system and their performance and the interaction with the flood event itself and the likelihood that protected areas may experience flooding. These relationships may be physical/causal, functional (systems operations), or probabilistic (correlation of different factors).
• Inundation Analysis Including Levee Breach and Inundation. For a flood event that is modeled, the inundation that may occur in a region is a function of the flooding level, the performance of the structures and components that provide protection to a region, the interrelationship of structures and their performance, and the hydraulic assessment of inundation that may occur. In the event of levee or floodwall failures, an assessment of the breaching that may occur is considered. This includes the size and timing of the breach, as well as its location. When a breach occurs, properties that are located in the resulting inundation are likely to experience greater damage than if the same area had been flooded in a without-levee situation. There are a number of reasons for this. First, if there are structures in the area adjacent to the breach, these structures will be impacted by high-velocity flows. In addition, these structures may be completely destroyed as a result of the scour that takes place as the breach forms. Similarly, structures located beyond the scour zone will also experience high-velocity flows that otherwise might not have occurred in a without-levee case.
• Consequence Assessment. The purpose of the consequence assessment is to estimate the damage to structures that are inundated in a flood event. At this time, the routine consequence analysis estimates direct damages to structures. Indirect losses and such aspects as loss of life and social cultural impacts are in the earlier stages of evolution (IPET, 2009). The assessment of damage is often based on models derived from empirical loss data. However, as part of the consequence analysis, it is important to consider the range and type of damages that may occur for events where there are limited loss data. For instance, if a levee breaches, structures in the flooded area will be inundated to a certain level. Some structures may experience greater damage if they are located close to the breach scour zone.
• Risk and Uncertainty Quantification. This part of the risk analysis probabilistically combines the elements of the analysis and calculates the frequency of occurrence of flood events and the consequences (damage to structures). As part of the quantification, the epistemic uncertainties, discussed below, are also propagated through the analysis to estimate the uncertainty in the risk results.
As a starting point for the development of an NFIP flood risk analysis, a conceptual framework for a flood risk analysis can be defined as
where risk (R) is a function of
H = flood hazard,
V = vulnerability, and
C = consequence (e.g., economic impact, public safety).
This basic conceptualization of flood risk analysis has been adopted by the Department of Homeland Security and is consistent with Box 1-3 (NRC, 2010). In this context, risk is a function of the flood hazard to which a community is exposed, the vulnerability of flood protection systems, and the consequences associated with flooding and system failures and the damage to a community, including economic impact and life safety. Implicit in this conceptual framework is the probability that consequences may occur.
This conceptual framework is extended to the assessment of the probability (also expressed as a frequency of occurrence) of adverse consequences that could occur as a result of a flood hazard. In other words, this extension incorporates the concept of uncertainty into the framework by an expanded, quantitative definition of risk (Kaplan and Garrick, 1981; ASME/ANS, 2009; IPET 2009):
ν = frequency of occurrence or exceedance and is a measure of the aleatory uncertainty (the randomness of events),
ρ = probability as a measure of the confidence to which an estimate of ν is the true value, or the epistemic uncertainty in the estimate of ν.
Equation 3-2 is consistent with the definition adopted for the entire report in the sense that the estimate of risk takes into account the occurrence of the flood hazard, the exposure and vulnerabilities of people and property to the hazard, the consequences caused by the hazard, and the uncertainty about all these parameters. But most importantly, it establishes the quantitative representation of risk.
As mathematically depicted above, a flood risk analysis is an evaluation that is made on the basis of available information, in which all aspects of the analysis are subject to uncertainties: the flood hazard analysis, the assessment of levee system performance, the assessment of flooding/inundation, and the estimate of consequences. In its recent review of the DHS approach to risk analysis, the National Research Council (NRC, 2010) noted that the evaluation of uncertainties is integral to a risk analysis. In building a risk analysis capability and risk-informed culture within the NFIP, how uncertainty is defined and incorporated is critical. Experience in other fields suggests that there is considerable benefit to defining these fundamentals so that they become a core element of the methods and tools that are developed (SSHAC, 1997; USNRC, 2011).
There are different types of uncertainty to consider in a flood risk analysis. The first type is attributed to the inherent randomness of events in nature—aleatory uncertainty—and is, in principle, irreducible. Examples of this type of uncertainty are the outcome of a roll of the dice, the occurrence (the time and place) and magnitude of a flood event, the spatial variation of levee properties, and the performance of a levee during a flood. In the context of engineering or statistical models, aleatory uncertainty can also correspond to unique (often small-scale) details that are not explained by a “model.” For a given model, one cannot reduce the aleatory uncertainty by collection of additional information. One may be able, however, to better quantify the aleatory uncertainty by using additional data.
The second type of uncertainty is attributed to lack of understanding (e.g., knowledge) about physical processes or a system that must be modeled. This source of uncertainty is referred to as epistemic (knowledge-based) uncertainty. There are a number of sources of epistemic uncertainty, including limited data to estimate model parameters, incomplete understanding of physical processes or systems, the limited ability of models (engineering or statistical) to predict events or outcomes of interest (modeling uncertainty), and the potential that multiple, alternative, technically defensible models or modeling approaches exist that give different results that affect the final assessment of risk. In principle, epistemic uncertainties can be reduced with improved knowledge and/or the collection of additional information.
The process of identifying and evaluating sources of epistemic uncertainties can vary, depending on the subject, the state of scientific or engineering understanding, observational and modeling experience, etc. For example, in a field or topical area where considerable observational experience exists and statistical models are used to develop predictive tools, the analysis of epistemic uncertainties may be an integral and in-depth part of
the state-of-practice. In other fields, direct observational evidence may be limited and predictive models are based on theoretical understanding (i.e., analytical models), estimates of the model parameters, the analysts’ experience, comparisons of model predictions with observations, etc. In areas where direct observation of events/parameters of interest is limited, competing models and/or scientific interpretations may exist, it is often necessary to elicit input from experts to evaluate and quantify epistemic uncertainties (Morgan and Henrion, 1990; SSHAC, 1997; USNRC, 2011).
One result from a flood risk analysis is the likelihood that a levee system will fail or a measure of the randomness of the flood hazard and the future performance of the levee system, that is, aleatory uncertainty (Figure 3-6, top). There is also epistemic uncertainty about the true value of the levee system failure, denoted as νLevee System Failure (ν in equation 3-2). As a result, there is a distribution of the estimate (Figure 3-6, bottom), which is an appropriate description of the true value of likelihood that the levee system would fail. This distribution can be viewed as a means to estimate confidence intervals on the estimate of the frequency of levee system failure.
With respect to levees, the current NFIP approach addresses epistemic uncertainty in the assessment of the frequency of flooding, that is, the flood hazard analysis, through requirements for levee freeboard. However, the
FIGURE 3-6 Estimate of the frequency of levee system failure as a single or best estimate, aleatory uncertainty (top), and an estimate that includes the quantification of both the aleatory and epistemic uncertainty (bottom) of one result from a flood risk analysis; the frequency of occurrence per year that a levee system would fail. The bottom figure can be used to make confidence statements about the estimate of the frequency of levee failure.
requirement for levee freeboard has been an ad hoc way of recognizing there are epistemic uncertainty in the assessment of the flood hazard (the one percent flood level). In other words, adding freeboard only implicitly accounts for uncertainty in a manner that does not make best use of the information and technology currently available that takes a “one size fits all” approach.7 Thus, in some communities, more freeboard is needed and in others, less freeboard may be required. A more thorough assessment of the uncertainties in the flood hazard assessment and in levee performance would derive a situation-specific estimate of necessary freeboard.
An NFIP flood hazard analysis estimates future flood damages to insured properties for rate-setting purposes by taking into account the frequency of flooding, the performance of a levee system (in a binary fashion), and the damages that might occur to insured residences. The result of the analysis will be a distribution on the economic consequences, or the probability of certain values of loss (Figure 3-7, top). The result can be used to estimate expected annual flood losses, which is a measure of the flood risks that is used to determine flood insurance rates.
However, because of epistemic uncertainties in each of the elements of the risk analysis—in the NFIP flood hazard analysis, the evaluation of levee system performance, and the assessment of consequences—there is epistemic uncertainty in the distribution of economic consequences. A more modern flood risk analysis captures how epistemic uncertainty manifests in the estimate of the expected losses (Figure 3-7, bottom). This allows consideration of a range of values related to the confidence of a certain dollar loss (rather than a dollar value, Figure 3-7, top), which can be explicitly considered in the determination of insurance rates.
A Modern Risk Analysis: The IPET Risk and Reliability Analysis
As part of the IPET study commissioned by USACE, a risk analysis was performed that included both the aleatory and epistemic uncertainties in the hurricane analysis (i.e., storm surge and wave height analysis) and the levee fragility analysis (IPET, 2009). The IPET analysis is a recent, cutting-edge example of a risk-based approach to flood analysis utilizing the two different types of uncertainty; thus it is used as an example to further illustrate a modern risk analysis.
The concepts of aleatory and epistemic uncertainty are not always intuitive and thus can be difficult to visualize. In the IPET example, aleatory uncertainty is characterized in terms of a single, best estimate of hurricane storm surge, which is expressed in terms of the annual frequency of exceedance of peak surge elevations (Figure 3-8A, top). Similarly, in the levee fragility analysis, the aleatory uncertainty in levee performance is quantified in terms of the estimate of the conditional probability of levee failure as a function of this peak elevation (Figure 3-8A, middle). The results of the hurricane and levee fragility analyses can be combined to determine a point estimate of the annual frequency of levee system failure (Figure 3-8A, bottom). This part of the IPET analysis is consistent with the current NFIP flood hazard analysis.
The sources of epistemic uncertainty in the IPET example were based on the knowledge available in assessing the occurrence of hurricanes and sources of uncertainty in the hurricane modeling. There were a number of sources of epistemic uncertainty that were considered in the IPET hurricane analysis:
• uncertainty in the storm surge model ability to predict the storm surge and wave height;
• limits in available data including the estimated annual frequency of occurrence of hurricane events in the Gulf of Mexico, the frequency of hurricane central pressure deficits, and accuracy of bathymetry and near-shore and onshore hydraulic properties; and
• limits in knowledge of atmospheric processes, specifically the hurricane central pressure deficit and the radius to maximum winds.
The estimate of levee fragility in the IPET analysis evaluated two primary failure modes: stability failure of
7 In section 65.10, the minimum standard for freeboard in NFIP communities is described: “(1) Freeboard. (i) Riverine levees must provide a minimum freeboard of three feet above the water-surface level of the base flood. An additional one foot above the minimum is required within 100 feet in either side of structures (such as bridges) riverward of the levee or wherever the flow is constricted. An additional one-half foot above the minimum at the upstream end of the levee, tapering to not less than the minimum at the downstream end of the levee, is also required.”
FIGURE 3-7 (Top) F/N curve illustrating the estimate of the frequency distribution on economic consequences without epistemic uncertainty. This shows the frequency that the annual maximum damage exceeds a specific value; this curve is the underpinning of the NFIP Hazus Multi Hazard (Hazus-MH) calculation of Average Annualized Losses due to flooding. (Bottom) Estimate of economic consequences that includes the quantification of the epistemic uncertainty.
FIGURE 3-8 IPET description of the flood hazard (top); performance of the levee, that is, levee fragility curve (middle); and potential for flooding (bottom). Incorporation of aleatory uncertainty only is shown on the left. Both aleatory and epistemic uncertainty are shown on the right, giving a more complete estimate of the likelihood of inundation.
the levee embankment (for flood levels below the levee crest) and failure due to overtopping. The assessment of the fragility for these failure modes is subject to epistemic uncertainties; that is, an estimate of the conditional probability of failure given a peak storm surge and wave height elevation cannot be estimated exactly (Figure 3-8B, middle). For instance, epistemic uncertainties that affect the levee stability analysis include:
• uncertainty in the geotechnical stability model (model uncertainty),
• limits in available data including the estimate of soil properties and overtopping resistance of levees,
• limits in the knowledge of the levee feature.8
Given the epistemic uncertainty in the estimate of the frequency of hurricane surge and wave heights and in the estimate of levee fragility, there is a corresponding uncertainty in the IPET estimate of the frequency of level system failure. That is, rather than making a single point estimate of the frequency of failure, a distribution of estimates is made that quantitatively reflects the epistemic uncertainty in both the hurricane hazard and in the levee system fragility analyses. This distribution of levee failure frequency estimates is shown in Figure 3-8B, bottom, as a probability density function on the estimate of the frequency of levee system failure. From this result, the mean frequency of failure can be determined. In addition, the analyst can make “confidence” statements about the uncertainty in the levee system frequency failure.
The NFIP is a unique government program that has a specific mandate and, at the same time, provides information to a wide range of users, establishes a standard of practice for flood hazard analysis, and floodplain mapping, and, by default, establishes what has been seen as a public safety standard for communities behind levees. The implementation of a more modern risk analysis as a part of the NFIP, as proposed and described earlier in this report, yields enhanced information about flood hazards, performance of flood protection measures, and consequences. With that information, FEMA can further support efforts to reduce flood losses in the United States (see Figure 3-9).
Table 3-1 shows a sample of the risk analysis products, uses, and users. Although uses are limited to FEMA’s main missions (risk communication, insurance, and mitigation; column 3), this information is useful to the public or to public agencies meeting other objectives. For example, a local or state agency seeking to reduce flood risk could use information on levee performance to make investment decisions for strengthening levees or implementing a property buy-out program in areas of greatest potential damage.
Realigning the NFIP to undertake a modern risk analysis, use products of such an analysis, and account for uncertainty in the inputs is a shift from “business as usual.” Current NFIP analyses focus on hazard—primarily on the 1 percent and 0.2 percent annual chance exceedance events. Current analyses limit consideration of consequence of flooding to insurance rate setting, without geographic specificity that is possible as proposed here. Levee performance is considered in current practice in a standards-based manner, with accredited levees considered as surely protecting against inundation and unaccredited levees as surely not protecting. What is proposed herein expands the risk assessment to consider explicitly the entire range of flood events, integrates the hazard assessment with economic consequence estimation throughout the analysis and decision making, and accounts in a more robust manner for levee performance throughout the range of hazards. The NFIP should move to a modern risk analysis that makes use of modern methods and computational mapping capacity to produce state-of-the-art risk estimates for all areas that are vulnerable to flooding.
The analysis of flood risks is multidisciplinary (hydrologists, geotechnical and structural engineers, operations personnel, etc.). As procedures for the implementation of a modern NFIP risk analysis are developed, these pro-
8 Estimate of the correlation length of levee properties. Correlation length refers to the length along a levee over which properties of the levee are estimated to be homogeneous and therefore its performance during the hurricane will be uniform.
FIGURE 3-9 With results of risk analysis as proposed, annual exceedence probability for individual elements of floodplains can be mapped, shading those elements with probability > regulatory threshold. Darker elements have a greater probability of inundation; lighter elements have less. The A transect illustrates a cross section of the channel at which the flow moves from the channel onto the floodplain. This illustrates the shading with a fixed-grid two-dimensional floodplain flow model, coupled with a one-dimensional channel flood model. Thus, the probability of inundation for any location in the floodplain behind a levee could be computed and mapped, considering the natural hazard and the performance of the levee. This would enable quick inspection to determine areas of greater hazard.
cedures need to clearly define the responsibilities of the professionals involved in the study, the interdisciplinary coordination that is required, the evaluations they are expected to perform, and the results they need to generate. This report does not address these issues specifically, but does discuss steps toward the development and implementation of a risk-based program.
Implementing a Risk-Based Approach
The following tasks are necessary in order for FEMA to implement a modern risk-based approach successfully:
1. Develop, test, and deploy—with the same care and rigor with which standard procedures for hydrologic and hydraulic analysis and mapping have been developed, tested, and deployed—a procedure for the complete risk analysis. This procedure should be consistent in concept with that described in the Summary Description of the Risk-Based Approach section of this chapter, including all components shown. This includes algorithms and computational methods that consider the likely users of the risk analysis procedures.
2. Develop, test, and deploy a consistent procedure for assessing levee systems (rather than individual levee segments) and for describing levee performance with a fragility curve.
3. Design, develop, test, and deploy a software application that can be used for FEMA’s risk analysis by agency staff, affected public agency staff, and contractors. This application may incorporate features of FEMA’s Hazus application—particularly the consequence assessment features and databases.
4. Emphasize and expand FEMA’s program of collecting and analyzing flood damage data, including insur-
TABLE 3-1 Products Useful for FEMA’s Insurance, Risk Communication, and Mitigation Programs Yielded by a Modern Risk-Based Analysis.
|Information Category||Risk Analysis Product||Use||Description|
|Flood hazard (floodplain)||Floodplain stage frequency distributions for locations in the floodplain; including at the individual property level. This result will also include an estimate of the uncertainty in the flood frequency estimate.||Risk communication||Report on likelihood of inundation to various depths for individual parcels, census tracts, or other subdivisions, better communicating risk to occupants, owners.|
|Flood hazard (floodplain)||Flood stage frequency distributions for locations in the floodplain; including at the individual property level. This result will also include an estimate of the uncertainty in the flood frequency estimate.||Mitigation||Provide information on depths, velocities, and other flood properties throughout floodplain, for specified events, to inform emergency evacuation and safe shelter planning by government agencies, industry, and public.|
|Flood hazard (floodplain)||Flood stage frequency distributions for locations in the floodplain; including at the individual property level. This result will also include an estimate of the uncertainty in the flood frequency estimate.||Risk communication, mitigation||Support wise land-use decision making by identifying annual exceedence probability of a parcel, tract, etc. in interior floodplain. This is consistent with earlier recommendations by NRC (2000) that “the Corps use annual exceedance probability as the performance measure of engineering risk … and the federal levee certification program focus not upon some level of assurance of passing the 100-year flood, but rather upon ‘annual exceedence probability.’”|
|For convenience, and in keeping with tradition, this information can be displayed on a map similar to existing FEMA maps (Figure 3-9).|
|Flood hazard (floodplain)||Flood stage frequency distributions for locations in the floodplain, including at the individual property level. This result will also include an estimate of the uncertainty in the flood frequency estimate.||Risk communication, mitigation||Similar to above, but conditional flooding can be displayed to answer question: Given channel flood flow or stage of specified annual exceedence probability, will a parcel, tract, etc. on the floodplain be inundated? For convenience, this answer too can be mapped. For example, properties with annual exceedence probability >0.01 can be identified and shaded, yielding a map similar to current FEMA maps, but taking into account performance of levees in other than a binary manner (Figure 3-9).|
|Deaggregation of the flood hazard (floodplain)||Conditional probability distribution on the relative contribution of different flood scenario types (levee overtopping, levee breach, breach locations, interior drainage and ponding, etc.) to the frequency and depth of flooding.||Risk mitigation, emergency preparedness||Report on the relative contribution of different events (flood events, levee failure events, etc.) that contribute to flooding in a community. This detailed information provides insight into the hazards to which a community is exposed and can guide the selection of mitigation measures and emergency preparedness and support risk communication.|
|Information Category||Risk Analysis Product||Use||Description|
|Levee system performance||Levee system fragility estimates, including an estimate of the uncertainty in the system fragility.||Risk communication||A levee system fragility curve provides quantitative information on the performance of a levee system, which may comprise multiple levee reaches, gate structures, etc. It provides direct information on the reliability of infrastructure performance and the protection it is intended to provide for a community. This can be integrated with hazard information to provide information on combined probability that a certain flood level will occur and that the levee will protect from interior area flooding.|
|Levee system performance||Levee system fragility estimates, including an estimate of the uncertainty in the system fragility||Mitigation||Provides information useful for identification of deficiencies and for setting priorities on repairs or upgrades to existing infrastructure, especially when integrated with hazard information.|
|Frequency distribution on consequences||Frequency distribution on the consequences of flooding. This will also include an estimate of the uncertainty in the frequency of exceeding levels of consequences.||Insurance||Enables computation of expected annual damage for use with premium determination procedures described in Chapter 5. This information can be provided at the individual structure level and community level.|
|Frequency distribution on consequences||Frequency distribution on the consequences of flooding. This will also include an estimate of the uncertainty in the frequency of exceeding levels of consequences.||Mitigation||Enables computation of expected annual damage for comparison with cost estimated for flood management alternatives to assess feasibility of investment for federal, state, and local mitigation planning.|
|Frequency distribution on consequences||Frequency distribution on the consequences of flooding. This will also include an estimate of the uncertainty in the frequency of exceeding levels of consequences.||Risk communication||Provides property owner or floodplain occupant with information about flood losses that they may expect on average for any year or over a longer duration.|
NOTE: Risk analysis products are results of or intermediate products of the risk analysis described in this chapter. Uses are limited to FEMA’s main missions: risk communication, insurance, and mitigation.
ance claim data and corresponding flood properties, and adding to that damage and flood property data collected from other agencies, including USACE and state and local government agencies. Depth-damage models commonly used for assessing consequence as a component of risk analysis are best calibrated and verified with actual damage data. Completing this task will ensure that models for consequence estimates are accurate and reliable.
5. Develop, test, and deploy consistent procedures for describing the uncertainty in all inputs to the risk analysis, include uncertainty about flow frequencies, water surface elevations, levee performance models, and consequence estimates.
6. Refine and enhance the methods of communication of information on flood risk, including now information on consequence. With the revised risk analysis approach proposed here, the information to be communicated offers an estimate of direct, tangible, economic impact of flooding—the cost to a property owner and information on potential life-safety consequences. (The risk analysis conducted by IPET for New Orleans indicated the potential
deaths that could be encountered for various levels of flooding.) The revised analysis will also offer information about the probability that water will exceed specified depths in any year; this can be expanded easily to describe the probability that flood depths will exceed specified depths over the life of a mortgage or another informative duration.
7. Work closely with other federal agencies to ensure that techniques are consistent across agencies so that communities are not faced with using multiple approaches to evaluate a levee system. (See Chapter 8 for further discussion.)
New Technology and Cost
FEMA can leverage new technology to share the information now available. For example, in 2000 the state of North Carolina was designated a Cooperating Technical State by FEMA, a decision largely driven by the impact of Hurricane Floyd in 1999 highlighting the need for accurate flood maps for the state. North Carolina’s Floodplain Mapping Program began updating and acquiring high-resolution flood hazard data, specifically light detection and ranging (Lidar) to generate digital elevation data, and creating new Digital Flood Insurance Rate Maps (DFIRMs) (North Carolina Floodplain Mapping Program, n.d.). This information is available on the Web through North Carolina’s Floodplain Mapping Information System, which includes access to DFIRMs and other features such as data export capability and a property address search function to enable viewing of flood risk to a specific property (Figure 3-10).
The cost to FEMA of developing procedures for and implementing a program based on modern risk analysis cannot be predicted with certainty without information from pilot applications, which are a valuable pursuit. However, in the absence of pilot studies, a comparison between the costs for a modern risk analysis that would be above that incurred for a Flood Insurance Study (FIS) using current methods (i.e., the incremental cost) is drawn.9
Current FIS procedures require many of the same hydrologic and hydraulic inputs required for a modern risk-based analysis, which indicates that the incremental cost for pilot studies should not be significant. Similarly, information on exposure and vulnerability of floodplain property is available in FEMA’s Hazus database, and so the incremental cost of gathering information should not be significant (unless a community prefers more recent information, in which case the cost of conducting a new structure inventory would be included). Topographic data are necessary for establishing the relationship between flood hazard and consequence; these are collected for current FIS studies, and so no additional cost will be incurred for this portion of the modern risk analyses.
The greatest cost, not currently incurred for a FIS, is the cost of developing system fragility curves—the functions that represent the likelihood that levees and appurtenant structures will provide the protection for which they were designed. Developing the curves requires information about the materials, methods of construction, operation and maintenance, performance history, and so on. Much of the information required to develop fragility curves is also required to meet the current requirements for certification and accreditation under 44 CFR §65.10. However, the manner of interpretation of the information is different when compared with the proposed modern risk analysis approach. No longer can an engineer compare the system or levee state to a standard, finding that the standard is or is not met. Instead, a qualified geotechnical engineer will evaluate the integrity of a levee system and incorporate this quantitatively in the conditional performance of the system. This can be done, as the State of California demonstrated with its Central Valley Flood Protection Plan. For risk analyses to support that study, engineers developed fragility curves at approximately 300 locations in approximately 1 year, following an extensive data collection effort (CA DWR, 2012). Similarly, USACE has developed levee fragility curves for use in ongoing planning studies, all of which now employ a risk analysis along with procedures and guidance for the development of appropriate levee fragility curves (USACE, 2006; Schultz et al., 2010).
9 A FIS is an examination of flood hazards with a specific legislative definition (Appendix C). From the FIS, FEMA’s FIRMs are produced. A flood hazard study, which is discussed in this chapter, is a consideration of the frequency of flooding included in the FIS.
FIGURE 3-10 A screen shot of an area subject to flood risk in North Carolina according to high-resolution flood hazard data gathered. The menu bar at the top of the image allows the user to explore exposure of an individual structures to the flood hazard as well as other options such as a DFRIM export function. Pink depicts areas subject to one percent annual chance flooding (AE zone, floodway), blue areas are subject to 1 percent annual chance flooding (AE zone, has base flood elevation established), and green areas are subject to the 0.2 percent annual chance flooding (shaded X zone).
SOURCE: http://www.ncfloodmaps.com, accessed January 7, 2012.
A Living, Risk-Based Analysis: Maintenance and Agents of Change
A modern risk-based analysis will serve a very different role than FEMA’s current analysis of the flood hazard, which yields a flood map that is not updated with regularity. The modern risk-based approach will be an integral part of all aspects of the NFIP: mapping, floodplain management, portfolio analysis, and insurance rate assessment. As such, the risk-based analysis will be a living, integral part of the NFIP; something that is used regularly by the program actuaries and analysts. By its very nature, the modern risk-based analysis will be a regularly updated, preferably online, resource for FEMA.
When a modern risk analysis is performed for a community, it would be based on information that is available at the time. Following its completion, there are factors that may change the risk profile of a community, and therefore require maintenance (updating of certain data) and possibly revision of hazard, performance, and consequence components and the resulting risk analysis products shown in Table 3-1. Given the role the risk analysis will play in the NFIP, it is important that FEMA develop and implement procedures to periodically review, maintain, and update the risk-based analysis, as necessary. There are a number of agents of change that may affect future flood risks: climate change, community development and changes in land use, changes in population, condition of flood mitigation systems (success of system maintenance, changes in system configuration, etc.), changes in the watershed, and so on. As such, FEMA’s implementation plan needs to take these factors into account.
The processes for maintenance and review needs to include
• periodic review of the elements of the risk analysis (hydrologic analyses that describe the flood hazard, geotechnical engineering studies that yield description of levee system performance, inventory of properties, etc.);
• standards that define the steps and basis to revise, refine, or replace one or more elements of the risk analysis for a community, and that establish schedules for periodically reviewing and revising
∘ individual property information,
∘ levee and flood mitigation system properties and condition, and
∘ hydrologic and hydraulic modeling.
Within its current program, FEMA has requirements for archiving hydrologic and hydraulic models used in the FIS (FEMA, 2003) and requiring that information on levees be maintained and certification be periodically updated. With the development and implementation of a risk-based analysis to evaluate community flood risks, FEMA needs to extend these practices to a community risk-based approach and faithfully maintain them.
FEMA policy with regard to levees is spelled out in 44 CFR §65.10. In addition to defining the base flood as the one percent annual chance exceedance flood, the regulation states that a community must provide evidence of a flood protection system’s ability to provide protection from the base flood. As noted in Chapter 1, the requirements that determine whether FEMA recognizes a levee system on NFIP maps include (1) design criteria, (2) operation plan criteria, and (3) maintenance plan criteria. If the levee system satisfies the criteria and is certified, it is recognized on NFIP maps, with the protected floodplain “mapped out” of the hazard area. If the levee system does not meet the criteria, the levee system is assumed to provide no protection.
Within the context of a risk-based analysis to evaluate the floodplain and flood protection systems in particular, the analysis would quantify the residual risk in the area behind every flood protection system, and develop insurance pricing based on the level of protection that is provided. For instance, communities will be given credit for the level of protection that is provided, even if this level is below the base flood standard—the estimated risk analysis and corresponding rates will recognize this change.
AIR (American Institutes for Research). 2012. AIR Models Overview. Available online at http://www.air-worldwide.com/Models/Overview/. Accessed November 28, 2012.
ASME/ANS (American Society of Mechanical Engineers and American Nuclear Society). 2009. Standard for Level 1/Large Early Release Frequency Probabilistic Risk Assessment for Nuclear Power Plant Applications, Addenda to ASME/ANS RA-S-2009. LaGrange Park, IL: ANS.
CA DWR (California Department of Water Resources). 2012. 2012 Central Valley Flood Protection Plan, Attachment 8E Levee Performance Curves. Sacramento, CA: CA DWR.
DOE (Department of Energy). 2002. Natural Phenomena Hazards Design and Evaluation Criteria for Department of Energy Facilities. DOE-STD-1020-2002. Washington, DC: DOE.
FEMA (Federal Emergency Management Agency). 2003. Guidelines and Specifications for Flood Hazard Mapping Partners. Available online at http://www.fema.gov/library/viewRecord.do?id=2206. Accessed December 12, 2012.
FLOODsite. 2004. Flood Risk Management. Available online at http://www.floodsite.net/html/flood_risk.htm. Accessed August 14, 2012).
ILPRC (Interagency Levee Policy Review Committee). 2006. The National Levee Challenge: Levees and the FEMA Flood Map Modernization Initiative. Available online at http://www.fema.gov/library/viewRecord.do?id=2677. Accessed August 8, 2012.
IPET (Interagency Performance Evaluation Task Force). 2009. Performance Evaluation of the New Orleans and Southeast Louisiana Hurricane Protection System: Final Report of the Interagency Performance Evaluation Task Force. Available online at https://ipet.wes.army.mil/. Accessed December 4, 2012.
Kaplan, S., and B. J. Garrick. 1981. On the quantitative definition of risk. Risk Analysis 1(1):11-27.
Morgan, G. M., and M. Henrion. 1990. Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis. Cambridge, UK: Cambridge University Press.
Moser, D., T. Bridges, S. Cone, Y Haimes, B. K. Harper, L. Shabman, and C. Yoe. n.d. Transforming the Corps into a Risk Managing Organization. White Paper. Available online at http://corpsriskanalysisgateway.us/data/docs/ref/Explore%20Resources/IWR%20Reports/White_Paper-Transforming_the_Corps_into_a_Risk_Managing_Org.pdf. Accessed February 5, 2013.
NOAA (National Oceanographic and Atmospheric Administration). 2012. Hydrologic Information Center—Flood Loss Data. Available online at http://www.nws.noaa.gov/hic/. Accessed February 27, 2013.
North Carolina Floodplain Mapping Program. n.d. LIDAR and Digital Elevation Data. Fact Sheet. Available online at http://www.ncfloodmaps.com/pubdocs/lidar_final_jan03.pdf. Accessed February 21, 2013.
NRC (National Research Council). 2000. Risk Analysis and Uncertainty in Flood Damage Reduction Studies. Washington, DC: National Academy Press.
NRC. 2010. Review of the Department of Homeland Security’s Approach to Risk Analysis. Washington, DC: The National Academies Press.
NRC. 2012a. Dam and Levee Safety and Community Resilience: A Vision for Future Practice. Washington, DC: The National Academies Press.
NRC. 2012b. Disaster Resilience: A National Imperative. Washington, DC: The National Academies Press.
RMS (Risk Management Solutions). 2012. RMS Models. Available online at http://www.rms.com/models/. Accessed November 28, 2012.
Rogers, J. D. 2012. GE 301 Lectures—Evolution and Development of Flood Control Engineering. Available online at http://web.mst.edu/~rogersda/umrcourses/ge301/. Accessed November 15, 2012.
Sayers, P., G. Galloway, E. Penning-Rowsell, F. Shen, K. Wang, Y. Chen, and T. Le Quesne. 2012. Flood risk management: A Strategic Approach—Consultation Draft. UNESCO on behalf of World Wildlife Fund-UK/China and the General Institute of Water Design and Planning, China.
Schultz, M. T., B. P. Gouldby, J. D. Simm, and J. L. Wibowo. 2010. Beyond the Factor of Safety: Developing Fragility Curves to Characterize System Reliability. Washington, DC: U.S. Army Corps of Engineers.
SSHAC (Senior Seismic Hazard Analysis Committee). 1997. Recommendations for Probabilistic Seismic Hazard Analysis—Guidance on Uncertainty and Use of Experts. Rockville, MD: U.S. Nuclear Regulatory Commission.
USACE (U.S. Army Corps of Engineers). 1996. Risk-Based Analysis for Flood Damage Reduction Studies. EM 1110-2-1619. Washington, DC: USACE.
USACE. 2000. Design and Construction of Levees. EM 1110-2-1913. Washington, DC: USACE.
USACE. 2006. Risk Analysis for Flood Damage Reduction Studies. ER 1105-2-101. Washington, DC: USACE.
USACE. 2008. Improving Public Safety—From Federal Protection to Shared Risk Reduction. Letter to J. Nussle, Director, Office of Management and Budget, April 16, 2008. Washington, DC: USACE.
USACE. 2010. USACE Process for the National Flood Insurance Program (NFIP) Levee System Evaluation. EC-1110-2- 6067. Available online at http://publications.usace.army.mil/publications/eng-circulars/EC_1110-2-6067.pdf. Accessed November 27, 2012.
USNRC (U.S. Nuclear Regulatory Commission). 2011. Practical Implementation Guidelines for SSHAC Level 3 and 4 Hazard Studies, NUREG-2117. Washington, DC: USNRC.
URS/JBA (URS Corporation/Jack R. Benjamin & Associates, Inc.). 2008. Delta Risk Management Strategy (DRMS), Phase 1, Risk Analysis Report (Draft). Sacramento, CA: California Department of Water Resources.