Third Report of the National Academy of Engineering/National Research Council Committee on New Orleans Regional Hurricane Protection Projects (2006)

Third Report of the National Academy of Engineering/National Research Council

Committee on New Orleans Regional Hurricane Protection Projects

Hurricane Katrina struck the U.S. Gulf Coast on August 29, 2005. The New Orleans metropolitan region was heavily impacted, as were large portions of the Alabama, Louisiana, and Mississippi Gulf coasts. New Orleans was especially affected because of the failure and overtopping of many sections of the city’s and the region’s hurricane protection system. Over one thousand lives were lost and property damage was extensive, making Hurricane Katrina one of the most devastating natural disasters in U.S history.

To improve scientific and engineering understanding of the structural performance of the hurricane protection system for the New Orleans metropolitan area during Hurricane Katrina, the U.S. Army Corps of Engineers established the Interagency Performance Evaluation Task Force, or IPET, in October 2005. The IPET consists of hundreds of experts from federal agencies, academia, and the private sector, working in 10 study groups. All IPET study groups are led or co-led by Corps of Engineers staff. The IPET is gathering and evaluating data directed toward better understanding in five broad areas:

1. design and status of the hurricane protection system pre-Katrina;

2. storm surges and waves generated by Hurricane Katrina;

3. performance of the hurricane protection system during and after the event;

4. societal-related consequences of Katrina-related damage; and

5. risks to New Orleans and the region posed by future tropical storms.

The Assistant Secretary of the Army for Civil Works requested the National Academy of Engineering (NAE) to convene an independent review committee to provide advice to the IPET effort. In December 2005 the NAE and the National Research Council (NRC) appointed the Committee on New Orleans Hurricane Protection Projects for this purpose.

The IPET has issued interim reports as it has proceeded with its analyses. The first two IPET interim reports were issued in January and March 2006. The NAE/NRC committee has followed the progress of the IPET, has met several times with IPET officials and other experts, and has issued two letter reports that provide advice on the first two IPET interim reports. The NAE/NRC committee reports were issued in February and May 2006. This is the NAE/NRC

committee’s third letter report and it reviews the IPET June 1, 2006, Final Draft (the IPET report, and all IPET documents, can be viewed at: https://ipet.wes.army.mil/; accessed October 3, 2006). This committee is scheduled to issue a fourth and final report in 2007, which will review the IPET final report. This report will also offer lessons learned from the Katrina experience, as called for in the committee’s statement of task.¹

Before commenting on the content of the IPET June 1, 2006, Final Draft, it is worth noting the extent of the study defined by the IPET. Each of the five IPET study objectives would by themselves define an ambitious scope of work. To merge all five objectives into a single project, to be completed in roughly one year’s time, has created one of the more extensive and complex studies ever undertaken by the Corps of Engineers (and the other participating IPET agencies and scientists). The IPET has accepted its charge with vigor and enthusiasm, collecting impressive amounts of data and conducting many detailed and useful analyses on a rigorous schedule.

The IPET has produced three main draft reports in 2006 and plans to issue its final report in 2007. Any study group given the charge and the time constraints of the IPET would be greatly challenged to produce a comprehensive report. Indeed, the five IPET study objectives define a set of studies that could have taken several years or even decades to complete. Thus, whereas the following report contains criticisms and suggestions for improvement, it is written with great appreciation for the realities and limitations inherent in an all-encompassing study slated for completion on a very short timeline. Our NAE/NRC committee continues to be impressed by the diligent efforts of the IPET team, and appreciates the IPET’s spirit of cooperation and openness in helping our committee conduct its reviews of IPET reports.

This report is divided into three main sections. The first section is entitled “Executive Summary of the IPET Report.” It comments on the organization, presentation, and content of the June 1, 2006, Final Draft Executive Summary. The second section is entitled “Geotechnical Concepts and Analyses.” The third section is entitled “Risk and Reliability”; it comments on Volume VIII of the IPET Final Draft, which addresses the topic of engineering and operational risk and reliability analysis. The report concludes with a short Summary.

The IPET June 1, 2006, Final Draft includes a 50-page Executive Summary and Overview that summarizes key concepts and findings. The report consists of over 6,000 pages of text, charts, figures, and appendixes. Given the size of the report, it is crucial to have an Executive Summary that is accessible and comprehensible by the public and decision makers. This section identifies concerns regarding the June 1, 2006, Final Draft Executive Summary and offers advice on how it might be improved in the IPET final report.

Those sections of the Executive Summary that present engineering and scientific terms and description of the storm and its impacts, and describe the scope of Hurricane Katrina, its impacts, and the IPET investigations (see pp. I-14-40) are clearly linked with the body of the IPET June 1, 2006, Final Draft. However, portions of the Executive Summary reach premature closure on important topics, gloss over uncertainties and differences of opinion, and assume a tone that is at times legal in nature. For example, the Executive Summary begins with discussion of the lack of a true hurricane protection system for the New Orleans region but shortly thereafter begins to backtrack and at times becomes defensive. An example of this is found in the opening paragraph under “Hurricane Protection System” (p. I-5), which states,

A determination of whether mistakes in structural design and systems performance constitute “negligence or malfeasance” was not addressed in the IPET report and is beyond this committee’s charge. “Negligence” and “malfeasance” are terms with strong legal overtones; the IPET studies were designed as technical investigations of structural performance and storm surge risks, not as a legal investigation.

• The IPET final report should focus on scientific and engineering issues, with less emphasis on legal concerns regarding negligence or malefeasance. The first two sentences within the Hurricane Protection System (Volume III) of the Executive Summary (p. I-5), and the corresponding section in the Executive Summary to Volume III (p. III-7) that make assertions regarding “negligence and malfeasance” and “design approaches” are inappropriate and should be removed.

The IPET June 1, 2006, Final Draft Executive Summary is presented in a multilayered structure, with “overarching findings,” “principal findings,” “detailed findings,” and “overarching lessons learned.” A summary might be organized in many different ways; but the relations among these varied layers are not clear and these tiers do not help explain the overall IPET report structure and key concepts and findings.

Much of the Executive Summary is written at a general level and is not well connected to the analyses that underpin the body of the report. On pp. I-10-12, for example, concepts of resilience, risk-based planning, awareness, and capability are discussed. Although the concepts are important, the discussion here is poorly linked to the body of the report. The Executive Summary places great emphasis on the concept of resilience, for example, but nowhere in the

main body of the report is the term clearly defined, nor is the concept extended and applied as a unifying principle throughout the report. For example, the IPET report states:

Resilience may be a relevant and useful concept in planning and managing hurricane protection; but it is inappropriate to introduce a central term or concept in an Executive Summary that is not developed significantly in the main body of the report. As a result, use of this term appears to be an afterthought, rather than a guiding theme within the study. Furthermore, reference to resilience as a “key objective” is confusing with regard to the existence of five IPET study objectives.

There are other gaps between the Executive Summary and key issues within the main body of the report. The IPET is grappling with many complex issues at various levels of conceptual and technical detail. These issues include strengths and vulnerabilities across the hurricane protection system, varying levels of risk of storm surge across the region, and uncertainties in storm and storm surge probabilities. The IPET report has five stated objectives (not a single objective, as suggested in the previous quote), one of which focuses on risks to the New Orleans region posed by future tropical storms. As stated in the IPET June 1, 2006, Final Draft, this objective is explained as “The Risk and Reliability: What was the risk and reliability of the hurricane protection system prior to Katrina, and what will it be following the planned repairs and improvements?”² Of the five report objectives, this is the most future oriented, as the others focus on evaluations of events before and during Hurricane Katrina.

The IPET June 1, 2006, Final Draft defers discussion of these issues until the risk analysis portion of the study is completed. Although the risk and reliability team has conducted a significant amount of background work for this evaluation, little evidence and no preliminary discussions of this work are found within the Executive Summary. Such a discussion could have included a summary of the goals of the IPET risk analysis; the decision to employ a joint probability method, alternative methods that were considered and why this method was selected; assumptions, strengths, and weaknesses of the method; a discussion of uncertainties that are likely to be part of the risk analysis results; and discussion—including preliminary maps of elevation and storm surge and flooding risks—of how future storm surge risks might affect the region.

Finally, if the IPET reports are to be useful in decisions regarding resettlement and reconstruction, they will have to present information in a manner that is both understandable to the general public and addresses primary questions of concern to the public and elected officials. Citizens and public officials in the New Orleans region are eager to know more about future risks and how the IPET investigations can help them make informed decisions. For example, two

important questions for homeowners, businesses, and elected officials are “What is the risk of being affected by hurricane storm surge?” and “What level of protection should be provided?” Even though answers to such questions entail many complexities, the IPET should strive to address such questions as clearly and candidly as possible. It is important to direct some portion of the IPET study to the public to ensure its relevance and usefulness. Communication of complex concepts to the public presents challenges. Nevertheless, the public is looking to the IPET to help inform resettlement decisions for New Orleans and the region. It is also important that the public appreciate the value of the IPET’s extensive efforts.

• The Executive Summary in the IPET June 1, 2006, Final Draft is not fully consistent with the body of the report. Summaries in future IPET reports should more closely match the bodies of those reports. Summaries should also be written in a style that officials and citizens can clearly understand.

• The Executive Summary should clearly and concisely present the report’s key findings and conclusions with regard to the study’s five primary objectives.

One IPET objective is to provide a better understanding of the design criteria and status of the pre-Katrina New Orleans hurricane protection system. To help meet this and the other IPET objectives, it will be important for the IPET final report to include a clear depiction and display of the hurricane protection system’s key features before, during, and after Hurricane Katrina. Readers of the report should be able to easily find figures and related tabular data that depict the extent and condition of the hurricane protection system before and after Hurricane Katrina.

The IPET June 1, 2006, Final Draft devotes a great deal of general discussion to the performance of “T-walls” vs. “I-walls.” Given this emphasis, one would expect a good summary and figure of the dimensions and locations of T-walls and I-walls. However, the report lacks a clear summary discussion of T-wall and I-wall performance, as well as a clear figure(s) and map(s) of these structures and their dimensions and locations. As a result, it is not clear whether, for instance, I-walls performed worse than T-walls because of inferior condition, or whether they were simply exposed to greater loads than were T-walls. Figure 4, p. I-20 in the Executive Summary includes some information about general levee location throughout the hurricane protection system, but it is incomplete and provides little specific information about levee heights, condition, and so on. It would also be useful to know the length of levees, levee design elevations, and best estimates of elevations just prior to Hurricane Katrina.

Where data on I-walls and T-walls are presented there are some inconsistencies. For example, in Appendix 12 of Volume V (p. V-12-10), the summary of damages from Hurricane Katrina states that 12,750 feet of the New Orleans East back levee suffered major damage. Table 12-2, located just below that summary, shows very different numbers. Furthermore, Appendix

12 shows damages to levees in New Orleans East, but there is no comparable analysis of St. Bernard Parish. The Executive Summary also should clearly explain where the system performed well and where it failed.

Volume V of the June 1, 2006, Final Draft is entitled “The Performance—Levees and Floodwalls.” Within Volume V, there are references to systemwide data; data in the appendixes (p. V-12-10), although useful, are limited. The main text fails to provide the type of an overview of regional geology and synthesis of geologic processes that are required to establish a context for evaluating the hurricane protection system. Attention was drawn to this deficiency in this committee’s first report (NRC, 2006a), and a summary of the regional geology was included in subsequent IPET reports. Although reference to regional geology is made in Appendix II, Volume V, the level of discussion is somewhat limited: regional geology is described in a single paragraph in Volume V in the June 1, 2006, Final Draft and is absent from the Executive Summary.

The IPET report (in Volume V) should include a summary of regional geology to explain respective elevation differences across the New Orleans area, identify localized problems with soft clay deposits and underseepage conditions, and offer reasons for present and future subsidence and settlement issues. A comprehensive assessment of the geology and its engineering implications establishes governing principles that affect performance of various hurricane protection structures across the region.

• The Executive Summary’s focus should be broadened beyond a few select breach sites and present higher-quality illustrations of the hurricane protection system. These should include more detailed description of the location and performance of levee, I-walls, T-walls, and other protective structures.

• The Executive Summary (and Volume V) should incorporate a more thorough discussion—including regional maps—of geologic conditions and dynamics and their implications for failure mechanisms and strengths and weaknesses of the hurricane protection system.

Uncertainties and risks are prominent concepts within the IPET analyses and are of paramount importance for future planning and decision making by New Orleans officials and citizens. There will likely be large uncertainties attached to the estimates of future storm recurrence intervals, risks to the city of storm surge, and reliability of the hurricane protection system. The Executive Summary does not explain and discuss these uncertainties and their implications, except at a conceptual level. For example, there is broad, conceptual discussion of the merits of “risk-based planning” (e.g., p. I-11), but no subsequent explanation of how

estimates of storm surge for the New Orleans region will be generated, why those estimates will include uncertainties, or what the concept of risk entails for future land use planning and development.

The Executive Summary is also weakened by attempts to be definitive at points where conditions do not justify such certainty. For example, a central topic in the IPET studies has been geotechnical investigations at key levee failure sites, notably at the 17th Street Canal, the Inner Harbor Navigation Canal, and the London Avenue Canal. The IPET has conducted geotechnical analyses at these sites, as has a team of engineers sponsored by the National Science Foundation (NSF). The IPET and the NSF-sponsored teams have exchanged information regarding possible levee failure mechanisms, but differences remain in views regarding the primary failure mechanism(s) at these sites. These differences are legitimate, and final agreement concerning these failures may never be reached (see NRC, 2006b, pp. 10-12, for a discussion of these IPET and NSF studies).

• The Executive Summary should place a stronger emphasis on explaining and illuminating key uncertainties within its analyses and the implications of uncertainties for future planning and decision making.

Volume V of the IPET June 1, 2006, Final Draft addresses the performance of the Hurricane Protection System during Hurricane Katrina, focusing on two lines of investigation: (1) studies of breaches that have been attributed to premature³ foundation failures of I-wall sections at the 17th Street, London Avenue, and Inner Harbor Navigation Canal (IHNC) sites,⁴ and 2) surveys of erosion along reaches of the system where levees were overtopped. Based upon these investigations, the IPET has drawn conclusions about soil and structural performance and raised issues that have implications for future practice not only in New Orleans but also for hurricane and flood protection structures in other parts of the country. These issues include causes of breaches, site characterization, analytical methods, factor of safety, and erosion of levee soils, each of which is addressed in the following sections.

Extensive documentation regarding the 17th Street Canal breach appeared in the first two IPET reports (January 2006 and March 2006) and was reviewed in this committee’s first two

reports (NRC, 2006a,b). The IPET June 1, 2006, Final Draft describes similar methods of data collection, analyses, and physical modeling for the two London Avenue breaches (Volume V, Appendixes 7, 8, 9), while a much reduced scope of investigation is provided for the IHNC (Volume V, Appendix 11). A lack of documentation for the IHNC that is consistent in scope and depth with that for the 17th Street Canal and London Avenue breaches is a weakness in the IPET June 1, 2006, Final Draft—especially given the damage in the Lower Ninth Ward that resulted from the IHNC breaches.

The Volume V Executive Summary concludes that the foundation failures at the 17th Street Canal and at the IHNC⁵ occurred in “weak foundation clay,” while those at the London Avenue Canal resulted from “intense seepage and high uplift water pressures in the sand foundation” with “a common element” relating to “development of a gap between the wall and soil on the canal side” (p. V-1). In contrast, the Executive Summary states that all foundation failures were “induced by the formation of a gap” (p. I-7) and that this was not considered in original design.

Gap formation, however, should not have been an unexpected mechanism; nor did it exclusively lead to all foundation failures. For example, the formation of a gap between I-wall and soil was demonstrated in full-scale field tests performed nearly 20 years ago by the Corps of Engineers (Jackson, 1988). Moreover, the consequences of gap formation are affected by shear strength of the underlying soils. Although the phenomenon of gap formation has been investigated in depth (including the use of centrifuge tests), undrained shear strength of marsh and clay deposits has been investigated in an uneven manner, with important data, especially direct simple shear strength data, absent in the IPET June 1, 2006, Final Draft. In meetings and in discussions with IPET members, the members have indicated that IPET has gathered these direct simple shear strength data. This useful information should be included in the IPET report and integrated into the report’s geotechnical analysis.

Based on review of the IPET analyses conducted to date, this committee concludes that:

1. Low undrained shear strength of low permeability foundation soils (referred to as marsh and grey/lacustrine clays) clearly contributed to the failure of the I-wall at the 17th Street Canal. The lack of reliable shear strength data, particularly laboratory test data, gathered by IPET does not allow for a clear resolution of resisting forces. Moreover, it inhibits reaching a firm conclusion regarding relative contributions of gap formation and soil strength to safety factor, and raises concerns about the characterization of clay and marsh deposit strength at other locations. In addition, the highly variable strength data do not support the assertion that the breach occurred at a location of local weakness in the underlying marsh/clay layers within the levee. Although the failure mechanism proposed by IPET is plausible, back-analyses only match the observed surge elevation at failure by introducing a full-depth gap and assuming very low shear strength in the clay beyond the toe of the levee.

2. Underseepage was a major contributing factor for the London Avenue Canal breaches. In the event of a storm surge in the canal, both the levee sections and the design I-wall embedment depths would not meet simple design criteria considered for upward hydraulic gradients (e.g., for Mississippi River levees; see Turnbull and Mansur, 1961). The IPET team

has performed elaborate deformation analyses, using finite element methods, which require reduced frictional strength within the sand to match the surge elevations at the time of breaching. The IPET team has not provided conclusive evidence that this corresponds to the critical mechanism of failure.

1. Physical model tests have successfully simulated failures of I-walls for both the 17th Street and London Avenue Canal sections. The IPET team should be congratulated for the successful completion of these complex tests.

2. The IPET June 1, 2006, Final Draft is the first of the IPET reports to provide information on the four breaches that occurred along the IHNC. Documentation of these breaches (Appendix 11) is brief and insufficient for independent review because (1) there is no explanation of the local geology (the subdivision of marsh deposits and introduction of “interdistributary” units); (2) the undrained strength profiles are highly questionable; (3) stability is based on rigid-body limit equilibrium analyses with circular arc failure mechanisms and a full-depth gap; and (4) there is no discussion of anomalies of the structural condition of the I-walls at the breach sites that might have been observed.

3. Examples of other lingering issues regarding alternative failure mechanisms include the impacts of large differences in settlement across the protection system, the toppling of large trees that had encroached into levees, and the presence of soft clay layers not identified in present geotechnical investigations. Without considering these types of factors, the proposal of a single failure mechanism could lead future designers to focus on narrowly drawn conclusions, leading to neglect of other, equally plausible failure modes. The IPET final report should include a broader discussion of these other possible modes of failure.

• The IPET report should include more substantial documentation to support the hypothesis that breaches along the IHNC were caused by overtopping and erosion or by foundation failure. It should also include information that examines other possible modes of failure and why these failure modes have been rejected.

• The IPET report should address the shear strength of marsh and clay deposits more comprehensively, and provide more thorough laboratory and field data for both quantifying the strength and identifying the soil layer in which failure occurred.

The quality of the site characterization studies is the key determinant for successful diagnosis of geotechnical failures. In previous reports this committee has expressed reservations about site investigation methods and interpretation presented in the IPET reports. The site investigations carried out by IPET are not consistent with current geotechnical engineering practice. This section summarizes these primary limitations.

Site stratigraphy should integrate observations from borehole logs, field probing tests (notably piezocone penetration records), and local geological knowledge. It is unclear how this process has been carried out to reconstruct cross sections of the state of the levees prior to Hurricane Katrina. The reports lack detailed plans showing how site investigations relate to

surveyed features of the breaches,⁶ and the cross sections are generally hand drawn. The soils data, which are provided through the public website, are not well organized. A parallel study by Seed et al. (2006) shows a more detailed interpretation of the stratigraphy at the 17th Street Canal breach site.

Laboratory test data can provide reliable information on shear strength properties of soils. The IPET team has not reported results from state-of-the-practice laboratory sampling and testing methods. The IPET June 1, 2006, Final Draft attributes the large “scatter” in previous laboratory strength measurements to natural soil variability. In fact, the sampling procedures, test specimen selection, and use of unconsolidated (UU and UC) triaxial shear tests likely contribute to this scatter. Best practices in soft-ground engineering were established more than 30 years ago—much of it in research sponsored by the Corps of Engineers—using procedures that minimize the effects of sample disturbance (Ladd and Foott, 1974).

The IPET has relied almost exclusively on cone penetration data⁷ to estimate undrained shear strength profiles in low permeability fill, marsh, and clay units (and frictional strength of sand at the London Avenue Canal). These data show quite uniform properties beneath the crest of the levee. There is, however, little information supporting the interpretation of undrained strength profiles at the toe of the levee. It is erroneous to assume normally consolidated properties for the marsh and clay units beyond the levee (this issue is also discussed by Seed et al., 2006). The risk assessment of the levee system (Volume VIII) incorporates geotechnical information derived principally from site investigation methods and data from the 17th Street and London Avenue Canal breaches. These data are limited and thus unlikely to cover the full range of subsurface conditions and variability affecting the entire New Orleans hurricane protection system. The IPET has conducted some simple soil strength tests (e.g., cone penetration data) at sites beyond the 17th Street Canal. These data should be used to help present a broader, systemwide picture of soil properties.

• The IPET should present site plans and soil profiles at the breach sites in a printed graphic format, containing stratigraphic interpretation and soil properties consistent with good geotechnical practice.

• The IPET report should estimate clay and marsh deposit strengths at locations other than the 17th Street Canal—both beneath and adjacent to levees—with available laboratory test data.

The IPET has used two independent analysis methods for evaluating mechanisms of failure at the breach sites—Limit Equilibrium Methods and nonlinear (deformation-based) Finite Element techniques. Several features of these methods deserve close attention. Limit Equilibrium Methods are standard geotechnical analysis techniques (e.g., Duncan and Wright, 2005; Krahn, 2001) that provide direct assessments of stability (e.g., for a specified loading condition or shear strength distribution within the soil) by searching for the critical failure surface within the soil mass. It is not clear why the IPET calculations have been restricted to circular arc failure mechanisms, which apparently are not the critical mechanisms associated with levee breaching, as they are inconsistent with Finite Element analyses and physical model tests the IPET has conducted. The IPET team has not reported on analyses for planar and other alternative sliding surfaces and has thus opened itself to criticisms of its conclusions.

Finite Element analyses are well established within geotechnical practice (e.g., Duncan, 1994; Potts and Zdravkovic, 2001) and offer unique capabilities to model both coupled ground deformations and seepage. The selection of constitutive models in the current analyses is limited by lack of reliable strength and stiffness data (usually obtained from laboratory tests). The IPET finite element analyses are able to simulate the large deformations of the I-wall that occur due to the storm surge, but they generate a full failure mechanism (at the assumed failure surge height) only when a gap is introduced at the soil-wall interface. Although this represents a plausible failure mechanism, it is but one of several possible failure mechanisms.

• The IPET report should integrate field observations, Finite Element analytical results, centrifuge test findings, and Limit Equilibrium analysis to assess safety factors for sliding surfaces that match deformation patterns both on-site and in centrifuge models. These deformation patterns include planar and composite sliding surfaces other than circular arcs.

Throughout the IPET reports and associated presentations, a factor of safety of 1.3 is used as the value in geotechnical stability analyses. This factor represents a margin of safety with respect to unknowns in order to obtain satisfactory performance of levees and other geotechnical facilities. The selection of this numerical value seems to reflect long-standing Corps of Engineers practice, as opposed to careful assessment of the critical role of levees in protecting lives and property in high storm surges.

The IPET should review the rationale for the criteria, such as factors of safety, that future designs must meet. The evaluation should consider whether some other criterion, such as margin of safety or a reliability index, might be more appropriate for these structures. Surely the safety factor or level of safety should be related to the loss if the structure fails. For example, a levee protecting hundreds of millions of dollars of property should have a higher safety factor than one protecting hundreds of thousands of dollars of property. And an urban levee designed to protect

large numbers of people should have a higher factor of safety than one designed to protect farmland. Furthermore, the evaluation should consider the well-known result that different methods of analysis yield different factors of safety, even for similar failure modes.

• The IPET should reexamine its criteria for selecting factors of safety for levees in the New Orleans region. It should provide guidance for appropriate safety factors as they relate to different land uses and levels of human occupation. It should also include alternate reliability criteria used in design.

The IPET investigations have shown that levee overtopping by Katrina was accompanied in many locations by severe erosion of levee soils. It is imperative that procedures be developed and applied for quantifying the suitability of levee soils with respect to erosion resistance and for establishing acceptance criteria for soil used in future levee construction and rehabilitation. The IPET June 1, 2006, Final Draft provides visual and qualitative descriptions of levee soils that were susceptible to erosion. This erosion is attributed to fill materials and placement methods on the basis of empirical survey evidence. However, no explicit recommendations for procedures that should be used in the future to quantify soil resistance to erosion are provided. Given the substantial amounts of soils that must be certified for future levee construction in New Orleans, there is a need for improved methods to quantify soil erosion resistance.

• The IPET report should provide additional guidance for:

  1. identifying erodible soils;

  2. quantifying the degree of soil resistance to erosion; and

  3. selecting and placing soils to resist erosion, including armoring.

Volume VIII of the IPET June 1, 2006, Final Draft entitled “Engineering and Operational Risk and Reliability Analysis” is structured to provide findings central to many of the five IPET report objectives, namely, estimates regarding risks of future storms and of the structural reliability of the hurricane protection system. Results from this chapter’s analysis are not presented in the IPET June 1, 2006, Final Draft (IPET intends to issue these results later in 2006). The following section thus does not comment on findings from Volume VIII of the IPET June 1, 2006, Final Draft, but rather on some methodological considerations that underpin that chapter.

The framework employed in Volume VIII of the IPET June 1, 2006 Final Draft to generate estimates of risk and reliability consists of five sequential components. The first is a model (Holland, 1980) for estimating wind and atmospheric pressure fields during hurricane events. This model requires the selection of six characteristic storm parameters at land fall: central pressure deficit, radius of maximum winds, a radial pressure profile parameter, storm forward speed, landfall location, and approach direction. IPET selected 1,155 combinations of the six characteristic hurricane parameters to cover a wide range of possible hurricanes. Probabilities of occurrence for the various hurricane parameters were assigned using a combination of published scientific papers and historical records of hurricanes in the New Orleans vicinity. The second component is a rainfall model that is based on the hurricane’s central pressure deficit and the radius to maximum winds. The third component is the ADCIRC storm surge model plus a correction factor for wave effects. This was used to compute the water level along the hurricane protection system for each hurricane. No probabilities are explicitly assigned to results from the rainfall or the ADCIRC/wave models. The fourth component is a set of fragility models for estimating structural responses of critical components of the hurricane protection system as a function of the water level and the specific location in the hurricane protection system. Based on available experimental data and expert judgment, probabilities of occurrence were assigned to the set of fragility models. A fifth component, the Joint Probability Method (JPM), was then used to compute the probability of occurrence of a specific outcome (e.g., failure of the hurricane protection system at a specific location) based on the probabilities assigned to each of the other four components.

Whereas the IPET framework is not an unreasonable approach to estimating risks and reliability of the hurricane protection system, it does contain potential limitations. The selection of the five models and the assignment of probabilities to variables within them are somewhat subjective; moreover, all of the models are subject to significant and often unknown uncertainty. For example, water level calculations using ADCIRC on a coarse or medium resolution grid together with the simple wave correction procedure is very different from the method used in Volume IV (ADCIRC on a fine resolution grid coupled to the WAM wave model and STWAVE wave models) to model Hurricane Katrina. The hurricane, rainfall, water level, and fragility models are run sequentially. Uncertainties thus will be propagated through the calculations. Final estimates from the Joint Probability Method are likely to have large uncertainties and hence may be of little use for establishing an absolute probability of failure for any section of the hurricane protection system. They may also be of little value for assessing relative probability of failure of different sections of the hurricane protection system.

Given the uncertainties that will be attached to results of the risk and reliability estimates, it is critical that the methods, assumptions, and errors in each model component are presented clearly. Furthermore, a thorough error analysis should be performed to determine the uncertainty in the final results obtained from the Joint Probability Method. A rigorous error assessment will be critical for establishing the level of confidence that can be placed in results of the risk and reliability analysis. Failure to report on the error contained in each model component will inhibit a better understanding of the primary source(s) of uncertainties in the final estimates. Results

from the Joint Probability Method applied only to the hurricane model should be compared to the historical record of Gulf hurricanes. The comparison could, for example, be based on storm category at landfall (in its first two letter reports this committee recommended that the IPET assemble a history of Gulf Coast hurricanes). This comparison should consider data for Atlantic Coast and Gulf of Mexico hurricanes including those in the 1990s and 2000s, to reflect an apparent increase in hurricane frequencies since the mid-1990s.

It would be useful to identify those model runs in the risk analysis ensemble that most closely resemble the standard project hurricane (SPH) and Hurricane Katrina, and then use these model runs as case studies on both the SPH design and the Katrina event. In other words, what would the newly formulated risk and reliability analysis suggest about the probability of catastrophic flooding from the SPH or Hurricane Katrina?

It is also important to ensure that the risk and reliability estimates have practical applications with regard to the hurricane protection system. It is also important that they be understood by a public that is looking to the IPET report to provide essential information for both public policy and individual decisions in a post-Katrina setting. All elements of the hurricane protection system, and hence the risks associated with that system depend on some level of hurricane storm surge protection. Yet, the IPET report does not provide information on this most important aspect of the risk analysis—what surge levels will the city be protected against? Instead, the IPET risk analysis relies on a joint probability approach that implicitly includes surge heights without explicitly providing this fundamental information.

• The IPET risk analysis approach is conceptually coherent and logical, and may be a useful risk evaluation tool. Nevertheless, levels of uncertainty in estimates of risks of flooding are expected to be large. The overall uncertainty within the analysis should be carefully evaluated and presented to establish the level of credibility of final results.

• Each model component used in the risk and reliability model should be thoroughly described, and its results reported and validated. Uncertainties within these components and their results also should be presented clearly.

• The IPET risk approach should be used to assess the probability of storms that most closely resemble both the Standard Project Hurricane and Hurricane Katrina; this will add useful context to the overall risk and reliability analysis.

• Results should be provided from each component of the risk and reliability analysis to help the public and elected officials understand the level of protection provided by the hurricane protection system in terms of storm parameters, surge level, and the overall system.

The summaries of outcomes and associated probabilities of the very large number of possible hurricane events investigated by IPET are likely to be difficult to explain clearly to the public, and may be of limited practical value to elected officials. As described in this committee’s second letter report (NRC, 2006b), a set of historical and “worst case” surge/wave

conditions throughout the region could be simulated using a select set of hurricanes (e.g., designed from historical storms such as Betsy, Camille, Katrina, and from expert judgment) and the models developed for the forensic component of the IPET study. This set of storms could be selected from the much larger set of storms being used for the risk and reliability analysis.

Knowledge of worst case scenario storm conditions can be a useful guide in reconstructing and redesigning hurricane protection structures and plans. Knowledge of historical storms would be helpful for providing context for the results of the worst case storm scenarios. The selection of a limited set of hurricanes would be somewhat subjective, and the end result would not be a formal probability of failure for the system. However, the results would be intuitive and helpful in strengthening the decision-making process for future resettlement and reconstruction.

In a June 18, 2006, letter from Lewis E. Link, IPET director, addressed to Dr. Wayne Clough, chairman of this NAE/NRC committee, Dr. Link responded to this NAE/NRC committee’s second report (NRC, 2006b). In this letter Dr. Link notes that IPET chose not to conduct such a “deterministic ensemble” analysis (this letter is included as Appendix B in this letter report). Although this IPET decision should be respected, Dr. Link’s letter does not explain fully how this decision was reached. Contrary to Dr. Link’s assertion, the analysis suggested in the NRC (2006b) report is not restricted to a specific period of record; rather its members can be tailored to reflect past or projected future conditions as deemed to be appropriate. Nor is it confined to definition by a specific set of parameters.

An issue of great importance is that the hurricane protection system design was not updated as technology advanced and as environmental conditions changed (e.g., losses of wetlands). One aspect of determining hurricane protection system failures during Hurricane Katrina is to assess whether current technology shows that the system is designed adequately for the SPH. To accomplish this, the IPET should re-evaluate the authorized levee/floodwall heights associated with the SPH using ADCIRC, WAM, and STWAVE models that were validated for Katrina. This comparative evaluation would identify whether these new models indicate the need for revised hurricane protection system heights. This exercise would have the added benefit of providing an assessment of how the surge and waves due to Katrina compare with those previously estimated for the SPH.

• To complement its joint probability method results, the IPET should create a set of hurricane scenarios that simulate a variety of possible future storm conditions in the greater New Orleans region. This set of scenarios should include both important historical storms and worst case scenario storms. These scenarios could either be generated as an additional set of possible storms or be selected from previous runs the IPET has generated. This would provide an assessment of whether current design is adequate to protect against a range of possible hurricane conditions.

• The IPET should evaluate the present system design using the best available contemporary models and methods, using Standard Project Hurricane storm parameters. This would provide an assessment of whether the authorized levee/floodwall heights are adequate to meet earlier design criteria.

There appears to be a lack of consensus on an appropriate method for estimating risk and reliability of the hurricane protection system among IPET and groups such as the Corps of Engineers (which is leading the Louisiana Coastal Protection and Restoration [LaCPR] study), the Federal Emergency Management Agency (FEMA, which is conducting mapping studies that are to include related risk estimate studies), and the National Oceanic and Atmospheric Administration (NOAA). The hazards of hurricane storm surge and coastal inundation are too important to be addressed with less than the best interagency coordination and scientific methods and knowledge. The IPET should work closely with these agencies to avoid complications that may result from inconsistent methods and results among federal studies and programs, and to avoid confusion in communicating results to the public.

• IPET should coordinate its risk and reliability analysis with other relevant programs and agencies, especially the Corps of Engineers LaCPR study, FEMA, and NOAA.

The post-Katrina context of New Orleans hurricane protection is subjecting the IPET investigation to a high level of scrutiny. The IPET is producing a report of keen interest to multiple audiences that include engineers, experts in a broad array of related disciplines (e.g., public policy, history, ecology), elected officials at different administrative levels, the media, and citizens and citizen interest groups. The IPET report not only should be technically sound but it must also provide clear answers to fundamental questions important to a broad audience. In the end it is important that the good efforts of the IPET not be regarded primarily as a technical and inaccessible exercise. Clear statements of the primary IPET outcomes—including uncertainties where they exist—will be of value not only for the public but also for future initiatives, such as the Louisiana Coastal Restoration and Protection program, which will build upon IPET study findings.

The IPET June 1, 2006, Final Draft contains weaknesses in its Executive Summary, geotechnical concepts and analyses, and risk and reliability modeling that will need to be remedied for the document to be both technically credible and a useful guide for citizens and elected officials. Connections between the report’s Executive Summary and the body of the report are often difficult to discern. The IPET’s final report will be strengthened by placing a strong emphasis on ensuring that the Executive Summary reflects clearly the content and main sections of the main body of the report.

The IPET is conducting a study of regional and national importance that will be referenced for years to come. The IPET experts have expended considerable efforts to bring their study to its current point, for which they should be commended. Our NAE/NRC committee offers this report in the spirit of contributing positively to the IPET effort. We look forward to future collaboration with the IPET as we continue our review of their important evaluations of the New Orleans hurricane protection system.