Review of Recommendations for Probabilistic Seismic Hazard Analysis: Guidance on Uncertainty and Use of Experts (1997)

“The future utility of PSHA in decision making depends to a large degree on our ability to implement the process in a meaningful and cost-effective way. Development of the SSHAC guidelines was planned with this goal in mind.”

—from Sponsors' Perspective, SSHAC Report

This review and commentary by the National Research Council's Panel on Seismic Hazard Evaluation presents the panel's evaluation of the report Recommendations for Probabilistic Seismic Hazard Analysis: Guidance on Uncertainty and Use of Experts (U.S. Nuclear Regulatory Commission, NUREG/CR-6372, Washington, DC, 1997). That report was prepared by the Senior Seismic Hazard Analysis Committee (SSHAC) (not a committee of the National Research Council) with sponsorship and oversight by the U.S. Nuclear Regulatory Commission (USNRC), the U.S. Department of Energy (DOE), and the Electric Power Research Institute (EPRI).

Earthquakes present a threat to people and the facilities they design and build. Seismic hazard analysis (SHA) is the evaluation of potentially damaging earthquake-related phenomena to which a facility may be subjected during its useful lifetime. An SHA is done for some practical purpose, typically seismic-resistant design or retrofitting. Although strong vibratory ground motion is not the only hazardous effect of earthquakes (landslides, fault offsets, and liquefaction are others), it is the cause of much widespread damage and is the measure of earthquake hazard that has been accepted as most significant for hazard resistance planning.

The level of effort put into an SHA depends on the investment in the facility that might be lost and the consequences to society should it fail. Critical facilities are those that are deemed so important to the functioning of society or whose catastrophic failure will have such disastrous consequences that a maximum (and necessarily costly) effort to assess seismic and all other natural hazards is justified. The SSHAC project was born in the context of SHA for such critical facilities, nuclear power plants in particular. Even though SSHAC broadened its concept of the applicability of its recommended approach to SHA, its report is strongly influenced by this orientation toward very large, costly facilities for which the end goal is to prevent catastrophic failure, even at great expense.

Two general approaches to SHA have been developed and applied. The first approach uses discrete, single-valued events to arrive at scenario-like descriptions of the hazard. Typically, a seismic source location, a maximum earthquake associated with that source, and a ground motion attenuation relationship are specified. The ground motion at the site of interest implied by the chosen inputs is then calculated. The frequency of earthquake occurrence is usually not taken into account, and there is no formal and open way of treating uncertainties. This approach has been labeled deterministic seismic hazard analysis (DSHA) and has been used for many years in the design of power plants, large dams, and other critical facilities.

The other approach is probabilistic seismic hazard analysis (PSHA) and is the subject of the SSHAC effort. PSHA allows the use of multivalued or continuous events and models incorporating the effects and frequencies of all earthquakes that could impact a site. PSHA can easily incorporate model and parameter uncertainties. The results of a PSHA, including the uncertainties, can be represented as a series of curves (mean, median, or selected fractiles), showing the annual frequency of exceeding different levels of the chosen measure of ground motion. The intent of high-level PSHA is to capture and display as much as possible of the knowledge provided by existing data, theory, and computational simulations.

It should be noted that the procedures recommended by SSHAC for the elicitation and aggregation of expert opinion as input to PSHA are equally applicable for compiling the input for DSHA. The only essential difference between DSHA and PSHA is that the latter carries units of time while the former usually does not (Hanks and Cornell, 1994). In the case of a specific design situation, both DSHA and PSHA result in estimates of

ground motion values or time histories that provide the basis for earthquake-resistant design. PSHA yields, in addition, the annual frequency of exceedance of that ground motion level together with attendant uncertainties. SSHAC's responsibilities did not extend to a discussion of the steps by which project engineers and sponsors use the output of a hazard assessment. One approach to this issue is presented in a recent paper by McGuire (1995).

Projection of the location, severity, and frequency of occurrence of future extreme natural events inherently involves a variety of uncertainties. Yet decisions on the siting and design of needed facilities must be made in the face of these uncertainties. No amount of statistical analysis, no matter how rigorously based and carefully done, can totally compensate for the incompleteness of available data and the defects of our evolving scientific knowledge. A primary objective of SSHAC was to acknowledge and document uncertainties explicitly so that users of PSHA will be able to make better-informed decisions.

The Panel on Seismic Hazard Evaluation was created under the Committee on Seismology of the National Research Council in October 1992. The panel was formed in response to a request from the USNRC to provide an independent review and evaluation of a report on PSHA to be produced by SSHAC.

The work of the panel was influenced by several factors. First, the USNRC asked the panel to provide an “interactive review,” that is, to submit feedback to SSHAC as it worked in order to avoid the production by SSHAC of a report in which the panel might find serious flaws after it was completed. This request raised serious questions as to how the panel could meet its requirement and not become so involved in the production of the SSHAC report that the objectivity of the panel's own review would be compromised. The panel agreed with the USNRC to provide “arms-length” interaction with SSHAC and developed methods of operation to achieve that goal.

Another factor affecting the work of the panel was a change in the charge to SSHAC after it began its work. The original task assigned by the sponsors concentrated on the reconciliation of two studies done in the mid-1980s by Lawrence Livermore National Laboratory (LLNL) and EPRI of the earthquake hazard at nuclear power plant sites in the United

States east of the Rocky Mountains. These studies were prompted by advice to the USNRC from the U.S. Geological Survey, based on its reconsideration of the likelihood that a major earthquake, such as the Charleston, S.C. earthquake of 1886, could occur again in Charleston or elsewhere along the eastern seaboard. The possibility of such an earthquake could have implications for the safety of nuclear power plants in the eastern United States. A brief history of the LLNL and EPRI studies is given in the SSHAC report.

Although the two studies ranked the many sites approximately the same (from most hazardous to least hazardous in terms of the mean hazard estimates), the absolute hazard values for specific sites, in terms of the mean value of the annual probability of exceeding a specified level of ground motion, differed greatly, with the LLNL results consistently greater.

The problem is illustrated in Figure 1.1, which displays the hazard at three widely separated sites as the annual frequency of occurrence of peak ground acceleration (PGA), the ground motion parameter chosen for this evaluation. The median hazard curve from each study is shown, as well as the 85th and 15th percentile curves. In two of the three cases shown, the median hazard calculated by LLNL is well above that derived by EPRI, and the “ uncertainty,” measured by the spread of the 15th and 85th percentile curves, is much greater for LLNL than EPRI. Also, the uncertainty is large, a factor of 5 or more at potentially damaging levels of ground motion (PGA greater than 200 cm/sec²).

The mean hazard curves, not shown in the figure, differ by even greater factors in many cases. This is because the LLNL median and 85th percentile curves are above the EPRI results, and arithmetic averages spanning several orders of magnitude give greatest weight to the largest numbers. This explains the relatively high values of the mean hazard derived by LLNL but it does not get at the fundamental cause for the differences in the estimates.

The desirability of discovering the cause(s) of the discrepancies was obvious, not only for intellectual reasons (why did competent scientists working from the same or similar knowledge and data bases get vastly different answers?), but also for the practical reason that the quantitative estimate of seismic hazard is important in judging whether earthquakes represent a substantial threat, as well as the weight of earthquakes relative to other natural hazards in making design and retrofitting decisions. The USNRC funded LLNL to investigate the

problem. LLNL's study (Bernreuter et al., 1987) concluded that the factors involved in the discrepancy were: (1) different values were chosen for the lower-bound earthquake when the groups were integrated over seismicity to calculate the hazard, (2) different ground motion models were used, and (3) LLNL included a correction for local site effects and EPRI did not. This explained why the two studies obtained different answers but does not explain why competent analysts arrived at significantly different inputs to the hazard calculations.

As SSHAC was being assembled, the underlying cause of the discrepancies between the two studies was identified by further study at LLNL. Researchers there concluded that the differences were due to the ways in which the inputs provided by experts had been elicited. Once this was recognized and taken into account, the differences in the outputs (mean hazard curves) were reduced from orders of magnitude to small factors that represented satisfactory agreement, given the many uncertainties in every step of the analysis. This resolution of the original problem led to changes in the SSHAC charter (1994), from which the following items are selectively cited to provide the context within which the SSHAC report was developed:

Objective: To develop implementation guidelines, including recommended methodology, suitable for the performance of PSHA for seismic regulation of nuclear power plants and other critical facilities.

Requirements and Guidelines (for the implementation guidelines and methodology):

• Be able to provide probabilistic seismic hazard results in the form of fractile probabilities and mean values over a range of ground motion levels suitable for use in probabilistic seismic risk assessments for nuclear facilities.

• Be defined in sufficient detail that, when independently applied by different organizations, no ambiguity exists on how the PSHA is to be performed and comparable results are obtained.

• It is specifically not the objective of this program to advance PSHA methodology or to develop a new PSHA methodology. Rather, an important step in reaching the

objective of this program is expected to be the completion of evaluations of independent PSHA applications by LLNL and EPRI as well as other relevant applications.

• The outcome of this process will be the recommended methodology and implementation guidelines for PSHA in nuclear power plant licensing.

The emphasis on methodology for doing PSHA as the central theme is reflected in the title of the SSHAC report. The focus on siting nuclear facilities, though not emphasized explicitly in the report, strongly influenced its concentration on high-level PSHA.

It should be recognized that the charges to SSHAC and to the panel did not call for the defense or promotion of PSHA as a method for evaluating earthquake hazards. SSHAC has produced a document that sets forth its conclusions and recommendations on the proper way to do a PSHA if that is the approach chosen by project developers and their analysts. Neither the SSHAC report nor the panel evaluates the efficacy of PSHA relative to other methods, DSHA in particular. The SSHAC report does provide criteria that can be used to decide the appropriate level of effort for a specific study. Some of the issues related to alternatives to a full-blown PSHA and alternatives to SSHAC's recommended procedures are discussed elsewhere in this report.

The panel offers its appraisal of the SSHAC report, with primary emphasis on the scientific validity of the work and its conclusions, with appropriate attention to the clarity of the presentation, possible sources of misinterpretation, and the report's contributions to PSHA.

The panel met with SSHAC three times (June 28-29, 1993; May 27-28, 1994; and December 9-10, 1994). Members of SSHAC, representatives of the three sponsoring organizations, and scientific and technical consultants to SSHAC attended the meetings. In addition, Thomas Hanks, a member of the panel, attended a number of SSHAC meetings as liaison observer.

By the nature of its charge, the panel was not able to begin its work until it received a draft product from SSHAC and could not finish its work until it had received the complete final SSHAC report. The June

1993 meeting was devoted primarily to briefings by agency representatives, SSHAC members, and scientific consultants, designed to educate the panel about the goals of SSHAC, the background of the problems being addressed, and the procedures SSHAC would follow. A spokesman for the USNRC explained that the agency wanted two products from SSHAC: (1) a set of guidelines for the process of seismic hazard assessment, and (2) a set of guidelines for the agency, using current data sets and computer codes, to reevaluate the hazards at existing sites. A SSHAC spokesman concluded that the central thrust of the project was to develop, justify, and illustrate methods for capturing both the inherent uncertainties in the parameters that go into an analysis and the disagreement among experts about the values of these parameters. At this time, the panel decided that it needed two additional members, one who could provide expertise in expert opinion analysis and decision science and one with extensive knowledge of both the deterministic and probabilistic approaches to seismic hazard assessment.

By May 1994 the focus of the SSHAC effort had changed, as noted above, from the reconciliation task to the more substantial and significant task of building on the lessons learned from prior experience in hazard assessment to develop scientifically sound procedures for doing PSHA. The SSHAC chairman explained that his committee's goal had been broadened to the development of a methodology that would be applicable not only to nuclear power plants but to other critical facilities as well. SSHAC members presented detailed technical briefings in their areas of expertise, so that the panel gained insight into the flavor of the report that SSHAC would produce. Vigorous discussions of both earth science and decision science issues provided a forum for the panel to explore details of the proposed SSHAC approaches and to convey in broad terms some concerns of the panel. Points raised in these discussions and the panel's evaluation of how SSHAC treated each are addressed elsewhere in this report.

The December 9-10, 1994, panel meeting was based on a detailed review of a draft report submitted by SSHAC. The draft was incomplete; in particular, the extensive appendixes, which on later examination proved to be essential and very valuable contributions of the SSHAC effort, were not available. But, the panel did conduct a detailed review of the main report. SSHAC members, as well as the agency representatives, were present for this review. The results of the review were submitted in the form of a formal letter report to the USNRC on March 16, 1995 (reproduced here as Appendix B). The USNRC forwarded this letter

report to SSHAC as part of its oversight of the final version of the SSHAC report.

The March 1995 letter report was the principal formal feedback from the panel to SSHAC. The letter report offered the panel's general comments on the SSHAC draft, a statement of concerns and problems, with suggestions for improvement, and a summary of specific scientific and technical concerns that the panel thought should be addressed. A draft of the final SSHAC report was sent to the panel on October 6, 1995. The present report is based on the panel's review of the October 6 draft, supplemented by several figures and parts of the appendixes that were submitted later. (Although the October 6 draft needed editing the panel was informed that the work of SSHAC was completed and that no further substantive changes in the SSHAC report would be made.)

The expectations of the sponsoring organizations are expressed succinctly in the last sentence of the Sponsors' Perspective that opens the SSHAC report, which is quoted at the beginning of this chapter. The panel has reviewed and evaluated the SSHAC report in light of these expectations and how well the goal has been achieved.

The panel determined that the SSHAC report could be reviewed under four main headings: (1) process (elicitation and aggregation) and documentation, (2) the treatment of uncertainty, (3) seismic source characterization, and (4) ground motion estimation. The first two concentrate on the decision science components of PSHA, the latter two on the earth science inputs. Following a chapter on each of these, the panel offers a summary of its findings and recommendations.