Review of Recommendations for Probabilistic Seismic Hazard Analysis: Guidance on Uncertainty and Use of Experts (1997)

A fundamental aspect of the Senior Seismic Hazard Analysis Committee 's (SSHAC) methodology is the distinct and separate treatment of aleatory and epistemic uncertainty. Throughout its report, SSHAC emphasizes the need to distinguish between these two types of uncertainty, the quantifications of their contributing sources, and the propagation and full display of the epistemic component to users (see, e.g., Sections 1.8 and 1.9). SSHAC deals with techniques to assess, elicit, combine, propagate, document, and display epistemic uncertainty, and it is clear that much if not most of the effort in any probabilistic seismic hazard analysis (PSHA) conducted according to SSHAC's recommendations would have to be expended in activities related to the handling of uncertainty.

The two fundamental types of uncertainty are defined by SSHAC as:

• Epistemic: the uncertainty attributable to incomplete knowledge about a phenomenon that affects our ability to model it.

• Aleatory: the uncertainty inherent in a nondeterministic (stochastic, random) phenomenon.

Epistemic uncertainty may be reduced with time as more data are collected and more research is completed. Aleatory uncertainty, on the other hand, cannot be reduced by further study, as it expresses the inherent variability of a phenomenon.

Making a rigorous separation between aleatory and epistemic uncertainty, as advocated by SSHAC, requires a level of effort and expertise much greater than that for most PSHA efforts. Therefore, the panel thinks it is appropriate to elaborate as to when and why such classification may be needed and indeed whether it is appropriate (these

issues are not addressed directly by SSHAC). In this regard, it is useful to consider separately two questions:

1. Is the aleatory/epistemic classification unique and clear?

2. Why is a separate treatment of epistemic and aleatory uncertainty needed and to what degree should it be pursued in a PSHA analysis?

Embedded in the second question are issues of utilization of results in which epistemic uncertainty and aleatory uncertainty are separated (i.e., of results stated in a “probability of frequency” format), either in the process of conducting the PSHA study or in the process of decision making by the ultimate user. In this chapter the panel briefly reviews SSHAC 's position on these issues and makes some recommendations.

SSHAC correctly points out that the classification of uncertainty as epistemic or aleatory depends on the model used to represent seismicity and ground motion. For example, epistemic uncertainty would be much greater if, in the assessment of seismic hazard at an eastern U.S. site, instead of representing random seismicity through homogeneous Poisson sources one used a model with an uncertain number of faults, each with an uncertain location, orientation, extent, state of stress, distribution of asperities, and so forth. As little is known about such faults, the total uncertainty about future seismicity and the calculated mean hazard curves would be about the same, irrespective of which model is used. However, the amount of epistemic uncertainty would be markedly different; it would be much greater for the more detailed, fault-based model. Consequently, the fractile hazard curves that represent epistemic uncertainty would also differ greatly.

A reasonable interpretation of the probabilistic models used in seismic hazard analysis is that they represent not intrinsic randomness but uncertainty on the part of the analyst about the actual states and laws of nature—for example, about the number of earthquakes of magnitude 6 to 7 that will occur in the next 50 years in a given crust volume. According

to this interpretation, all or most of the uncertainty in PSHA is due to ignorance. In certain cases, uncertainty due to ignorance may be expressed numerically by long-term relative frequencies. For example, with a very long record of seismicity, one could extract the long-term relative frequency with which earthquakes of magnitude 6 to 7 occur in a generic 50-year period. In the absence of other relevant information, it is reasonable to use this long-term relative frequency as a measure of epistemic uncertainty about the occurrence of the event in the next 50 years. Note that as interest in PSHA is typically in the occurrence of rare events in the near future and because the occurrence of such events depends to a large extent on the current physical conditions of the earth's crust near the site, ignorance or epistemic interpretation of the occurrence probability is more appropriate than the long-term relative frequency or aleatory interpretation. In certain parts of its report, SSHAC concedes that in reality there may be just one type of uncertainty. For example, Section 2.2.3 reads, in part:

. . . Even though we have discussed probabilities appearing in the model of the world and the epistemic model, and we have given them different names, leading philosophers of science and uncertainty (e.g. de Finetti 1974; de Groot 1988) believe that, conceptually, there is only one kind of uncertainty; namely, that which stems from lack of knowledge.

Other statements support this position. For example, Section 2.2.6 states that “. . . the different terminology [aleatory versus epistemic] is not intended to imply that these uncertainties are of fundamentally different nature.” Similarly, Section 1.8 points out that in the context of seismic hazard analysis, “the division between the two different types of uncertainty, epistemic and aleatory, is somewhat arbitrary.” The panel concludes that, unless one accepts that all uncertainty is fundamentally epistemic, the classification of PSHA uncertainty as aleatory or epistemic is ambiguous.

Reference to a particular class of seismicity models (e.g., the models described in Sections 2.1 and Chapter 4 of the SSHAC report) produces some stability in the epistemic/aleatory distinction. However, if such distinction is to have any impact on the decisions, the basis for choosing any particular model type should be made clear, as alternative

and equally valid choices would lead to different decisions. In view of this undesirable dependence of epistemic uncertainty on the models selected for PSHA, one may question whether the epistemic/aleatory uncertainty decomposition is actually called for in a PSHA study and the extent to which it is needed for decision making by the users. These questions are addressed in the following section.

SSHAC does not provide a clear rationale for the need to separate aleatory uncertainty from epistemic uncertainty, although the report refers to several uses of this separation. Sections 2.2.5 and 2.2.6 of the report cite facilitated communication of results, discipline on the part of the analyst, and completeness of results. A “theoretical foundation” for the aleatory/epistemic distinction is offered in Section 2.2.6 by quoting a result by de Finetti in probability theory that shows how to combine epistemic and aleatory uncertainty to quantify total uncertainty for a particular (the binomial) model. However, the same result indicates neither how to separate the two uncertainties in practice (this is acknowledged by SSHAC) nor how to make decisions considering epistemic uncertainty. Therefore, the panel finds reference to de Finetti's result not relevant to whether or why the aleatory/epistemic distinction is necessary.

Reference to the decision-making implications of the epistemic/aleatory character of the uncertainty is made at the end of SSHAC's Appendix F, where it is stated that: “because epistemic and aleatory uncertainties are treated differently in making design and retrofit decisions, and because the median hazard is sometimes the preferred central measure of hazard due to its stability, it is also important to allocate uncertainties in the proper category.” While it is true that the median curve is often preferred to the mean curve, a clear rationale for this practice or, more generally, a procedure for dealing with epistemic uncertainty in decision making is not presented in the SSHAC report. Finally, in Section 7.6 reference is made to the need for multiple hazard curves in the context of probabilistic risk assessment studies.

It is not the purpose of this discussion to analyze in detail each of the reasons for quantifying epistemic uncertainty. However, the panel observes that different uncertainty representations are appropriate for

different applications. To add focus to this discussion, we consider and contrast three main uses of quantified epistemic uncertainty in PSHA:

1. In the elicitation and experts/model combination process, quantitative estimates of epistemic uncertainty are used to characterize the credibility of alternative hypotheses and models, to assess the statistical variability of parameters, and to communicate this information among the experts and between the experts and the TFI.

2. In the course of a properly conducted analysis, the effect of epistemic uncertainty on the final hazard is used to assess the relative importance of different models (e.g., of the seismicity model versus the ground motion model) and parameters and to guide the analyst in seeking further information (data, expert opinion, etc.) to reduce uncertainty in the most cost-effective way.

3. A project's sponsor typically accounts for uncertainty in a hazard when making decisions (e.g., about the design of a new facility or the retrofitting of an existing one).

For ease of reference, we label these three phases of uncertainty consideration as the elicitation/combination phase, the PSHA planning phase, and the final utilization phase. Different needs for uncertainty representation characterize these phases.

In the elicitation/combination phase, experts need to be aware of all pertinent sources of uncertainty, including parameter and model uncertainties and their correlations, and the limitations and errors of the available data, so that they can make an informed assessment of the validity of alternative hypotheses, the accuracy of alternative models, and the value of data and can convey such uncertainties to the TI/TFI. The panel finds the type of epistemic uncertainty analysis recommended by SSHAC to be most useful at this stage of a PSHA study.

In the PSHA planning phase (which refers to resource allocation for the purpose of maximizing the reduction of uncertainty on the final hazard results), there is no need for a detailed analysis of uncertainty. In fact, such analysis is usually not available when the PSHA effort is structured. For this purpose it may be sufficient to conduct limited sensitivity analyses, using bounding hypotheses, and to consider the level of effort that would be required to substantially reduce each component of uncertainty.

The final utilization phase is critically important and arguably the one phase that should drive the level of uncertainty analysis and mode of uncertainty representation in a properly conducted PSHA. SSHAC's position is that the final results of a study should represent the epistemic uncertainty of the informed scientific community. This is roughly defined by SSHAC as the average of the uncertainties of the experts that make up the community (possibly weighted according to their degree of expertise, their outlier status, etc.).

A fundamental problem with this way of presenting the final results is that, as previously noted, the epistemic uncertainty in the hazard depends on which among many legitimate models one uses—for example, a deterministic or stochastic model of earthquake occurrence. What changes with the model is not the mean hazard but the amount of epistemic uncertainty and, therefore, all the fractile hazard curves—including the median. Therefore, any decision that is based on the fractile curves rather than the mean curve depends on the essentially arbitrary choice of how much epistemic uncertainty is included in the seismicity and ground motion models. This well-known fact has often been taken to mean that the only admissible decision rules are those based on the mean hazard and that other decision rules are wrong and should be excluded. In fact, this is not quite correct. As the study by Veneziano (1995) quoted in the SSHAC report shows:

1. If the mean hazard can be assumed to remain constant over the lifetime of the project (e.g., because only a small amount of relevant new information is expected to become available in the near future), decisions should be based exclusively on the present mean hazard.

2. On the other hand, if the mean hazard cannot be assumed to remain constant over the lifetime of the project, decisions should depend on possible future fluctuations of the mean hazard (Veneziano, 1995, p. 121).

These results show why the common practice of using mean probabilities is appropriate in certain cases but also explain why in other cases one should act conservatively. Notice that the distinction does not depend on the total amount of current epistemic uncertainty but on the amount of total uncertainty that might be explained in the future and thus might cause the mean hazard to fluctuate.

This is consistent with intuition. As a classic example of the irrelevance to decision making of the aleatory/epistemic classification, the betting attitude of a rational individual on the outcome of a coin flip should not change from before flipping, when all the uncertainty is aleatory, to after flipping (but before the outcome is revealed), when the same total amount of uncertainty is epistemic. On the other hand, the importance of temporal fluctuations of a mean hazard may be illustrated by considering the retrofitting problem, which occurs when, at some time after completion of a project, the estimated mean hazard changes and exceeds a regulatory limit. The reason why future volatility of the mean hazard should in this case affect present decisions is that the utility of each decision depends in an asymmetric way on future positive and negative changes in the mean hazard: large penalties are associated with retrofitting if the mean hazard increases, whereas only modest gains may result from future reductions in the mean hazard. The decision maker should consider the potential future volatility of the mean hazard and include it in his/her deliberations.

In the future, fundamental advances in PSHA may come from adopting this time-dependent view of earthquake safety decisions. However, explicit quantification of future volatility of a mean hazard would require a level of analysis even more sophisticated than that proposed by SSHAC, and the panel does not advocate such an extension at the present time, even for critical facilities.

Short of explicitly quantifying the future variability of the mean hazard, what could be done to provide the decision maker with a useful representation of epistemic uncertainty? One possibility, but certainly not the only one, is to calculate the mean hazard according to the uncertainty of each participating expert, when that expert acts as an evaluator (not integrator) of alternative models, data sets, etc. To the degree that the beliefs held now by different members of the scientific community reflect possible future fluctuations in the overall community mean hazard, this should be useful input to the decision maker. For example, this information would allow the decision maker to see how the decision he/she must make would vary if different experts in the informed scientific community had to make that same decision. Notice that the hazard curves derived from each expert do not suffer from the limitations of the fractile curves observed earlier; each of them is a mean hazard curve and therefore is insensitive to the choice of model type used by the expert.

Some observations should be made on presenting the final hazard results through the community mean hazard and the interexpert variability in the mean hazard, as just described:

1. One might argue that full epistemic uncertainty quantification is needed anyway, to calculate the mean hazard of the community and the mean hazard of the individual experts. However, this is true only in theory, as it is clear that different amounts of information are needed to estimate with confidence the mean value of a random variable, as opposed to its complete distribution. For example, the use of best estimates for recurrence and ground motion models often leads to hazard values that are close to the mean hazards obtained by considering a large number of alternative models. Moreover, there is no need when calculating the mean hazard to label accurately each component of uncertainty as epistemic or aleatory, provided that the total uncertainty is accounted for. Therefore, the elaborate machinery needed to carefully separate uncertainties of different types is no longer needed.

2. Much emphasis is given in the SSHAC report to intensive interaction among experts, discussion of alternative models, and exclusion or downweighting of outliers. These are all appropriate and remain valid under the format proposed here. In essence, what changes is that the TFI quantifies not the total uncertainty of the scientific community, as done in the SSHAC approach, but the variability of the mean hazard according to the experts that make up that community. In so doing, weights can be applied and outliers can be removed for the same reasons and in the same way as discussed by SSHAC.

3. The multiple interpretations, models, and model parameters at the basis of the elicitation process are not “lost.” They remain part of the documentation of the PSHA study and should be made available to interested users. The panel anticipates that users will primarily be technical experts—for example, in the context of a regulatory review or an update of a PSHA study. However, that information should, for the most part, be irrelevant to the decision maker.

As observed previously, the correct way to represent epistemic uncertainty for decision making would be through the uncertain fluctuations of the mean hazard in future assessments. The expert-to-expert variability of the mean hazard at the time of the analysis is only a surrogate for this variability and is not entirely satisfactory because using

it this way implies that, during the time interval of interest, new evidence and knowledge may end up “proving right” one member of the present group of experts. While this may not be a valid assumption, documentation of the expert-to-expert variability in the mean hazard may be preferable to the full display of epistemic uncertainty proposed by SSHAC.