Risk of Vessel Accidents and Spills in the Aleutian Islands: Designing a Comprehensive Risk Assessment - Special Report 293 (2009)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

CHAPTER 2 Fundamentals of Risk Assessment Risk is the combination of the likelihood and consequences of an undesirable event. For example, the risk of pollution from a vessel accident could be expressed as the likelihood of a spill com- bined with the impact of that spill. As noted in Chapter 1, to cal- culate risk, situations must be evaluated to answer the following questions: • What can go wrong? • How likely is it? • What are the impacts? The first question involves creation of a risk scenario; the sec- ond, determination of likelihood; and the third, specification of consequences. The process for answering these three questions is called “risk analysis,” and the answers derived, for all possible scenarios, are a complete expression of the risk being assessed. This chapter provides an overview of risk assessment; describes the overall organization of and approach to risk assessment; and summa- rizes the committee’s proposed approach for a risk assessment of shipping operations in the Aleutian Islands, which is detailed in Chapters 5 and 6. 28 

Fundamentals of Risk Assessment • 29 OVERVIEW OF RISK ASSESSMENT Risk assessment combines risk analysis with risk management, the latter term denoting the processes of establishing risk tolerance cri- teria and selecting and implementing risk reduction measures. Risk assessment is a rational and structured approach for identifying hazards, analyzing risk, and identifying risk reduction measures. Properly implemented within an organization that follows a long- term risk management process, it provides a cost-effective basis for maintaining risk within appropriate limits. In the marine industry, various risk assessment frameworks exist. One established approach is the International Maritime Organiza- tion’s (IMO’s) Formal Safety Assessment (FSA). FSA is described by IMO as a “rational and systematic process for assessing risks relat- ing to maritime safety and the protection of the marine environment” (IMO 2002, 1). This process is also used by IMO for evaluating the cost and benefits of options for reducing risks (IMO 2002). The results of risk assessments, including those employing FSA approaches, can be used to compare options, weigh costs against benefits, and aid in making decisions among options. Figure 2-1 outlines the FSA process. Most risk assessment processes, including those applied in other fields, such as the aviation and nuclear power industries (NRC 1997; NRC 1994; NRC 1983), use the same overall approach as FSA and generally comprise the following steps: • Hazard identification, • Risk analysis, • Risk control options, • Cost–benefit assessment, and • Recommendations for decision making. Step 1: Hazard Identification The hazard identification step, in the IMO approach, might more properly be called the hazard and accident scenario identification step. Hazards are materials or conditions with the potential to result in harm to human life or health, property, or the environment. During this preliminary hazard identification stage, analysts use a combination of techniques aimed at identifying all relevant hazards 

30 • Risk of Vessel Accidents and Spills in the Aleutian Islands Decision Makers Step 1 Step 2 Step 5 Hazard Risk Decision-Making Identification Analysis Recommendations Step 3 Risk Control Options Step 4 Cost–Benefit Assessment FIGURE 2-1 IMO’s FSA process. (Source: IMO 2002.) and associated scenarios within the scope of the risk assessment study. In the case of shipping operations, the objectives of hazard identification are to • Identify specific hazards involved in shipping that have the poten- tial to harm human life and health, property, or the environment;1 • Identify accident types (e.g., drift groundings, powered groundings, collisions) and scenarios and provide an understanding of the causal factors (e.g., loss of steering, inadequate stability) and conditions (e.g., sea state, weather, current) leading to these accidents; • Provide an understanding of the likelihood and consequences of these accidents and scenarios; and • Identify the high-risk scenarios and conditions under which they may occur. Hazard identification generally involves both high-level analyti- cal and qualitative assessments. Various techniques are applied, such as checklists, HaZID (Center for Chemical Process Safety 2008), and expert judgment. (The formal use of expert opinion and evi- dence is summarized in Appendix C. The discussion covers the use of expert opinion, the “facilitator,” and the issue of controlling bias.) The analytical assessment helps ensure that historical expe- 1 All other consequences of concern to stakeholders that are discussed later in this report are direct impacts of such harm or fear that it will occur. 

Fundamentals of Risk Assessment • 31 rience and accident data are taken into account; it is performed at a coarse level, sufficient to help identify the principal hazards and scenarios. The hazard identification should not be restricted to situations that have occurred in the past; rather, the approach used should allow for creative thinking such that potential hazards not previously encountered are also postulated. Keeping the analysis as broad as possible at this stage is essential to a quality assessment (Atwood et al. 2003; DNV 2002; NRC 1994; O’Hara et al. 2004). Step 2: Risk Analysis Once hazards and accident scenarios have been identified, detailed analysis of risks can begin. This step involves more rigorous inves- tigations into the conditions and causes of the most significant scenarios. It commonly includes processing and analyzing large quantities of data and performing modeling. The analysis relies on historical experience, analytical methods, and expert knowledge or judgment. To conduct a risk assessment, analysts must make practical deci- sions about the techniques to be used, such as hazard and operability analysis (HaZOP) (CCPS 2008), event and fault trees, elicita- tion of expert judgment, human reliability analysis (discussed in Appendix D), simulation, and consequence (fate and transport) analysis. Analysts must also determine the effort necessary to achieve a level of precision from the risk analysis that will ultimately result in beneficial, usable results for all concerned or potentially affected. Thus analysts must determine whether quantitative, semiquan- titative, or qualitative techniques or a combination thereof will provide the most appropriate risk estimates. Regardless of what techniques are used, careful identification of the sources of uncer- tainty is required, along with estimates of the uncertainty in stated results (Atwood et al. 2003; DNV 2002; O’Hara et al. 2004). (Appen- dix E examines issues associated with uncertainty, including sources of uncertainty, sensitivity analysis, propagation of uncertainty, and Bayesian statistical analysis.) The choice of techniques is influenced by the nature of the avail- able information and the precision necessary to determine a credible risk value. Figure 2-2 illustrates how qualitative or quantitative tech- niques can be used for risk analysis (ABS 2000). Regardless of the techniques chosen, the goal of the analysis remains the same: to derive 

32 • Risk of Vessel Accidents and Spills in the Aleutian Islands QUALITATIVE TECHNIQUES QUANTITATIVE TECHNIQUES FREQUENCY ASSESSMENT Model Estimate Causes Likelihoods • Absolute and relative risks Hazard Estimate • Major risk Identification Risk contributors CONSEQUENCE ASSESSMENT • Comparisons Model Estimate with other Effects Impacts risks Qualitative ranking Quantified benefits and costs of recommendations of risk reduction alternatives FIGURE 2-2 Risk analysis techniques. (Source: ABS 2002.) estimations of risk and to provide detail sufficient for examining risk reduction measures that can achieve a tolerable level of risk (NRC 1989). The output of the risk analysis should be a refined char- acterization of scenarios, their likelihood, and their consequences, allowing risks to be ranked in order of consideration for risk control options. Scenarios Scenarios are initially narrative descriptions of what can happen. In the case of shipping operations, developing scenarios requires exten- sive experience in those operations, good engineering knowledge, and a grasp of the modeling required to develop scenarios that can be analyzed efficiently. (See Appendix F for a detailed description of event sequence diagram methodology and risk scenario develop- ment.) Figure 2-3 illustrates the primary aspects of marine scenar- ios. The scenario begins with an initiating cause, such as a loss of propulsion, a fire, or adverse weather. The next step is to develop a sequence of events that represents the response of the “system” (the ship, its hardware and software, its crew) to the cause. The safeguards in place (barriers, operational controls, and risk control options) are delineated. If the cause is not controlled by the safe- guards, failures may occur (hardware failures, human and organiza- tional failures, or failures caused by environmental stressors). This 

Fundamentals of Risk Assessment • 33 C11 C12 Accident d Causes an rt te Categories Fa nspo C1 Tra C2 C3 Failures, Human and Organizational Errors, Environmental Stressors Safeguards, Barriers, Operational Controls, Risk Control Options C Consequences FIGURE 2-3 Primary aspects of marine scenarios. sequence of events either is arrested or leads to an accident that can have immediate consequences, such as loss of life, physical damage to the ship, and spills of hazardous materials. If a spill is involved, the scenario continues through transport of the mate- rial and its deposition in the environment. Should a spill occur, mitigation measures (additional safeguards) can limit the envi- ronmental and subsequent economic and social consequences. Remediation, or cleaning up the contamination, can limit harm to life in the area. Likelihood Estimates of the likelihood of the identified scenarios come first from experienced judgment and second from simple statistics based on analysis of accident reports. Finally, when needed, likelihood estimates are derived from evaluation of detailed models of the scenarios. Consequences The consequences of concern to stakeholders are identified through literature reviews and interactions with stakeholders (NRC 1994; NRC 1989). For the present study, the committee identified pre- liminary consequences of concern following a series of informa- tional meetings (see the “Risk Assessment Approach” section later in this chapter). Analysts will need to refine this list. Historical conse- quences related to loss of life and damage to ships and cargoes can be 

34 • Risk of Vessel Accidents and Spills in the Aleutian Islands quantified from accident data. Consequences to the environment can be identified through modeling efforts. The few historical events with significant consequences can indicate the potential extent of consequences but are not adequate for prediction purposes. One aim of the risk analysis is to determine and characterize the risk levels of various scenarios. Often this characterization will use categories such as the following to determine the importance of risk reduction for a given scenario: • Negligible—no risk reduction methods required; • Tolerable—risk should be reduced to “as low as reasonably practical”; and • Intolerable—risk reduction must be undertaken irrespective of cost. Such characterization allows comparison across scenarios and risks and provides a means for properly considering risk reduction for situations outside acceptable boundaries given the concerns and needs of the various stakeholders. Step 3: Risk Control Options The next step is to identify possible risk control measures, priori- tize and identify those that are more promising, and analyze their effectiveness. The results of the screening process associated with hazard identification and the risk analysis of the existing system allow the assessment of risk control measures to focus on scenarios identified as having the highest risk, considering the combination of likelihood of occurrence and consequences. However, it is also important to consider scenarios identified as having the highest likelihood of occurrence even if their consequences are modest, and scenarios having the highest consequences even if their like- lihood is small. Once screened, the more promising risk control measures are subjected to risk analysis as described in Step 2 above to quantify their impact on the likelihood and consequences of accidents. Step 4: Cost–Benefit Assessment The purpose of cost–benefit assessment is to provide an addi- tional tool for decision making that identifies the implementation 

Fundamentals of Risk Assessment • 35 costs and the expected benefits of risk reduction measures. Cost- effectiveness is often expressed in terms of net cost per unit reduction in risk, enabling the ranking of risk reduction measures. While determining implementation costs and understanding the relationship between costs and benefits yield valuable input for the decision-making process, that process is inevitably more com- plex than simply selecting the most cost-effective solutions. For example, certain benefits, such as damage to natural resources and societal impacts, are difficult to quantify in monetary terms yet need to be considered in the overall assessment. In cost–benefit assessment, costs usually are discounted to present value. Benefits generally are not discounted; rather, the cumulative benefits over the study period are applied. Thus, a cost-effectiveness index for a risk reduction measure is calculated as the net cost of the measure divided by its gross benefits. For shipping opera- tions, typical indices are dollars per fatality avoided or dollars per gallon of oil spill avoided. Alternatively, a multidimensional com- parison of costs and risk curves or risk matrices (described later in this chapter) can be more informative than calculation of a cost– benefit ratio. Step 5: Recommendations for Decision Making The final step in IMO’s FSA methodology is to present decision makers with a set of well-defined recommendations. Those rec- ommendations should reflect all relevant findings, including the following: • Comparison and ranking of the hazards and risk scenarios, • Comparison and ranking of risk control measures as a function of costs and benefits, and • Consideration of risk control measures that keep risks as low as reasonably practical. Documentation of the recommendations should include a description of the evaluation criteria used in ranking the risks and risk reduction measures. It should also include an explanation of significant uncertainties associated with the recommendations (NRC 1989)—in the case of costs, for example, the interest rate used for discounting (see the discussion of addressing uncertainty in Appendix E). 

36 • Risk of Vessel Accidents and Spills in the Aleutian Islands ORGANIZATION OF RISK ASSESSMENT Definition of the Problem Before beginning a risk analysis, it is important to define the problem carefully. The purpose of problem definition is to identify objectives and set the bounds for and focus of the analysis. As an example of defining the problem at hand, the risk assessment addressed by the present study focuses on accidents and spills rather than intentional operational releases. This is but one of many dimensions that must be defined for this risk assessment. The charge to the committee and this report define the problem and scope of the approach for this risk assessment study. Management of the Assessment The previous section described the sequence of steps to be fol- lowed in a risk assessment. Other important analytical choices include whether the assessment should be tiered in a way that permits broad-brush qualitative aspects of risk to be examined first, on the chance that easily identified risks can be addressed by measures that are relatively easy to implement, saving both time and expense. If this approach is applied, measures with high benefit and relatively low implementation costs may prove suf- ficient in some circumstances, obviating the need to extend the assessment into areas of greater precision whereby quantitative estimates of risk are developed. When a risk assessment is intended to aid decision makers in identifying and reducing technological risks of considerable pub- lic concern, some elements of how best to organize the study are matters of choice that are not easily prescribed. Primary among these is the relationship to be developed among managers and decision makers, analysts, those with local knowledge of the tech- nological system undergoing analysis, others with a detailed under- standing of the potential local environmental and socioeconomic impacts associated with the risks of concern, and the broader stakeholder community of interested and affected parties. The modern approach to risk assessment increasingly emphasizes formal roles for all these parties. 

Fundamentals of Risk Assessment • 37 Stakeholder Engagement Recent years have seen a trend in risk assessment toward extensive engagement of stakeholders throughout the process of defining and analyzing risks and identifying risk reduction measures (Bonano et al. 2000; NRC 1996; Presidential/Congressional Commission on Risk Assessment and Risk Management 1997; Omenn 2006). For example, the Presidential/Congressional Commission on Risk Assess- ment and Risk Management (1997) divided the risk assessment and management process into six stages. Only the final “evaluation” stage (which involves assessing the effectiveness of measures adopted to address the identified risks) is cited as being appropriately conducted without explicit stakeholder involvement (see Figure 2-4). Problem/ Context Evaluation Risks Engage Stakeholders Actions Options Decisions FIGURE 2-4 Engagement of stakeholders in the risk assessment and management process. (Source: Omenn 2006. Reprinted with permission from the American Association for the Advancement of Science.) 

38 • Risk of Vessel Accidents and Spills in the Aleutian Islands Engaging stakeholders, decision makers, and analysts—typically contractors—in the design and conduct of a risk assessment has been termed “collaborative risk assessment” (Charnley 2000). This was the approach taken in the Prince William Sound Risk Assessment Study (PWS study) (Merrick et al. 2002), in which a “highly interactive and cooperative” steering committee (NRC 1998) played a significant role in shaping the overall study through frequent meetings with the analytical team. The steering commit- tee operated by means of consensus decision making. In the end, although it had begun as an advisory body with many members skeptical about the outcome of the study, it fully endorsed the study results and volunteered to be the publisher of record for the final study report (Merrick et al. 2002; PWS Steering Committee 1996). The PWS study’s steering committee was constituted to be broadly representative of the main groups with an interest in risk reduction in Prince William Sound, groups that, in the aftermath of the 1989 Exxon Valdez oil spill, had highly adversarial relation- ships. The committee’s unanimous acceptance of the study results, together with self-reports by the study team (Merrick et al. 2002), suggest that stakeholder engagement accomplished an important goal of collaborative risk assessment—organizational learning that led not only to new understanding of the nature of risks within the system but also to a new collaborative decision-making approach to managing the identified risks. Stakeholders contributed resources, knowledge, and information to the study, and the resulting collabora- tive learning induced not only policy but also organizational change (Busenberg 2000). In the PWS study, local stakeholders played another impor- tant role—supplying substantive domain expertise that helped the study team quantify the relative importance, in terms of rela- tive conditional probabilities, of various situational factors that could influence risk in the Prince William Sound shipping sys- tem (Merrick et al. 2002). A group consisting of pilots, deck offi- cers, and shipboard engineers who had worked aboard trade vessels of the Trans-Alaska Pipeline System rated the relative like- lihood of a large number of different scenarios resulting in acci- dents. The results of questionnaires in which 120 scenarios were rated (Merrick et al. 2002) became a primary data source for the PWS study. 

Fundamentals of Risk Assessment • 39 Identification of Stakeholders The question of whom to consider stakeholders, and by extension whom to invite to be engaged in the analytic and deliberative phases of a risk assessment, is ultimately answered in part by the nature of the problem and in part by the extent to which the decision- making organization views itself as inclusive (Mitchell et al. 1997). The modern tendency in environmental management is certainly in the direction of inclusiveness—in the words of Mikalsen and Jentoft (2001, 282), going “from user groups to stakeholders.” The basis for stakeholder identification should not be closeness to the problem itself (i.e., user groups) but rather who has power, legitimacy, and urgency given the problem’s defining characteristics, particularly the distribution of benefits and costs in relation to the problem as it stands today or might stand in the future (Mikalsen and Jentoft 2001; Mitchell et al. 1997). By these measures, stake- holders can be identified as definitive (having unequivocal claims by virtue of direct engagement in the problem domain, in other words, possessing all three of the above attributes); expectant (having legiti- mate expectations to be involved because they possess two of the three attributes); or latent (possessing just one of the three attri- butes) (Mikalsen and Jentoft 2001). The implication is that groups geographically removed from the problem arena may nevertheless have a stake in addressing the problem (e.g., fishing companies headquartered in the lower 48 states or national and international nongovernmental organizations representing environmental inter- ests), just as may local populations, such as native communities or immigrant workers in local seafood processing operations. As is clear from Ritchie and Gill’s (2006) study of the poten- tial social impacts of the M/V Selendang Ayu spill and the com- mittee’s meetings with community leaders in Dutch Harbor, the Dutch Harbor community has enjoyed considerable economic benefit from its position as home port to major Bering Sea–Gulf of Alaska fisheries that are among the most economically valu- able in the world. These economic benefits have translated into substantial social and cultural benefits to the community at large, making it a place that residents value for the high quality of life it now affords (Ritchie and Gill 2006). In the end, how- ever, these community attributes and the economic activity that supports them underscore the high degree of resource dependency on the Bering Sea–Gulf of Alaska fisheries, rendering both the 

40 • Risk of Vessel Accidents and Spills in the Aleutian Islands local community and the fisheries themselves highly vulnerable to the effects of oil spills. As the above discussion suggests, some aspects of stakeholder vul- nerability to oil spills are difficult to quantify given the approaches typically used to account for and value social impacts in risk assess- ments (Murphy and Gardoni 2006). The least-connected latent stakeholders—perhaps immigrant workers in seafood-processing plants with comparatively low levels of both power and legitimacy but high levels of urgency—are most likely to be subject to under- counting with respect to the losses they would suffer were a large oil spill with major detrimental impacts on local fisheries to occur. Methods derived from the field of economic development have been proposed as means of accounting for such impacts in risk anal- ysis (Murphy and Gardoni 2006), but their utility has yet to be demonstrated through application. Possible Limitations and Biases Stakeholder engagement in risk assessment clearly confers many benefits. These include, as noted above, trust building, clarification of the values and goals that should inform the assessment, the pro- vision of local information and knowledge that would otherwise be easily missed, and potentially a path to organizational learning and policy change that might not otherwise be available. Stakeholder engagement also comes with potential limitations, however. Fre- quent interactions with stakeholders can compromise the study team’s objectivity, lead to “common denominator” study framing when analytical understanding might indicate different choices, and introduce bias into the analysis (NRC 1998). The heavy reli- ance on poorly controlled and documented elicitation of expert views in the PWS study, for example, led to results that proved difficult to validate (Merrick et al. 2002). The choice was made to pursue this path because the study’s steering committee insisted that only local data be used in the study, rather than the worldwide data often used to support estimates of the likelihood of rare events in risk studies. Reliance on individuals with knowledge of or direct involvement in previous incidents, such as those involving the Exxon Valdez or M/V Selendang Ayu, can introduce “availability bias,” by which repeat events are deemed highly likely simply because they have already occurred (Merrick et al. 2002). In the Aleutians’ maritime system, 

Fundamentals of Risk Assessment • 41 for example, transiting cargo ships and freighters are easily iden- tified as the risk “problem” given that vivid imagery of the M/V Kuroshima, M/V Selendang Ayu, and Cougar Ace accidents is fresh in local memory. The fishing fleet is more easily seen as vulnerable to spills associated with vessels in the cargo trade, rather than a source of additional risk. Yet in the PWS study, the primary source of collision risk was found to be a collision between a fishing vessel and a Trans-Alaska Pipeline System trade tanker, a risk traceable to the large number of fishing vessels present within Prince William Sound during fishing seasons. RISK ASSESSMENT APPROACH This section presents a brief exposition of the general ideas behind risk assessment to set the stage for discussion of the specific tasks required for the risk assessment of shipping operations in the Aleutian Islands. Recall that the approach to risk assessment begins with risk analysis, a systematic process for answering the three questions posed at the beginning of this chapter: • What can go wrong? • How likely is it? • What are the impacts? The formal definition of a risk analysis proceeds from these simple questions, where a particular answer is Si, a particular scenario; pi, the likelihood of that scenario; and Ci, the associated consequences. In mathematical parlance, this answer is known as the risk triplet <Si, pi, Ci>, and the complete set of answers—that is, all possible scenarios—is, in fact, the “risk analysis” (NRC 1997). The analysis describes and quantifies every scenario. The calculational link between the full set of triplets and the “risk curve” (e.g., the risk matrix) that displays summary results for frequency versus consequences is developed later in this section. Approaches used for defining the scenarios always begin with qualitative descriptions. These descriptions become more thorough, detailed, and analytical as structured search and analysis tools are applied. Estimations of likelihood and consequences gener- ally begin with rough ideas based on experience and judgment. They then progress, stepwise, through ranges taken from event 

42 • Risk of Vessel Accidents and Spills in the Aleutian Islands records;2 to statistics based on the data; and finally to full-fledged analysis based on logic models, engineering and physics calcula- tions, simulation, human performance modeling, and fate and trans- port modeling. In some cases, the analysis may end with qualitative techniques if risks do not lend themselves to quantification, if discrete or sufficient credible data required for quantitative assess- ments are unavailable, or if obtaining or analyzing data is not cost-effective. Analysis results include qualitative descriptions of the scenarios (narratives), their likelihood (e.g., frequent, infrequent, rare), and their consequences (e.g., mild, moderate, severe, catastrophic). As the analysis is extended, it produces models of the scenarios; calculations of the frequencies of particular events, outcomes, and classes of scenarios; and estimates of specific consequences (oil spills, deaths and injuries, damage to natural resources, socioeconomic impacts, and damage to reputation). These outputs become more quantitative and the uncertainty of the results is narrowed as more detailed information is developed. Summary measures of risk are presented in qualitative, semi- quantitative, and quantitative formats, becoming more quantitative as the level of analysis deepens. However, results in all of these formats should be reported because they speak to different audi- ences and to different purposes. One popular format for presenting the results of qualitative risk analysis is a matrix (see Figure 2-5) with columns corresponding to various levels of consequence and rows to different levels of likelihood. The number of columns and rows depends on the depth of the completed analysis and the intended use of the results. Risk scenarios analyzed are placed in the cells of the risk matrix according to their level of likelihood and conse- quences. Color schemes are used to indicate different risk levels (e.g., red for high risk). The risk scenarios are thereby placed in a limited number of risk categories for ranking and comparison in risk management, also on the basis of qualitative acceptance crite- ria. Sometimes numerical scales (e.g., 1 through 5) are used for one or both dimensions of the matrix, but the numbers are not meant 2 These ranges must be tempered with judgment to allow for the limited nature of historical events: not every possible scenario has yet happened, and some have happened so seldom that one can have little confidence that the observed range is fully representative of the possibilities. 

Fundamentals of Risk Assessment • 43 Frequency of Severity of Incident (or Consequences) Occurrence/ Likelihood Incidental Minor Serious Major Catastrophic (1) (2) (3) (4) (5) Frequent (5) High Risk Occasional (4) Seldom (3) Remote (2) Low Risk Unlikely (1) FIGURE 2-5 Example risk matrix. to be estimates of actual frequencies of scenarios or the magnitude of their consequences. Note that uncertainty can be represented in the risk matrix by having results span multiple cells rather than being scored in a single cell (NRC 1997). A number of formats can be used to represent results of quan- titative risk analysis. One is the “expected value” of risk. If scenario Si has a consequence of magnitude Ci and a probability of pi, then the scenario risk (Ri) is defined as Ri = pi * ci, and the total risk is calculated by summing over all risk scenarios: R = Σ pi * ci. A major drawback of this format is that it masks the potentially important difference between “low probability–high consequence” and “high probability–low consequence” scenarios when they result in the same values for probability times consequence. A far better method for representation of risk is known as a “risk curve” or “risk profile.” The risk curve is developed from the complete set of risk triplets. The triplets are presented in a list of scenarios rearranged in order of increasing consequences, that is, C1 ≤ C2 ≤ C3 ≤ . . . ≤ CN, with the corresponding probabilities. A fourth column is included showing the cumulative probability, Pi (uppercase P), as shown in Table 2-1. When the points <Ci, Pi> are plotted, the result is the staircase function illustrated in Figure 2-6. Since the scenarios in Table 2-1 

44 • Risk of Vessel Accidents and Spills in the Aleutian Islands TABLE 2-1 Risk Table Scenario Probability Consequences Cumulative Probability S1 p1 C1 P1 = P2 + p1 S2 p2 C2 P2 = P3 + p2 … … … … Si pi Ci Pi = Pi + 1 + pi … … … … SN−1 pN−1 CN−1 PN−1 = PN + pN−1 SN pN CN PN = pN Source: NRC 1997. generally are really categories of scenarios, one could argue that the staircase function should be regarded as a discrete approximation of a nearly continuous reality. If a smooth curve is drawn through the staircase, that curve can be regarded as representing the actual risk, and it is the risk curve or risk profile (Figure 2-6). Often a combination of qualitative and quantitative analyses is needed to establish risk estimates when the problem under analysis is diverse and complex. Indeed, the majority of risk assessments fall P1 Staircase function P = Probability of exceedance P2 P3 Smoothed risk curve P4 PN C1 C2 C3 C4 CN Consequences FIGURE 2-6 Risk curve. (Source: NRC 1997.) 

Fundamentals of Risk Assessment • 45 into this category. In some cases, actual numerical values are used to discriminate among different levels of likelihood or consequences expressed with qualitative descriptors (NRC 1997). The first activity in the actual risk analysis step is the analysis of historical events. In the case of the present study, accidents that have occurred in the Aleutians are of primary concern, and if these histori- cal data were sufficient, they would be all the analysis would examine. Because of special conditions that occur in the Bering Sea, the sig- nature of accidents in that region differs from worldwide averages. However, to ensure that limited data for the Aleutians do not cause the risk analysis to ignore rare, high-consequence events, those data must be supplemented by worldwide data. The most appropriate way to integrate the data is by using Bayes’ theorem, as described in Appendix E. In the preliminary assessment, however, results for the Aleutians may be adjusted by using expert judgment, on the basis of information extracted from the worldwide data. Reviewing historical data on accidents and spills provides a pic- ture of the accident types likely to occur and an indication of the types that pose the greatest risk. To characterize the risks and begin to understand how the likelihood and consequences of spills can be mitigated, it is then necessary to understand the accident scenar- ios, that is, the series of steps leading up to these dominant accident types. The development of accident scenarios begins with identification of the conditions that affect the progress of the sce- narios and limit their consequences. As shown in Figure 2-7, for each ship type (including fuels and cargoes), the analyst asks what causes can lead to various accident categories and subsequent damage. Qualitative methods (such as checklists, HaZID, and HaZOP) can be used to help identify hazards and scenarios of concern. These methods can also be useful screening tools. Screening is particularly important since hazard identification and accident scenario identifi- cation can yield numerous scenarios that could result in losses. Since analyzing all such scenarios in detail may not be realistic or even possible, high-level risk screening may be desirable. Screening allows Accident Immediate Ship Type Causes Categories Damage FIGURE 2-7 Simplified accident scenarios. 

46 • Risk of Vessel Accidents and Spills in the Aleutian Islands analysts to review various scenarios and risk controls and safeguards in place and to compare them against broad risk criteria with estab- lished thresholds to determine which scenarios require further assess- ment. The further assessment can be conducted with either qualitative or quantitative methods, again depending on the nature of the infor- mation available and the level of precision required. At the same time, it is important to retain the list of screened scenarios. In fact, it is bet- ter to think of this process as one of setting priorities, because assump- tions used in the screening process need to be tested later in the analysis to ensure that important scenarios have not been set aside. In addition, new information often emerges that challenges early assumptions. Analysts must expand the potentially important, high-level, simplified accident scenarios with detailed information from the available data sources. To extract the most useful information from the historical record, a model is needed. For this purpose, the committee proposes an extension of the simplified accident scenario model illustrated in Figure 2-7. It begins with the three elements shown in Figure 2-8 that represent the initial or bound- ary conditions for the scenario: the ship type (including its fuel and cargo); its location in the Aleutian chain; and the conditions, such as sea state and weather, before and during the sequence of events of the accident. All ship types must be considered; those of importance will surely include tankers, containerships, service and refueling support ships, fishing boats, local commercial ships, and passenger ships. As for locations, the Risk Analysis Team will likely need to break up the areas near the Aleutians into zones mapped onto the sea, identifying areas of similar hazard and sensitivity, such as passes and harbors (see Figure 2-9). Conditions of importance identified by the com- mittee include weather (sea state, freak waves, icing, wind, rain, and fog), traffic, season of the year, and time of day. Incorporated next are the additional elements identified in the simplified accident scenario of Figure 2-7: the cause, the accident Location Ship Type Conditions (area) FIGURE 2-8 Initial conditions. 

Fundamentals of Risk Assessment • 47 Zone j Zone i FIGURE 2-9 Illustrative zones in the Aleutians. category, and the immediate damage. Adding the opportunities for crew/rescuer control, the environmental consequences, and pos- sible remediation yields the basic scenario model for the risk anal- ysis (see Figure 2-10). This model can be used in several ways to facilitate the risk analysis, as described below. The elements of the model can be defined as follows: • Cause [fire or explosion, flooding, human error, loss of propul- sion, loss of steerage, and weather (from the conditions identified earlier)]. • Accident category (drift grounding, powered grounding, collision, allision, structural failure). Ship Location Opportunity Conditions Causes Type (area) for Control Accident Immediate Opportunity Environmental Remediation Categories Damage for Control Consequences FIGURE 2-10 Basic scenario model for Aleutian shipping risk analysis. 

48 • Risk of Vessel Accidents and Spills in the Aleutian Islands • Immediate damage [spill (material, amount, rate, duration), loss of life (crew and rescuers), physical damage to property]. • Opportunities for control. [Crew and rescuers usually have multiple opportunities to control the accident, and the analysis team must identify and model them. They are grouped into two general types in the basic scenario model: the opportunity to con- trol events (a) before the causal event actually becomes an acci- dent and (b) after the accident has caused immediate damage but before subsequent consequences accrue.] • Environmental consequences. (Because of the rare nature of seri- ous spills, modeling is needed to evaluate environmental and sub- sequent socioeconomic damage; anecdotal evidence is available in the data.) • Possible remediation (the final opportunity to control long-term losses). Event analysis proceeds with cataloging of the results of the review of accident records within the framework of the scenario model. For this purpose, a table with headings corresponding to the elements of the scenario model can be used (see Table 2-2). Once analysts have populated the table (referred to as the event database) by using the available data, they will find that many of the cells are empty because of incompleteness in the accident reports. Nevertheless, a variety of useful analyses can be performed: • Major accident categories can be grouped on the basis of events in the database. TABLE 2-2 Elements of the Scenario Model Environmental Consequences Opportunity for Control Opportunity for Control Ship Type and Cargo Immediate Damage Accident Category Remediation Conditions Location Cause Event Event 1 Event 2 

Fundamentals of Risk Assessment • 49 • The frequencies of representative sequences of events through immediate damage can be determined by combining data from the Aleutian events table, the generic (worldwide) table, and expert judgment. • Pairs of consequences and conditions can be examined, and con- ditional probability estimates can be developed, such as the likelihood of drift groundings involving bad weather or collisions occurring in passes compared with other locations. Finally, the basic scenario model provides a useful structure for evaluating and comparing risk control options. Figure 2-11 illus- trates how risk control options can intervene at every stage of the scenario. Interventions before the accident occurs are known as “prevention” and are clearly preferred. However, it is impossible, economically and in principle, to prevent every accident. Some unanticipated events will occur, and one must be able to control such events. Moreover, in many cases it is more feasible and economically viable to control an event than to try to prevent it. Therefore, the best approach is to distribute risk control options throughout the scenario, some offering prevention and others providing mitiga- tion of accident consequences (Atwood et al. 2003; DNV 2002; O’Hara et al. 2004; USNRC 1981). An approach for evaluating competing options qualitatively is to evaluate each option against each stage of the model. In this process, favored solutions must be considered on the same Ship Location Opportunity Conditions Causes Type (area) for Control Intervention at these stages provides prevention, but cannot achieve complete prevention. Accident Immediate Opportunity Environmental Remediation Categories Damage for Control Consequences Intervention at these stages provides mitigation. FIGURE 2-11 Risk control can intervene at every stage of the scenario. 

50 • Risk of Vessel Accidents and Spills in the Aleutian Islands basis as all others. Overstated claims must be proved. Table 2-2, based on the scenario model, provides a tool for this evaluation. As each option is considered, analysts ask for which scenarios and where in each the option offers improvement. They then enter in the table the effectiveness of the option versus the stages. Also included are the basis for that claim and the feasi- bility and practicality of the option and its expected costs. These evaluations can be based on judgment, but it must be informed and documented judgment. Many proposed options can be expected to be seen as offering improvements for similar effects. In such cases, it is likely that only one of the competing options should be instituted. Careful cost–benefit analysis will suggest which one to choose. Note that after qualitative analysis and preliminary quantitative analysis, it may be possible to select some particularly obvious options for implementation. In most cases, however, more thorough, detailed models and quantification will be required. RISK ASSESSMENT OF ALEUTIAN SHIPPING OPERATIONS The approach for the Aleutian Islands risk assessment proposed by the committee encompasses all the steps in IMO’s FSA identified earlier in Figure 2-1: hazard identification, risk analysis, risk control options, cost–benefit assessment, and recommendations for deci- sion making. However, the organization and sequencing of the spe- cific tasks necessary to complete these steps need to reflect lessons learned from many previous risk assessments. The progress of the PWS study illustrated many problems that need to be avoided. Risk analysts tend to attack the problem in bottom-up fashion, attempt- ing to perform the best and most complete analysis possible. By the time they make their first attempt to quantify their model, the majority of the available funding has been spent. Many corrections, reframings, and additions are required, but there are no resources to complete the work. Experience has revealed that a phased approach can avoid many of these problems, better focus the detailed analysis effort, and provide useful results at an early stage. The committee’s plan for the risk assessment of Aleutian ship- ping operations begins with a Phase A Preliminary Risk Assess- 

Fundamentals of Risk Assessment • 51 ment that structures the overall problem. It is as complete as possible in formulating the range of possible scenarios, but modeling is lim- ited. The Phase A assessment relies heavily on data analysis and expert judgment. The follow-on Phase B Focused Risk Assessment is aimed at providing careful and detailed comparisons of risk before and after risk control options are applied. The committee proposes an organizational structure for the risk assessment consisting of four groups or panels—a Management Team, an Advisory Panel, a Risk Analysis Team, and a Peer Review Panel. The Management Team would assume overall responsibility for ensuring that the work is carried out in an effective and useful way. The Advisory Panel would consist of stakeholders and experts who could provide local knowledge and expertise. The Risk Analysis Team would be provided by the contractor. Finally, the Peer Review Panel would provide technical oversight. The four groups would interact to move the project through the risk management process shown in Figure 2-12. Details are provided in Chapters 5 and 6. The entire risk assessment must encompass the steps outlined in Figure 2-13. The work begins with the Phase A risk analysis, which provides a high-level estimate of the likelihood and consequences of Responsible Party Define the Problem and Risk Management Approach This committee Collaborative effort of Advisory Panel, Perform the Risk Assessment Management Team, Risk Analysis Team, and Peer Review Panel Final Report Documenting Collaborative effort of Management Team Assessment and Recommendations and Advisory Panel Decision-Making Process Decision makers Implementation of Risk Reduction Measures Monitoring FIGURE 2-12 Steps in the risk management process for the Aleutian Islands. 

52 • Risk of Vessel Accidents and Spills in the Aleutian Islands Phase A Preliminary Risk Assessment Risk analysis: characterize risks by performing a high-level estimate of the likelihood and consequences of accidents and dominant accident scenarios Rank accidents and accident scenarios by level of risk Develop list of potential risk reduction measures Perform qualitative assessment and prioritization of risk reduction measures Phase B Focused Risk Assessment Perform quantitative risk analysis to estimate effectiveness and benefit–cost for risk reduction measures Rank risk reduction measures and recommend measures for implementation FIGURE 2-13 Steps in the proposed tiered risk management process. accidents and dominant accident scenarios. This is followed by a ranking of accidents and accident scenarios by level of risk and development of a list of potential risk reduction measures. Next are a qualitative assessment and prioritization of risk reduction measures. In Phase B, detailed analysis provides more rigorous comparisons of risk with and without specific risk control measures. The analysis includes quantitative risk analysis to estimate the effectiveness and benefit–cost of risk reduction measures, ranking of the measures, and the recommendation of measures for implementation. To avoid misleading results, groups of control measures must be examined to ensure that the potential improvements offered by one measure are not already provided by others. The basic task structure of the proposed risk assessment approach is shown in Table 2-3, which indicates how the Aleutian Islands risk assessment tasks relate to IMO’s FSA steps. Phase A includes the 

TABLE 2-3 How the Tasks of the Aleutian Islands Phased Risk Assessment Relate to the Steps in IMO’s FSA IMO FSA Step 3. Risk 1. Hazard 2. Risk Control 4. Cost–Benefit 5. Decision-Making Task Identification Analysis Options Assessment Recommendations Phase A Preliminary Risk Assessment 1. Traffic Study 2. Baseline Spill Study 3. Identification of High-Risk Accidents 4. Phase A Consequence Analysis 5. Accident Scenario and Causality Study 6. Development of Rankings for Accident Scenarios 7. Development of List of Potential Risk Reduction Options 8. Evaluation of Risk Reduction Options 9. Prioritizing of Risk Reduction Measures 10. Peer Review Phase B Focused Risk Assessment: Comparative Analysis of Risk Control Options 1a. Detailed Risk Comparison: Base Case Versus Option Set 1 1b. Cost–Benefit Assessment: Base Case Versus Option Set 1 2a. ... Decision-Making Recommendations 

54 • Risk of Vessel Accidents and Spills in the Aleutian Islands FSA’s hazard identification step; the qualitative and initial quantita- tive portions of the risk analysis step; and preliminary portions of the risk control options, cost–benefit assessment, and decision-making recommendations steps. Upon completion of Phase A, the risk ana- lysts will have identified the major accident categories and esti- mated their likelihood. The analysts will have defined the full range of scenarios that may be of interest and investigated the fate of a rep- resentative set of spills in a representative set of locations along the Aleutian chain. Local experts and stakeholders will have proposed a set of risk reduction options, evaluated their feasibility and potential impacts on each element of the scenarios, and made preliminary recommendations for prioritizing the options. This approach will ensure that a well-defined subset of the full risk assessment with a closely controlled scope is performed initially. Phase A will provide useful preliminary results and a sound basis for scoping future work while retaining a substantial portion of the budget for specific analyses. Phase B is expected to be performed in a series of follow-on tasks aimed at refining the Phase A results for evaluation of specific risk reduction options. Organizing the steps of a risk assessment in a series of phases is a well-tested approach for improving the quality and cost-effectiveness of the endeavor. Careful structuring of tasks is required to ensure that the initial phase provides useful information, does not mask impor- tant aspects of the problem, and does not bias future work (Atwood et al. 2003; DNV 2002; O’Hara et al. 2004). REFERENCES Abbreviations ABS American Bureau of Shipping CCPS Center for Chemical Process Safety DNV Det Norske Veritas IMO International Maritime Organization NRC National Research Council PWS Prince William Sound USNRC U.S. Nuclear Regulatory Commission ABS. 2000. ABS Guidance Notes on Risk Assessment. Houston, Tex. Atwood, C. L., J. L. LaChance, H. F. Martz, D. L. Anderson, M. Englehardte, D. Whitehead, and T. Wheeler. 2003. Handbook of Parameter Estimation for 

Fundamentals of Risk Assessment • 55 Probabilistic Risk Assessment. NUREG/CR-6823, SAND2003-3348P. Sandia National Laboratories for U.S. Nuclear Regulatory Commission, Washington, D.C. Bonano, E. J., G. E. Apostolakis, P. F. Salter, A. Ghassemi, and S. Jennings. 2000. Application of Risk Assessment and Decision Analysis to the Evaluation, Ranking and Selection of Environmental Remediation Alternatives. Journal of Hazardous Materials, Vol. 71, pp. 35–57. Busenberg, G. 2000. Innovation, Learning, and Policy Evolution in Hazardous Systems. American Behavioral Science, Vol. 44, No. 4, pp. 1–11. CCPS. 2008. Guidelines for Hazard Evaluation Procedures, 3rd ed. John Wiley and Sons, N.J. Charnley, G. 2000. Enhancing the Role of Science in Stakeholders-Based Risk Management Decision-Making. Health Risk Strategies, Washington, D.C. DNV. 2002. Marine Risk Assessment. Offshore Technology Report 2001/063. Health and Safety Executive, London. IMO. 2002. Guidelines for Formal Safety Assessment (FSA). IMO MSC/Circ. 1023, MEPC/Circ 392. April 5. Merrick, J. R. W., J. R. van Dorp, T. Mazzuchi, J. R. Harrald, J. E. Spahn, and M. Grabowski. 2002. The Prince William Sound Risk Assessment. Interfaces, Vol. 32, No. 6, pp. 25–40. Mikalsen, K., and S. Jentoft. 2001. From User-Groups to Stakeholders? The Public Interest in Fisheries Management. Marine Policy, Vol. 25, No. 4, pp. 281–292. Mitchell, R. K., B. R. Agle, and D. J. Wood. 1997. Toward a Theory of Stakeholder Identification and Salience: Defining the Principle of Who and What Really Counts. Academy of Management Review, Vol. 22, pp. 853–886. Murphy, C., and P. Gardoni. 2006. The Role of Society in Engineering Risk Analysis: A Capabilities-Based Approach. Risk Analysis, Vol. 26, No. 4, pp. 1073–1083. NRC. 1983. Risk Assessment in the Federal Government: Managing the Process. National Academy Press, Washington, D.C. NRC. 1989. Improving Risk Communication. National Academy Press, Washington, D.C. NRC. 1994. Science and Judgment in Risk Assessment. National Academy Press, Washington, D.C. NRC. 1996. Understanding Risk: Informing Decisions in a Democratic Society. National Academy Press, Washington, D.C. NRC. 1997. Risk Assessment and Management at Deseret Chemical Depot and the Tooele Chemical Agent Disposal Facility. National Academy Press, Washington, D.C. NRC. 1998. Review of the Prince William Sound, Alaska, Risk Assessment Study. Marine Board, National Academy Press, Washington, D.C. O’Hara, J. M., J. C. Higgins, J. J. Persensky, P. M. Lewis, and J. P. Bongarra. 2004. Human Factors Engineering Program Review Model. NUREG-0711, Rev. 2. U.S. Nuclear Regulatory Commission, Washington, D.C. 

56 • Risk of Vessel Accidents and Spills in the Aleutian Islands Omenn, G. S. 2006. Presidential Address: Grand Challenges and Great Oppor- tunities in Science, Technology, and Public Policy. Science, Vol. 314, Dec. 15, pp. 1696–1704. Presidential/Congressional Commission on Risk Assessment and Risk Management. 1997. Framework for Environmental Health Risk Management. Final Report, Vol. 1. Washington, D.C. PWS Steering Committee. 1996. Prince William Sound, Alaska, Risk Assessment Study Final Report. DNV, George Washington University, Rensselaer Polytechnic Institute, and Le Moyne College, Dec. 15. Ritchie, L., and D. Gill. 2006. The Selendang Ayu Oil Spill: A Study of the Renewable Resource Community of Dutch Harbor/Unalaska. Quick Response Report 181. Natural Hazards Center, University of Colorado, Boulder. USNRC. 1981. Fault Tree Handbook. NUREG-0492. Washington, D.C.