Hazardous Materials Transportation Risk Assessment: State of the Practice (2013)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

7 S e c t i o n 3 Current applications of risk assessments and the tools available for conducting them were explored through the literature review, organizational interview process, and an on-line survey. A summary of the results of these efforts is presented in the following section. Section 3.1 presents a summary of relevant hazmat trans- portation risk analysis literature. Additional details of the hazmat transportation risk analysis activities and tools from individual articles are provided in Appendix A. Sections 3.2 – 3.7 report information provided by inter- view subjects and survey respondents arranged into sections according to organization type (e.g., carriers, shippers, fed- eral agencies, etc.) and are listed according to the respond- ing organization (as opposed to the developer of the model or approach being used). Many of the contacted organiza- tions (see Appendix B for a complete list) were involved in hazardous materials-related research activities, but had no direct involvement in developing, enhancing, or using hazmat transportation risk assessments. As such, only infor- mation from those organizations that provided substantive comments about hazmat transportation risk assessment is presented. Included is a summary of the risk assessment prac- tices and capabilities of these organizations structured under the following headings: • Current uses, users, modes, and decision making • Models, tools, methodologies, approaches • Key sources of data • Assumptions, limitations, biases, and availability • Updates • Risk communication • Desired improvements • Implementation barriers Risk assessment approaches discussed in this section involving concrete models or methodologies with sufficient documentation or available information are further charac- terized in Section 4 to facilitate selection of an approach for given decisions, data constraints, and desired output types. 3.1 General Literature Results The literature review formed the basis of the project’s Technical Memorandum and can be found in its entirety in Appendix A. Following the Literature Review Methodol- ogy, the documents were reviewed and sorted into four main categories according to theme or area of impact on hazard- ous materials transportation risk assessment: new modeling techniques and approaches, data-driven risk assessment, use of risk analysis and route choice, and economic risk analy- sis. Though some documents could have been included in multiple categories, they were listed in the most applicable section to avoid duplication. 3.1.1 New Modeling Techniques and Approaches The first category, New Modeling Techniques and Approaches, contains a total of 15 sources, seven of which are reviewed in detail in Appendix A. Documents contained in this section deal with high-level approaches to risk assessment and modeling. The literature discusses a variety of initial or govern- ing approaches to risk. In the case of Trépanier et al., the paper discusses data deficiencies through a comparison of accident databases that are collected and maintained by various agen- cies within Canada and found that only 41 true matches existed between two databases. Those matches accounted for 28.1% of the total reported accidents in one database and 2.9% in the other, which lead to the authors concluding that accidents are being under-reported. Another study, by Ghazel, builds a complex behavioral model to understand the causes of acci- dents between vehicular traffic and trains. The complex model was created using two simpler models that focused on trains’ relationship to signals and signals’ relationship with vehicular Current Uses

8traffic. An article by Gheorghea investigates release-incident data and groups accidents into common themes based on the accident’s cause. The authors conclude that identifying com- mon themes allows for improved risk assessments and future planning. Additionally, the National Research Council’s (NRC) report on Department of Homeland Security (DHS) risk assessments was reviewed. The report suggests that the DHS’s assessments need to incorporate third-party peer reviews, extend to all hazards, and reduce the subjectivity of the assess- ments. Additionally, the NRC report found that DHS assess- ments heavily emphasized quantitative analysis, which may not be appropriate when dealing with adaptive adversaries—those that make adjustments in their strategies as security counter- measures are deployed—resulting in reduced effectiveness of those countermeasures over time. 3.1.2 Data-Driven Risk Assessment Data-Driven Risk Assessment is the second category used to categorize documents, and contains five resources, three of which are summarized in depth in Appendix A. The article by Romano and Romano reviewed under this section focuses on quantifying risk elements in terms of the population affected based on the material being transported, the conditional release probability of all materials, and population data for surrounding areas. Akin to that article is Glickman’s investi- gation of the conditional release and accident probabilities of the use of two different containers, both over two routes, and with the average case and worst case scenarios. The article by Clark and Beserfield-Sacre discusses a methodology that was developed to identify the cause of loading/unloading inci- dents and found the “causing object” variable (as opposed to failure mode, contributing actions, etc.) in such events to be the most important contributing factor. Additional source documents included information pertaining to techniques for gathering vehicle crash data. 3.1.3 The Use of Risk Analysis and Route Choice The third section, The Use of Risk Analysis and Route Choice, is comprised of literature that specifically discusses the use of risk to inform route decision making. Unlike the first two categories, this section is focused on only one out- come of risk assessment. In most cases, the routing of haz- ardous materials is the key decision being made with regard to transporting those materials, so it was expected that a number of documents would deal with this issue. The seven articles summarized include information regarding trans- portation through tunnels, rural highways, routes with lower populations but fewer response resources, and the Boston metropolis; the four additional resources described offer more insight into regulations and risks surrounding trans- port through population centers and tunnels. The first article in the section deals with modeling population-based routing decisions using data on population, population density and accident/exposure-mitigation practices. The authors theorize that population-minimizing decisions are not always optimal as population centers are more likely to have the resources to mitigate accident consequences. Philippe Cassini authored a paper describing a model which used F/N curves to com- pare truck routing choices of going through an urban area or using a detour that involved tunnels. An article by Bubbioco et al. finds that compared to open-air routes, tunnels reduce the societal risk for rail hazmat transport but increase the risk involved with hazmat trucking. Additional articles describe how toll policy can be used to change the routing behavior of hazmat transportation or discuss the route-decision princi- ples promoted by professional associations or municipalities. 3.1.4 Economic Risk Analysis Finally, the Economic Risk Analysis category contains two summarized articles and two sources of additional informa- tion. The literature in this section focuses on calculating and financially quantifying the various components in a risk assess- ment, particularly with regard to consequence elements. Work by Wijnen et al. discusses the Value of a Statistical Life (VoSL) that is used to quantify human casualties in monetary terms. This conversion allows for human casualties to be added or included in the consequence calculations with economic costs. An article by Verma, meanwhile, discusses ways to measure the economic costs and benefits of decision making. By low- ering the costs, or consequences, of transporting a hazardous material, industry stakeholders may also lower the overall risk. The articles and information in this section can assist indus- try stakeholders to quantify the consequence component to a common unit, which will allow decision makers a better under- standing of risk between certain scenarios, including routing, packaging, modes, and mitigation strategies. As a whole, the literature review established a baseline understanding of the key issues and assessment methodolo- gies within the hazardous material transportation industry. This initial research helped to inform the remainder of the data gathering process for this project, including stakeholder interviews and surveys. 3.2 Carriers 3.2.1 Association of American Railroads (AAR) Current uses, users, modes, and decision making. The Railway Supply Institute (RSI), the RRF, and the AAR all per- form research and analyses that support hazmat transpor-

9 Assumptions, limitations, biases, and availability. As with many data-driven approaches, the RCRMS methodol- ogy assumes that certain data derived at the national level are appropriate to use at a more localized level. Where there are significant deviations from those representative values at the local level, the analysis would not fully account for them. The methodology has been annually reviewed, as discussed below, to determine if new data or approaches can be adopted to improve the model. Addressing uncertainty. A key concept is that the desired data are often missing, but users try to determine whether they have a representative sample that would support a mean- ingful analysis. It is often the case that a representative sample is available and users apply statistical analyses to quantify the biases or uncertainties for the decision makers. Of course, communicating the results to decision makers is often more difficult when the analyses are more engineering-focused. For the RCRMS, uncertainty in the input data is addressed by grouping output route alternatives into broad categories of relative attractiveness, rather than having the user directly compare numerical output. This approach ensures that users do not make routing decisions based on seemingly high- precision output values when such confidence is not war- ranted by the quality of the input variables. Updates. Currently, the railroads go through an annual review process for all of their high-hazard materials specifi- cally listed in 49 CFR 172.800, which includes TIH, explo- sives, and radioactive materials. The regulation requires all movements of these materials that occurred in one year to be reevaluated for the following year. Desired improvements. One area of current work that is progressing slowly relates to estimating the resulting impacts from specific changes in train and track operations or main- tenance, such as changing track class or speed. RSI believes there is a need for cross-modal research to ascertain the poten- tial consequences from spills, such as evacuations. A related desired improvement is understanding how dispersing clouds interact with the environment. 3.2.2 Class I Railroad Current uses, users, modes, and decision making. A large Class I railroad conducts route risk assessments for its hazmat shipments as directed by federal regulations. Hazmat routing is largely dictated by the TSA and positive train control (PTC) requirements and TSA audits. As a result of these regulatory constraints, routing decisions have become more politically and economically driven than risk based. The railroad regards information about hazmat risk assessments as sensitive secu- rity information (SSI) and therefore cannot be disclosed. tation risk assessment. Their work is primarily intended to support the railroads that comprise their membership. Uses include supporting regulatory requirements for routing and determining the appropriate strategies to reduce the likeli- hood of a release when accidents occur (mostly related to rail car design and safety features). Their focus is on continuously improving the data and models that support their decisions. Models, tools, methodologies, approaches. AAR, through the RRF, supports the development and operation of the RCRMS. This tool has grown nationwide to meet the fed- eral regulatory requirements of 49 CFR 172.800–Additional Planning Requirements for Transportation by Rail.6 RCRMS allows rail operators, as part of their route analysis (including assessment of route alternatives), to consider the 27 required criteria, including network infrastructure characteristics, railroad operating characteristics, human factors, and envi- ronmental and terrorist-related parameters. The regulatory requirements apply to high-hazard materials, which include toxic inhalation materials (TIH), explosives, and radioactive materials. Accident rates are incorporated for both main- line track and rail yard activity. Both safety and security risks are considered and consequences include potential human health, critical infrastructure, and environmental impacts. Safety and security risks are presented separately, but also combined into a single risk metric. Key sources of data. The RSI/AAR Safety Research and Test Project utilizes 28 different data sources, as no one entity has reliable data on all aspects of design and features of spe- cific tank cars, how those cars perform in accidents (e.g., did they leak, was there damage and, if so, where was it and how much damage was there?), and specifics of the accidents (e.g., speed, number of cars derailed—which is a proxy for sever- ity, ambient temperature, and so on). Engineers interpret the data from these sources to integrate them into a complete picture. Where data are still not available, the RSI/AAR typi- cally goes back to the engineers’ interpretation and expert knowledge to fill the gaps. Much of the data used in the RCRMS come from the indi- vidual railroads themselves. This includes commodity types and volumes, as well as some specific track characteristics (e.g., track class, grade, and defect detectors). Some data are more generic, such as Federal Railroad Administration (FRA) accident data. Other data, such as the inputs into the conse- quence estimates, are obtained from public sources, includ- ing the Federal Emergency Management Agency (FEMA). 6 Commonly referred to by the PHMSA rulemaking docket: HM-232E – Enhancing Rail Transportation Safety and Security of Hazardous Materials Shipments.

10 most detailed outcome when there is significant uncertainty in the other parameters. This company has examined some environmental, critical infrastructure/key resource, and economic consequences, but most of its formal distribution risk assessments have focused on human impact. For materials that may pose significant impact to the environment if spilled, but with less severe human health hazards, they may use the same risk review methodology to make decisions based on environmental consequences, exploring packaging and mode alternatives. Key sources of data. For rail frequency data, this com- pany currently uses data from the FRA website and current rel- evant publications.8 For highway accident rates, they attempt to obtain these from their actual carriers since there can be considerable diversity in performance between trucking seg- ments and individual companies. They separate accident fre- quency and conditional release probability into two elements to consider areas of influence within each. Rail probability data are derived from RSI/AAR work, and they support ongoing industry efforts to correlate conditional release probabilities to puncture resistance. For highway probabilities—and until the HM-07 project results are implemented—they may utilize data from a variety of sources, including data on pressurized gases and propane road tankers. However, they avoid use of data from studies where adequate delineation by tank design is lacking. Where the level of the hazard is sufficiently high to war- rant it, they may use proxies for some data. For example, as detailed road cargo tank and portable tank data are difficult to acquire, rail tank car data may be modified as appropriate where the thicknesses might be somewhat comparable but the forces applied to the tanks would be different. Assumptions, limitations, biases, and availability. Many transportation risk assessments are constrained by a lack of reliable information, so conservative assumptions are used. However, overestimating risk can potentially lead to putting significant resources toward ineffective or unwarranted miti- gation strategies. Transportation risk assessments involve assumptions around essentially every factor (frequency and consequence). Prediction of when and where incidents will occur, and the conditions at the time of the incident (weather, speed, obstacles, population present, etc.) means that even select- ing scenarios involves numerous assumptions. Fault tree assessments for transportation can become too large and unwieldy; developing a comprehensive tree and then assign- Risk communication. Risk analysis results are commu- nicated externally only through federal audits. Implementation barriers. Federal regulations constrain- ing route choice and potential methods of hazmat route risk analysis are a barrier to alternative and more in-depth assess- ments, as well as to disclosing any information about further risk assessment activities. 3.3 Shippers 3.3.1 Large Chemical Manufacturer #1 Current uses, users, modes, and decision making. This company applies their risk assessment approach to mode and route choice, packaging selection, application of security measures, manufacturing locations, alternate product selec- tion, operational changes, and emergency response resource planning. Their risk assessments might influence carrier selection if there were resulting requirements, such as tandem drivers, to implement risk reduction measures. Models, tools, methodologies, approaches. This com- pany follows an approach similar to that described in Guide- lines for Chemical Transportation Safety, Security, and Risk Management.7 Foundational safety and security risk man- agement practices (such as compliance with regulations and strong company internal standards as well as root cause inves- tigation and corrective/preventative actions to incidents) are built upon and enhanced with increasing levels of risk. Formal risk reviews are required for certain material/mode combinations based on a screening process that considers the hazards of the chemical shipped, the number of shipments, the container size, and the estimated consequence of a release. Chemicals of most concern include toxic inhalation hazard materials, flammable gases, and packing group I materials. The risk review process is largely qualitative/semi- quantitative, and includes both safety and security elements. Full quantitative risk analyses are rarely performed due to data limitations, but may be conducted on very specific issues in order to assess the impact of certain risk mitigation options in more detail. This company generally does not estimate fatalities in its assessments, but rather uses consequence analysis to under- stand the potential impact of different release sizes to an order of magnitude. Given the uncertainty as well as lack of specificity in certain critical data elements, they believe cau- tion should be used to avoid evaluating one parameter to the 7 Center for Chemical Process Safety (CCPS) of the American Insti- tute of Chemical Engineers and John Wiley & Sons, Inc., Hoboken, New Jersey. 2008. 8 Anderson, R. and C. P. L. Barkan 2004. Railroad Accident rates for use in rail transportation risk analysis. Transportation Research Record: Journal of the Transportation Research Board, No. 1863: 88-98.

11 to every five years. Additional reviews and special studies are performed as necessary, for example, with significant changes in the supply chain (new geographical market, new package, etc.). Desired improvements The most pressing need is for improved data—more accurate and more detailed—for all aspects of the risk equation (accident frequency, conditional release probability, and consequence). Without better data, it is important to rely more on screening level analyses and not to attempt estimation of very detailed frequencies or conse- quences, such as fatalities. Better data to support bulk truck risk assessment would be the most valuable. These data would be helpful not only in informing modal decisions, but in evaluation of design improvements for road tankers and portable tanks. 3.3.2 Large Chemical Manufacturer #2 Current uses, users, modes, and decision making. This company conducts committee-based hazmat transportation risk assessments in order to reduce the risk of transporting given chemicals by selecting shipment parameters. These assessments consider transport by barge, rail, and truck, and are used to review shipment variables such as methods of shipment, potential hazards and threats, chains of responsi- bility, and operational guidelines for carrier operators. Analy- sis results are used primarily to inform shipping decisions by senior managers. Models, tools, methodologies, approaches. This com- pany approaches hazmat transportation risk assessments through a committee review process adhering to inter- nally developed standards and procedures. An assessment is carried out for each shipped chemical, with an assigned champion to oversee the assessment process. Each review committee is composed of a variety of company personnel, including representatives from operations, transportation, and internal emergency responder teams, as well as external shipment carriers. These committees review and recommend shipment parameters, such as timing, route path, container, employed technologies, etc. While shipment modes are gen- erally predefined for existing chemicals, switching modes for a given commodity may be considered for increasing safety or security. Assessments consider potential consequences to human populations and the environment. Considering potential economic impacts to the company is specifically excluded from the assessment process. This company’s method of analysis considers both safety and security. While safety analyses are primarily quantitative, security analysis is largely qualitative. Qualitative security ing an appropriate probability on each node in the tree can range from challenging to impossible. For fixed facilities, where assets generally do not move around, failure scenarios are more limited and failure data are generally more available. This company uses an approach for security risk assess- ments that considers threat, vulnerability, and consequence, but recognizes that using threat and vulnerability as a proxy for frequency has limitations. As risk assessment is designed to be forward looking, current threat data may be not appli- cable in the future, and one needs to have effective mitigation strategies against current threats. This company accomplishes that by using a variety of information sources to assess cur- rent threats, and implementing a tiered program requiring added levels of security measures for certain materials as the threat level increases. The company also participates in pro- grams such as C-TPAT, PIP, and AEO to institute strong secu- rity practices across its supply chain. Some additional areas of concern mentioned by this com- pany include the need for a holistic view of transportation risk. For example, transportation risk assessments may not always consider the loading and unloading operations, but this can be important in making mode selections and consid- ering risk mitigation options within a given mode, whether in transportation or on site. The selection of scenarios can also create bias. Not all sce- narios that can occur are included in most risk assessments. Additionally, only some elements of a scenario can be quanti- fied. Sometimes the elements that cannot be quantified can significantly impact the risk. It is important to understand the “big picture” (past just specific scenarios) to be aware of the important contributors to risk and mitigation, so that inappropriate conclusions are not drawn. Addressing uncertainty. Consideration of uncertainty is extremely important. For example, if error bounds are not placed on F-N curves,9 it may be difficult to discern that two options, while appearing different, are really statistically the same. One method used by this company is to evaluate results using high and low estimates for key variables to understand the range of results. Even assuming good data, and whether using qualitative/ semi-quantitative or full quantitative analyses, it can be dif- ficult to answer the subsequent question about whether the risk (either before or after mitigation) is acceptable. Updates. This company conducts risk reviews on a cycle that depends on the hazard and varies from every two 9 F-N curves are graphs in which the Y axis represents the cumulative frequency of fatalities (or other consequences) and the x axis represents the consequences. The X axis often uses a logarithmic scale.

12 3.3.3 Institute of Makers of Explosives Current uses, users, modes, and decision making. Rep- resentatives from the IME and constituent member organiza- tions indicated that there is no industry-standard methodology for route risk assessment and that such assessments are left to each individual carrier. Carriers generally follow routes pre- scribed by the DOT, the DHS, and the DOE, making route risk assessment less of an operational and financial priority. As a result, hazmat transportation risk assessment for IME constituent members focuses primarily on explosives risks at transportation-related facilities, such as safe havens, ports, and industrial origins and destinations. Users of the IME risk analysis model and software include senior managers of IME’s constituent member organiza- tions in all modes of transportation, as well as regulatory and enforcement officials, such as the U.S. Coast Guard, U.S. Bureau of Alcohol, Tobacco, Firearms, and Explosives (ATF), and several Canadian governmental authorities. Models, tools, methodologies, approaches. IMESAFR is an industry standard tool used to assess explosives risks at facilities. The model was developed by the DOD for mili- tary use [Safety Assessment for Explosives Risk (SAFER)], and IME has translated it into a software tool for more gen- eral use. IMESAFR can be applied to hazmat transportation through its use in assessing current or potential “safe haven” locations, calculating allowable limits of materials at trans- portation facilities, and through its evaluation of loading and unloading activities at industrial facilities and ports. The model was developed with a focus on safety risks and, as such, employs the general risk equation: Risk = Frequency × Probability × Consequence. In order to account for security risks, the Frequency term in this equation can be multiplied by a predetermined factor according to the federally defined threat level (e.g., Red = ×10). Data considered by the IMESAFR software include activ- ity types (e.g., loading, unloading, repackaging, short-term storage, etc.), type and quantity of explosives, building construction information, exposed populations, and event frequencies. Consequences metrics include human fatali- ties, major and minor injuries, and a group-risk metric that describes the potential for complete fatalities to an entire given population. Key Sources of Data. Frequency data are based on a DOD/IME database of historical events and nonevents. Con- sequence data are based on a controlled-testing database of explosion tests conducted by DOD/IME. All of the data nec- essary for performing model calculations are stored within the software and are accessible to users through a series of dropdown menus. analysis takes the form of reviewing management procedures for given scenarios. Some qualitative security aspects are incorporated as well, however, including developing trip pro- files with strictly defined locations along a route at which a driver may stop. Examples of data considered by review committees in risk assessments include internal toxicological profiles, con- tainer characteristics, previous incidents and lessons learned, emergency response capabilities, publicly available response guidelines, locations of population centers, and locations of water, among others. Key sources of data. Data used in risk assessments are sourced primarily from internal company databases and, to a lesser extent, from carrier operators. Assumptions, biases, limitations, and data availability. This company characterizes its risk assessments as being overly conservative. Common assumptions in the assessment process include the automatic selection of a company-owned trucking firm for extremely hazardous materials and that, in general, shipping by barge is safer than by train, which is, in turn, safer than by truck. The company generally has access to the data necessary for performing risk assessments through its internally main- tained databases. When it becomes necessary for them to collect data from outside sources, the lack of openness and information sharing throughout the chemical industry can present an obstacle. Addressing uncertainty. Missing and insufficient data is noted in assessment committees’ final reports to business managers. In some cases, noting the use of flawed or incom- plete data is sufficient, while in other cases managers may call on assessment committees to collect or develop the data and incorporate them into an updated report. Updates. Risk assessments for each chemical are sched- uled for update every five years. Risk communication. Risk assessment results are com- municated to senior managers through formal reports and are shared with and agreed upon by carrier operators. Their risk analyses are not published for the general public. Desired improvements. While hazmat containers have been well characterized with regard for failures and inci- dents, similar analysis is lacking for the hoses used to transfer chemicals to and from these containers. The development of hose failure and incident data for incorporation into the risk assessment process would help this company make better- informed decisions regarding the transfer of chemicals to and from carrier containers.

13 IMESAFR could be made a more useful tool for hazmat transportation analysis by incorporating in-motion risk assess- ment capabilities that include transport specific elements, such as the potential for highway accidents. Implementation barriers. Barriers to the wider use of the IMESAFR model include the general lack of understand- ing of quantitative risk assessment by potential users, a lack of guidance on how to use model output values, potential political backlash from explicitly quantifying risks, a lack of focus and funding on the part of shippers, and the dearth of regulatory incentives. 3.3.4 Large Chemical/Plastics Manufacturer Current uses, users, modes, and decision making. This organization responded to the online survey and evaluates the risk to human health and the environment from an acci- dent or chemical release during loading and transporting their products. Models, tools, methodologies, approaches. The risks posed by loading and unloading operations are evaluated within a materials-of-concern process involving operations; engineering; maintenance; and environmental, health, and safety. En-route transportation risk is considered by envi- ronmental, health, and safety professionals with experience in hazardous materials transportation. This evaluation is done globally, at a high level from headquarters, as well as at the plant level on a chemical- and route-specific basis. Most of its evaluations are qualitative with a few semi- quantitative assessments for specific transportation activities. The main risk assessment inputs are chemical toxicity and other properties of hazardous materials, route infor- mation, travel times and trip duration, probability, and consequence. Risk communication. Risk assessment results are usu- ally only communicated to immediate project stakeholders. Not much information is shared internally and is an area for improvement. Desired improvements. For its purposes, an easy-to- access database of route-specific information would be very useful. Implementation barriers. The biggest constraints to more accurate and repeatable results are awareness of and access to relevant data concerning route information. The availability of data overseas is particularly lacking. Another constraint is the availability of qualified risk assessment professionals. Assumptions, biases, limitations, and data availability. In order to overcome limitations in available frequency data for commercial activities, IMESAFR assumes that military event frequencies are a suitable proxy for commercial event frequencies. Of the 17 activities that IMESAFR considers, only three or four have enough associated frequency infor- mation from commercial applications to provide a suitable level of confidence in the data. Comparisons of these frequen- cies to military use data have shown that event frequencies for commercial and military applications tend to be similar. Consequently, frequencies associated with military activities are used in place of commercial data throughout the model. A scarcity of explosion data has led to IMESAFR being designed as a strongly conservative model. The explosion database includes only a small number of tests (n × 30), and these tests generally represent the explosions of quantities of material much larger than most shippers would typically have on hand. To address the lack of data for small-quantity explosions, these explosions are assumed to behave similarly to large-quantity explosions. Limits to the use of IMESAFR for hazmat transportation risk analysis include the lack of in-motion transport analysis and the lack of reliable frequency data for use beyond Canada and the United States. Addressing uncertainty. Uncertainty in event prob- ability data is handled by incorporating its upper and lower bounds into the model and returning results for the range of the input data. Additionally, IMESAFR addresses uncertainty by tending toward strongly conservative estimates of model parameters. One such example is the use of large quantity explosion data for modeling small quantity events. A simi- larly conservative tactic involves calculating fatalities at the intra-facility level to be 100%, despite historical evidence indicating fatalities at this level tend to be more along the lines of 10 to 20%. Updates. Risk assessments are generally carried out on an as needed basis by IME member organizations and associ- ated agencies. Risk communication. Model results are typically used internally by IME constituent members and are rarely pub- lished for public review. Desired improvements. The most meaningful improve- ments to the current IMESAFR approach involve generating or collecting better model input data. Data needs include additional testing of explosives, with a focus on smaller quan- tities of explosives, and improved event probability data, with consideration given to locations outside of the United States and Canada.

14 3.4.1.2 Savannah River Site Current uses, users, modes, and decision making. The DOE has developed and uses several complimentary risk- related tools that inform everything from packaging and securing radioactive materials for transportation, to select- ing routing methods and itineraries focused on decisions to promote regulatory compliance when transporting haz- ardous materials between sites. The tools range from mode (road, rail, and water) and route selection to ensuring that the shipped materials are properly marked. The DOE ‘risk’ tools—RADCALC, TRAGIS, and RADTRAN—were mainly developed for internal use in transporting radio- active materials between points. RADCALC is not a risk assess- ment tool. RADCALC was designed as a pre-transport safety compliance/placarding tool, TRAGIS (Transportation Rout- ing Analysis Geographic Information System) as a mode/ route selection tool, and RADTRAN is fed by TRAGIS data and used to determine the risk of exposure of large campaigns (numerous shipments of the same material). Models, tools, methodologies, approaches. RADCAT is the input file generator for the RADTRAN program and code. RADTRAN calculates risks of transporting radioactive mate- rials for both routine/incident-free transport, as well as for the risks of transportation accidents. RADTRAN (versions I and II) was designed for the 1977 EIS FOR THE TRANSPOR- TATION OF RADIOACTIVE MATERIALS (NUREG 0170) and was thus initially supported by NRC. RADTRAN (cur- rently Version 6) is now supported by DOE/EM. RADTRAN, bundled with its input file generator RADCAT, is available for download from https://radtran.sandia.gov/radcat. RADCALC is an independent tool that helps packagers and shippers determine the classification of radioactive mate- rials for shipping from the radionuclide shipment inventory. DOE sites, such as Savannah River Site (SRS), use the RADCALC tool to comply with DOT placarding and safety regulations pertaining to transport of hazardous materials. TRAGIS minimizes travel time over preferred routes and minimizes distance over nonpreferred routes. RADTRAN runs risk analysis over a route (can be TRAGIS recommended routes) based on consequence/exposure data. Key sources of data. The inputs to the RADCALC tool are: shipped hazardous material’s radionuclide informa- tion, the amount of material being shipped, and information about the shipment container. SRS augments RADCALC with information about the frequency of shipments and the route’s length and terrain. In addition to the radioactive characteristics of the ship- ment, TRAGIS uses census data over the potential route (population density, day/night population) to calculate the 3.4 Federal Agencies 3.4.1 Department of Energy 3.4.1.1 National Nuclear Security Administration Current uses, users, modes, and decision making. The DOE’s National Nuclear Security Administration (NNSA) serves a lead role in supporting the Federal Radiological Monitoring and Assessment Center (FRMAC), which is an inter-agency group charged with conducting the assessment of any radiological release in the United States. It is also called upon to assist in assessing radiological releases elsewhere, such as in Japan recently. The FRMAC assessment tool is, however, not a true ‘risk assessment’ tool but serves only to characterize the release. The outputs from FRMAC could be used to conduct elements of a quantitative risk assessment, but that is not the goal of the interagency group. The mission of the FRMAC is to coordinate and manage all federal radio- logical environmental monitoring and assessment activities during a nuclear or radiological incident, within the United States, in support of state, local, tribal governments, DHS, and the federal coordinating agency. Models, tools, methodologies, approaches. The FRMAC assessment manual is a tool used by DOE to perform calcu- lations centered on the assessment of radiological incidents and releases. The tool operates independently of the etiology of the event that produced the release or potential release, so it does not consider the specific events, whether natural or man-made, that might lead to a release. The assessment results are then presented to an inter-agency team which con- siders the exposure and other consequences of such release in light of their particular programs. The team’s members are: Food and Drug Administration (FDA), Environmental Protection Agency (EPA), U.S. Department of Agriculture (USDA), Nuclear Regulatory Commission (NRC), DOE’s National Nuclear Security Administration (NNSA), and the Centers for Disease Control (CDC). CDC convenes the inter- agency group as needed. The assessment produces results in five areas: (1) Plume Phase Evaluations, (2) Population Protection, (3) Emergency Worker Protection, (4) Ingestion Pathway Analysis, and (5) Sample Management. Key sources of data. For FRMAC, the tool uses health physics data along with embedded analytic assumptions about radiation exposure effects. Implementation barriers. As the DOE is part of the federal government, the department’s budget and division’s share of the budget are the key constraints to any improve- ments or increased use of its tools.

15 3.4.1.3 Oak Ridge National Laboratory Center for Transportation Analysis Current uses, users, modes, and decision making. The Oak Ridge National Laboratory Center for Transportation Analysis (CTA) supports a wide range of hazmat transporta- tion risk analyses in support of governmental and private- sector applications. CTA tools for hazmat transportation risk analysis address risks related to barge, rail, highway, pipeline, and transportation facilities. Output from these tools inform routing, security planning and countermeasure application, and emergency response and planning. Models, tools, methodologies, approaches. TRACC is a web-based tool developed by CTA and Mississippi State Uni- versity for tracking the location of barges carrying dangerous cargo and identifying high-risk transport situations. This sys- tem tracks hazmat barges using a Global Positioning System (GPS) to monitor their locations more consistently than the current, widely used methods, which include operators periodi- cally reporting their locations by fax or phone. TRACC uses this GPS information to identify safety and security risks by detect- ing anomalous stops or movements and predicting hazardous conditions, such as multiple barges passing in a high-traffic area or the buildup of incompatible cargo at a given location. GeoCTA is a geographic information system (GIS)-based analysis tool designed to facilitate planning for and responding to natural disasters and terrorist activity. GeoCTA specifically focuses on transportation and other critical infrastructure within high threat urban areas (HTUAs) and is described as a useful tool for preparing for and responding to hazmat shipment spills. For each HTUA, GeoCTA provides detailed, up-to-date digital maps; offers location data, information for mapped entities, and emergency contacts for critical infra- structure (including hazmat facilities); calculates potentially exposed populations and population risk indices; and pro- vides 3-D visual navigation tools for reviewing geospatial data. The Readiness and Resilience Assessment System (RRAS) is a tool developed for the Transportation Security Network Management office (TSNM) of the TSA. This system is used to gauge the ability of transportation facilities, systems (e.g., highway, rail, and pipeline networks), services, and security to withstand and recover from terrorist attacks, including those involving chemical, biological, and radiological hazmat. Key sources of data. TRACC utilizes current and histori- cal route and commodity data provided by carriers as well as spatial information gathered using GPS-enabled tracking devices. GeoCTA employs population statistics, HTUA, sensi- tive location, and critical infrastructure spatial data sourced from government-maintained databases. RRAS uses data describing security measures, technology, personnel, train- ing, etc., gathered locally for each assessment. preferred route. The preferred transportation mode can be determined before TRAGIS (as a constraint) or afterwards (as a result of the recommended route). RADTRAN supplements TRAGIS data with historical/ incident probability data and dispersal methodologies along with potential factors such as presence of fire or elevation of release. Furthermore, since RADTRAN is used for ‘cam- paigns’ which potentially increase the overall exposure along the selected route, the tool primarily looks at chronic and acute consequences of human exposure. Finally, RADTRAN allows for changes in route conditions like the addition of traffic delays, fuel stops, or construction. Assumptions, limitations, and biases. All three of the DOE tools are limited by not including nonlethal human health effects, economic (such as business interruptions and indirect economic effects) or environmental consequences into the model. Furthermore, the tools calculate risk based on the maximum exposed individual, which leads to a higher estimated number of Latent Cancer Fatalities than are likely to occur. Also, RADTRAN relies heavily on user-input data, which can affect the results. Addressing uncertainty. RADTRAN addresses uncer- tainty by allowing the user to change various parameters, including meteorological conditions, traffic patterns along the route, preferred mode of transportation, and exposed population. While this approach is essentially a method of sensitivity analysis and does not help in estimating absolute uncertainty, it does provide a better understanding of the importance of the parameters being applied to the decision that the user is making. Availability and updates. Besides data updates that occur when data is re-released by third parties, the tools have not undergone major changes in the past five years. In 2006, DOE successfully attempted to adapt the tools for use by the international community. Risk communication. As part of a large campaign where DOE uses RADTRAN, DOE describes the details of the cam- paign, including the potential risk, to emergency response/ emergency management teams along the proposed route. Desired improvements. A DOE representative stated the desire to have RADTRAN and TRAGIS validated, and possibly improved upon beyond solely updating data-sets, by security and safety experts. Implementation barriers. As the DOE is part of the federal government, the department’s budget and division’s share of the budget are the key constraints to any improve- ments or increased use of its tools.

16 The FAA is also moving toward using a risk ranking approach to assess shippers in the Hazmat Intelligence Portal (HIP) using thirty values and identifying those that might warrant some inspection or further investigation. Included factors are expected to include past inspection history, violations, serious incidents, and the materials they ship. Key sources of data. The FAA has utilized a Volpe report on specific materials that could, under the right circum- stances, cause an aircraft accident. The FAA Tech Center conducts engineering studies and experimental research to determine whether specific materials or items are too risky to allow onboard or to establish limits for them. The FAA uses Hazardous Materials Information System (HMIS) and FAA data mining where possible. They utilize contracted work but also obtain useful data from the industry regarding specific regulations in their attempt to influence a regulatory change. The FAA reaches out to packaging experts for very specific issues and expertise, but these outreach effort are typically not systematic. Assumptions, limitations, biases, and availability. One of the significant issues in hazmat aviation transportation is the lack of data on the amount of regulated hazmat that moves through the system on FedEx and other carriers. Much of the prior research did not address very important issues, such as the differentiation between passenger and cargo aircraft or bulk vs. small overnight packages. The specific nature of the materials was also not known or reported in that research, for example, whether the material was paint or another, more hazardous, flammable liquid. Overall, the FAA does not focus on a specific formula or equation for their risk-related assessments. Addressing uncertainty. The FAA does not compute a risk number or score and does not address uncertainty in its analyses. Updates. Updates to assessments or supporting analyses are done on an issue specific basis. Risk communication. When initiating a new analysis, the FAA begins by investigating existing related research. Depend- ing on the specifics analyses, the FAA Tech Center then reaches out to similar businesses to those involved. The FAA headquar- ters staff would contact other packaging experts for detailed needs. For more generic information, they would reach out to industry groups, such as the Dangerous Goods Panel. Desired improvements and implementation barriers. Obtaining better numbers to use (including denominator data) were ranked high on the FAA list of desired improve- ments. They desire to include all of the risk components into Addressing uncertainty. The RRAS model addresses uncertainty quantitatively through the inclusion of confidence measurement values for its vulnerability, emergency response capabilities, and organizational awareness components. Updates. TRACC analyses are performed continuously in real time. Standard update intervals for GeoCTA and RRAS analyses are not defined. Risk communication. TRACC output is reported directly to carriers, government agencies, and responders as necessary. 3.4.2 Department of Transportation 3.4.2.1 Federal Aviation Administration (FAA) Current uses, users, modes, and decision making. The FAA risk assessment focus is on ranking the potential con- sequences from the failure of some component of the trans- portation system or package. They are mostly concerned with high-consequence events and identifying appropriate risk mitigation strategies to reduce their probability of occur- rence. The FAA would like to use the risk assessment process to manage overall hazardous materials transportation risk. A critical issue for the FAA now is the overall scale of industry operations. A system that worked well in the past for 6 million packages might not be scalable to the current level of 12 mil- lion packages. With undeclared shipments being a significant concern, the FAA wants to understand the risk of these ship- ments and the appropriate requirements for carriers when they are given an undeclared package. In exploring the potential for risk mitigation strategies, FAA focuses on opportunities to control potential events so that they become noncatastrophic. This includes fire suppression, emergency response, instructions to the pilot, etc. The FAA does look at differences with the other modes in trying to understand risk. There are many differences and the key similarities are marking and labeling. Models, tools, methodologies, approaches. As prob- ability data are difficult to obtain, the FAA tries to find pre- cursors and incidents/accidents with similar characteristics to the issue at hand. They are trying to develop ways to pre- dict the probability and determine large-scale trends, such as whether the probability will change over the next decade. A good example relates to accident probabilities involving lith- ium batteries, which have quickly been getting more power- ful, yet retaining the same package sizes. More heat can be generated with these more powerful batteries. Consequence data focus on expected deaths and injuries. Critical infrastructure and the huge economic impacts aris- ing from the loss of large air cargo sorting facilities (e.g., Memphis or Louisville) have not been considered to date.

17 modal comparisons, to at least compare rail to highway trans- portation. Such a comparison would allow them to examine how modal shift affects the overall hazmat transportation risk. Models, tools, methodologies, approaches. Often, FRA employs risk assessment models developed by the industry, such as the TIH work being conducted for FRA by ICF Inter- national. Current work is focusing on one material at a time, as opposed to a comprehensive approach. For rail risk, the likelihood of an accident is not considered to be a hazmat-specific factor. FRA focuses on the factors that affect the probability that the package will be involved in a derailment or a major accident and the probability that it will be damaged or punctured and release the product. FRA is working to reduce the number of assumptions involved to pave the way for better risk assessments. Consequence assessment is generally focused on acute human health and most often measured by potential expo- sure. The risk assessment considers environmental exposure to water and land but does not include potential economic consequences. FRA relies on the industry use of the Rail Corridor Risk Assessment Model (RCRMS) for railroad selection of routes for security-sensitive materials. RCRMS is described in more detail in the AAR discussion in this document. Key sources of data. Data are drawn from the FRA acci- dent database of reportable accidents, from AAR/RSI data on damaged cars (irrespective of the cause), and the HMIS’ 5800.1 incident report form, which is primarily used for non- accident releases. This information draws primarily from rail industry research to get conditional release probability data. Assumptions, limitations, biases, and availability. The key limitations and assumptions pertain to missing data. There is a dearth of information on the types and number of intermodal hazmat shipments. The FRA’s risk assessment assumes that intermodal hazmat shipments are small pack- ages, ignoring larger intermodal bulk containers (IBCs). Additionally, the accident data is missing information on car or package type in many accident reports. As its work is mostly focused on releases, the level of expo- sure for consequences is not being considered in a significant way. The probability of the release is the primary focus and consequences are determined by materials and impact range. They focus on accidents that may result in a large release, even if the probability for such incidents is low. Addressing uncertainty. The FRA has a goal to compute uncertainty in their current risk assessment work on prioritiz- ing its efforts. In the case of missing data on car or package type, FRA uses a waybill sample to apply assumptions to the accident data to determine information about the unknown cars. their analyses. This also represents the largest barrier they face in being able to perform risk assessments. 3.4.2.2 Federal Motor Carrier Safety Administration (FMCSA) Current uses, users, modes, and decision making. In recent years, FMCSA has primarily been an enforcement authority and has not been focused on hazmat transporta- tion risk assessment, which has fallen mostly within the pur- view of the PHMSA which has the regulatory authority. A significant number of the decision-making processes that consider risk assessments are driven by issues conveyed by external parties or through observations from field investi- gators. For example, a recent problem in the field with certain types of packaging triggered a detailed investigation to deter- mine if the problem posed an imminent hazard along with the underlying root cause, so that the appropriate mitigation approach could be determined and applied. Models, tools, methodologies, approaches. The imple- mentation of the Compliance, Safety, Accountability (CSA) program and the Safety Measurement System (SMS) for mea- suring the safety of motor carriers and commercial motor vehicle drivers collectively take the place of a risk assessment tool. Items that are predictors of crashes get elevated empha- sis, creating triggers for moving carriers to a higher level of scrutiny. While the CSA program covers all aspects of motor carrier operation, hazmat considerations are represented currently through the cargo-related Behavior Analysis and Safety Improvement Category (BASIC). Development of a separate, hazmat-specific BASIC would be a risk assessment activity itself. An important aspect of the methodology being developed for the HM BASIC is the determination of perti- nent carriers (carriers transporting placarded loads). Key sources of data. Field data from inspections and enforcement actions provide much of the information used by the FMCSA hazmat division. Where applicable, this information is combined with incident report data from the Federal Motor Carrier Safety Administration (FMCSA) and the PHMSA. 3.4.2.3 Federal Railroad Administration (FRA) Current uses, users, modes, and decision making. FRA uses risk assessment to help identify potential risk reduction strategies, including those that consider route choice, packag- ing selection, application of security measures, operational changes, research prioritization, and inspection and enforce- ment prioritization. Its focus is on research prioritization. Current FRA work focuses exclusively on rail; however, there is potential for exploring other modes in the future. The FRA would like to update a 10-year-old study that examined

18 PHMSA risk assessments are focused on very specific issues, such as the transportation of lithium batteries, particularly by air. Others are general or strategic, such as identifying the com- modities that pose the greatest safety risk or understanding the outcomes that are being observed in industry. For regula- tory evaluations, the focus is on benefit-cost analysis and the security benefits of their safety-based proposed regulations. For special permits and approvals, PHMSA performs analy- ses, but would not characterize them as traditional risk assess- ments. It ensures that proposed approaches for transportation demonstrate an equivalent level of safety and their evaluations rely on information provided by the applicant. Models, tools, methodologies, approaches. PHMSA focuses on the risk to the public from an unintentional release of hazardous materials. Typically, the focus is on a compara- tive risk assessment, with the current regulations establishing a baseline of acceptable risk. If a given design has limitations, for example, PHMSA would consider operational constraints that could adjust for those limitations. In other words, the combination of the package and the parameters of transport- ing it are considered together. For most risk assessments performed by PHMSA, a sep- arate analysis process is used, based on the specifics of the analysis and the available data. For regulatory evaluations, the Office of Management and Budget (OMB) guidelines are used. Their resource allocation is similar to FMCSA’s Com- prehensive Safety Analysis model. In some cases, there are externally available risk assess- ments, such as those provided by industry in response to a proposed rulemaking, but they are rarely used unless there is transparency of the methodology and associated to avoid any additional bias. Key sources of data. Data on prior incidents, inspec- tions, violations, and complaints are used to assess the safety risks for specific companies. Other factors include the types of materials, quantities handled, and the size of the company. For targeting inspections, the HIP is the primary source of information and includes separate incident, inspection, and report data. As for inspections and enforcement activity, shippers are the primary focus for PHMSA and carriers are most directly addressed by the modal administrations. Sometimes, industry data is obtained and is usually sanitized before being deliv- ered to PHMSA. To improve data at the record level, National Response Center (NRC) data are used. Assumptions, limitations, biases, and availability. While it is accepted that data are abundant, some of the key data elements are missing and the remaining are not particularly useful in the context of application. Analyses are limited by a lack of data. A risk evaluation to support a rulemaking is Another approach considers the results of accidents involv- ing a specific type of rail car to infer potential applicability to another material in another type of rail car. Updates. Risk assessment components are currently not updated on a fixed schedule partly due to the ongoing nature of the work as well as dependency on the level of effort required for the updates. Risk communication. A significant portion of the FRA’s risk assessment-driven work is accessible in the public domain. There will be a publicly available report for their current proj- ect. The Tank Car Committee or the Advanced Tank Car Col- laborative Research Program (ATCCRP) will provide updates on their activities and approaches. FRA obtains input and suggestions from those two sources. Desired improvements. FRA would like to increase coop- eration with AAR for obtaining more information. Other data sources with valuable information include tank car build- ers, owners, and the individual railroads. It wishes to obtain more data on hazmat flow in general and also data on rail cars, including availability and prevalence of each rail car type in the rail transportation system. For example, if movement and billing data were available suggesting that 60% of a certain type of rail car carried hazardous materials, it would provide good denominator data for risk calculations. Builders and owners could potentially provide information on the average trips and mileage for different types of cars, which would support esti- mation of car miles for different types of materials. Implementation barriers. Key barriers that preclude greater use of risk assessments include the lack of accurate data; agreement on the types of metrics that should be used when computing and communicating risk; and agreement on an acceptable level of risk. Challenges also lie in determining the specific entities that can design and implement a method- ology for defining the acceptable level of risk. 3.4.2.4 Pipeline and Hazardous Materials Safety Administration (PHMSA)/Office of Hazardous Materials Safety (OHMS) Current uses, users, modes, and decision making. For PHMSA, risk assessment is the starting point for a wide vari- ety of issues. It extends beyond application of the risk equation and includes gaining an appreciation for the quantitative and qualitative dimensions of risk. Some issues, such as affected entities and risk distribution, are important policy consider- ations that are not typically addressed in benefit-cost analyses because they do not alter the overall societal numbers. In other cases, PHMSA considers risk factors in the decision processes that focus on other factors, such as grant allocations.

19 they occur in relatively small numbers, so there is a lack of explanatory value. Updates. PHMSA updates its assessments on an ad hoc basis. A few new analyses, such as for the most important commodities, will be regularly updated. Risk communication. The results of regulatory evalua- tions get reported as part of the rulemaking process. Some of PHMSA’s assessments and analyses are conducted for inter- nal decision making and are not communicated externally. Other analyses are documented in reports and made publicly available. Desired improvements. PHMSA’s Office of Hazardous Materials Safety will identify its desired improvements after it has implemented its research plan. In general, being able to explore insurance data to get a bet- ter understanding of hazmat incident underreporting would be highly desirable. There would be other useful applications of insurance data as well. Performing analyses will allow PHMSA to address data quality and data gap issues. This effort could be part of an iterative process that improves the analyses while compiling pertinent but missing data. Additionally, it would be useful to create metadata for the existing data so that analysts gain a clear understanding of possible code values and the order in which they were pre- sented to the reporting entity. In addition, it is important to document who collects the data, who reports them, and what PHMSA does to capture them. Implementation barriers. Data quality is a significant implementation barrier. Lack of analytical resources for processing the data can potentially restrict data use even if the data are available. Systematically evaluating the known and the unknown errors within the data, and in each step in the analysis, supported by creating processes for this evalua- tion, will improve the value of the analysis and further define its scope. 3.4.2.5 Pipeline and Hazardous Materials Safety Administration (PHMSA)/Office of Pipeline Safety (OPS) Current uses, users, modes, and decision making. The PHMSA/OPS does not conduct risk assessments of pipelines itself, but rather oversees the individual pipeline operators who are required to ensure the safety, integrity, and reliability of their own pipelines. OPS is tasked with assuring pipeline safety, exclusively. Issues of security are of peripheral interest to the OPS mission, and are relevant only for managing third- party strikes, which may include terrorist events. very different than an initial risk assessment to determine if a rulemaking should be even proposed. Sometimes a risk assessment is performed after the fact to support a rulemaking which constrains true applicability and induces unintended bias. Rarely are the assumptions used in a rulemaking revis- ited to determine whether they still apply. Limitations of the hazmat transport incident data pertain to underreporting and missing data elements. Many of the elements of interest to PHMSA are subjective, such as the “cause” data field. Non-descriptive values (e.g., unknown, other, blank) are often reported. Additionally, the hazmat being transported and its quantity are sometimes unknown, which constrains the exposure analysis. OHMS often makes assumptions due to the lack of data, but clearly states them in its work. It is implicitly assumed that in OHMS’ inspection targeting models, the judgment of inspectors is useful in predicting future risk; this assumption has never been tested. The output from the targeting models is a ranked list of companies, but the inspectors have discre- tion in how they use that list. As the hazmat program focuses on prevention, emergency response is largely ignored in its analyses. No assumptions are made in relation to response. Often in its regulatory evaluations, it is assumed that the regulated entities will automatically come into compliance. In reality, different industry segments may be driven by benefit- cost ratios that do not necessarily translate into compliance. A large number of biases can potentially impact risk assess- ments. Incident data reported by carriers may have an inher- ent bias. There is potential for institutional bias in the target inspection data as approaches are refined over time. There are potential psychological biases from operators that tend to affix blame to individuals rather than the system. Addressing uncertainty. PHMSA addresses uncertainty in two ways. For risk assessments, instead of using a Monte Carlo approach, uncertainty is explored through sensitivity analysis, by varying key elements of the analysis and examining how the results change. This method does not quantify uncer- tainty within the model or consider the uncertainty inherent in the model’s assumptions, but it provides a better understand- ing of the influence of the parameters on the user’s decision. The second area where uncertainty is addressed is in the reporting of its performance measures on deaths and major injuries. Since the numbers are small, there is a lot of fluc- tuation from year to year. Therefore, PHMSA started deter- mining the longer-term trend and applying a one standard deviation above and below the trend line to see whether the risk fell outside that range. It is interesting to note that PHMSA does not believe that better denominator data will help as much as is generally perceived. This is because incidents are stochastic events and

20 chemical exposure, DHS for identifying the need for detec- tors for certain chemicals, and the National Security Council for developing communication processes. Models, tools, methodologies, approaches. The Chemi- cal Infrastructure Risk Assessment (CIRA) examines the human health risks from chemicals in the chemical supply chain. The program includes a review of atmospheric trans- port and dispersion models, along with application of the most appropriate model to a release at any point in the sup- ply chain, including transportation using a probabilistic risk assessment. The CTRA provides an end-to-end assessment of the threat due to terrorist use of toxic chemicals. Specifically, the assess- ment examines the terrorist use of chemical warfare agents and toxic industrial chemicals as it applies to, but not limited to, the transportation sector. The CTRA is a combination of separate models that exam- ine all routes of exposure: inhalation, ingestion, and per- cutaneous, and includes lethal and non-lethal effects. The underlying framework is probabilistic risk assessment (PRA) that includes terrorist intention. The event tree is refined to great detail and each of the branches can be combined as required by the user. Each event tree branch defines a sce- nario and the frequencies are applied along the path down that branch. Consequences are determined by the appropri- ate model for that scenario and multiplied by the overall fre- quency. The methodology supports large and small accidental releases as well as large intentional releases. Specific models used as part of the CTRA include Health Prediction and Assessment Capability (HPAC) (for outdoor inhalation consequences) and Contaminant Multizone Mod- eling Software (CONTAM) (for indoor inhalation conse- quences), the Self Consistent Integral Puff (SCIPUFF) model (which also is a collection of models), a statistical model for percutaneous exposure, and a stock and flow model for inges- tion that considers the food supply. Key sources of data. Expert elicitation is used to gather information about likelihood (combining threat and vulner- ability), using a methodology developed at USC’s CREATE. Consequence data include toxicity data for over 120 chemi- cals that are categorized by route of exposure (inhalation, percutaneous, and ingestion) and injury severity (lethal, severely injured, and moderately injured). Assumptions, limitations, biases, and availability. CSAC will be much improved by probability slopes and the toxic load exponent for LD50 toxicology values. While lethal dose data are more readily available, for the injury categories, CSAC is lacking some of the data related to the severely and moderately injured categories and it makes assumptions where necessary. Models, tools, methodologies, approaches. Pipeline oper- ators are required by Integrity Management and Distributed Integrity Management regulations to conduct risk assessments for high-consequence areas in accordance with approaches outlined by the American Society of Mechanical Engineers (ASME)/American National Standards Institute (ANSI). The Pipeline Risk Management Manual is the industry standard methodology for conducting pipeline risk assessments. Key sources of data. A key data source is the PHMSA database of pipeline incidents and accidents. Since 2000, operators have been responsible for reporting incidents and when reporting new incidents, operators must review pre- vious database records to identify similar incidents on their system for investigation and analysis. Assumptions, biases, limitations, and data availability. Accident/incident data prior to the creation of the PHMSA database in 2000 is insufficient for long-term trend analysis of failure causation. Desired improvements. Some commonly used pipeline risk assessment models are many decades old and use out- dated curve-based methods. Updating these models using modern computer technology could provide more realis- tic modeling and better define acceptable deviations from optimal operating conditions. New cross-modal initiatives, such as nondestructive evaluation (NDE), would have wide- ranging benefits. Finally, improvements in modeling and characterization of high-strength steel pipes (e.g., ×80, ×100, and ×120) are needed in order to be able to take full advantage of their improvements in strength and weight. 3.4.3 Department of Homeland Security 3.4.3.1 Science and Technology Directorate (S&T) Current uses, users, modes, and decision making. DHS S&T includes many components that are relevant to hazmat transportation risk assessment. The Transportation Security Lab was contacted, but did not specifically address risk assess- ment issues. The key DHS S&T component was the Chemical Security Analysis Center (CSAC). The component models in the CTRA are used by exter- nal (to CSAC) policy makers across government agencies to assess the relative risk of representative scenarios (e.g., what is the riskiest scenario for a given chemical?), the relative risk of representative chemicals, and how chemical risks change in the context of different scenarios (e.g., different chemi- cals may be better suited for indoor, outdoor, or food-based scenarios). The purpose of the assessments is to raise aware- ness as well as determine the relative risks. Examples of uses include HHS for developing medical countermeasures for

21 given attack scenario would succeed, and the ultimate impacts of the total loss of the assets on the agency’s mission.”10 Consequence estimation is primarily focused on acute public health impacts. They generally do not consider criti- cal infrastructure/key resources in their risk assessments, but these do inform their activities that directly relate to those enti- ties. They do not consider environmental impacts to a great degree and have the ability to look at economic risk but have some reservations about the available models. They have used IMPLAN, though, for their input-output-based modeling. A fundamental question is determining the endpoint of the analysis, with direct effects, indirect effects, or induced effects. Key sources of data. Data sources are varied and infor- mation is not always available. Most data are obtained from other government sources; however, a lot of recent attention has been focused on obtaining data from industry that would address denominator issues: quantity of material being trans- ported, associated containers, frequency, etc. Assumptions, limitations, biases, and availability. TSA uses a range of parameters, often by assigning them through Monte Carlo simulation, typically running from 500 to 1,000 simulations for each scenario. TSA attempts to eliminate biases from their assessments, but a source of bias could be the dif- ference in the inputs from some industry representatives. For example, there is more information available to them on arsine and phosphine than on acrylonitrile. The Likelihood component is prone to more bias than the consequence component, which uses well-known toxicity val- ues and the Monte Carlo simulation approach for varying the release amounts. Addressing uncertainty. Uncertainty is captured and pre- sented to decision makers through whisker plots that display the mean and the uncertainty band for each resulting value. The Latin hypercube sampling method is also used in their uncer- tainty analyses. In surveys, responses to questions are captured with the respondent’s level of certainty of their response. Updates. TSA generally updates its risk assessments every two years, but they are now moving to a four-year update cycle as there are relatively fewer changes at the two-year interval. TSA is focusing its updates to the data based on the current state of the art, rather than making adjustments to their mod- els. Barring any revolutionary advancements, it does not have any expectation that that will change. CTRA uses average container sizes for the materials as actu- ally transported. It uses a modified Latin hypercube Monte Carlo approach for sampling the range of container sizes pos- sible, centered on the mean, but not using only the mean size. Addressing uncertainty. The CTRA probabilistic risk assessment allows for computing uncertainties; CSAC identi- fies unreliable data points and captures uncertainties around that point. It often reports risk with an error range using the t distribution. Updates. The CTRA and CIRA have been updated every two years, but are moving to a four-year cycle to reflect infre- quent changes in underlying information. Risk communication. The CTRA and CIRA are classi- fied models, but outputs are shared with other entities that can take action to address identified risks. Desired improvements. The CTRA does not currently consider intermodal shipments. Additionally, there can be some improvements in the model components that address transportation. Enhanced ability to share results without compromising security is desired. Implementation barriers. There are numerous data needs; one of the biggest is the need for better toxicology data. 3.4.3.2 Transportation Security Administration (TSA) Current uses, users, modes, and decision making. Many elements within TSA are focusing on different aspects of terrorism risk and for different scenarios. Generally, TSA is focused on a broader level than specific countermeasure implementation; it focuses on overarching and intermodal issues such as the relative risks of rail and highway transport, cross-modal comparison of other types of terrorist threats (e.g., rail transit), etc. Resource allocation across regulation, education, and identifying and promoting best practices are issues of primary focus. Models, tools, methodologies, approaches. TSA uses the traditional threat, vulnerability, consequence construct for risk assessments. Risk assessment models are generally externally developed, but they have built some internally as well. In general, they integrate external models to suit their needs. One toolkit used in the transportation sector is the Terrorism Risk Assessment and Management (TRAM) tool- kit. This software-focused approach is geared toward asset owners and operators to “identify their most critical assets, the vulnerability of those assets to attack, the likelihood that a 10 National Research Council. “Review of the Department of Homeland Security’s Approach to Risk Analysis,” National Academy of Sciences. 2010.

22 when commodity flow information tends to be an essentially shared component. Assumptions, biases, limitations, and data availability. While commodity flow information is often essential to risk assessments used by Cal EMA, the data currently must be col- lected “piecemeal, on a local-level,” in the absence of statewide commodity flow survey data. Updates. The update schedule of each assessing entity’s analysis varies. In general, rural jurisdictions update their assessments less frequently than major population centers. Risk communication. The statewide response plan devel- oped from the many risk assessments carried out within the state is available online through Cal EMA. Assessment results and emergency response plans for individual jurisdictions are often made available online, but that may not apply to all jurisdictions. Desired improvements. Consolidated commodity flow information for the entire state of California is needed. Cur- rently commodity flow information is collected locally, with no continuity across the state. Having a statewide commod- ity flow survey would help local and rural planners who often lack the resources to conduct these surveys for them- selves. This research would similarly benefit local emergency responders who could use it to better prioritize resource uti- lization and funding while ensuring that they have access to the equipment necessary for potential hazards in their area and that the equipment is efficiently deployed. Implementation barriers. Impediments to hazmat trans- portation risk assessment include high expense and lack of local/rural resources and the need to gather proprietary data from private entities such as railroads. 3.5.2 State Transportation Department Current uses, users, modes, and decision making. The emergency manager for this survey respondent receives the results of hazmat corridor studies performed in their state. These studies have been included in county planning pro- cesses for most of the areas analyzed. These corridor studies have not been performed on all cor- ridors. They are planning on reviewing the data from these studies and from internal data on crash and hazardous mate- rials incidents to determine if any actions are warranted, such as the potential for implementing hazmat route restrictions. The agency’s primary focus is on the highway transporta- tion of hazmat. They hope to employ risk assessments as they develop a better enforcement and rerouting program over the next few years. Risk communication. Risk communication is generally determined by the DHS decision makers. The relevant federal government agencies are usually briefed, but most of the infor- mation is considered sensitive and is not widely distributed. A significant portion of the industry stakeholders have clearances and can obtain relevant risk assessments. Some data, however, are provided by organizations or communities that do not want it given to others. This particularly applies to the intelligence community. Desired improvements. The TSA desires more accurate and comprehensive data on the types, quantities, and fre- quencies of hazmat shipments. TSA believes that the application of game theory to hazmat transportation risk assessment is an avenue worth pursuing. Currently, game theory cannot yet be used but can potentially reduce the time collecting information from subject matter experts since there are no stand-alone models or capabilities to estimate the activity or intent of “intelligent adversaries.” Implementation barriers. There is a concern about risk assessments that identify areas of potential concern although no action is taken. Understanding the uncertainty is critical because the implementation of risk mitigation strategies relies on assessments that are based on incomplete data. 3.5 State Agencies 3.5.1 California Emergency Management Agency Current uses, users, modes, and decision making. The California Emergency Management Agency’s (Cal EMA) relationship to hazardous material transportation risk assess- ment lies in the use of an all hazards assessment approach that includes hazmat transport risks. Cal EMA officials use these assessments to develop emergency response and resource plans on a state-wide level. Assessments employed by Cal EMA consider hazmat transport risk by rail, road, and, in coastal areas, intermodal transportation. Models, tools, methodologies, approaches. Cal EMA approaches risk assessment primarily through the aggrega- tion of risk assessments conducted by subordinate govern- ment entities within the state, such as regional, countywide, city, and local emergency planning committees (LEPCs). Both safety and security components are considered, through the use of all-hazards analysis methodologies. There is no com- mon risk assessment methodology or a statewide standard available to all assessors; each local entity individually decides how the assessments are carried out. Because risk assess- ments are performed using methodologies specific to each local authority, the data required for each approach vary, even

23 ability of a release. Environmental consequences are rarely considered; most of their scenarios focus on human health and property damage. Their legislative authority is directed to immediate impacts and not long-term impacts, which would include many of the potential environmental impacts. Key sources of data. Data specifics vary with the nature of the problem being considered. TDG produces a report on the movement and handling of dangerous goods in Canada. That is compiled with the accidents that occurred and pro- vides a context for understanding the relative probability of an accident. Their reported accident statistics are delineated by severity level, mode, contributing factor, phase, type of release, material class, packaging type, and region. These reports are used to help focus enforcement actions at the regional/local levels. Desired improvements. Two areas of desired improve- ment are to better understand all of the companies involved in hazmat transportation and to better understand the vol- umes that are transported. The TDG Directorate also would like to try to define the accidents for which further informa- tion is needed. Implementation barriers. The lack of data is one of the main barriers to conducting risk assessments. For example, they would like to obtain more information on accidents. Police reports would be useful, but would have some vari- ability. For specific issues there might be industry data that is typically not collected by the government. Other barriers include strategic priorities and a common understanding of risk in the program. Transparency across different departments would allow for better sharing and leveraging data. One potential example is an economic affairs analysis group that might generate a database of electronic shipping documents that would include information that can be used to support risk analyses in the future. This scenario was actually used in the context of airline data. 3.6.2 Foreign Security Agency Current uses, users, modes, and decision making. This foreign survey respondent has a variety of tools to analyze the various types of risk they deal with, from corporate risk analysis to facility-based assessments and to strategic secu- rity risk assessments. This agency examines security risks from a high-level strategic viewpoint and from facility- and operational-level security assessments. Much of their risk assessment research revolves around methodologies. They are revisiting their strategic security risk assessment methodology in an effort to ensure that it provides current and up-to-date information on security risks. Models, tools, methodologies, approaches. The data element mainly utilized is state highway incident data and it is provided to counties on request. Risk communication. They share the corridor studies done through the state Emergency Response Commission with other governmental agencies to support planning activities. Desired improvements. They desire further informa- tion on how other state departments of transportation or county agencies conduct their risk assessments and how they implement any necessary actions to reduce their risks. Implementation barriers. Primary barriers include the paucity of time for performing risk assessments, and relative costs that accrue from conducting risk assessments. 3.6 International Organizations 3.6.1 Transport Dangerous Goods (Transport Canada) Current uses, users, modes, and decision making. Risk assessment tends to be focused on corporate risk (at the pro- gram level) and there are separate efforts underway to exam- ine how they think about risk. The Transport Dangerous Goods (TDG) Directorate is interested in the overall risk pro- file for dangerous goods transported in Canada. They want to use this profile to focus their efforts and integrate their sepa- rate programs, which include accident reporting, inspection program, ad hoc requests, emergency response assistance [Canadian Transport Emergency Centre (CANUTEC)], and the emergency response action plan (ERAP) program. For some programs, such as their certificates of equivalency process (analogous to special permits and approvals in the United States), they are examining what information they need to perform a proper assessment. While risk is a key concept in their regulatory program, risk perception is also given an important role. For example, a specific accident may gener- ate enough emphasis through public attention to support risk reduction measures even if that area was not at the top of the priority list. Their regulatory-based risk assessments often come into play when publishing proposed regulations. Risk assessments are used to understand potentially viable options and to decide with proceeding on a proposed regulation. Subsequent to a decision to move forward, regulatory impact statements, known aspects of the problem, considered alternatives, and benefit-cost analy- ses are all simultaneously published for public review. Models, tools, methodologies, approaches. While the historical record does not provide much statistical basis for analysis on deaths and injuries, TDG tries to focus on the prob-

24 the largest risks discovered in the risk assessment process, but also examines some of the potential actions that can be taken to mitigate risk levels. Desired improvements. One recommended improve- ment is the development of a taxonomy for security risk. Stan- dardized methodologies for looking at risk should be developed and shared among security risk practitioners internationally. There should be better distinction between high-level strategic security risk assessments (e.g., industry-level), opera- tional security risk assessments (e.g., organization-level), and tactical security risk assessments (e.g., facility-level). Implementation barriers. The lack of a risk manage- ment cycle within which risk assessments are conducted is an important issue. Barriers for using risk assessments are grounded in underdeveloped risk management processes that include identifying, analyzing, evaluating, and mitigat- ing risk. The ability to conduct dynamic risk assessments was mentioned as a barrier as well. 3.6.3 Public Foreign Research Organization Current uses, users, modes, and decision making. This survey respondent initially focused their hazmat transporta- tion research on road transportation in their country. They are evaluating potential threats during hazmat transportation to people, core infrastructure, and the environment. Most of their research in this area is directly applied to the legislative process; they seek other implementation approaches where that is not possible. Current work relates to finding the safest transport routes for domestic hazmat shipments. Models, tools, methodologies, approaches. They pay special attention to transport through highway tunnels. Their goal is to prepare a general methodology for evaluating high- way tunnels according to international standards [the Euro- pean Agreement concerning the International Carriage of Dangerous Goods by Road (ADR), in this case]. Assumptions, limitations, biases, and availability. One notable constraint they face is in modeling or estimating transnational transportation flows and those shipments that transit through their country. They have been able to create a model of their main national hazmat flows. They have used estimations for the international flows that have larger devia- tions than their national hazmat model. An important, high-priority focus of their risk assessment model development is determining the appropriate impor- tance (weight) for each input. They are developing a set of risk assessments models with different element weights and are trying to determine the best combination for modeling their conditions. They assess risk through a process that includes (a) event identification and scenario identification, (b) threat assess- ment, (c) vulnerability assessment, (d) impact assessment, and (e) risk calculation. The three assessment stages of the process involve workshops where a variety of stakeholders from within the federal government, industry, state and local governments, and other key stakeholders achieve a consensus on the relative risks of each scenario. The analysis that emerges from these risk assessments is used by senior decision makers to prioritize and efficiently allocate resources. They are also used by the participants to develop a better understanding of the main security risks rel- evant to their organization. Models, tools, methodologies, approaches. The main input in their security risk assessment methodology is the combined knowledge and expertise of subject matter experts. For example, in conducting a risk assessment on hazmat transportation, they involve experts from the security and intelligence community, hazmat shippers and carriers, emer- gency services providers, state and regional government rep- resentatives, representatives from the public health agencies, the nuclear safety agency, the natural resources agency, and various other experts in the field of hazardous materials. In a workshop environment, these experts evaluate and discuss various considerations around the relative threat, vulnerabil- ity, and impact of the scenarios being considered. Collecting input this way ensures that the best opinions and insights of subject matter experts are captured. They feel that an issue that needs to be addressed is the ability to collect this type of information in a dynamic fashion. Assumptions, limitations, biases, and availability. Their largest constraint is the inability to dynamically analyze risk. Since their risk assessment methodology requires subject matter experts in a workshop environment, conducting risk assessments cannot be conducted in a continuous fashion. They are currently exploring options to develop an “ever- green” methodology that would provide information in a timely and cyclical fashion. Another constraint is the subjectivity of the rating criteria. They broadly look at threat, vulnerability, and impact cat- egories, but they believe that they are not addressing all of the relevant parameters and scenarios. There are uncertain limits within which they analyze economic, health, response and recovery efforts, and other criteria. Risk communication. Risk assessment results are pre- sented in a classified internal report used by their organiza- tion to set future policy and priorities. They also produce a more general report that does not include the security sensi- tive information and is shared with workshop participants and interested stakeholders. The report highlights some of

25 Key sources of data. Data feeding their models would vary based on the industry and client they are working for. Some of their work includes estimating the direct and indi- rect consequences from potential attacks, such as the impacts on tourism and on the freight industry. Assumptions, limitations, biases, and availability. Data requests often require some form of authority to be effective. This can be regulatory authority, legislation, or in return for grant funds. All threats are based on reporting and the subjective inter- pretation of the reports. The assumption is that the threats are from an intelligent adversary able to achieve maximum impacts. Assessments therefore assume reasonable worst-case consequences with representative assets. In addition, there is a modal bias with respect to threat as the highway mode has the most incidents (not necessarily hazmat specific) and the greatest terrorist capability. At the other extreme, aviation has the most bias. As with many other entities, there is a bias introduced by the primary focus on human health impacts for some clients, including the U.S. Coast Guard and the Maritime Security Risk Analysis Model (MSRAM). ABS notes an observed bias between the private and pub- lic sectors. The private sector tends to rate things higher and worse than does the public sector. If a risk assessment is for strategic planning and is scenario based, there would be some assumptions about the applicable scenarios and others would be ruled out. Addressing uncertainty. Addressing uncertainty is depen- dent on the purpose of the assessment. In some elicitations where there is particular concern, low, best, and high estimates are obtained to offset the uncertainty introduced by the sub- jectivity of the input. In some cases, it is still appropriate to estimate the consequences of a failed attack. There may be sig- nificant economic impacts, for example. Risk communication. For some of the federal, cross- cutting work, each mode can only see the generated reports for their mode. Full reports may only be available to DHS and Congress. Risk communication is a necessary step whenever risk assessments are used as an explanation for a decision or action. Corporations regularly engage in risk tolerance dis- cussions to support making hard decisions and tend to be more advanced than the public sector in this area. Desired improvements. The biggest challenge is the lack of a knowledge capture effort. There is so much related work, but there is a lot of “rebuilding the wheel.” A repository of knowledge to build from would be helpful. They generally believe that their data are sufficient for their needs, including all the details of domestic hazmat accidents. Some data, however, is restricted to their research focus area. Risk communication. To date, they have only been communicating internally within their organization. Plans for external communication in the future include publishing their results in scientific journals. Desired improvements. They believe that research on risk assessment is very scattered throughout the world and they would like to be able to benchmark their approaches with those of other countries. Implementation barriers. They state that data are always a key aspect of successful risk assessment. After sufficient and accurate data are acquired, the limitation is in terms of fund- ing and availability of personnel. The difference in opinions about the importance of different model inputs can also be a barrier. 3.7 Consulting and Research Organizations 3.7.1 ABS Consulting Current uses, users, modes, and decision making. ABS Consulting (ABS) primarily builds models in-house for their industry and government clients. They note that the Inter- national Standards Organization (ISO) standard 31000 for risk management contains a methodology framework for risk assessment. Models, tools, methodologies, approaches. Each client’s risk assessment needs are different and the models they cre- ate are usually designed for a specific purpose. It might be for port security in general, specific facility types, pre-defined criteria. In some cases, their client’s focus is entirely on a spe- cific kind of security threat, for example, an al Qaeda-type adversary, rather than an all-hazards approach. In general, the consequences estimated are targeted to the customer, with some focused primarily on human health impacts and possibly critical infrastructure/key resources while others, such as a facility, might be more interested in the economic consequences of losing operating capability. Some clients are considering altering the monetization of impacts and may consider lesser injuries in addition to more serious injuries and fatalities. Where environmental impacts are con- sidered, they are often included as indirect impacts. An internal group at ABS has developed custom blast modeling and these models are used for estimating conse- quences from blasts. All vulnerability estimates are based on the Kent scale, which uses linguistic terms to represent the different values.

26 3.7.3 Engineering Systems Inc. (ESI) Current uses, users, modes, and decision making. ESI relies on risk assessments done by others to support different positions in their work, but do not conduct these assessments themselves. This includes understanding the true risk of an accident occurring and what factors are most influential. When looking at a case, were the right things appropriately considered? They do, however, develop and evaluate database processes related to hazmat container performance. Key sources of data. When working with specific cases, lit- erature searches can often find relevant academic or industry- published works. Sometimes, the parties to a legal case have prepared proprietary research or analyses. Assumptions, limitations, biases, and availability. There are institutional barriers that can affect the quality, accuracy, and completeness of data and assumptions need to account for these barriers. In addition, biases are introduced because of the various parties’ willingness to contribute complete data. Data are often incomplete and not updated as new infor- mation is discovered. This necessitates the use of inferred information from sources like clipping services. Risk communication. They communicate externally through research reports made public through the TRB and provided directly to clients. Desired improvements. They recommend a set of tools to communicate the risk assessment process to business peo- ple to help them understand how risk assessments can benefit and improve their businesses. Implementation barriers. There can be some fear in the industry about how the data in a risk assessment can or will be used. They focus on the potential uses against them rather than focusing on the benefits that might accrue. Many of the individuals that they have contacted in their work are reluc- tant and fearful. The business management aspects of the process are the stumbling blocks. Providing a better under- standing of the process, how the data would be collected, and how they would be used would help. Industry consensus standards, options, and guidelines are needed that provide a framework for industry members to complete risk assessments while providing a basis for protec- tion in the event of litigation related to decisions informed by the assessments. 3.7.4 Private Consulting Firm Current uses, users, modes, and decision making. The current risk assessment focus for this survey respondent is 3.7.2 Booz Allen Hamilton Models, tools, methodologies, approaches. Booz Allen Hamilton believes that when considering risk assessment from a government perspective, the process encompasses a general, high-level quantification of risk and a carefully done prioritization of specific incident risks, which are required for the process to be useful. This includes estimating probabili- ties and consequences at a granular level in terms of specific failure modes—in other words, breaking problems down to specific failure modes and assessing the relative risks of vari- ous failure modes; absolute risks are not all that important. Current uses, users, modes, and decision making. When applying risk assessment to hazardous materials, they have incorporated elements of work done for other agencies and clients in non-hazmat areas, such as FAA or National Aero- nautics and Space Administration (NASA) or in general engi- neering processes (design and development). For example, they have modified the traditional frequency (F), probabil- ity (P), and consequence (C) models based on their work for FAA on airframe airworthiness issues, particularly with respect to failure modes. The risk tolerance for hazardous materials transportation is still focused primarily on fatalities, but does consider inju- ries as well. For pipeline, environmental consequences are given much greater consideration than on the general hazmat side; this applies to economic impact as well. Key sources of data. Most of the company’s hazardous materials work uses existing PHMSA data. They mine that data and use additional research to augment samples. Their current work includes developing a sampling basis. Assumptions, limitations, biases, and availability. Research on prior work can lead to appropriate assumptions in areas such as the uniform distribution of accidents. In their regulatory work, they use assumptions on costs based on the standards that the government uses, including the value of a life, the proportional value of cargo in an airplane (passen- ger and freight). Also, unique distributions are likely for each failure mode analyzed. Biases include the focus on fatalities and the lack of focus on some modes—maritime, for example. There is a signifi- cant attention on the rail mode that is centered on the con- tainment of materials in tank cars—a fairly narrow focus. Addressing uncertainty. Sensitivity analysis is impor- tant as is the consideration of significant digits. The latter is important because much of the risk computation uses esti- mates. Booz Allen Hamilton believes that there is a need to test for a wide range of estimates to see how they impact the final results.

27 risk assessments on their own and integrate them into the reg- ular project preparation process. Implementation barriers. Time to do a proper analysis can be a barrier since many of their clients want quick results and do not want to take the required amount of time. 3.7.6 National Center for Risk and Economic Analysis of Terrorism Events (CREATE) Current uses, users, modes, and decision making. The CREATE is conducting research and developing a risk assess- ment model to inform emergency management and terrorism officials’ decisions about the type and placement of radiological/ nuclear detection devices in order to prevent or deter terrorist attacks using these materials. The CREATE approach can be used for detection resource deployment on a local level, such as at a port or airport, or for building a systemwide detection network on a statewide or multiple metropolitan area level. Models, tools, methodologies, approaches. The general risk methodology employed by CREATE is a Threat × Vulner- ability × Consequence scheme. This approach is used to con- struct a detection resource allocation strategy that optimizes any of a variety of parameters including the probability of detection, the costs associated with deployment of equipment or traffic congestion/delays, or the human and economic costs of the failure to detect or deter. Data requirements vary depending upon the application of the model and the desired output metrics, but can include traffic flow information, population and population density data, iconic target locations, economic statistics, and spatial transportation network information. Key sources of data. The U.S. Census Bureau is a source of data for population, demographic, and traffic flow information. 3.7.7 National Pipeline Safety and Operations Research Center Current uses, users, modes, and decision making. The National Pipeline Safety and Operations Research Center conducts risk assessments that are focused exclusively on safety and applicable only to pipeline hazmat transportation. These assessments are used to gauge the risk to the holistic environment surrounding pipelines, as opposed to the pipe- line itself. Pipeline risk assessments by the Research Center are generally carried out to support governmental planning decisions and legal reviews. Models, tools, methodologies, approaches. The Research Center follows the general industry-standard methodology for targeted at emergency managers and LEPCs to better under- stand the quantity and volume of priority hazardous materi- als transported and stored in a specific study area. Efforts include identifying and mapping critical facilities, activity centers, chemical companies, and transportation networks as well an identifying and mapping chemical ship- ments to and from chemical companies and other critical facilities by motor carrier, rail, pipeline, and barge. Models, tools, methodologies, approaches. The pri- mary data used in the company’s analyses include chemi- cal facility locations, chemical storage quantities, chemical transportation volumes, direction, frequency, and mode. The company uses published research to help improve their assessments and hope that the TRB will continue to sponsor and publish in this area. Assumptions, limitations, biases, and availability. Obtaining proprietary data needed to support analyses is a significant issue. Risk communication. Internal communication is for emergency planning purposes only. Their information is pro- prietary and not shared with third parties or the public. They aggregate their results by chemical and mode to show annual volumes only. Implementation barriers. The data limitations due to proprietary issues are their biggest barrier to implementing hazmat transportation risk assessments. 3.7.5 Private Consultant Current uses, users, modes, and decision making. This survey respondent works with clients to evaluate different practices and to develop the best alternatives that combine all the aspects that have the potential for harm or negative out- comes. They include safety, security, productivity, and morale in their risk assessments. Models, tools, methodologies, approaches. A major consideration in their analyses is weighing frequency and probability with the severity of consequences. The data to show short- and long-term costs of low-probability/high- consequence scenarios or events are difficult to find. Risk communication. Their results are communicated internally to clients, usually at the mid-manager level and to workers during their regular training. Desired improvements. A requested improvement is the development of easy-to-follow guidelines for workers, supervi- sors, and mid-level managers so they can conduct some basic

28 TSA’s scenario-based TSSRA is designed to identify the biggest risks and this is one of the key sources of information for some of their members. Key sources of data. Much of the data to support secu- rity risk assessments comes from the owner-operator of the facility or operation. Having the proper regulatory authority is often crucial to obtaining that information, but providing grant money in exchange for it can also be effective and is a much softer approach. Consequence estimates are determined from models, research, and reports. Economic consequences are often obtained from REMI and similar models. Assumptions, limitations, biases, and availability. Fun- damental areas of assumption are about the nature of the risk being evaluated and the nature of the various scenarios (which are informed by subject matter experts). While vul- nerabilities can be more accurately estimated, understanding the full nature of the threats can be difficult. Consequences also present some assumptions, such as the value of a human life and how to account for the psychological impacts of lethal attacks. Updates. SARMA believes that using risk as a decision support tool needs to be an ongoing process. This includes accounting for the risk buy-down from implementing secu- rity countermeasures and then seeing what happens to the risk for other segments of the organization’s operations. Risk communication. Communicating risk can be a challenge. In one example, a federal grant program’s approach changed dramatically and there was the need for a hard con- versation with the stakeholders about risk tolerance on the front end and a discussion on how to communicate the results on the back end. These were both difficult conversations to have in the public policy context. Desired improvements. The biggest challenge is to get the data needed to do risk assessments. Solving this prob- lem is very complex from the state or national perspective. It is less challenging at the local level. This is why it is hard to establish a risk baseline. The state fusion centers could be leveraged to capture some of these data. At a higher level, there is a need for better information and guidance on data collection and how to make the best use of the data. There has been a lot of discussion about the time horizons for risk assessments. Generally, they focus on a single year. It would be beneficial to use a five-year time period and com- pare against organizational objectives. pipeline risk assessment outlined in the Pipeline Risk Manage- ment Manual. The Research Center’s approach evaluates risks to the environment, human health, society, infrastructure, etc., and varies model parameter weightings as appropriate to the focus of their assessment. Key sources of data. Model input data is generated through research for each assessment and, to a lesser extent, sourced from publicly available PHMSA databases. Rarely, a pipeline operator may provide information in support of an assessment, typically at the request of the funding entity’s legal representation. Assumptions, biases, limitations, and data availability. The Research Center generally assumes that assessed pipelines are buried. This assumption is due to above-ground pipeline tending to be located far from developed areas, precluding the need for risk analyses. Addressing uncertainty. Where data is unknown, the most conservative risk value is used. Updates. Risk assessments carried out by the Research Center are not updated. Risk communication. Risk analyses carried out by the Research Center tend to be covered under attorney-client privi- lege, prohibiting the sharing of analysis results. Implementation barriers. A lack of public funding is the main barrier to more widespread pipeline risk analysis. 3.7.8 Security Analysis and Risk Management Association (SARMA) Current uses, users, modes, and decision making. The focus of risk assessment from the SARMA perspective is to consider a broad all-hazards approach (which still generally focuses on security risk) and identify and measure the needed investments to drive them down or move them to another sec- tor. A past survey conducted by SARMA determined that the security risk assessment discipline was not well defined. Even the focus of the analysis varied substantially, from a broad brush strategic level to a tactical approach at a facility level. Models, tools, methodologies, approaches. Generally population and population density are used as proxy mea- sures for both vulnerability and population consequence. Creating a risk baseline is difficult. One project examined using the Target Capabilities List (TCL) and optimizing response and recovery capability with the dollars available. The issue was determining whether the right capability had been identified in the first place.

29 ology would include data about vehicle location, shipment characteristics, and dynamic operating conditions. Assumptions, limitations, biases, and availability. The risk scores will be dynamically changing and will rely a lot on the underlying, geospatially referenced data. The risks of hijacking in rural areas will be overshadowed by high risks influenced by high population areas. Also, there is an initial assumption that the hazard class-specific impacts and con- sequences are representative of all the materials in that class. Updates. The situational awareness aspect of the meth- odology will be updated with each vehicle location update. The methodology itself will be subject to periodic reviews and will consider new and emerging ways to acquire data. Quar- terly reviews are expected early in the implementation phase. Risk communication. In this methodology, risk is pre- sented as numeric values for safety and security. In the situa- tional awareness context, the risk values are displayed by color. Different risk levels are represented by different colors and can provide an easy-to-grasp picture of the distribution of risk across an area of interest. Desired improvements. Improvements that were men- tioned included the ability to better measure uncertainty. At a high level, it would be beneficial to get a nationally accepted methodology to use in similar systems and to conduct research to determine if the methodology is effective and which changes would create additional value. Implementation barriers. A big barrier is the ability to get funding to implement an accepted, cost-effective system that addresses industry privacy concerns. Industry itself is a barrier to getting overall risk management implemented because of privacy/security concerns. 3.7.10 University of Illinois at Urbana-Champaign (UIUC) Current uses, users, modes, and decision making. The UIUC evaluates the probability and the consequences of hazmat transportation at both the macro-level (nationwide/ regional network) and the micro-level (route or segment- specific). Risk analyses conducted by the university are pri- marily safety-focused and support a number of national-level research initiatives in the rail industry. Models, tools, methodologies, approaches. A recent ini- tiative has focused on estimating the risk tradeoffs involved in switching from standard tank cars to alternative designs. This analysis employed a model that considers historical shipment 3.7.9 The Kentucky Transportation Center (KTC) at the University of Kentucky Current uses, users, modes, and decision making. KTC is conducting research and development of a methodol- ogy and tool to provide data and information for use by the federal government for real-time situational awareness of high-security risk highway hazmat shipments. Both safety and security are being considered, but the focus is on secu- rity. By understanding the relative risks of different ship- ments, the users of the methodology (law enforcement and anti-terrorism officials) can make better decisions about the appropriate security countermeasures that would help reduce risk. Models, tools, methodologies, approaches. The meth- odological basis for the approach derives from the best prac- tices used across a number of different systems, including the RCRMS. Safety methodology considers frequency, probability, and consequence and security methodology considers threat, vul- nerability, and consequence. Both risk measures are relative scores and are not combined together, but considered sepa- rately. Security risk would be dynamically computed and the safety risk would be static for a given planned route. As with most safety based route risk assessment approaches, this work utilizes roadway type to apply appropriate per-mile accident rates that are applicable and consistent across the country. New work is being considered to update the prob- ability of release given an accident for different types of mate- rial and packaging combinations. Consequences are focused on population exposure, critical infrastructure/key resources, environmentally sensitive areas, and economic impact. One of the key benefits of the approach being developed is that it supports identifying the risk reduction potential of various security risk mitigation strategies. Threat considers the attractiveness of different locations based on both static and dynamic factors, including a range of potential attack modes, hazardous material and packag- ing, population density, and presence of certain types of tar- gets. Vulnerability is based on the attack mode, material and packaging, and the security posture. Consequences are deter- mined as for safety for consistency. Key sources of data. The specific elements of this work are still being developed, but population data are Census- based, critical infrastructure and some threat information would be obtained from the federal government, and road- way network data would be used with historical accident data to derive the potential accident frequencies. Other probability and vulnerability information would be elicited from subject matter experts. In addition, implementation of the method-

30 critical data, such as railroad waybill information, is restricted to government or industry for security reasons. UIUC describes constraints in data for evaluating new tech- nologies and their effects on risk, such as new railroad way- side defect detectors to reduce mechanical failures and positive train control. There is a need to assume a certain level of effec- tiveness in improving safety that may or may not exist. This organization describes difficulties in accounting for litigation costs in their risk models. Another serious limita- tion is the lack of highway container conditional release prob- ability data. Risk communication. The results of UIUC risk analyses are reported internally to private carriers and shippers and externally through presentations at major conferences and published research articles and papers. Desired improvements. A detailed database of damages from accidents involving cargo tanks and ISO tanks would help to better perform multi-modal risk analysis. Implementation barriers. Data availability is a barrier for conducting risk assessments for highway and multi-modal applications. routes and volumes of individual TIH chemicals, the fre- quency of car derailments along those routes, expected condi- tional release rates, and potential human receptors along each route. Risks were calculated for a number of high-volume chemicals using their standard tank car configurations and compared against similar calculations using alternative tank car technologies. The results of this study informed decisions made by the AAR on TIH tank car standards and were incor- porated into TIH transportation regulations by the U.S. DOT. Key sources of data. Probability and frequency data are obtained from the FRA Accident Database, the RSI-AAR Tank Car Database, AAR TRAINII Waybills, and Surface Transportation Board (STB) Waybills. Consequences and the potential severity of incidents are derived from U.S. census data, USDOT ERG response guidelines, and GIS analyses. Assumptions, limitations, biases, and availability. The level of data sufficiency differs across the various data ele- ments employed by UIUC in risk analyses, though “close to 90 percent” of its data needs are satisfied for railroad risk analysis. While much of the data is available through public access of government databases and academic journals, some