Frontiers of Engineering: Reports on Leading-Edge Engineering from the 2008 Symposium (2009)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Human Reliability Analysis in Cognitive Engineering and System Design Ronald Laurids Boring Sandia National Laboratories Albuquerque, New Mexico Human factors engineering (HFE) combines elements of several engineering disciplines, psychology, and computer science into a single discipline (Boring, 2002). Two major subdisciplines of HFE include: • cognitive engineering (CE), which focuses on the cognitive aspects of human-system interactions to maximize system usability (Nielsen, 1993), safety (Palanque et al., 2007), and user enjoyment (Norman, 2002) • human reliability analysis (HRA), typically part of an overall probabilistic risk assessment (PRA), which focuses primarily on verifying the safe performance of human actions Despite similarities in focus, the main difference between CE and HRA is in the timing of when they are used. CE is typically implemented in the design phase of   he T submitted manuscript has been authored by a contractor of the U.S. government under con- tract No. DE-AC04-94AL85000. The U.S. government retains a nonexclusive, royalty-free license to publish or reproduce the published form of this contribution, or allow others to do so, for U.S. government purposes. 103 

104 FRONTIERS OF ENGINEERING the engineering cycle, whereas HRA is often used in the verification and validation phase, after systems have already been built. However, the application of HRA primarily to as-built systems is a historical artifact. Analysts have included assessments of human reliability in military system evaluations since the 1960s (Swain, 1963), but the first widely publicly available guidance for HRA was described in the WASH-1400 report (U.S. Nuclear Regula- tory Commission, 1975), which addresses the safety of nuclear power plants. The Technique for Human Error-Rate Prediction (THERP) HRA method (Swain and Guttman, 1983) provided the first systematic method of identifying, modeling, and quantifying human errors. THERP and subsequent HRA methods developed in the aftermath of the Three Mile Island nuclear incident in the United States were accompanied by a call for risk-informed decision making using PRA and HRA (Kadak and Matsuo, 2007). Together, HRA and PRA produced assessments of existing systems with less emphasis on design than was typical with HFE and CE. HUMAN RELIABILITY PROCESS MODEL The three phases of contemporary HRA methods are depicted in Figure 1. As shown, HRAs can be characterized as qualitative or quantitative. A qualita- tive HRA includes the identification and modeling phases described below. It converges on other assessment approaches such as root-cause analysis, which is used to determine the causes of human errors. A subsequent quantitative HRA uses these qualitative insights to estimate the likelihood of these errors. HRA Phase 1: Identify the Sources of Errors This phase typically consists of a task analysis to determine human actions and a review of those actions to identify opportunities for errors. Performance- FIGURE 1  The three phases of HRA. Boring Figure 1 R01394 bitmapped fixed image 

HUMAN RELIABILITY ANALYSIS 105 TABLE 1 Performance-Shaping Factors in Good Practices for Implementing HRA Applicability and suitability Workload, time pressure, and Accessibility or operability of of training and experience stress equipment Suitability of relevant Team and crew dynamics Need for special tools procedures and administrative controls Availability and Available staffing and Communications strategy and understandability of resources coordination instrumentation Time available vs. time Ergonomic quality of human- Special fitness needs required system interface Complexity of required Environment Off-normal operations and diagnosis and response situation Source: U.S. Nuclear Regulatory Commission, 2005. shaping factors (PSFs), aspects of behavior and context that may impact the outcome of a task, are then identified. For example, a PSF might be the presence or absence of clearly defined, well-understood procedures, which can greatly enhance or hinder human performance of a given task. Good Practices for Implementing HRA, a report sponsored by the U.S. Nuclear Regulatory Commission (2005), provides a standardized list of 15 PSFs believed to have an impact on human performance in the nuclear domain (see Table 1). An individual HRA method may have as few as three PSFs (Galyean, 2006) or as many as 50 PSFs (Chang and Mosleh, 2007), depending on the level of detail required for capturing human actions. HRA Phase 2: Model the Errors in an Overall Risk Model Human activities of interest in an HRA are not generally performed in isola- tion; they are interactions with hardware systems. The hardware systems modeled in a PRA feature reliability curves for both systems and components to provide mean times before failure. A failed hardware system can cause humans to fail at their prescribed tasks, or a human error can cause a hardware system to fail prematurely or unexpectedly. A hardware system may be designed as a failsafe backup for human actions errors, such as an automatic pressure-venting valve that can mitigate system damage if the human operator fails to regulate pressure properly. Conversely, the human operator may save a failed hardware system. For example, positive human intervention can recover a failure or prevent the escalation of a hardware failure. 

106 FRONTIERS OF ENGINEERING FIGURE 2  A logical “OR” gate connecting hardware-system failure and human error in the form of a fault tree (top) and event tree (bottom). The fault tree is read from bottom to top. The event tree is read as a sequence from left to right. Boring Figure 2 R01394 In an HRA, human activities are modeled as part of a fault tree, or event tree (see bitmapped fixed image Figure 2), to show their interactions with the hardware system. Phase 3: Quantify the Errors The object of many HRAs is to provide a probabilistic expression of the likeli- hood of a failed human action, called the human error probability (HEP). HRAs are primarily differentiated by their approaches to error quantification. Although dozens of approaches have been developed, they tend to follow a common pat- tern, beginning with a nominal HEP (i.e., a generic or default error rate for human 

HUMAN RELIABILITY ANALYSIS 107 reliability 0 < PSF < 1 HEPoverall < HEPnominal increases reliability HEPoverall = HEPnominal x PSF PSF = 1 HEPoverall = HEPnominal stays same reliability PSF > 1 HEPoverall > HEPnominal decreases EQUATION 1 activities) and followed by a modification of the nominal HEP according to the specific PSFs. PSFs are often treated as multipliers. For example, if the effect of good proce- dures has a PSF value less than one, the product of the nominal HEP and the PSF multiplier would be less than the nominal HEP, resulting in an overall decrease in HEP and corresponding increase in human reliability. Conversely, if the effect of poor procedures has a PSF value greater than one, the product of the nominal HEP and the PSF multiplier would be greater than the nominal HEP, resulting in an overall increase in HEP and corresponding decrease in human reliability (see Equation 1). APPLICATION OF HUMAN RELIABILITY ANALYSIS TO SYSTEM DESIGN HRAs can be either retrospective or prospective. The purpose of a retrospec- tive HRA is to assess the risk of something that has already happened, such as an incident or accident, to determine the likelihood of it happening the way it actu- ally did. Was it an anomalous accident, or is it to be expected that it could occur again, given the same situation? A prospective HRA is an attempt to assess the risk of something that hasn’t actually happened, such as an extremely rare event (e.g., human performance in a nuclear power plant control room during a seismic event or fire). Note that, even though a prospective HRA can be extremely helpful for anticipating breakdowns in the human-system interface, prospective HRAs have not commonly been used to provide information that can be incorporated into the early-stage design of a system. Rather, as noted in Hirschberg (2004), prospective HRAs are usually used to improve existing processes and systems by pinpointing weaknesses and providing a basis for prioritizing “fixes.” Thus, they are typically used in assessing and making iterative improvements in existing technologies. This after-the-fact use of prospective HRAs is artificially limiting. If they were used not just on as-built systems but also on systems that are still being designed, they could be design tools used in combination with CE and HFE. Three recent developments show how HRAs could be used in the design phase of system development. 

108 FRONTIERS OF ENGINEERING The Need for Human-Certified, Safety-Critical Systems Recent regulatory guidance documents, such as the Human Factors Engineer- ing Program Review Model (O’Hara et al., 2004) for nuclear power plants and Human-Rating Requirements (NASA, 2005) for aerospace systems, suggest using HRAs as part of the design process to complement existing human-factors design best practices (Boring, 2007a). As new nuclear power and aerospace systems are built, qualitative HRAs can complement other HFE and CE techniques to antici- pate sources of human errors and, ultimately, to help design the system to prevent those errors from occurring. In addition, quantitative HRAs may be used to help determine the likelihood and consequences of specific errors and to prioritize the error-likely design issues according to their impact on safety. The Emergence of Resilience Engineering A recent development is a growing awareness that the negative consequences of an incident can be greatly mitigated by the quality of underlying human inter- actions with the system. The goal of resilience engineering (Hollnagel, 2006; Sheridan, 2008) is to identify the qualities that make humans, processes, and systems robust or resilient in the face of adverse events. Resilience engineering differs from HRA in that it argues for the unpredictability of adverse events, but it shares many conceptual underpinnings with HRA. Resilience engineering can be reconciled with HRA in the context of system design. HRA provides a standardized way of assessing vulnerabilities in human actions, which make actions less robust. An HRA can even be used to define the characteristics of resilience (e.g., PSFs that characterize resilient, as opposed to brittle, actions). In the context of system design, the goals of resilience engineer- ing and HRA are complementary, and HRA can help identify and build resilient processes and systems. Development of Human Reliability for Modeling Human Performance Cacciabue (1998) and others (e.g., Boring, 2007b; Lüdke, 2005) have explained the importance of the simulation and modeling of human performance for HRA. In human-performance modeling, a virtual human (in the form of a cognitive simulation) interacts with virtual systems to reveal areas where human performance is degraded or enhanced in human-system interactions. Simulations address the dynamic nature of human performance in a way that has not been possible with classic static HRA methods. A chief advantage of incorporating HRA into human-performance modeling is that it provides a way of estimating the safety of novel equipment and configu- rations. It is reasonable to assume there will also be significant cost advantages to using modeling to screen new equipment virtually instead of configuring a 

HUMAN RELIABILITY ANALYSIS 109 FIGURE 3  The four phases of HRA integrated with CE. Boring Figure 3 simulator with new equipment and enlisting appropriate personnel (e.g., control R01394 room staff) to perform representative tasks (Boring et al., 2008). Human-performance modeling, utilizing insights from CE to provide a bitmapped fixed image reasonable and reliable simulation, has already been shown to be a powerful system-design tool in HFE (Foyle and Hooey, 2007). When elements of HRA (such as dynamically assigned PSFs) are included in human-performance model- ing, simulations can not only show if humans will interact successfully with a system, but can also provide a basis for determining the performance decrements and enhancements for particular system configurations. CONCLUSION In this brief paper I have outlined the three process phases typically associated with HRA, namely identification, modeling, and quantification. These three phases represent a historic evolution that should now evolve to include a fourth phase, error prevention, particularly in the design phase of systems (see Figure 3). Insights based on 25 years of experience with formal HRAs can now be applied to a process more closely aligned with HFE and CE. Insights derived from HRAs on the types and causes of human errors, as well as the likelihood and con- sequences of those errors, will ultimately facilitate the design of safer systems. REFERENCES Boring, R.L. 2002. Human-computer Interaction as Cognitive Science. Pp. 1767–1771 in Proceedings of the 46th Annual Meeting of the Human Factors and Ergonomics Society. Santa Monica, Calif.: Human Factors and Ergonomics Society. Boring, R.L. 2007a. Meeting Human Reliability Requirements through Human Factors Design, Testing, and Modeling. Pp. 3–8 in Risk, Reliability and Societal Safety. Vol. 1: Specialisation Topics. Proceedings of the European Safety and Reliability Conference (ESREL 2007), edited by T. Aven and J.E. Vinnem. London, U.K.: Taylor and Francis. Boring, R.L. 2007b. Dynamic Human Reliability Analysis: Benefits and Challenges of Simulating Human Performance. Pp. 1043–1049 in Risk, Reliability and Societal Safety. Vol. 2: Thematic Topics. Proceedings of the European Safety and Reliability Conference (ESREL 2007), edited by T. Aven and J.E. Vinnem. London, U.K.: Taylor and Francis. 

110 FRONTIERS OF ENGINEERING Boring, R.L., D.I. Gertman, T.Q. Tran, and B.F. Gore. 2008. Framework and Application for Model- ing Control Room Crew Performance at Nuclear Power Plants. Pp. 930-934 in Proceedings of the 52nd Annual Meeting of the Human Factors and Ergonomics Society. Santa Monica, Calif.: Human Factors and Ergonomics Society. Cacciabue, P.C. 1998. Modelling and simulation of human behaviour for safety analysis and control of complex systems. Safety Science 28: 97–110. Chang, Y.H.J., and A. Mosleh. 2007. Cognitive modeling and dynamic probabilisitic simulation of operating crew response to complex system accidents. Part 2: IDAC performance influencing factors model. Reliability Engineering and System Safety 29: 1014–1040. Foyle, D.C., and B.L. Hooey. 2007. Human Performance Modeling in Aviation. Boca Raton, Fla.: CRC Press. Galyean, W.J. 2006. Orthogonal PSF Taxonomy for Human Reliability Analysis. Pp. 1–5 in Proceed- ings of the 8th International Conference on Probabilistic Safety Assessment and Management, May 14–18, 2006, New Orleans, Louisiana. Paper PSAM-0281. Washington, D.C.: American Society of Mechanical Engineers. Hirschberg, S. 2004. Human Reliability Analysis in Probabilistic Safety Assessment for Nuclear Power Plants. CSNI Technical Opinion Papers No. 4, OECD NEA No. 5068. Issy-les-Moulineaux, France: OECD Nuclear Energy Agency. Hollnagel, E. 2006. Resilience—The Challenge of the Unstable. Pp. 9–17 in Resilience Engineering: Concepts and Precepts, edited by E. Hollnagel, D.D. Woods, and N. Leveson. Burlington, Vt.: Ashgate Publishing Company. Kadak, A.C., and T. Matsuo. 2007. The nuclear industry’s transition to risk-informed regulation and operation in the United States. Reliability Engineering and System Safety 92: 609–618. Lüdke, A. 2005. Kognitive Analyse formaler sicherheitskritischer Steuerungssysteme auf Basis eines integrierten Mensch-Maschine-Models. Berlin: Akademische Verlagsgesellschaft Aka GmbH. NASA (National Aeronautics and Space Administration). 2005. Human-Rating Requirements for Space Systems. NPR 8705.2A. Washington, D.C.: NASA Office of Safety and Mission Assurance. Nielsen, J. 1993. Usability Engineering. San Francisco, Calif.: Morgan Kaufman. Norman, D.A. 2002. Emotion and design: attractive things work better. Interactions Magazine 9(4): 36–42. O’Hara, J.M., J.C. Higgins, J.J. Persensky, P.M. Lewis, and J.P. Bongarra. 2004. Human Factors Engineering Program Review Model. NUREG-0711, Rev. 2. Washington, D.C.: U.S. Nuclear Regulatory Commission. Palanque, P., S. Basnyat, A. Blandford, R. Bernhaupt, R. Boring, C. Johnson, and P. Johnson. 2007. Beyond Usability for Safety Critical Systems: How to Be Sure (Safe, Usable, Reliable, and Evolvable)? Pp. 2133–2136 in CHI 2007 Conference Proceedings, Extended Abstracts. New York: Association for Computing Machinery. Sheridan, T.B. 2008. Risk, human error, and system resilience: fundamental ideas. Human Factors 50(3): 418–426. Swain, A.D. 1963. A Method for Performing a Human Factors Reliability Analysis. Monograph SCR- 686. Albuquerque, N.M.: Sandia National Laboratories. Swain, A.D., and H.E. Guttman. 1983. Handbook of Human Reliability Analysis with Emphasis on Nuclear Power Plant Applications (THERP). Final Report. NUREG/CR-1278. Washington, D.C.: U.S. Nuclear Regulatory Commission. U.S. Nuclear Regulatory Commission. 1975. Reactor Safety Study: An Assessment of Accident Risks in U.S. Commercial Nuclear Power Plants. WASH-1400, NUREG-75/014. Washington, D.C.: U.S. Nuclear Regulatory Commission. U.S. Nuclear Regulatory Commission. 2005. Good Practices for Implementing Human Reliability Analysis (HRA). NUREG-1792. Washington, D.C.: U.S. Nuclear Regulatory Commission.