Brazil-U.S. Workshop on Strengthening the Culture of Nuclear Safety and Security: Summary of a Workshop (2015)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

III Performance Assessment and Improvement of Safety and Security Culture The third session of the conference covered performance assessment and improvement of safety and security culture: methodological and deployment issues, and how to get a virtuous cycle of the process to achieve safety and secu- rity culture and those related to other organizational goals. The session included presentations from Michael O’Brien, Antonio Barroso, and Ivan de Souza Azevedo and was moderated by Marcos A. Viana Tavares. Mr. Tavares, representing Embraer, saw great similarity between the aero- nautics and nuclear fields, which can form the basis for an exchange of experi- ences. Experiences in Nuclear Security Culture – Michael O’Brien, U.S. Lawrence Livermore National Laboratory Mr. O’Brien provided background on his perspective beginning in 1989, when, after approximately 14 years working in the military and for the military at U.S. Department of Defense agencies, he entered the Department of Energy (DOE) when it was undergoing very rigorous inspections. Admiral James Wat- kins, the secretary of energy at the time, strongly emphasized safety culture and security culture, and the U.S. National Laboratories were found deficient. Mr. O’Brien was hired to correct deficiencies, conducting inspections of U.S. National Laboratories and contractors that did work for the Department of Energy. He was part of a DOE working group tasked with rewriting the regula- tions (called Orders). At that time, there was a policy emphasis, though it was defined more in terms of responsibilities than culture. People had responsibili- ties for safety and security all the way from the top down to the bottom. The working group placed an emphasis on performance-based practices and put more ownership of an organization at a lower level and on individuals’ respon- sibilities. It also started conducting self-assessments, which were a really critical 29 

30 Brazil-U.S. Workshop on Strengthening the Culture of Nuclear Safety and Security component. The organizations started to take responsibility, to police them- selves, and to identify and then remedy deficiencies and security issues before receiving an inspection from the government. Mr. O’Brien initially took some issue with the term culture in the context of safety and security deficiencies, but over time found that it really does reso- nate as culture. Different organizations do have different cultures. An agency could promulgate a certain attitude and belief system, and it does take a while to change those. He also discussed the common ground between safety and security. Both require a coordinated approach and there are sometimes conflicts, but safety and security have to work together. One common scenario in vulnerability analyses is planning for emergency evacuation. Work stops because there is a safety is- sue, and people egress out of the environment without securing their work. From a security standpoint, material is now vulnerable, and people could use the emergency situation as a means to take material out of the facility. We have to design procedures to meet both goals, to allow people to safely egress out of buildings, but to do it in a contained environment. Implementing a security culture has to start at the top. The Nuclear Security Summit process among heads of state resonates through all of our organizations in every participating country. Providing a safe and secure environment for nuclear material instills public confidence. Moving from a national-leadership level to operational considerations, many aspects of a strong security culture begin through hiring practices. During their careers the best applicants migrate into more sensi- tive positions with stricter requirements. The organization adds to their job func- tions and provides additional measures to assure that their work achieves the best results supported by a positive work environment. But we can also minimize problems by separation of duties, so that employ- ees do not have free access and their authorities are limited. One of the common problems in dealing with insider threats is access authorities, or access to sensitive areas. The basic principles we try to adhere to from a security standpoint are deter- rence, detection, delay, and response. Applying these principles to an insider threat protection program is more difficult than addressing external threats. The people that work in this environment are expected to have good, strong security culture, but they could also themselves become adversaries. It is very difficult to have de- terrence against an insider. Coworkers and systems can help with detection. Delay is difficult, but through separation of duties, some of those measures can be ap- plied. And response is done by a guard force or protective force, but oftentimes coworkers themselves provide that response. Mr. O’Brien stated that the security culture of a facility is reflected in peo- ple’s attitudes and the way that they conduct their business. We can tell whether an operation is done with efficiency and awareness, or lackadaisically. Opera- tion, how systems are maintained, is a reflection of their overall security culture. We tend to overemphasize technology and underemphasize the human aspect. Most of the focus in physical protection is on systems, but it takes people to 

Performance Assessment and Improvement of Safety and Security Culture 31 design, install, operate, test, and repair the systems properly. The operating pro- cedures for these systems have to be correct. Most security incidents point back to the human element. It is rare for a system itself to fail. Sometimes a system or sensor is misapplied in the wrong environment. Sometimes it is not maintained. Sometimes cameras are not as- sessed properly. But it generally points back to the human element. Oftentimes the solution is not more technology or better technology, but the human aspect that might need to be addressed in a particular situation. Mr. O’Brien presented several illustrative examples: About the incident at the Y-12 National Security Complex, Mr. O’Brien reported that there was a sense that everything was fine with security culture at the site, which is generally true for most U.S nuclear facilities. But due to fac- tors such as changes in management, decisions that result in apathy among workers, security forces, or operators, security culture can degrade rather quick- ly. It is thus critical that security culture be monitored closely; that we look for indicators that security culture might be degrading. And generally, when people examine major incidents such as at Y-12, they will often find ignored indicators. Another example is the major breach of security at Pelindaba in South Af- rica in 2007. Subsequent interviews with the management of this facility sug- gested a lack of recognition of the threat. It was stated that “people broke in, and they were just looking for some computers.” We must ask why anybody would breach a major secured facility just to steal computers. Adversaries will often exploit the weakest moments. At the time of the breach, attention was not fo- cused on security. A party was going on onsite, and in the Operations Center that was ultimately attacked, the individual that was supposed to be working at that time was handicapped. Instead, the adversaries were confronted with a woman, who had replaced the scheduled worker, and her fiancé, the man who was ultimately shot, who was there to keep her company. He tried to resist the adversaries, and he notified security, but security didn’t respond quickly, and he nearly died from his wounds. In addition, a separate team penetrated the facility and was confronted by the response force, and all the adversaries departed the facility without being apprehended. After the fact, questions arose about the security culture at this particular organization. Good security policies and regulations need to include all levels of man- agement, especially the worker. All must understand their responsibilities. Mr. O’Brien described the integrated work scope program his laboratory employs: For any new activity, managers must define the safety hazards as well as the security measures to put in place so that everyone involved in the activity under- stands the safety and security aspects. When that activity is conducted, any em- ployee has the right to stop work. Managers also look at the security environ- ment: If an activity requires a security guard or a security escort to be present, and if someone notices that the security measures are not correct, they will stop work until security corrects it. Empowering the employee is a powerful tool, and it can help avoid many problems. Having these types of mechanisms in place 

32 Brazil-U.S. Workshop on Strengthening the Culture of Nuclear Safety and Security and adhering to them with support from management are good indications of a strong security culture. Mr. O’Brien went on to provide a personal example of flagging security culture while conducting vulnerability assessments at a foreign nuclear facility. A manager wanted to speak with him, and while he was waiting for an escort to the office the other people said, “Aw, Mike, you know the way. Just go on in. We don’t need to go with you.” This was a severe security breach because here I am a foreigner and I am allowed to walk through their nuclear facility unescorted. I walked from where we were. And we were looking at some of the alarm sensors that were covered up by equipment with cobwebs growing over them. I went to this manager’s office and walked by a guard and waved, and he recognized me, so he told me to continue. Then, I walked by a guard asleep on the stairwell. So, I walked up to the office and met with my col- league. And the first thing he said was, “Why are you here? There’s no- body with you.” It just shocked him. This episode highlighted the differences between security culture and security awareness. Security awareness involves training conducted for employees, so that they understand security, but security culture goes deeper than that. The integrated work scope program links roles and responsibilities to cer- tain work activities and to training requirements. The training requirements go into a lab-wide database. And this training is also integrated with an access con- trol system (ACS). For example, if someone has not fulfilled the training that is required to enter a Category I nuclear area, access is not allowed by the ACS. Work cannot be conducted until training requirements have been met. There are also built-in, engineered features to keep adherence to training or other require- ments for work in controlled areas. If security requirements are not being met in the training system, a badge can be revoked. Lastly, he discussed the human reliability program, which evaluates in depth the risks, if any, that an employee in a sensitive position poses. As em- ployees migrate to more and more sensitive positions, they may need to adopt additional requirements. The human reliability program has not advanced as far as safety culture, but the goal is to add more filters, so that the potential to have an insider threat is further and further mitigated and reduced. Mr. O’Brien summarized by emphasizing that security awareness should permeate everyone from the director of an institute through every line of man- agement, down to the individual employees. This permeation can be assessed by examining among workers their attitudes, ability to understand security proce- dures, roles and responsibility for security, and ability to act accordingly when confronted with anomalies. We want people to be inquisitive and to question, but also to act responsibly and expeditiously. Action could be by an employee if he or she is empowered to act or it could be by someone who is in a security position. 

Performance Assessment and Improvement of Safety and Security Culture 33 Assessment Instruments for Safety Culture: What Are We Measuring? – Antonio Barroso, Instituto de Pesquisas Energéticas e Nucleares (Institute of Nuclear and Energy Research) Dr. Barroso outlined his presentation, focusing on measuring nuclear safe- ty culture by discussing the origin of the term safety culture and the guidance of the International Atomic Energy Agency (IAEA), the concept of security culture and how it is used by practitioners, the assessment instruments, and what we are measuring. The nuclear field has systems, equipment, controls, procedures, manage- ment, and people. The first conclusion of the International Nuclear Safety Advi- sory Group (INSAG) 1 is the importance of placing both authority and responsi- bility for the safety of the plant on a senior member of the operations staff of the plant. Of equal importance, formal procedures must be properly reviewed, ap- proved, and supplemented by the creation and maintenance of a nuclear safety culture. From that point on, the IAEA started to try to understand safety culture. They read Edgar Schein’s levels of culture, 1 which discusses the relationship between safety culture and real culture within an organization (see Figure 3-1). Dr. Barroso cited Monica Haage illustrating the three levels: basic assumptions, guiding principles, and espoused values. There are also beliefs, known strate- gies, and physical artifacts. Dr. Barroso then presented a time line of work on these principles, begin- ning with INSAG 1 in 1986. INSAG 4 appeared in 1991, and safety culture was recognized in INSAG 7, which also corrected, complemented, and expanded the work of INSAG 1. INSAG Series 15 was also about safety and security. After Fukushima in March 2012, Safety Report 74 discussed safety, the nuclear power plant projects, and a new TECDOC in 2013. These guidance documents were written by specialists, and they represent a consensus, but it is almost impossible to guarantee universal agreement. Creat- ing a standards document is a different process. It is much slower, and must en- sure greater consistency. People are the basic agents: Nothing happens in an organization if a per- son does not make a decision or take action. Dr. Barroso therefore suggested an addition to the standard definition “Safety culture is the assembly of systems, characteristics, mindsets, and attitudes of the organization and individual levels which assures that, as an overriding priority, nuclear safety issues receive the attention warranted by their significance.” He added, “and adequate resources, information, and actionable knowledge are empowered at the decision or action points where safety issues are dealt with.” He noted that the chief operations officer at the Fukushima nuclear power plant said that if he were in charge he would have flooded the cooling system 1 Schein, E. H., 2010. Organizational Culture and Leadership, 4th ed. Jossey-Bass: San Francisco. 

34 Brazil-U.S. Workshop on Strengthening the Culture of Nuclear Safety and Security FIGURE 3-1 Edgar Schein’s levels of culture. SOURCE: Adapted from Haage, Monica, 2010. “Oversight of Management Systems, Leadership and Safety Culture” First ICTP- IAEA School of Nuclear Energy Management, November 8, 2010, International Centre for Theoretical Physics, Trieste, Italy. with seawater, but at the time the utility was still thinking about the economic value of the plant. This is an example of lax culture, Dr. Barroso said. The report from the Japanese Commission 2 noted that the events that caused the Fukushima crisis happened before the earthquake. Decisions of minimal economic expense could have been made to help prevent the Fukushima accident. The accident was partially due to a Japanese culture that had shame and protection at its core. We need to direct resources to where safety and security issues are a problem. Believing that change starts with an awareness of safety culture, Dr. Bar- roso assessed the frequency of the term nuclear safety culture in the Scopus da- tabase of relevant journals and conferences. From 1997 to 2011, nuclear safety culture was largely absent from the nuclear industry literature. During the first period he looked at from 1990 to 2000, the term appeared in an average of 1.82 publications per year, with this number decreasing every year. During the sec- ond period from 2001 to 2011, this number increased slightly, but citations de- creased even more. This suggests that nuclear safety culture is not part of the agenda. Better measurement leads to better management. Documents from agen- cies may be valuable with in-depth and thorough insights, but they contain in- complete quantitative assessment models. 2 National Diet of Japan Fukushima Nuclear Accident Independent Investigation Commission (NAIIC), 2012. Available at http://warp.da.ndl.go.jp/info:ndljp/pid/3856371/ naiic.go.jp/en/report/. 

Performance Assessment and Improvement of Safety and Security Culture 35 He inquired whether safety culture is a reflective construct or a formative construct. Examples of the latter are the U.N. Human Development Index, which yields a number indicator. If a researcher asks about a well-balanced diet, regular exercise, and sleep every night, the formative result is informative for human health. It is something else if one asks, “Do I have a healthy lifestyle?” which is a reflective construct. To evaluate safety culture, one can try to identify the key variables that influence an organization’s safety performance, which leads to structural equations that are very different from the reflective construct. The way to deal with this is statistically from observation. His third example involved the direct link between blood alcohol level and intoxication. A formative construct would involve taking notes at a party on how many drinks each guest has and then quantifying blood alcohol level. On the other hand, a reflective construct is how drunk a guest is, measured by variables that are manifestations of this top variable. In this case, such indications could be a breathalyzer test, a physical reaction test, or a clocked memory test. In the formative construct, observed variables like drink number (which are not corre- lated) define the alcohol content. On the other side, observed variables are indi- cators of mental capacity and drunkenness level. These variables need to be cor- related. It is almost impossible to pass a breathalyzer test and fail a physical reaction test. With safety culture, he argued, we need a reflective construct to conduct re- liability tests and validity tests, and we need valid and reliable assessment instru- ments. We need to know the dimensionality of the construct, content validity, convergent validity, discriminant validity, and then predictive validity. Does the instrument indicate that the culture is indeed better? Validation is cumulative. Dr. Barroso looked at the nuclear safety literature on another similar data- base, the Web of Science, for validated assessment instruments. He also read abstracts and papers on assessment instruments and their validation and found only two articles. The first was by Lee from 1998. 3 The model has a top con- struct and several variables that are able to be measured individually: 172 items were observed through questionnaires from almost 6,000 participants. Lee did the exploratory factor analysis with a matrix rotation model and came to 38 fac- tors—a very high number—that cannot be directly measured. Working on vari- ances came to 19 factors and 81 items. This was a first validated instrument to build on, but nothing else appeared on the assessment of nuclear safety culture until an article last year from Lopes de Castro. 4 In order to assess the IAEA safety culture model, Lopes de Castro based the model on Table 1 of the IAEA Safety Report Series no. 42 (2005), in which the agency describes 5 characteris- tics and 37 attributes that should be used in evaluation and self-evaluation in- 3 Lee, T., 1998. “Assessment of Safety Culture at a Nuclear Reprocessing Plant.” Work & Stress. Vol. 12. n. 3 217-237. 4 Accident Analysis & Prevention 60, 231–244 (November 2013). Available at http:// www.sciencedirect.com/science/article/pii/S0001457513003291. 

36 Brazil-U.S. Workshop on Strengthening the Culture of Nuclear Safety and Security struments. The characteristics were used as factors or latent variables and attrib- utes as indicators or sampling variables. The validity was low with only moder- ate convergent validity and very poor discriminating validity. So there is not an instrument to assess nuclear safety culture that has been duly validated. Dr. Barroso advocated applying instruments from other fields to help as- sess safety in the nuclear field. Hospitals and medical organizations have taken many steps towards cumulative validation and are assessing the safety culture for the patient using a model developed in 2005. It has been applied and validat- ed in at least 5 countries and 10 different organizations. 5 The Synergy Between Safety Culture and Operational Excellence – Ivan de Souza Azevedo, Eletrobras Eletronuclear Mr. Azevedo stated the need to improve continuously to achieve a good safety culture and proceeded to discuss safety culture principles at Eletronuclear power plants. He said that Eletronuclear does not have specific indicators to assess safety culture because it is difficult to create such indicators, but he said that the company has other means to assess safety culture. One of the 16 companies of the Eletrobras system, Eletronuclear is subor- dinated to the Brazilian Ministry of Mines and Energy. Its mission is to generate safe and reliable electric power to an interconnected system throughout a coun- try about the size of Europe. The share of power produced by nuclear is very small, just over 3 percent, and most power comes from hydroelectric generation. Two nuclear power plants, ANGRA 1 and ANGRA 2, built with American and German technology respectively, are in operation. About 70 percent of the more than 1,800 employees work at the power station in the city of Angra dos Reis. The remaining workers are employed at the headquarters in Rio de Janeiro. Eletronuclear is working on the 1,405-megawatt ANGRA 3 reactor, with con- struction foreseen to be completed by the end of 2018. When ANGRA 3 is com- plete, 60 percent of all energy consumed in the state of Rio de Janeiro will come from these plants. Eletronuclear embraces the defense-in-depth concept. Human and organi- zational factors are at the root of all the accidents, he said. Fukushima was not a natural disaster, but a profoundly man-made disaster, as the investigation report concludes. Eletronuclear has adopted the most recent IAEA definition of safety culture from 2006. It conveys the same classic concept of INSAG 4, but adds “protection and safety.” These concepts are applied to all steps and procedures, from project development to special operations. The five main characteristics that the IAEA establishes as an indication of a strong safety culture are that safety (1) is a clearly recognized value, (2) is a 5 See Halligan, Michelle and Aleksandra Zecevic, February 8, 2011. “Safety culture in healthcare: a review of concepts, dimensions, measures and progress” Quality & Safety in Health Care. 

Performance Assessment and Improvement of Safety and Security Culture 37 leadership responsibility, (3) requires clearly established accountability, and (4) must be integrated into all activities, and (5) the organization is learning driven. The safety culture assessment program was only carried out in five power plants with promising results, but the program was interrupted. Eletronuclear requested one of these assessments, but, Mr. Azevedo said, the IAEA was unable to carry it out. Mr. Azevedo suggested that in addition to the IAEA, U.S. Nuclear Regu- latory Commission, and Institute of Nuclear Power Operations definitions of safety culture, one should also look to the World Association of Nuclear Opera- tors, which is a partner of Eletronuclear. The nuclear safety culture definitions from all these organizations talk about values, behaviors, leadership, and safety as a priority. Safety culture at organizations is internalized by individuals. It considers individual and technological variables and their organization. Founded in 1997, Eletronuclear was created from two companies with more than 20 years of experience: Furnas Centrais Elétricas (a regional power utility) built and operated ANGRA 1 and built ANGRA 2; and Nuclebrás Equi- pamentos Pesados (Nuclebrás Heavy Equipment) was an engineering and design company that created projects for ANGRA 2 and ANGRA 3. In 2009, Eletronu- clear became fully integrated into the Eletrobras System and changed its brand once again. This merger is an important case study in creating safety culture. It took a long time for assimilation of the two companies, and worker animosity became a concern. The chief executive officer asked for support from the IAEA to help the company to assess the level of its safety culture. The group developed a sur- vey instrument with 22 safety categories, a questionnaire with 70 questions that was distributed throughout the company. Eighty percent of the company partici- pated. The results of this first assessment showed that there was a satisfactory safety culture. The poor performance areas were motivation; job satisfaction; view of mis- takes; absence of safety versus production conflict; and working conditions in regard to time pressure, workload, and stress. The survey allowed management to address these issues. The ensuing integrated management policy was updated in 2004, will be updated again this year, and remains part of an ongoing review. Eletronuclear also established an integrated policy for the whole company, stating: “Nuclear safety is a priority, precedes productivity and economic aspects, and should never be impaired for any reason.” The IAEA helped convene a suc- cessful conference on safety culture in Rio de Janeiro with power plant operators from all over the world. The company increased involvement with the nuclear industry, adopted partly developed indicators, and developed a self-assessment cycle of internal and external assessments and audits. These include regulatory and nonregulatory self-assessments, and internal audits, both corporate and at the plants. When assessing nuclear safety culture, it is important to communicate with the greater world nuclear community to benefit the entire industry. Mr. Azevedo presented a performance indicator from the plants and a power history developed from experiences in other countries such as project and 

38 Brazil-U.S. Workshop on Strengthening the Culture of Nuclear Safety and Security design modifications, maintenance, alarms, and manual controls. Several plant operational indicators, such as readiness to supply energy to the system, were consolidated in a single indicator that also reflects the safety practices adopted by the company. The safe operation of Eletronuclear’s plants starts with good design and extends through operational practices. Reporting small events, train- ing personnel, and maintaining a good relationship with the regulating agencies are essential to a safe environment. Mr. Azevedo said that Eletronuclear needs to continue the efforts of com- municating the established safety policy so that everyone in the company can understand and internalize it. Good training is absolutely essential for this so that this policy is adopted throughout the company, so people understand why we are talking about safety all the time. It should also reflect a respect towards society, the surroundings, the environment, the important relationships with in- ternational organizations and the adherence to international conventions. When we talk about synergy of the operations and safety culture, we mean the integra- tion of all of this, Mr. Azevedo said. DISCUSSION A participant noted the importance of forensics in nuclear safety and its absence in the workshop. Forensics is essential to global safety and security. Brazil is starting to conduct studies on nuclear risk analysis based on articles on nuclear security failure. He asked whether Dr. Barroso had found related infor- mation in his work. Dr. Barroso responded that he believes forensics is a lot more relevant to se- curity than to safety, but it has a relationship to safety as well. But risk analysis is not part of the culture assessment. Quantitative analysis of nuclear safety culture should be far more developed than what is found in literature. Nuclear forensics is a useful intelligence tool and can be incorporated mainly in security culture. Mr. O’Brien commented that Dr. Barroso may be a bit too harsh on the nuclear community, and that the medical community is in fact not doing a better job. In the United States, the third leading cause of death with disease is iatro- genic disease, or disease caused by medical mistakes. A culture that produces in excess of 20,000, and maybe as much as 200,000, deaths a year is obviously not operating to a high level of safety, and there is much room for improvement. A National Academies report has been very critical in this area as well. 6 By a measure of outcomes, the nuclear industry has been quite successful because the level of fatal accidents is minuscule. 6 IOM, 1999. To Err is Human: Building a Safer Health System. Available at https:// www.iom.edu/~/media/Files/Report%20Files/1999/To-Err-is-Human/To%20Err%20is%20 Human%201999%20%20report%20brief.pdf. 

Performance Assessment and Improvement of Safety and Security Culture 39 Dr. Barroso responded that considering different assessment methods is valuable. Patient safety may not be very good in terms of total death, but patient safety culture is much more widely discussed than nuclear safety culture. Dr. Haber mentioned that after Chernobyl the U.S. Nuclear Regulatory Commission and the U.S. nuclear industry did not use the term safety culture, but a search for “organizational management” might find more documentation. The term safety culture was not widely used until the mid-late-nineties in the U.S. nuclear industries. A participant inquired what the right balance between preparing for insider and outsider threats is, given that each are very different. An outside terrorist might be interested in attacking a nuclear power plant and causing a lot of dam- age, not in consequences 10 years from now. On the other hand, investigating nuclear plant workers goes on continuously, and although it is very difficult to initiate, workers inside the plant know what they have to do to cause a real acci- dent. In addition, insiders can immediately cause an accident because of access beyond protection systems. Mr. O’Brien explained that assessments examine all threat possibilities, with outsiders trying to penetrate a facility and insiders acting alone to sabotage it. They also examine the possibility of an insider colluding with an outsider and at targets of opportunity within a nuclear facility. The goal is to balance the threat spectrum through measures that will reduce risks overall. Assessments look at the entire threat spectrum and develop scenarios that prevent sabotage of all the targets within a given location. Admiral Ellis outlined several themes from across the different sessions. He noted: • The importance of interaction among designers, operators, and overse- ers or regulators. This is important in the context of reactor safety, nu- clear materials control, accounting, and even aircraft safety. Today, systems are not just technical, but sociotechnical; technical systems are as important as the people who operate them. • The importance of leadership to take responsibility for implementation. • That measuring the effectiveness of organizations with respect to safety culture is critical to improving safety culture. The methodologies need to be sound and validated, but few exist. • The value of sharing experiences across different industrial sectors with thorough lists of best practices. • The need to balance openness with the need to keep some information, and even discussions, secret. As different countries have taken different approaches, these discussions with the United States and across indus- trial sectors are helpful. • That the transition to digital systems is another challenge, as many fa- cilities were designed before the Internet. This transition can lead to in- sider vulnerability by not fully understanding how the system behaves, 

40 Brazil-U.S. Workshop on Strengthening the Culture of Nuclear Safety and Security or through cyber intrusions and cybersecurity breaches. Countries can share best practices on insider threats as solutions are developed. • That the arrogance of excellence leads to complacency and a belief that we fully understand the systems. Safety culture and security culture are mutable, and can be changed for the better and for the worse. Change starts with leadership, its decisions, and its responses to failures. Admiral Ellis continued by mentioning the communication gaps between the risk analysis communities and the safety and security communities. On the importance of linking leadership to responsibility for implementation, he brought up the seven C’s of leadership, the first of which is commitment. A leader has to believe that an initiative is important and remain committed to it. We should factor in the idea of empowerment, he said. On continuing to aspire to better measurements of safety culture, he drew an analogy to the struggle to find a metric to measure readiness in the armed forces. Outputs, attributes, and aspects of good cultures can be measured, but there is no culture meter on a reactor control panel. It is worth considering and exploring and continuing to research, but if we wait for a perfect metric begin to build, encour- age, or in some cases demand a better culture, we will be waiting a while. Other members of the audience commented on Admiral Ellis’ remarks: • If we want to improve safety and security, we need an objective. • Formal surveys do not necessarily convert to an improvement in safety culture. There are definitive elements that contribute to and enhance good cultures, but a score of 89.2 on a safety culture exam should not necessarily be indicative of the quality or the integrity of the process. • There are some behavioral indicators that can signal that an organization has a good or bad culture of safety or security, such as worker satisfac- tion. Complaints to the Nuclear Regulatory Commission from employees onsite, for example, are an indicator, and there are other indicators that ought to trigger further assessments. • While there is no safety culture meter, there are many other indicators of a problem. Imperfect measurement does not mean that we should wait. We have to use whatever indicators we have today, even if they are not solid, to track progress. When we set goals, we have to have in- dicators. Assessors should be aware that we do not have a solid founda- tion for measurement in the nuclear area, just a scale. • Safety needs to be applied to the entire fuel cycle from design to disposal. • There are four M’s of risk assessment and management that are useful to follow: measure; minimize the risk through system design; manage the remaining risk; and then, in the case of failure, mitigate it in an ef- fective way. • Safety indicators and performance indicators cannot be joined with safety culture indicators. 

Performance Assessment and Improvement of Safety and Security Culture 41 A participant noted that embedded safety or security culture leads to ad- herence to requirements. Quoting an old admiral, he said, “I have seen clean ships that could not shoot, but I’ve never seen a dirty one that could.” A trend in failures, accidents, or mishaps is worth exploring further to uncover a safety culture issue. It may be just an incredible string of bad luck, but that is unlikely. A participant cautioned the group about indicators for measuring safety culture. Indicators are ways to examine outcome or performance, but the culture we are trying to understand is how you got to the performance. In a nuclear power plant, one indicator for evaluating maintenance technicians is how many times a task has to be redone due to an initially incorrect execution. The goal is to have a small amount of rework. A maintenance technician can follow proce- dures, do everything according to specifications, and perhaps issue the rework. He could also cut corners and do workarounds to meet the goal (to reduce re- work), and he will still meet the maintenance rework indicator. Indicators or metrics are not the same as understanding culture in an organization. A participant mentioned that often during a labor strike, particularly in the aviation sector, employees start to “follow the rules” to the letter, causing many flights to be delayed and canceled. These very small rules may need to be reex- amined, as safety and security are the properties of the reactions of the compo- nents of the system. Safety should make sense more at the higher or system level than at the lower level. A participant noted that a questioning attitude may be the single most im- portant element of safety culture. We cannot measure the attitude, but you can measure whether or not questions and issues are raised. Any organization that goes for an extended period of time with no one raising a safety issue, whether or not it is a real issue, exists with a nonfunctioning safety culture. You can at least benchmark attempts to raise issues and bring them to the attention of the management in one organization relative to other organizations. A participant said that regarding a questioning attitude in the military, people have a stereotypic view of the military, perhaps just as they do of chief executive officers in business. They believe that everything is declaratory. In any business there are leaders that are good and leaders that are bad. A bad lead- er was fond of saying three things. “When I want your opinion, I will give it to you.” “What good is power if I can’t abuse it?” And the final one was, “When the king is unhappy, one of the little people must die.” From this, one can get a sense of what the organization was like. On the other hand, a good leader has an open-door policy, values people in the organization, demands and asks questions himself, and does not construct situations ahead of time. While crisis can demand time compression and execu- tive judgment, when contributions from every level in the organization are al- lowed, the best outcomes are attained.