3

Progress, but Continuing Concerns

The National Nuclear Security Administration (NNSA) has made progress in implementing changes aimed at addressing some of the concerns raised by A New Foundation for the Nuclear Enterprise (the “Augustine-Mies” report) and other reports. In these areas it should continue building on the progress achieved. The necessary change will be impossible without leadership commitment. The panel has been impressed by the people who have played important roles in improving governance and management of the nuclear security enterprise, including former Administrator Klotz, Acting Deputy Administrator White, Acting Administrator Erhart, and several of the top staff members. This group is capable, positive, thoughtful, and forward-thinking. In addition, the panel was generally impressed by the level of partnering and collegiality between management and operating (M&O) contractor leadership and the field office managers (FOMs) at the three sites visited in 2017. Field offices appeared empowered to perform a good degree of localized decision making, and lines of authority seem to be clearer than what was described in previous studies. These encouraging changes were partially attributed to the elevation of FOMs to report directly to the Administrator, putting them on the same level with other senior NNSA officials. Communications between the FOMs and M&O leadership, and between FOMs and the Administrator, appear to be strong at the sites visited. The nascent work to improve site governance, and thereby a healthier relationship between the government and its M&O contractors, is an example of a promising step. This is discussed further in this chapter.

However, in spite of the signs of progress reported, there continue to be reports of ongoing frustrations and inefficiencies of the kind that the Augustine-Mies panel and other groups identified. The panel heard a number of such reports during its site visits, evidencing persistent concerns about NNSA’s management of the large-scale change necessary to implement the Augustine-Mies report’s recommendations. Some of these concerns are discussed later in this chapter. More generally, though, there is an urgent need for NNSA to develop a true vision to guide its efforts to reform governance and management and adjust some aspects of culture throughout the nuclear security enterprise. A much improved implementation plan for governance and management reform can be developed once such high-level planning has occurred.

NNSA’S DECEMBER 2016 GOVERNANCE AND MANAGEMENT IMPLEMENTATION PLAN IS INADEQUATE

NNSA has undertaken a number of activities aimed at responding to the recommendations of the Augustine-Mies report and other studies, as detailed in the December 2016 Department of Energy (DOE)-NNSA report to Congress, Governance and Management of the Nuclear Security Enterprise (the “Implementation Plan”), and elsewhere. Many of these activities appear to be constructive and useful. However, the panel concludes that this collection of activities, and its documentation in the Implementation Plan, is not sufficient to address the major systemic concerns identified by the Augustine-Mies report. Specifically, DOE/NNSA have not provided clear statements of how one would know that success has been achieved, nor why these and other ongoing activities are promising means for achieving

the desired outcomes. In addition, the panel finds that despite Administrator Klotz’s personal focus on “Mission First,” successful mission performance is not clearly and consistently articulated as a goal or constraint in the discussions of some key activities included in the Implementation Plan.

The Augustine-Mies report and Securing America’s Future: Realizing the Potential of the Department of Energy’s National Laboratories (the “CRENEL” report), along with others, have said that the nuclear security enterprise urgently needs some fundamental changes, not mere adjustments to management processes. (The panel believes the word “foundation” was chosen deliberately for the title of the Augustine-Mies report.) Long-term problems require a long-term strategic approach. However, the December 2016 Implementation Plan instead provides primarily a set of activities to address discrete issues. NNSA did not articulate a comprehensive view of the problems and of the necessary crosscutting changes, nor did it adequately describe how or why specific proposed activities will in fact address the concerns raised by the Augustine-Mies report. The Implementation Plan does not describe how NNSA will determine whether the steps it takes are successful. The panel is reminded that the Augustine-Mies report said, “If action is reasonably prompt, measurable progress should be observed very quickly—in a matter of a few months,”¹ and it went on to suggest 15 “concrete indicators of change” that should be observable within 2 years.

It is reasonable to ask whether it is even possible for DOE/NNSA to correct the problems identified by the Augustine-Mies report and elsewhere, given their long persistence. The panel has seen some evidence of improvement, but leadership needs to continue to give a high priority to reform of governance and management. Those in charge of reform efforts need to avail themselves of the best practices used by other organizations that have successfully changed their practices and culture, including for the strategic planning emphasized in this report. The panel has not examined whether a structural change in the organization is necessary.

Any attempt to create an operational plan for reforming governance and management needs to build on an overall vision of what success would look like—addressing the five serious concerns revisited in Chapter 1 in a way that is informed by the enterprise strategy recommended in Chapter 2 of this report. NNSA leadership should devise and own a clear strategy and plan for addressing long-term systemic governance and management problems. It is important that both the strategy and plan be recognized as crucial to the success of the nuclear security enterprise and to overall mission success.

Finding 3.1. The panel considers the December 2016 DOE-NNSA report to Congress, Governance and Management of the Nuclear Security Enterprise, to be inadequate in several dimensions. Rather than following a careful process of specifying goals and then articulating a plan to achieve them, NNSA has laid out actions it would take without linking them clearly to desired outcomes or explaining why the actions were selected. It does not consider how the various activities will interact to effect the needed changes nor does it convey how the activities will impact mission success. Of equal concern, it gives little indication of how change will be measured—there are no baselines—or how one would know that success has been attained. Furthermore, there is no plan for communicating and socializing the overall goals and progress throughout the enterprise. Such communication is necessary in order to promulgate changes, embed responsibilities for carrying out steps in the plan, and prepare for necessary adjustments to the culture across the enterprise.

___________________

¹ Congressional Advisory Panel on the Governance of the Nuclear Security Enterprise, 2014, A New Foundation for the Nuclear Enterprise: Report of the Congressional Advisory Panel on the Governance of the Nuclear Security Enterprise, http://cdn.knoxblogs.com/atomiccity/wpcontent/uploads/sites/11/2014/12/Governance.pdf?_ga=1.83182294.1320535883.1415285934, p. xvi.

NNSA has already embarked on useful activities aimed at reforming governance and management in response to the serious concerns identified by previous reports, but it still lacks a concept of operations and a real implementation plan, which would include the following elements:

1. A well-articulated statement of the intended concept of operations and goals (e.g., mission focus, simplicity, and clarity, as well as alignment of resources, organizations, and incentives) and what the intended result will be;
2. A plan for how to achieve the goals and intended results;
3. Active commitment to the goals and vision by senior-most leadership (at both NNSA and DOE);
4. A plan for how to accomplish the change, including centralized leadership and decentralized implementation;
5. Active involvement and engagement of personnel across the enterprise in planning and achieving the change;
6. Regularly scheduled reviews of progress against predetermined measures of effectiveness—with a visible cadence and a sense of urgency—that are conveyed across the enterprise and course corrections to be made as needed to accomplish the pre-set goals; and
7. A plan for communication and reinforcement of the desired attributes of the change through training, leadership activities, performance reviews, and ongoing continuous improvement programs.

The panel’s first report included the following Finding: “The change management process in place within NNSA is promising—it has addressed many foundational elements, such as obtaining top-level direction and involving participants from across the subcultures of the nuclear security enterprise. But the first steps of change are not yet fully embedded.”² This Finding reflected the panel’s recognition of the following foundational steps toward successful change: strong support from the NNSA Administrator and the previous DOE Secretary, appointment of a senior person (the head of the NNSA Office of Policy) to lead the change-management efforts, and that person’s success in enlisting several strong senior leaders to play key roles. However, in the time since its first report, the panel has become increasingly concerned about other important foundational elements of change management, items 1–7 above, which have not been established and promulgated. The absence of these elements after 2 years of effort is a major concern.

Recommendation 3.1. NNSA should expeditiously create an implementation plan to enable achievement of the governance and management changes driven by NNSA’s enterprise-wide strategic goals. This new implementation plan should link proposed actions explicitly to specific goals, including a timeline associated with each action, specification of who is responsible for which parts of the execution and who is accountable for the outcome, and measures to be used to gauge progress and impact.

This implementation plan is distinct from, but connected to, the enterprise strategic plan discussed in Chapter 2 and, specifically, Recommendation 2.1. Successful development of this plan, and execution of the actions described in it, will combine to create a path toward major change.

___________________

² National Academies of Sciences, Engineering, and Medicine and the National Academy of Public Administration, 2017, Report 1 on Tracking and Assessing Governance and Management Reform in the Nuclear Security Enterprise, Washington, DC: The National Academies Press, pp. 33-34.

SOME PROMISING DEVELOPMENTS ON WHICH TO BUILD

Improvements to Site Governance

One of the most promising initiatives to date has been NNSA’s work to improve site governance, particularly improving communication and transparency so that federal oversight can be applied most effectively and efficiently. This helps address the fourth serious concern of the Augustine-Mies report: “Dysfunctional relationships between the government and its [M&O] site operators, which has encouraged burdensome transactional oversight rather than management focus on mission execution.”³

One step toward improving site governance was the NNSA Administrator’s decision to give NNSA FOMs direct reporting lines to the Administrator. This step seems to be well targeted to the goals of Augustine-Mies Recommendation 9, which calls for the establishment of a simple, clear line-management operating structure that both synchronizes activities across programs, mission-support functions, and operating sites and provides leadership focus for key programs. (The Implementation Plan does not make that connection.) The text in the Augustine-Mies report that directly follows that recommendation provides an apt description of the purpose of that change: “An effective mechanism would solidify the decision authority of the Director and coordinate the efforts of all the key officials accountable for executing the program . . . . An effective mechanism will permit the participants to share information regularly across sites, programs, and functions. It will provide a clearinghouse for raising issues in the execution of programs and for considering strategies for resolving them.”⁴

To the extent that communication is coordinated through the FOMs, these goals may be met. As stated in that same text, “Over time, the discipline of exercising leadership and management roles through this mechanism will reinforce the needed management culture by improving communications, understanding, and working relationships.”⁵ That goal of adjusting management culture—to reestablish trusted relationships between government and its M&O contractors—is indeed the ultimate purpose of reform efforts.

Another promising step for site governance is NNSA’s development of an initiative that uses peer review to improve practices at individual sites. In a memorandum dated March 22, 2016, the NNSA Administrator laid out his top five governance and management initiatives for transforming NNSA into a highly effective organization. One of those initiatives briefly outlined how site governance should be restructured. NNSA elaborated on and memorialized this vision by promulgating a supplemental directive on August 12, 2016, entitled “Site Governance” (NNSA SD 226.1B). On November 23, 2016, DOE issued a policy statement to further clarify various federal officials’ roles and responsibilities in the governance of the National Laboratories across the Department (DOE P 112.1). Then, by memoranda issued on December 16, 2016, and June 7, 2017, Administrator Klotz established and chartered the Site Governance Executive Steering Committee to drive implementation of the NNSA’s site governance initiative.

The vision for site governance reform, as laid out by Administrator Klotz and elaborated in subsequent materials, has the following key elements:

• The government, the M&O site operator, and the contractor’s corporate parents are expected to work cooperatively to ensure mission performance.
• The roles of the federal personnel in headquarters program offices, functional support offices, and field offices in oversight of the site operation are clarified.

___________________

³ Congressional Advisory Panel, 2014, A New Foundation, p. x.

⁴ Ibid., p. 55.

⁵ Ibid., p. 55.

• Peer review will be used at each site to collaboratively review the contractor assurance system (CAS) and the effectiveness of the site governance arrangements there and to identify lessons learned and opportunities for improvement.

During its first year of operation, the Site Governance Executive Steering Committee developed guiding documents for its process, developed a design document for governance training for NNSA’s federal workforce (to be made available to the M&O workforce, too), set in motion the design for governance and leadership training to NNSA’s senior executives, and conducted two pilot peer reviews at Nevada National Security Site (NNSS) and Lawrence Livermore National Laboratory (LLNL). These are called peer reviews because the invited participants are explicitly asked to critique the CAS and its use so as to prompt improvements, and participants are likewise expected to contribute their varied perspectives to assess how effectively the partners at a given site are addressing typical governance challenges. Equally explicitly, these site reviews are not cast as “evaluations” or “assessments” wherein the site’s CAS is rated or ranked.

A report is developed by the peer review team in the weeks following each site’s session, highlighting notable practices and making recommendations for improvements. In some cases, these recommendations may apply across all sites or result in changes to the peer review process, or—as planned for 2018—lead to changes to key directives such as SD 226.1B. In addition, those involved are explicitly charged with improving the process each time, and interviewees explained that significant improvements were made between the first pilot peer review at NNSS and the second pilot peer review at LLNL.

Finding 3.2. Although measures of effectiveness have not yet been established to assess the benefits of the site-governance and management peer review process, the panel believes it represents a useful and promising approach that is already contributing to improved communication, better-defined roles and responsibilities at individual sites, and cross-enterprise learning.

Recommendation 3.2. The NNSA Administrator should ensure that measures of effectiveness are defined and tracked, and then use the site governance and management peer review process across NNSA as a mechanism for communicating and reinforcing shared values/behaviors, strengthening processes and relationships at each site, and improving the usefulness of the sites’ contractor assurance systems.

Other Promising Developments

Other actions taken in response to the Augustine-Mies report and other reports—as described in NNSA’s May 2015 Report to Congress on the final Augustine-Mies report, the December 2016 Implementation Plan, and elsewhere—are also promising. While the panel is unable to assess the efficacy of most of these actions because no initial goals were set and no measures of success have been developed, the steps described in the following sections appear to show progress, and they can be the foundation on which to build a more complete and strategic plan for change. The key step to ensuring that these actions result in the desired improvements is to implement the list of seven elements that follow Finding 3.1.

Sustained National Leadership

The first serious concern from the Augustine-Mies report, “A lack of sustained national leadership focus and priority,”⁶ is not within the control of NNSA. However, NNSA reports that it has taken steps to help raise the profile of nuclear security, by increasing the number of interactions it has with Congress.⁷ This effort, plus the recent release of the 2018 Nuclear Posture Review, should help raise the level of national attention to the nuclear security enterprise and its mission. However, NNSA’s actions cannot guarantee an increase in attention or involvement of national leadership.

Infrastructure and Project Management

Concern about infrastructure needs and the impact of deferred maintenance has been recognized within the nuclear security enterprise for years. Under the direction of the Associate Administrator of the Office of Safety, Infrastructure, and Operations (NA-50), an Infrastructure Roadmap was issued in March 2017 to improve NNSA infrastructure management.⁸ The panel was told that, using new data-driven, risk-informed infrastructure tools, NA-50 has made improvements and progress in tackling significant infrastructure needs, a fact that was also noted by field office and laboratory personnel during site visits.

The Infrastructure Roadmap is an encouraging development, in light of what the Augustine-Mies report said about NNSA’s infrastructure:

In addition, Augustine-Mies Recommendation 13 called for improvement in construction project management capabilities and adoption of improved practices. In 2013, before that report was released, the DOE Secretary began studying how to improve management of major projects, and certain practices were codified in policy in 2014–2015. According to Section III.B.8 of the Implementation Plan, “NNSA has been removed from the GAO high-risk list for projects under $750 million for the fourth year in a row [2013–2016, and] . . . has evolved, on a portfolio basis, from delivering its projects over budget to delivering them 5 percent under budget . . . .”¹⁰

___________________

⁶ Congressional Advisory Panel, 2014, A New Foundation, p. x.

⁷ NNSA’s Office of External Affairs recorded more than 250 briefings, meetings, hearings, and Administrator or Principal Deputy Administrator engagements with Congress from January through November 2017.

⁸ National Nuclear Security Administration, 2017, NNSA Infrastructure Roadmap: Safe Operations, Effective Infrastructure, Enterprise Services. https://nnsa.energy.gov/sites/default/files/nnsa/inlinefiles/nnsa_infrastructure_roadmap.pdf.

⁹ Congressional Advisory Panel, 2014, A New Foundation, p. 47.

¹⁰ DOE and NNSA, 2016, Governance and Management of the Nuclear Security Enterprise, Report to Congress, http://www.osti.gov/scitech/servlets/purl/1357927, p. 19.

Culture Change

In its first report, the panel suggested that NNSA should issue a clear statement of “commander’s intent” to provide guideposts for reforming and shaping behaviors across the enterprise. To date, the panel is not aware that this has been done, but it has learned that as part of developing training for the site governance and management peer review system, an online course is being developed that will include specified cultural behaviors for all governmental (and, ultimately, M&O) parties. This training, as it is made available, has the potential to help shape NNSA’s culture in beneficial ways.

Further, the head of the NNSA Office of Policy has used the panel’s observations from site visits to create a more uniform awareness among the Administrator’s direct reports that governance and management problems persist, which is a first step toward their recognizing that change is necessary.

The need for reconciling disparate attitudes and assumptions—workplace culture—was also illustrated anecdotally by a field office manager in reference to Administrator Klotz’s specific instruction that each field office should improve its relationship with its laboratory, to the extent of being “an advocate for the lab” when appropriate. Employees in that field office had mixed reactions to that goal, and the FOM had to work to develop a shared understanding among the staff with regard to their appropriate oversight roles. This FOM thus considered cultural change to be essential and worked with the staff to ensure that their working relationships with the laboratory were properly aligned with the Administrator’s values.

NNSA as a Desirable Workplace

Based on publicly available results from the annual Federal Employee Viewpoint Survey,¹¹ overall morale (satisfaction and engagement) among NNSA federal employees appears to have improved from 2015 to 2016, and again from 2016 to 2017. While morale and employee satisfaction of NNSA employees was not raised as an issue by the Augustine-Mies report, satisfaction and engagement are generally seen as important indicators of “organizational health.” Thus this improvement is an encouraging indication that may improve the chances of success for change initiatives.

Greater Involvement in Site Management and Operations by the Corporate Parents

NNSA has told the panel that it is working to encourage corporate parents of M&O contractors to take a more active and productive role in applying corporate methods of management, as was recommended by the Augustine-Mies report. NNSA’s Supplemental Directive on “Site Governance” (SD 226.1B) presents a conceptual framework that explicitly increases the role and visibility of corporate parents. As part of this, a representative of the site’s corporate parent participates in the peer review process for each site, and special attention is paid to identifying the strengths of CAS and management assurance systems that the corporate parent brings to the functioning of each location.

OPERATING CONCERNS IDENTIFIED FROM SITE VISITS AND INTERVIEWS

In addition to the crosscutting concern about NNSA’s change-management efforts, explained in the first section of this chapter, the panel observed some specific operating concerns from its site visits and interviews, broadly grouped into the following categories:

___________________

¹¹ Survey reports are available at https://www.opm.gov/fevs/.

• Uneven awareness of leadership’s overall message and focus;
• Burdensome and uneven oversight, including from non-NNSA sources, and continuing concerns about data calls;
• Incomplete or confused understanding of roles and responsibilities;
• Narrow interpretations and or/enforcement of contract provisions or rules that frustrate M&O personnel;
• Risk aversion by some federal personnel;
• Loss of values underlying the federally funded research and development center model: Scientists and engineers report feeling that as “contractors” they must focus on short-term tasks and schedules instead of pursuing innovative solutions or advances with longer-term payoffs;
• Fragmentation of funding;
• Occasional human resources issues (e.g., constraints that hinder rotational assignments, delays obtaining clearances) that may limit laboratories’/sites’ ability to attract and maintain the caliber and quantity of needed scientific and research talent; and
• At field offices, tight staffing to meet responsibilities and potential impact of retirements.

Anecdotal accounts of concerns are only suggestive, and systematic data collection is needed to properly examine these topics. Here the panel cites two illustrative examples of such concerns:

Interpretation of contract provisions. At one location, multiple laboratory staff raised concerns about the stringency with which certain federal rules are interpreted, especially with regard to contractor decisions about purchasing and construction. Contractor personnel felt they had to jump through as many requirement “hoops” as they would if they were federal employees, which undercuts the intended advantage of a government-owned/contractor-operated facility.

In contrast, a senior headquarters official expressed doubt that more stringent contract interpretations were driven by NNSA. He said that the recent trend has been to delegate more and more decision making to the M&O contractors, and it was unlikely that the government was imposing extra requirements and oversight. This indicates the need to continue to improve communications between contractors and field offices and between field offices and headquarters in order to sort out such issues and mitigate this kind of frustration.

Audits and data calls. While NNSA reports that the number of audits and data calls is decreasing, employees at several sites told the panel that the number of data calls seems to be unchanged. At least one laboratory is working to make CAS information accessible to field office staff, which may help the latter access more information directly rather than having to issue a data call.

Frustration with external audits and oversight was reported by personnel at every location. The number of investigations launched by DOE’s Office of the Inspector General, the Government Accountability Office, the Defense Nuclear Facilities Safety Board, as well as by state and local regulators is outside of NNSA’s control and poses an ongoing challenge. Staff at multiple sites said that external reviews are time-consuming, uncoordinated, burdensome, inefficient, typically compliance-based, focused on process rather than outcomes, and not always of much use when completed because they take so long.

In spite of these and other operational concerns, staff members at the sites the panel visited are nevertheless clearly engaged in and proud of their work. They tend to describe the frustrations associated with governance and management as annoyances they can adjust to—at the loss of some efficiency—rather than roadblocks to success. But, as explained in Chapter 1, problems with governance and management pose a real risk to the nation’s ability to provide adequate nuclear security.

THE NEED FOR DATA AND OTHER OBJECTIVE EVIDENCE

Data and other objective evidence (e.g., from structured interviews) are necessary in order to (1) effectively characterize situations that need corrections and (2) track whether changes under way are producing the desired effect. There is a fundamental need for NNSA to characterize its progress and demonstrate success in addressing the serious problems raised by the Augustine-Mies report and other reports. Currently this need, however, is often thwarted by a lack of data and other objective evidence needed to truly understand problems and evaluate progress. A specific illustrative example was mentioned in the panel’s first report:

Absent a solid base of evidence, NNSA appears to be relying on piecemeal and anecdotal information. For example, the reduction in complaints raised to the Acting Deputy Administrator and counts of particular types of data calls and audits were cited as a way to assess whether burdensome practices have declined. But some activities that feel burdensome are not captured through these counts, and numbers alone are not sufficient in any case because “burden” can result from attributes that are not captured by tracking the number of instances. Thus, more informative metrics are needed.

Another opportunity for improved understanding exists in connection with decisions about risk. Data on instances of disagreements about the acceptable level of risk, as apparently occurs occasionally between federal employees and laboratory or plant employees, could provide insight into the frequency and nature of such disputes, the positions of individuals who tend to disagree, and the pathway and time frame by which disputes are handled and resolved. Absent such basic data in an accessible format, it is difficult to find policies and actions that can help forestall or simplify future disputes.

Data and metrics are necessary to inform understanding of at least the following aspects of the nuclear security enterprise:

• Is the enterprise meeting its commitments, such as for mission deliverables?
• Is it providing state-of-the-art research and intelligence support?
• Is the enterprise investing sufficiently and effectively to sustain infrastructure and other capabilities?
• Are workers engaged? Is the work environment attracting and retaining the right caliber of people?
• Are operations understood and controlled well?
• Are associated risks reported and managed appropriately?
• Are available resources used effectively and focused on the right objectives?

Finding 3.3. NNSA lacks systematic data collection—tailored to inform well-specified questions in order to assess the scope and severity of its governance and management challenges and the effectiveness of its improvement efforts.

___________________

¹² National Academies of Sciences, Engineering, and Medicine and the National Academy of Public Administration, 2017, Report 1, p. 26.

For example, the apparent improvements in communication within the nuclear security enterprise—between field offices and M&O contractors, between field offices and headquarters, and between laboratories and others involved in regular meetings of boards, councils, and working groups—might offer more opportunities for disputes and process inefficiencies to be shared and resolved. But the panel is not aware of any concerted effort to monitor trends or to draw links between such issues and the governance and management problems. NNSA does not know whether such activities have improved trust, increased collaboration, improved transparency, and so on. Such data and links are necessary in order to define metrics of success or to clarify whether and how success can be defined and monitored.

Further, because currently governance and management problems are largely revealed anecdotally—through individual experiences—NNSA needs to monitor the level of satisfaction of employees throughout the nuclear security enterprise regularly. Just as NNSA benefits from insights revealed through the annual Federal Employee Viewpoint Survey, it needs to learn about the attitudes and levels of satisfaction of non-federal employees across the enterprise. Many of the M&O contractors may already conduct such employee satisfaction surveys, so there could be ways for NNSA to tap into or augment existing data and questions. This is not to say that workforce data are all that needs to be collected. Rather, they are highlighted here because they appear to offer useful basic information that might be available with relative ease.

Because surveying is an art as well as a science, useful data collection may best be contracted out to an experienced professional firm. Such a firm can also navigate privacy constraints and policies and laws that aim to protect workers from intrusive burdens.

Recommendation 3.3. As a first step toward meeting the need for objective evidence and data, NNSA should begin surveying the entire workforce of the nuclear security enterprise (possibly by leveraging existing surveys) so as to gain understanding of attitudes and engagement throughout the enterprise and insight about specific worker concerns.

Such an enterprise-wide survey would provide a foundation for tracking the effects of governance and management reform while helping to clarify what additional data are needed. In addition to gathering basic information about attitudes throughout the enterprise, NNSA should also routinely assess its own progress toward meaningful change. But this will be useful only to the extent that it reflects an insightful understanding of how much progress has been made against the ultimate goals. Merely indicating that a near-term action has been completed, for example, without also reflecting how effective that action has been against a meaningful element of change, is not helpful.

Any such regular, routine review and assessment also must be widely shared across NNSA so as to continue to engage all in achieving the strategic goals NNSA has set. Thus, it would be important that such scoring be rigorous if it is to be credible and effective in providing incentives that produce lasting results.

This page intentionally left blank.

Appendixes

This page intentionally left blank.